Friday, April 7, 2017

LineageOS: Rebuilding the XT1095 Victara Image to work with the Chinese version XT1085 (Hardcore Raw Edition)

NOTE: The text that follows is not edited and may not even be coherent.  I don't have time to make it nice and pretty for the public, but these are my "dump notes" I use when rebuilding the XT1095 ROMs to work on the XT1085.  They're better from a technical perspective than the other article I wrote, although they lack the explanation as to why it's necessary...hopefully I'll have the energy (or focus) to come back to this later and edit it.  If I remember correctly, these are my personal edits to an XDA article on a very similar subject...

UPDATE: With Lineage there are two extra steps now to convert the image to a sparse image and to convert the file_contexts.bin file to file_contexts (no .bin).  I just followed these instructions to convert a March 2018 image of LineageOS 14.1 and I realize that these instructions are pretty rough.  Very sorry - I hope to update them soon.

XDA: [DEV] sdat2img 1.0 - img2sdat 1.2 ◦ Unpack/re-pack android DAT files

Ubuntu x64 - Guide
A. Prerequisites | Getting Ready
1. You must run all of these commands from an Ubuntu x64 operating system.  I am using Ubuntu LTS 16.04.
2. 10 GB of free space (for the temporary files)
3. Download all of the scripts listed on the XDA site.  You'll need and put them in a folder on your Linux system. I created one in my home directory called \BUILD.  Here are the download links as of 2017 Aug 9:

          Note: Make each script executable (including the sefcontext binary under lib64 for the convert tool

 4. Install all package dependencies that you'll need to run the scripts, namely:
  • python3
  • build-essential
  • android-tools-fsutils
5. Download the ROM ZIP file that you want to edit. I use LineageOS 14.1 for Victara
6. Extract the ROM ZIP file into a new folder on your computer. I make a new folder called \VictaraLineage in my home directory. The typical ROM will contain the following files:
  •  boot.img
  •  file_contexts.bin
  •  system.new.dat
  •  system.patch.dat
  •  system.transfer.list
  •  \install
  •  \META-INF
  •  \system
7. Copy the system.transfer.list, system.new.dat, and file_contexts.bin files into the \BUILD folder with your scripts.

B. Converting the Image to EXT4
1. You will first use the sdat2img binary; the usage is as follows. CD into the BUILD folder:
          ./sdat2img.py <transfer_list> <system_new_file> <system_ext4>
            - <transfer_list> = input, system.transfer.list from rom zip
            - <system_new_file> = input, system.new.dat from rom zip
            - <system_ext4> = output ext4 raw image file
          an quick example:
            sudo ./sdat2img.py system.transfer.list system.new.dat new_system.img

 Note: By running this command the script will create a new file called my_new_system.img which is the raw ext4 image.  This IMG file will be about 2.9GB.

 2. Delete the system.transfer.list and system.new.dat files from the \BUILD directory to free up space and avoid conflicts or confusion later.

 C. Convert the file_contexts.bin file
Nougat now uses a file_contexts file instead of the old file_contexts.bin. Run the following script to create the older file format:

           sudo ./convert.sh
Copy the file from the "convert" subfolder back into the BUILD folder.

D. Decompress and Re-compress the image to a new size
3. Mount the ext4 raw image into an output folder so we can see apks/jars etc.  To do this we need to type this command:

          sudo mount -t ext4 -o loop new_system.img LinOS14

 Note: To increase the likelihood of success, I first create the mount point's folder...I'm not a seasoned Linux guy, so this might not be necessary, but it works well for me when I do.

As you can see there is a new folder called output which we can edit/modify/delete your files. I see the following on a recent LineageOS 14.1 image:

  • build.prop
  • recovery-from-boot.p
  • \addon.d
  • \app
  • \bin
  • \etc
  • \fake-libs
  • \fonts
  • \framework
  • \lib
  • \media
  • \priv-app
  • \tts
  • \usr
  • \vendor
  • \xbin

E. To compress it back to a raw ext4 image, to do this we need the make_ext4fs binary. Make sure you have the file_contexts file (taken from the Rom zip) inside the make_ext4fs path. Then type this (got issues? see here)
./make_ext4fs -T 0 -S file_contexts -l 1073741824 -a system system_new.img output/
You will get the new raw ext4 image called 'system_new.img' ready for the next step.
<<My command for the XT1085 looks like:
sudo make_ext4fs -T 0 -S file_contexts -l 2046820352 -a system new_raw_system.img LinOS14/
Unmount the image file

sudo umount LinOS14

 Step 3 - Converting = EXT4 (raw image) -> IMG (sparse image)

 Now we need to convert the ext4 raw image into a sparse image. For this you need img2simg binary in the BUILD directory (see pre-requisites)
The usage is simple:  img2simg <raw_image_file> <sparse_image_file>


 My command is
img2simg new_raw_system.img new_sparse_system.img

 Step 4 - Converting = EXT4 (raw image) -> DAT (sparse data)
Now we need the img2sdat binary, the usage is very simple:
./img2sdat.py <system_sparse_img>
- <system_sparse_img> = name of input ext4 raw image file
and a quick example:
sudo ./img2sdat my_new_system.img
<<My command for the XT1085 looks like:
 ./img2sdat.py new_sparse_system.img
Type "4" and enter for Android Nougat 7.0
The script output will be a new

  • system.transfer.list
  • system.patch.dat
  • system.new.dat


 Step 5 Move the New Files Back into the ZIP
A. Copy the system.transfer.list and system.new.dat into the original unzipped ROM image folder, overwriting the originals (ignore the system.patch.dat file)
B. Rezip the files into a new ZIP, and for consistency's sake rename the ZIP exactly like the original ZIP but with XT1085VERSION at the end
C. Copy that ZIP file away from your BUILD and SCRIPT folders to avoid deleting it when cleaning up
D. CLEANUP
 Delete build information...it's big
 .img
at