Bundling data files with PyInstaller (–onefile)

Posted on

Solving problem is about exposing yourself to as many situations as possible like Bundling data files with PyInstaller (–onefile) and practice these strategies over and over. With time, it becomes second nature and a natural way you approach any problems in general. Big or small, always start with a plan, use other strategies mentioned here till you are confident and ready to code the solution.
In this post, my aim is to share an overview the topic about Bundling data files with PyInstaller (–onefile), which can be followed any time. Take easy to follow this discuss.

Bundling data files with PyInstaller (–onefile)

I’m trying to build a one-file EXE with PyInstaller which is to include an image and an icon. I cannot for the life of me get it to work with --onefile.

If I do --onedir it works all works very well.
When I use --onefile, it can’t find the referenced additional files (when running the compiled EXE). It finds the DLLs and everything else fine, just not the two images.

I’ve looked in the temp-dir generated when running the EXE (Temp_MEI95642 for example) and the files are indeed in there. When I drop the EXE in that temp-directory it finds them. Very perplexing.

This is what I’ve added to the .spec file

a.datas += [('images/icon.ico', 'D:\[workspace]\App\src\images\icon.ico',  'DATA'),
('images/loaderani.gif','D:\[workspace]\App\src\images\loaderani.gif','DATA')]

I should add that I have tried not putting them in subfolders as well, didn’t make a difference.

Edit: Marked newer answer as correct due to PyInstaller update.

Answer #1:

Newer versions of PyInstaller do not set the env variable anymore, so Shish’s excellent answer will not work. Now the path gets set as sys._MEIPASS:

def resource_path(relative_path):
    """ Get absolute path to resource, works for dev and for PyInstaller """
    try:
        # PyInstaller creates a temp folder and stores path in _MEIPASS
        base_path = sys._MEIPASS
    except Exception:
        base_path = os.path.abspath(".")
    return os.path.join(base_path, relative_path)
Answered By: Arboreal Shark

Answer #2:

pyinstaller unpacks your data into a temporary folder, and stores this directory path in the _MEIPASS2 environment variable. To get the _MEIPASS2 dir in packed-mode and use the local directory in unpacked (development) mode, I use this:

def resource_path(relative):
    return os.path.join(
        os.environ.get(
            "_MEIPASS2",
            os.path.abspath(".")
        ),
        relative
    )

Output:

# in development
>>> resource_path("app_icon.ico")
"/home/shish/src/my_app/app_icon.ico"
# in production
>>> resource_path("app_icon.ico")
"/tmp/_MEI34121/app_icon.ico"
Answered By: max

Answer #3:

All of the other answers use the current working directory in the case where the application is not PyInstalled (i.e. sys._MEIPASS is not set). That is wrong, as it prevents you from running your application from a directory other than the one where your script is.

A better solution:

import sys
import os
def resource_path(relative_path):
    """ Get absolute path to resource, works for dev and for PyInstaller """
    base_path = getattr(sys, '_MEIPASS', os.path.dirname(os.path.abspath(__file__)))
    return os.path.join(base_path, relative_path)
Answered By: Shish

Answer #4:

Perhaps i missed a step or did something wrong but the methods which are above, didn’t bundle data files with PyInstaller into one exe file. Let me share the steps what i have done.

  1. step:Write one of the above methods into your py file with importing sys and os modules. I tried both of them. The last one is:

    def resource_path(relative_path):
    """ Get absolute path to resource, works for dev and for PyInstaller """
        base_path = getattr(sys, '_MEIPASS', os.path.dirname(os.path.abspath(__file__)))
        return os.path.join(base_path, relative_path)
    
  2. step: Write, pyi-makespec file.py, to the console, to create a file.spec file.

  3. step: Open, file.spec with Notepad++ to add the data files like below:

    a = Analysis(['C:\Users\TCK\Desktop\Projeler\Converter-GUI.py'],
                 pathex=['C:\Users\TCK\Desktop\Projeler'],
                 binaries=[],
                 datas=[],
                 hiddenimports=[],
                 hookspath=[],
                 runtime_hooks=[],
                 excludes=[],
                 win_no_prefer_redirects=False,
                 win_private_assemblies=False,
                 cipher=block_cipher)
    #Add the file like the below example
    a.datas += [('Converter-GUI.ico', 'C:\Users\TCK\Desktop\Projeler\Converter-GUI.ico', 'DATA')]
    pyz = PYZ(a.pure, a.zipped_data,
         cipher=block_cipher)
    exe = EXE(pyz,
              a.scripts,
              exclude_binaries=True,
              name='Converter-GUI',
              debug=False,
              strip=False,
              upx=True,
              #Turn the console option False if you don't want to see the console while executing the program.
              console=False,
              #Add an icon to the program.
              icon='C:\Users\TCK\Desktop\Projeler\Converter-GUI.ico')
    coll = COLLECT(exe,
                   a.binaries,
                   a.zipfiles,
                   a.datas,
                   strip=False,
                   upx=True,
                   name='Converter-GUI')
    
  4. step: I followed the above steps, then saved the spec file. At last opened the console and write, pyinstaller file.spec (in my case, file=Converter-GUI).

Conclusion: There’s still more than one file in the dist folder.

Note: I’m using Python 3.5.

EDIT: Finally it works with Jonathan Reinhart’s method.

  1. step: Add the below codes to your python file with importing sys and os.

    def resource_path(relative_path):
    """ Get absolute path to resource, works for dev and for PyInstaller """
        base_path = getattr(sys, '_MEIPASS', os.path.dirname(os.path.abspath(__file__)))
        return os.path.join(base_path, relative_path)
    
  2. step: Call the above function with adding the path of your file:

    image_path = resource_path("Converter-GUI.ico")
    
  3. step: Write the above variable that call the function to where your codes need the path. In my case it’s:

        self.window.iconbitmap(image_path)
    
  4. step: Open the console in the same directory of your python file, write the codes like below:

        pyinstaller --onefile your_file.py
    
  5. step: Open the .spec file of the python file and append the a.datas array and add the icon to the exe class, which was given above before the edit in 3’rd step.
  6. step: Save and exit the path file. Go to your folder which include the spec and py file. Open again the console window and type the below command:

        pyinstaller your_file.spec
    

After the 6. step your one file is ready to use.

Answered By: Jonathon Reinhart

Answer #5:

Instead for rewriting all my path code as suggested, I changed the working directory:

if getattr(sys, 'frozen', False):
    os.chdir(sys._MEIPASS)

Just add those two lines at the beginning of your code, you can leave the rest as is.

Answered By: dildeolupbiten

Answer #6:

Slight modification to the accepted answer.

def resource_path(relative_path):
    """ Get absolute path to resource, works for dev and for PyInstaller """
    if hasattr(sys, '_MEIPASS'):
        return os.path.join(sys._MEIPASS, relative_path)
    return os.path.join(os.path.abspath("."), relative_path)
Answered By: Bobsleigh

Answer #7:

Using the excellent answer from Max and This post about adding extra data files like images or sound & my own research/testing, I’ve figured out what I believe is the easiest way to add such files.

If you would like to see a live example, my repository is here on GitHub.

Note: this is for compiling using the --onefile or -F command with pyinstaller.

My environment is as follows.


Solving the problem in 2 steps

To solve the issue we need to specifically tell Pyinstaller that we have extra files that need to be “bundled” with the application.

We also need to be using a ‘relative’ path, so the application can run properly when it’s running as a Python Script or a Frozen EXE.

With that being said we need a function that allows us to have relative paths. Using the function that Max Posted we can easily solve the relative pathing.

def img_resource_path(relative_path):
    """ Get absolute path to resource, works for dev and for PyInstaller """
    try:
        # PyInstaller creates a temp folder and stores path in _MEIPASS
        base_path = sys._MEIPASS
    except Exception:
        base_path = os.path.abspath(".")
    return os.path.join(base_path, relative_path)

We would use the above function like this so the application icon shows up when the app is running as either a Script OR Frozen EXE.

icon_path = img_resource_path("app/img/app_icon.ico")
root.wm_iconbitmap(icon_path)

The next step is that we need to instruct Pyinstaller on where to find the extra files when it’s compiling so that when the application is run, they get created in the temp directory.

We can solve this issue two ways as shown in the documentation, but I personally prefer managing my own .spec file so that’s how we’re going to do it.

First, you must already have a .spec file. In my case, I was able to create what I needed by running pyinstaller with extra args, you can find extra args here. Because of this, my spec file may look a little different than yours but I’m posting all of it for reference after I explain the important bits.

added_files is essentially a List containing Tuple’s, in my case I’m only wanting to add a SINGLE image, but you can add multiple ico’s, png’s or jpg’s using ('app/img/*.ico', 'app/img') You may also create another tuple like soadded_files = [ (), (), ()] to have multiple imports

The first part of the tuple defines what file or what type of file’s you would like to add as well as where to find them. Think of this as CTRL+C

The second part of the tuple tells Pyinstaller, to make the path ‘app/img/’ and place the files in that directory RELATIVE to whatever temp directory gets created when you run the .exe. Think of this as CTRL+V

Under a = Analysis([main..., I’ve set datas=added_files, originally it used to be datas=[] but we want out the list of imports to be, well, imported so we pass in our custom imports.

You don’t need to do this unless you want a specific icon for the EXE, at the bottom of the spec file I’m telling Pyinstaller to set my application icon for the exe with the option icon='app\img\app_icon.ico'.

added_files = [
    ('app/img/app_icon.ico','app/img/')
]
a = Analysis(['main.py'],
             pathex=['D:\Github Repos\Processes-Killer\Process Killer'],
             binaries=[],
             datas=added_files,
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          [],
          name='Process Killer',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          upx_exclude=[],
          runtime_tmpdir=None,
          console=True , uac_admin=True, icon='app\img\app_icon.ico')

Compiling to EXE

I’m very lazy; I don’t like typing things more than I have to. I’ve created a .bat file that I can just click. You don’t have to do this, this code will run in a command prompt shell just fine without it.

Since the .spec file contains all of our compiling settings & args (aka options) we just have to give that .spec file to Pyinstaller.

pyinstaller.exe "Process Killer.spec"
Answered By: Krishna Balan

Answer #8:

The most common complaint/question I’ve seen wrt PyInstaller is “my code can’t find a data file which I definitely included in the bundle, where is it?”, and it isn’t easy to see what/where your code is searching because the extracted code is in a temp location and is removed when it exits. Add this bit of code to see what’s included in your onefile and where it is, using @Jonathon Reinhart’s resource_path()

for root, dirs, files in os.walk(resource_path("")):
    print(root)
    for file in files:
        print( "  ",file)
Answered By: Jtonna

Leave a Reply

Your email address will not be published.