Consider publishing releases to PyPI

[jeremycline]

Hi,

As background, we're considering using LISA to run tests on Fedora's Azure cloud images (and possibly other cloud images later) so I'm investigating getting that set up to run automatically when a new Fedora image is built.

For various reasons, it would be convenient to have LISA releases available on PyPI, or at least installable from a wheel. Based on my experimentation it seems that currently LISA relies on at least the examples and microsoft directories that are outside the package. My question is, are you open to the idea of tweaking the repository layout so wheels and sdists can be built/installed without needing to git-clone the repository, and then set up publishing to PyPI for each tag?

I am happy to help with the work to make that happen, but wanted to see if that's something the project was open to (or has previously tried and hit a blocker)?

[squirrelsc]

Thanks for suggesting the improvements. We tried as you mentioned, but the main concern is restructuring the folder. There are about 10 downstream repos, many of which are outside our control. Still, I think it's worth doing now if you're able to help. But if you've changed your mind, you could try using LISA dockers instead—it might be easier than cloning and installing.

The new structure would be like below. (lisa is in pipy already)

mslisa.<current lisa.xxx like mslisa.tools>
mslisa.<current microsoft.xxx like mslisa.microsoft.xxx>
mslisa.examples

[jeremycline]

Thanks for suggesting the improvements. We tried as you mentioned, but the main concern is restructuring the folder. There are about 10 downstream repos, many of which are outside our control. Still, I think it's worth doing now if you're able to help. But if you've changed your mind, you could try using LISA dockers instead—it might be easier than cloning and installing.

Ah, interesting. Is symlinking an option? Something like this? With that commit I can build lisa with python -m build, install the wheel which includes the examples and microsoft directories within the package, and run lisa which loads the examples/runbook/hello_world.yml. I'm sure there's other references to the files I'd need to fix up to make things work only from the wheel, but that's the general idea.

[squirrelsc]

Thanks for the thorough investigation and proposal!

1. What if we create the links in reverse, so no files need to be moved?
2. Have you tested whether it works with pip install ., nox commands for development?

[jeremycline]

Thanks for the thorough investigation and proposal!

1. What if we create the links in reverse, so no files need to be moved?

I'm pretty sure this won't work for two reasons. The first is that the files will be outside the package, so wheels won't include them (but they will be part of the sdist). The second is that wheels also don't support symbolic links particularly well (see the related PEP). Out of curiosity I tried it and the wheel just doesn't include the symlink at all (in addition to not having the actual directories they point to, of course).

What do you anticipate breaking if the directories are moved and symlinked to? Knowing the types of things you're concerned about might help me think of alternatives or mitigations.

3. Have you tested whether it works with pip install ., nox commands for development?

I just checked and both work fine, I think:

$ nox
<snip>
nox > Session dev was successful.                                                                                                                                                                                                                                                          
nox > Ran multiple sessions:                                                                                                                                                                                                                                                               
nox > * test: success                                                                                                                                                                                                                                                                      
nox > * example: success                                                                                                                                                                                                                                                                   
nox > * coverage: success                                                                                                                                                                                                                                                                  
nox > * black: success                                                                                                                                                                                                                                                                     
nox > * isort: success                                                                                                                       
nox > * flake8: success                                                                                                                      
nox > * pylint: success                                                                                                                      
nox > * mypy: success                                                                                                                        
nox > * docs: success                                                                                                                        
nox > * dev: success  

and

❯ pip install -e .[azure]

❯ pip freeze | grep lisa
-e git+https://github.com/microsoft/lisa.git@c41c21aa45c167894201b7ab03bc9be8661cce43#egg=lisa

❯ python
Python 3.13.7 (main, Aug 14 2025, 00:00:00) [GCC 15.2.1 20250808 (Red Hat 15.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lisa
>>>

❯ lisa
baremetal package is not installed. [No module named 'smb']
aws package is not installed. [No module named 'boto3']
libvirt package is not installed. [No module named 'libvirt']
Python version: 3.13.7 (main, Aug 14 2025, 00:00:00) [GCC 15.2.1 20250808 (Red Hat 15.2.1-1)]
local time: 2025-09-12 13:50:58.346860-04:00
git head: fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
submodules: fatal: not a git repository (or any parent up to mount point /)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
run log path: /home/jcline/.virtualenvs/lisa-editable/runtime/log/20250912/20250912-175058-346, working path: /home/jcline/.virtualenvs/lisa-editable/runtime/working/20250912/20250912-175058-346
2025-09-12 17:50:58.363[139828228976384][INFO] lisa.init[runbook] loading runbook: /home/jcline/devel/github/microsoft/lisa/lisa/examples/runbook/hello_world.yml
2025-09-12 17:50:58.364[139828228976384][INFO] lisa.init[module] loading Python extensions from /home/jcline/devel/github/microsoft/lisa/lisa/examples/testsuites
2025-09-12 17:50:58.365[139828228976384][INFO] lisa.init[test] registered test suite 'HelloWorld' with test cases: 'hello, bye'
2025-09-12 17:50:58.365[139828228976384][INFO] lisa.init[test] registered test suite 'MultipleNodesDemo' with test cases: 'os_info, verify_tcp_connection'
2025-09-12 17:50:58.366[139828228976384][INFO] lisa.init[test] registered test suite 'WithScript' with test cases: 'script'
2025-09-12 17:50:58.366[139828228976384][INFO] lisa.init[runbook] run name is 'lisa-None-20250912-175058-346'
2025-09-12 17:50:58.464[139828228976384][INFO] lisa.init[selector] selected count: 5
2025-09-12 17:50:58.464[139828228976384][INFO] lisa.init[selector] HelloWorld.hello, area: demo, category: functional, tags: [], priority: 0
2025-09-12 17:50:58.464[139828228976384][INFO] lisa.init[selector] HelloWorld.bye, area: demo, category: functional, tags: [], priority: 1
2025-09-12 17:50:58.464[139828228976384][INFO] lisa.init[selector] MultipleNodesDemo.os_info, area: demo, category: functional, tags: [], priority: 1
2025-09-12 17:50:58.464[139828228976384][INFO] lisa.init[selector] MultipleNodesDemo.verify_tcp_connection, area: demo, category: functional, tags: [], priority: 2
2025-09-12 17:50:58.464[139828228976384][INFO] lisa.init[selector] WithScript.script, area: demo, category: functional, tags: [], priority: 1 
2025-09-12 17:50:58.483[139827859904192][INFO] lisa.env[customized_0].deploy deploying environment: customized_0
2025-09-12 17:50:58.484[139827859904192][INFO] lisa.env[customized_0].deploy deployed in 0.001 sec
2025-09-12 17:50:58.488[139827868296896][INFO] lisa.env[customized_0].node[0] initializing node '' local
2025-09-12 17:50:58.543[139827859904192][INFO] lisa.case[hello][lisa_0_0] test case 'HelloWorld.hello' is running
2025-09-12 17:50:58.571[139827849414336][INFO] lisa.case[hello][lisa_0_0] before test case
2025-09-12 17:50:58.574[139827849414336][INFO] lisa.case[hello][lisa_0_0] release: '#1 SMP PREEMPT_DYNAMIC Thu Sep 11 17:46:54 UTC 2025', version: '6.16.7-200.fc42.x86_64', hardware: 'x86_64', os: 'GNU/Linux'
2025-09-12 17:50:58.595[139827841021632][INFO] lisa.case[hello][lisa_0_0] after test case
2025-09-12 17:50:58.596[139827859904192][INFO] lisa.case[hello][lisa_0_0] result: PASSED, elapsed: 0.053 sec
2025-09-12 17:50:58.606[139827859904192][INFO] lisa.case[bye][lisa_0_1] test case 'HelloWorld.bye' is running
2025-09-12 17:50:58.610[139827849414336][INFO] lisa.case[bye][lisa_0_1] before test case
2025-09-12 17:50:58.629[139827868296896][INFO] lisa.case[bye][lisa_0_1] after test case
2025-09-12 17:50:58.630[139827859904192][INFO] lisa.case[bye][lisa_0_1] result: PASSED, elapsed: 0.024 sec
2025-09-12 17:50:58.636[139827859904192][INFO] lisa.case[script][lisa_0_4] test case 'WithScript.script' is running
2025-09-12 17:50:58.652[139827868296896][INFO] lisa.case[script][lisa_0_4] first run finished within 0.012 sec
2025-09-12 17:50:58.664[139827868296896][INFO] lisa.case[script][lisa_0_4] second run finished within 0.012 sec, total: 0.025 sec
2025-09-12 17:50:58.667[139827859904192][INFO] lisa.case[script][lisa_0_4] result: PASSED, elapsed: 0.031 sec
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner ________________________________________
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner                                   HelloWorld.hello: PASSED   
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner                                     HelloWorld.bye: PASSED   
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner                          MultipleNodesDemo.os_info: SKIPPED  no available environment: ['no enough nodes, requirement: 2, capability: 1.']
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner            MultipleNodesDemo.verify_tcp_connection: SKIPPED  no available environment: ['no enough nodes, requirement: 2, capability: 1.']
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner                                  WithScript.script: PASSED   
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner test result summary
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner     TOTAL    : 5
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner     QUEUED   : 0
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner     ASSIGNED : 0
2025-09-12 17:50:58.677[139828228976384][INFO] lisa.RootRunner     RUNNING  : 0
2025-09-12 17:50:58.678[139828228976384][INFO] lisa.RootRunner     FAILED   : 0
2025-09-12 17:50:58.678[139828228976384][INFO] lisa.RootRunner     PASSED   : 3
2025-09-12 17:50:58.678[139828228976384][INFO] lisa.RootRunner     SKIPPED  : 2
2025-09-12 17:50:58.679[139828228976384][INFO] lisa.notifier[Html] report: /home/jcline/.virtualenvs/lisa-editable/runtime/log/20250912/20250912-175058-346/lisa.html
2025-09-12 17:50:58.685[139828228976384][INFO] lisa. completed in 0.341 sec

Obviously, these are pretty shallow smoke checks and there may need to be a few additional tweaks (I see it's doing some git operations when I just run lisa, for example).

[squirrelsc]

Thanks for digging deeper! If everything works as expected, the only interruptions would be the active PRs. The remaining question is about the package name—if we name it "mslisa," what changes or breaks might occur?

[jeremycline]

Thanks for digging deeper! If everything works as expected, the only interruptions would be the active PRs. The remaining question is about the package name—if we name it "mslisa," what changes or breaks might occur?

It's hard for me to be certain, but I think that change wouldn't be too disruptive. You expect all your users to be interacting via the command line and not through a Python API, right? My memory on the fine details of Python packaging are hazy, but I'm pretty sure you can have a package named "mslisa" and the importable name be "lisa", but that's unusual and a little confusing so I wouldn't recommend it. That means all your imports change from "import lisa...." to "import mslisa", but that's a pretty mechanical change.

The documentation doesn't mention the package name when installing since it's done from inside the git repository. It could theoretically break someone who's building the package and installing it by name, but that's definitely not something the documentation mentions or encourages so I think that's okay.

So, to summarize, I think it wouldn't be particularly painful and now is the best time to do it if you're looking to change the name.

[squirrelsc]

My goal is to avoid breaking downstream usage, which currently uses LISA as a submodule and installs it via git+nox. I want existing import lisa... statements to continue working in downstreams. Once we support PyPI, downstream users can switch to pip install and update the namespace then, avoiding any breaking changes.

I prefer the name "mslisa" because "lisa" is already taken on PyPI. Changing the root namespace to "mslisa" will ensure consistency between the PyPI package and the codebase.

[jeremycline]

My goal is to avoid breaking downstream usage, which currently uses LISA as a submodule and installs it via git+nox. I want existing import lisa... statements to continue working in downstreams. Once we support PyPI, downstream users can switch to pip install and update the namespace then, avoiding any breaking changes.

I prefer the name "mslisa" because "lisa" is already taken on PyPI. Changing the root namespace to "mslisa" will ensure consistency between the PyPI package and the codebase.

I see. Well, you can publish it under mslisa by setting the name field to mslisa in your pyproject.toml, and then leaving the lisa directory as-is. For example, with a2b3a08 I'll get a wheel like mslisa-20250804.1.post70-py3-none-any.whl, which I can install and then import as lisa:

❯ pip freeze | grep lisa
mslisa @ file:///home/jcline/devel/github/microsoft/lisa/dist/mslisa-20250804.1.post70-py3-none-any.whl#sha256=9d89d6422b01ca404a4f4d8cc1450cbb84d331f98b26c2ceded31bbe354e27f6
❯ python
Python 3.13.7 (main, Aug 14 2025, 00:00:00) [GCC 15.2.1 20250808 (Red Hat 15.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import lisa
>>> 

Your import name likely conflicts with https://pypi.org/project/lisa/ but I don't think anyone would need both those packages installed at the same time so that's no problem.