Updating software on Arctic
General
When logging on to work with arcticICC, first execute “lsst”. This is an alias set in the ~/.bash_profile file that sources ~/lsst_home/loadLSST.bash. This will setup the environment required for development and management of arcticICC.
eups: this is a package manager that tracks tagged versions and their dependencies. Most common eups commands: eups list, setup. Some information on eups here: https://developer.lsst.io/stack/eups-tutorial.html
scons: this is a high-level utility for building software and automatically handling the eups tagging and installing. scons will build any compiled code, run unittests, and optionally install and declare new tags using eups commands under the hood.
to view available versions of all code managed in eups:
$ eups list [-v]
Restarting arcticICC
$ lsst
$ setup arctic [version] # if version is specified a specific version is setup, otherwise the current version is setup
$ arctic stop
$ arctic start
Running a development version
Development for arcticICC should happen in ~/code/arcticICC. This is a git repo. First update the code there
$ git pull
If development is happening on a branch be sure to check out the correct branch
$ git checkout branchname
Clean the directory to be sure that all code is built from scratch (EDIT: You seem to need to do a setup -r . before scons recognizes the command and will run)
$ scons —clean
Build the code (this compiles all C/C++ extensions and automatically runs unittests)
$ scons
If any unittests failed, your code probably broke something important. Be sure to fix any unit test failures before proceeding. If your code changes the behavior of a specific tested feature, change the test to reflect the new behavior.
If the code builds and the unit tests pass, you’re ready to run it. Ask eups to setup of the code in the current working directory (~/code/arcticICC)
$ setup -r . # setup code in this (.) directory
Finally run the actor and test with TUI
Tagging a new version
Once on sky testing is complete and observers are happy that things are working correctly tag a new version. If you’re currently working on a branch, merge it back to master
$ git checkout master
$ git merge branchname
Delete the development branch so it doesn’t hang around and cause confusion
$ git branch -d branchname # delete branch locally
$ git push origin --delete branchname # delete branch remotely (on github)
Update the .__version__.py file to the new version number
Update the doc/versionHistory.html
Commit these changes with a commit message including the tag
$ git commit -am ‘x.y.z’
Add a github tag
$ git tag x.y.z
Push the tag to github
$ git push —tags
Installing and running a new tagged version
Begin in the development repo ~/code/arcticICC. Ensure it is in an up-to-date state
$ git log | head # this will show you the last commit message, it should be ‘x.y.z’ indicating that you have the correct state of the code.
If the development state of the code has progressed since the tag (unusual) you can explicitly force the code to the tagged version you want
$ git checkout x.y.z
Next build the code and run the tests in place
$ lsst (if needed)
$ setup -r .
If everything looks good you’re ready to tag/install with scons/eups
$ scons install declare version=x.y.z —current # be sure that x.y.z matches the git tagged number!
This last command a few things:
It copies the compiled code to a new location on disk (/home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/x.y.z)
It declares the location for this version in eups, pointing to the newly installed directory
It updates the table ups/arcticICC.table to force exact dependencies to be setup (these were the dependencies at the time the code was built). This way if a dependency (eg twistedActor) is updated, there is no risk to breaking the current version of arctic.
Finally, run. Usually I’ll log in using a fresh shell to make sure I get a blank environment
$ setup arcticICC
Verify that the correct version is setup automatically after your install
$ eups list —setup
You should see that arcticICC x.y.z is setup (along with all of its dependencies)
Finally start the new actor
From TUI run the command:
> hub startNubs arctic
to re-attach the hub to the actor.
Updating the code base and testing it
Log into the arctic-icc as arctic user.
Then type lsst to set up the environment to use eups
After that we can list the arctic versions installed on the system and managed by eups using:
eups list arcticICC -v
This will result in a list of installed versions and show which one is the current and which one is the user/test also it lists their locations
Ex:
0.0.1 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/0.0.1
0.0.2 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/0.0.2
0.1.0 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/0.1.0
0.1.1 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/0.1.1
0.2.0 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/0.2.0
0.2.0b1 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/0.2.0b1
1.0 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/1.0
1.0b1 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/1.0b1
1.0b1+1 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/1.0b1+1
1.1 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/1.1
1.2 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/1.2
1.3 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/1.3 current
2.1 /home/arctic/lsst_home /home/arctic/lsst_home/Linux64/arcticICC/2.1
git /home/arctic/lsst_home /home/arctic/code/arcticICC user:test
The purpose of eups is to allow one to quickly switch versions. To set the currently installed version you can type:
setup arcticICC
With the currently installed version running we might have to stop (arcticICC in particular) the process to be ready to update the software. The command for arctic is:
arcticICC stop
You can test this by using arcticICC status
Once arctic is stopped, we change our directory into the user/test git folder and can checkout the version we want to work on and set as a new installed running version using:
git checkout ‘branchname’
Before we install this new version, we want to do a quick test to make sure it is going to work, we can temporarily set this as the working version. Since we are in the user/test folder we can tell eups to setup this particular folder using:
setup -r .
This will set this to our temporary user/test folder to be a working version so that a call to arcticICC status or start or stop will invoke this temporary version not the installed version.
Once we start test arctic using:
arcticICC start we can the let it run for a while and have tests done in tui etc by connecting arctic and seeing how the new version of code performs.
If we like what is happening, we can then permanently install and run this version. But first we want to tag this branch in git which means want to make sure:
The __version__.py file reflects the tag we will be tagging in git.
Installing the tested codebase
We need to now set this version to the installed running version and we do that with our git user/test directory setup. The step above where the call to setup -r . needs to have been done so that when we run our scons command it is getting the right version.
So once we are sure our test/user directory is setup we can run the following commands from there:
scons –clean
Then
scons
This will execute a build on the code base including the proper building of lower level code pieces (usually in C/C++) and then allow us to send that fully compiled version to an installed folder managed under eups.
The build will run some user tests which should finish just and once a successful build has been done we want to type the following command to deploy it:
scons install declare version=packageVersion current
where packageVersion is the version we put in the __version__.py and under the tags.
Once this has been completed the new version should be running and setup and all that is required is to restart the arctic ICC with a call to
arcticICC start
we can do a ps aux | grep python to be able check and make sure it is executing out of the correct folder. Once this is confirmed, the hub startNubs will have to be called from tui or the hub.
Rolling back to a previous version
It is conceivable that some things still are not quite right and to allow for science in the up coming evening we need to rapidly roll back to a working version. To do this we can run
To get a listing and then we can navigate to the version of folder we want to have current and say…
$ setup arcticICC versionYouWant
