make '--production' mode explicit; prevent tests from messing up data/runDB

Created by: jorana

Explicit production mode This pull request builds on #96. With this pull request we make it absolutely explicit that you are running the production mode of bootstrax. Run this by e.g.:

bootstrax --production

Testing mode: ommit '--production' If one does not specify that the bootstrax is used for --production some features are disabled that might be hard/impossible/annoying to fix.

This prevents bootstrax from interacting with the runDB to:

• Register new data
• Update the status of bootstrax
• Delete live_data
• Write data to any of the xenonnt-processed data directories on the eventbuilders

To this end we have to overwrite the mongo frontend in the strax.DataDirectory and point it to a folder in the directory where strax is called from. This might not be ideal but allows sufficient flexibility. As it is checked each run that there is sufficient storage, I deem it save enough. Of course this means that the DAQ people have the responsibility to clean up this folder themselves.

Furthermore, as we don't keep track of the processing using the runsDB for these test mode bootstrax instances I've added a simple counter in the main() loop that checks how many runs it has processed (or tried to) such that we still loop over the events. This is not a infinite loop (as one shouldn't want to build for testing). This could in principle give funny edge cases of the testing mode, what if another 'production bootstrax' is processing the same run and deletes the live_data before a 'test bootstrax' finishes processing? However, such a scenario doesn't seem sufficiently likely/important that I think it is required to have this handled differently: the test will fail and production will continue unaffected.

The DAQ database is still used to register the bootstrax instance such that it can be killed in the main() loop.

Additionally, I've added another safeguard that prevents us from accidently deleting 'raw_records' from a disk if 'live_data' is no longer available.

How to run bootstrax production After this PR bootstrax should be better protected from accidently removing/overwriting important data. The best mode to operate bootstrax during production after this PR will be:

(base) xedaq@eb1:~$ conda activate py37
(py37) xedaq@eb1:~$ python bootstrax --infer_mode --production --delete_live

How to run bootstrax for testing Testing is assumed one can simply do single runs by:

(base) xedaq@eb1:~$ conda activate py37
(py37) xedaq@eb1:~$ python bootstrax --process <run_id>