eVOLVER Wiki + Software Release 2.0.0

Hi everyone!

We have a couple of new developments in software/documentation that I would like to outline here.

eVOLVER Documentation Wiki

First, I’d like to announce that we now have an official open-source eVOLVER wiki (eVOLVER Documentation Wiki - eVOLVER)! It’s still under construction, but we’re hoping for the wiki to be home for guides and general documentation about hardware/software structure and usage. We’re going to keep editing private while we finish up some of the main pages, but will eventually open it up so that anyone can contribute. The wiki documents are all synced to a Github repository (GitHub - khalillab/evolver-docs: Gitbook documentation for eVOLVER), so going forward pull requests can be made there if you would like to contribute!

If you find any errors or have anything you’d like to see added/prioritized, please let us know here in the forum.

Software Release 2.0.0

Second, we recently released version 2.0.0 of the eVOLVER software with a bunch of new of upgrades/features! This release introduces some major changes to the GUI and DPU, as well as handling some minor bug fixes throughout all of the repos.

You can download the latest release on each individual Github repo by clicking the releases link on the right side of the page for a given repo.

Updating existing systems

If you want to use these new features (briefly described below), you will need to upgrade the server, dpu, and GUI all together. You should not need to update the Arduino microcontrollers. Make sure you do step 4 for the GUI to function properly.

We are also now recommending venv for installing the DPU. Instructions on the wiki outline the process.

Lastly, if you update, you must add a default pump_cal to the calibrations.json file or else the setup page will have issues running. Please see the example file on github - you can copy the following into the beginning of your own calibrations.json file after the initial opening bracket.

{"name": "pumpcal_20210217", "calibrationType": "pump", "timeCollected": 1613602540591, "measuredData": [10, 10, 10, 10, 10, 10, 10, 10,10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10], "fits": [{"name": "pumpcal_20210217", "type": "constant", "timeFit": 1613602540591, "active": true, "params": ["pump"], "coefficients": [0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75", "0.75"]}], "raw": [{"param": "pump", "vialData": [["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"], ["7.5"]]}]}

If you have other questions about updating please ask them here.

GUI updates

We improved the calibration process and added a new way to edit, run, and view data from your experiments all in the GUI.

Calibrations

First, we made it such that the GUI will now automatically make fits for you data and save those on the server. This means you no longer need to run the python script calibrate.py after you do your calibrations in the GUI!

Second, data from your calibrations can be viewed in real time at any point during the calibration by clicking the VIEW COLLECTED DATA button on the bottom left of the calibration pages. This will bring up a set of graphs for all of the vials displaying all data collected up that point in the calibration process, allowing you to detect problems before going through the entire process.

Lastly, pump calibrations are now integrated with the GUI/server. To do a pump calibration, first fill the tubing for the pumps you’d like to calibrate with water. Next, select the vials you’d like to calibrate, put the tubing for those pumps into the vials, run the pumps, then note the amount of liquid pumped in the application. You no longer need a pump_cal.txt for the DPU - the calibrations will be acquired similarly to the temp/od cals in json format from the server. A guide for this process will be on the wiki soon.

Experiment Editor Page

You can now manage your experiments within the GUI from the experiment manger page. This works similarly to how it was described in the JoVE paper in 2019, but the differences are highlighted on the wiki guide.

If you set the experiment directory to be a dropbox or google drive mounted location (by clicking the pen button at the top of the experiment manager page) you can easily share data between collaborators and view your data remotely in real-time by selecting that directory on a GUI running on a different computer.

Experiment Editor Page

You can also now edit experiments directly within the GUI. Click the pen icon in the row on the experiment manager table to bring up this page. There are three pre-defined experiment editor pages you can use for basic turbidostat, chemostat, or growth curve experiments which you can select in the top right drop down. The page also allows you do pick the eVOLVER you’d like to run on (just be sure to change the calibrations if you need to on the setup page before running) and modify the experiment name. Select vials similarly to the setup page and apply experimental parameters via the sliders and buttons on the left side of the page. When you finish setting up everything, click Save Expt. at the bottom right of the page (NOTE: the floppy disk icon is meant only for the file editor described below. We’ll make this more intuitive in a future release.)

We also allow you to directly modify the eVOLER python scripts within the application if you’d like to, but please note doing this will lock the experiment out of the predefined functions. You should not need to modify the files normally unless you have something more custom you’d like to do. The save icon on the bottom left is used for saving the files.

You can also bring up a file browser to interact with the files directly if you need to by click the folder icon on the bottom of the page.

After you finish setting up the experiment, you can click the play icon on the experiment editor or manager page to start the experiment!

If you’d like to modify a running experiment, stop the experiment (click the square button on either the manager/editor/data visualization page), make any desired changes, and restart the experiment. The experiment will continue appending data to the files where it left off.

Data Visualization Page

Clicking the graphs icon on the experiment editor or manager page will bring up the data visualization page. On this page you can view OD and Temp values for all vials, as well as view the experiment logs. You can bring up individual plots by clicking the vial number on the right, and select data ranges either using the radio buttons or by moving the sliders on the individual plot. If you move the slider, the values will be persisted if you go back to viewing all vials.

If you click the DOWNLOAD button, you can save a copy of the experiment/data to any desired location on your computer.

Note that if you are viewing the data remotely, you should not try to use the play/stop buttons, they will not work and could cause problems with your data. We will make this more robust in a future update.

NOTE: If you close the GUI, your experiments will continue to run in the background on your computer. You can reopen the app and it will detect this experiment and allow you to interact with it as normal. There will be popups when this occurs letting you make decisions about how to handle the experiments.

DPU Updates

The DPU has been updated to accommodate the GUI managing running experiments. For the most part it is the same, but there main difference you will notice is that the IP is no longer supplied as a constant variable within custom_script.py - it is instead provided now on the command line. You can still run your experiments as normal on the command line if you don’t want to use the GUI. If you run them on the command line, you can still view your data through the application by pointing the GUI to the experiments directory where the experiment is running. Just make sure you keep the experiment name as data if you want to use this feature. To maintain backwards compatibility with the old cloud evolution django webapp for viewing data we left this as a variable that can be modified.

There is a new file nbstreamreader.py that is for receiving messages from the GUI while the DPU is running. It currently is only for receiving stop messages to stop pumps before quitting, but it could be used in the future for more interesting applications.

Server Updates

There are minimal updates here except for some bug fixes. Please see the note about the pump calibration up above.

Final thoughts

I’m hoping that we maintain a faster development cycle going forward - I know it’s been a while since our last release. If you find any problems or bugs in the new software do not hesitate to let us know here on the forum so we can get them fixed as soon as possible!

I would also like to acknowledge and thank @ezirayw, @bgwong, @Nate_Borders, and @danhart for their help with the software and in working on the wiki.

5 Likes

This is awesome, great work y’all!!!

1 Like

Hi all, I am just following the instructions for the installation of the DPU (using venv as you suggest) on Windows, but ran into an issue. The python version for my virtual environment is python 3.6 (as suggested by the GitHub). As I run python -m pip install . I get the error “Could not find a version that satisfies the requirement bokeh>=2.4.3 (from dpu)”. I have a feeling this may be related to the older version of python, where the latest version of bokeh available is 2.3.3 and therefore flags that error.

Do you use a newer version of python for the new v 2.0.0 release or is it still python 3.6? If so, do you know the workaround for this?

1 Like

Hi!

The examples on Github for the DPU shouldn’t say python3.6, I’ll correct that today and add links to the wiki with more detailed examples.

Follow the directions here:

You can just remove the dpu-env directory that you created using venv on python3.6 and start over.

Zack

2 Likes

Thanks for that! Got everything working nicely on our computer (both the GPU and GUI) however I am struggling with updating the GUI in the eVOLVER. I followed your instructions for updating the eVOLVER server after which I also transferred the new GUI app image via scp, renamed the old file and new file, and rebooted.

After rebooting I’m simply seeing a blank lit up screen. This has happened both when the new app image file was named evolver-electron-2.0.0.AppImage as well as evolver-electron-1.0.0.AppImage. The GUI works when I go back to the old file but, unsurprisingly, it is the outdated GUI. Any tips?

Thanks for alerting me - it’s also giving me trouble when I try to update. You can continue using the old one for now until I figure it out, it should work for the setup page. If you need to do calibrations, use the Desktop GUI.

Zack

1 Like

Ok figured it out - I’ll write a guide up on the wiki. It’s just very picky about how the file is placed there. You can scp the AppImage file over, ssh over to the pi, remove the old file (or backup using the mv command), then use mv to put the new AppImage in place, naming it evolver-electron-1.0.0.AppImage. The autostart script is looking explicitly for this filename at /users/pi/.

Zack

Hey Zack, thanks for the help but I’m still having issues. I tried the renaming strategy of the new evolver-electron-2.0.0.AppImage file to evolver-electron-1.0.0.AppImage whilst backing up the older version with mv evolver-electron-1.0.0.AppImage evolver-electron-1.0.0.AppImage-old but I’m still getting a blank, slightly lit-up screen with the following contents of the /users/pi directory:


That being said I have simply scp’d the AppImage file that I downloaded from the GitHub instead of packaging it myself with yarn; not sure if that’s the problem. I can give that a shot and report back.

Hi, I too am having this same problem. I tried packaging it myself and that didn’t fix anything. One thing worth noting is that for me the original .AppImage file is at /home/pi, not /users/pi. I don’t think this should make a difference but am unsure. I checked and I don’t have a /users directory at all.

Sorry, I wrote my original response from memory. I go between different operating systems and mix the paths up sometimes. I’ll make a more detailed post with screenshots on Monday.

Did you remove the old file (by either using mv or rm) before putting the new one in place? And did you place the file while being ssh’d on the PI (i.e. not simply by scp’ing the file)?

I think there are some weird permissions issues that depend on how the file was put in place. So if you place it (using mv) while being the pi user while ssh’d on the pi the permissions are set differently than if the file is placed via scp. I’ll dive into it more next week and do a more detailed post. Again, you can simply use the old version of the GUI on the rpi for now, functionally the new version changes nothing on the setup page, and on the new version that is the only button that appears on the pi.

Zack

I followed these, as listed on the evolver-electron github:
$ scp release/evolver-electron-1.0.0.AppImage pi@<RPI_IP>:~/evolver-electron-1.0.0.AppImage-new
$ ssh pi@<RPI_IP>
$ mv evolver-electron-1.0.0.AppImage evolver-electron-1.0.0.AppImage-old
$ mv evolver-electron-1.0.0.AppImage-new evolver-electron-1.0.0.AppImage
$ sudo reboot now

1 Like

Thanks, that’s helpful! I’ll get back to you Monday or Tuesday.

Hi All,

The following worked for me:

1. SCP File to Pi

For me, on my mac, I typed the following:

scp Downloads/evolver-electron-2.0.0.AppImage pi@your_evolver_address:

2. Change permissions of the AppImage

sudo chmod +x evolver-electron-2.0.0.AppImage

3. Change what AppImage the Pi uses to start in kiosk mode

sudo nano /etc/xdg/openbox/autostart

Scroll down and change the following lines to what version of the App you are using:


# Start Electron App in Kiosk Mode
./evolver-electron-2.0.0.AppImage

Reboot eVOLVER.

Hope this helps! Thanks for all the feedback!

Best,
Brandon

2 Likes

Good ole’ chmod +x :slight_smile: I’ll add this to the wiki in the installation instructions for the GUI and link it in the server update page. Thanks Brandon!

Hi, unfortunately this still didn’t fix it. After doing this and rebooting the only choice available on the raspberry pi GUI is “settings”.

For the time being, I have decided to revert everything to v1.2.1; I was also having an issue when I tried updating the server where it could no longer communicate with the arduinos.

There should only be a “SETUP” button on the most recent version. If you aren’t just seeing a black screen it’s likely working the way it’s supposed to be.

We temporarily removed “CALIBRATIONS”, as they require some additional dependencies and configurations that aren’t on the RPi image we send out by default in order to do them without the need to subsequently run a script on a connected lab desktop/laptop. We will be adding it back soon.

Hi @kawidney,

Tracked down the problem with the connection not working - I’ll put out a fix tomorrow! Thanks for helping us out!

Zack

1 Like

I updated the binary file in the release on github and pushed the changes to hotfix. This, combined with Brandon’s instructions for adding the executable flag to the file permissions, should resolve all issues. Let me know if you still have problems!

Unfortunately still not working, after having updated everything, when I reboot the server (or the machine), I stop receiving updates on OD and temp, and cannot send commands to change stir rate