Pumps not working in chemostat mode

[ericlyall]

Some background:
Over the weekend I had a chemostat experiment running. I used the pycharm IDE to start the experiment. I stopped the experiment in pycharm and changed the dilution rate / starting OD a couple times, an restarted the chemo state experiment under the same name. I also tried a the turbidostat mode briefly to see if it was working- all vials were dilluted at my upper threshold, as expected.

I then resumed the chemostat experiment. The starting conditions were time >10 min, and and OD greater than 0.4. Only 4 of the vials (vials 5,6,7, and 12) had their pumps working and reached steady state.
At some point during the weekend, someone closed the pycharm IDE. Strangely, pumps were still running on vials 5,6,7 and 12 even though the script was seemingly closed & stopped. I have since realized that the dpu scripts should be executed through the terminal so it can properly execute the “stop pump” command. I then turned the eVOLVER off.

Here’s my current issue: When running the eVOLVER.py script through the terminal, none of the pumps are pumping in chemostat mode

• I can run the pumps manually from the GUI either on the computer or the evolver itself, and all pumps work.
• The pumps work in turbidostat mode
• I still get OD and temperature data
• I zeroed the vials at the start of the experiment
• I have water in the vials, and starting conditions as t=0 and OD>-1. Print statements have indicated these starting conditions are met for all 16 vials.
• rate_config = [1.1538] * 16
• Bolus = .5 mL, meaning the chemostat should pump every minute

Here are some of the things I’ve tried / debugging results:

• Power cycling everything for 19 seconds.
• Adding a print statement for the message in the updata_chemo function:
• Here are the results of that print statement.
  
  

-It seems like the pi is getting these commands when the conf.yml file is checked:

• I tried editing the conf.yml file to replace the pump line with null, as Brandon suggested.

[heinsz]

Hi @ericlyall , can you start a chemostat experiment and then tail the evolver log file for me on the rpi? After starting the experiment, ssh to the rpi and run sudo supervisorctl, then run tail -f evolver. Let that go for about a minute then paste the output here. Press ctrl+c to stop the tail, the type exit to get back to the regular prompt on the rpi.

Zack

[ericlyall]

Here’s that output: True, ‘value’: [‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’]}, ‘pump’: {‘fields_expected_incoming’: 49, ‘fields_expected_outgoing’: 49, ‘recurring’: True, ‘value’: [‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’]}, ‘stir’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: True, ‘value’: [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10]}, ‘temp’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: True, ‘value’: [‘1634’, ‘1629’, ‘1608’, ‘1626’, ‘1611’, ‘1612’, ‘1606’, ‘1625’, ‘1632’, ‘1623’, ‘1618’, ‘1624’, ‘1612’, ‘1626’, ‘1625’, ‘1631’]}}, ‘ip’: ‘10.0.0.100’, ‘timestamp’: 1613769971.8181431}
od_135r,1000,!
od_135b,55800,51461,55387,54013,56281,55119,55491,52980,55490,55252,56526,56164,56683,51955,55152,52343,end
od_135a,!
od_90r,1000,!
od_90b,25876,28693,28730,28403,28123,27938,28071,28445,29093,27976,28510,26865,27717,28650,29869,28685,end
od_90a,!
od_ledr,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,!
od_lede,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,end
od_leda,!
pumpr,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,–,--,–,--,–,--,–,--,–,--,–,--,–,--,–,--,!
pumpe,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,–,--,–,--,–,--,–,--,–,--,–,--,–,--,–,--,end
pumpa,!
stirr,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,!
stire,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,end
stira,!
tempr,1634,1629,1608,1626,1611,1612,1606,1625,1632,1623,1618,1624,1612,1626,1625,1631,!
tempb,1638,1631,1610,1627,1606,1605,1615,1625,1618,1625,1616,1627,1608,1628,1624,1634,end
tempa,!
Broadcasting data
{‘data’: {‘od_135’: [‘55800’, ‘51461’, ‘55387’, ‘54013’, ‘56281’, ‘55119’, ‘55491’, ‘52980’, ‘55490’, ‘55252’, ‘56526’, ‘56164’, ‘56683’, ‘51955’, ‘55152’, ‘52343’], ‘od_90’: [‘25876’, ‘28693’, ‘28730’, ‘28403’, ‘28123’, ‘27938’, ‘28071’, ‘28445’, ‘29093’, ‘27976’, ‘28510’, ‘26865’, ‘27717’, ‘28650’, ‘29869’, ‘28685’], ‘temp’: [‘1638’, ‘1631’, ‘1610’, ‘1627’, ‘1606’, ‘1605’, ‘1615’, ‘1625’, ‘1618’, ‘1625’, ‘1616’, ‘1627’, ‘1608’, ‘1628’, ‘1624’, ‘1634’]}, ‘config’: {‘lxml’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: False, ‘value’: [‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’]}, ‘od_135’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 2, ‘recurring’: True, ‘value’: ‘1000’}, ‘od_90’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 2, ‘recurring’: True, ‘value’: ‘1000’}, ‘od_led’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: True, ‘value’: [‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’]}, ‘pump’: {‘fields_expected_incoming’: 49, ‘fields_expected_outgoing’: 49, ‘recurring’: True, ‘value’: [‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’]}, ‘stir’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: True, ‘value’: [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10]}, ‘temp’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: True, ‘value’: [‘1634’, ‘1629’, ‘1608’, ‘1626’, ‘1611’, ‘1612’, ‘1606’, ‘1625’, ‘1632’, ‘1623’, ‘1618’, ‘1624’, ‘1612’, ‘1626’, ‘1625’, ‘1631’]}}, ‘ip’: ‘10.0.0.100’, ‘timestamp’: 1613769991.7348042}
od_135r,1000,!
od_135b,55931,51379,55426,53846,56306,55096,55462,52915,55611,55275,56620,56121,56703,51797,55163,52473,end
od_135a,!
od_90r,1000,!
od_90b,25975,28662,28736,28328,28131,27936,28069,28380,29076,27960,28691,26848,27700,28642,29872,28832,end
od_90a,!
od_ledr,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,!
od_lede,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,end
od_leda,!
pumpr,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,–,--,–,--,–,--,–,--,–,--,–,--,–,--,–,--,!
pumpe,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,0|0,–,--,–,--,–,--,–,--,–,--,–,--,–,--,–,--,end
pumpa,!
stirr,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,!
stire,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,end
stira,!
tempr,1634,1629,1608,1626,1611,1612,1606,1625,1632,1623,1618,1624,1612,1626,1625,1631,!
tempb,1633,1627,1611,1627,1614,1611,1605,1623,1627,1622,1619,1622,1608,1622,1627,1628,end
tempa,!
Broadcasting data
{‘data’: {‘od_135’: [‘55931’, ‘51379’, ‘55426’, ‘53846’, ‘56306’, ‘55096’, ‘55462’, ‘52915’, ‘55611’, ‘55275’, ‘56620’, ‘56121’, ‘56703’, ‘51797’, ‘55163’, ‘52473’], ‘od_90’: [‘25975’, ‘28662’, ‘28736’, ‘28328’, ‘28131’, ‘27936’, ‘28069’, ‘28380’, ‘29076’, ‘27960’, ‘28691’, ‘26848’, ‘27700’, ‘28642’, ‘29872’, ‘28832’], ‘temp’: [‘1633’, ‘1627’, ‘1611’, ‘1627’, ‘1614’, ‘1611’, ‘1605’, ‘1623’, ‘1627’, ‘1622’, ‘1619’, ‘1622’, ‘1608’, ‘1622’, ‘1627’, ‘1628’]}, ‘config’: {‘lxml’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: False, ‘value’: [‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’]}, ‘od_135’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 2, ‘recurring’: True, ‘value’: ‘1000’}, ‘od_90’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 2, ‘recurring’: True, ‘value’: ‘1000’}, ‘od_led’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: True, ‘value’: [‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’, ‘4095’]}, ‘pump’: {‘fields_expected_incoming’: 49, ‘fields_expected_outgoing’: 49, ‘recurring’: True, ‘value’: [‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘0|0’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’, ‘–’]}, ‘stir’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: True, ‘value’: [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10]}, ‘temp’: {‘fields_expected_incoming’: 17, ‘fields_expected_outgoing’: 17, ‘recurring’: True, ‘value’: [‘1634’, ‘1629’, ‘1608’, ‘1626’, ‘1611’, ‘1612’, ‘1606’, ‘1625’, ‘1632’, ‘1623’, ‘1618’, ‘1624’, ‘1612’, ‘1626’, ‘1625’, ‘1631’]}}, ‘ip’: ‘10.0.0.100’, ‘timestamp’: 1613770011.8132913}
Received COMMAND
pumpi,0.64|60,0.65|60,0.66|60,0.66|60,0.64|60,0.62|60,0.64|60,0.65|60,0.65|60,0.67|60,0.66|60,0.62|60,0.65|60,0.64|60,0.64|60,0.67|60,1.28|60,1.30|60,1.32|60,1.32|60,1.29|60,1.23|60,1.27|60,1.31|60,1.30|60,1.33|60,1.32|60,1.25|60,1.29|60,1.28|60,1.27|60,1.33|60,–,--,–,--,–,--,–,--,–,--,–,--,–,--,–,--,_!

Traceback (most recent call last):
File “/home/pi/evolver/evolver/evolver_server.py”, line 242, in run_commands
returned_data = serial_communication(command[‘param’], command[‘value’], command[‘type’])
File “/home/pi/evolver/evolver/evolver_server.py”, line 285, in serial_communication
raise EvolverSerialError('Error: Response has incorrect address.\n\tExpected: ’ + param + ‘\n\tFound:’ + address)evolver_server.EvolverSerialError: Error: Response has incorrect address.
Expected: pump
Found:
od_135r,1000,!
od_135b,55924,51498,55422,53968,56325,55111,55488,52903,55628,55234,56564,56145,56696,51868,55158,52454,end
od_135a,!
od_90r,1000,!
od_90b,25949,28702,28740,28379,28139,27947,28091,28407,29071,27978,28697,26864,27715,28569,29864,28729,end
od_90a,!
od_ledr,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,!
od_lede,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,end
od_leda,!
pumpr,0.64|60,0.65|60,0.66|60,0.66|60,0.64|60,0.62|60,0.64|60,0.65|60,0.65|60,0.67|60,0.66|60,0.62|60,0.65|60,0.64|60,0.64|60,0.67|60,1.28|60,1.30|60,1.32|60,1.32|60,1.29|60,1.23|60,1.27|60,1.31|60,1.30|60,1.33|60,1.32|60,1.25|60,1.29|60,1.28|60,1.27|60,1.33|60,–,--,–,--,–,--,–,--,–,--,–,--,–,--,–,--,_!

Traceback (most recent call last):
File “/home/pi/evolver/evolver/evolver_server.py”, line 242, in run_commands
returned_data = serial_communication(command[‘param’], command[‘value’], command[‘type’])
File “/home/pi/evolver/evolver/evolver_server.py”, line 285, in serial_communication
raise EvolverSerialError('Error: Response has incorrect address.\n\tExpected: ’ + param + ‘\n\tFound:’ + address)evolver_server.EvolverSerialError: Error: Response has incorrect address.
Expected: pump
Found:
stirr,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,_!

Traceback (most recent call last):
  File "/home/pi/evolver/evolver/evolver_server.py", line 242, in run_commands
    returned_data = serial_communication(command['param'], command['value'], command['type'])
  File "/home/pi/evolver/evolver/evolver_server.py", line 285, in serial_communication
    raise EvolverSerialError('Error: Response has incorrect address.\n\tExpected: ' + param + '\n\tFound:' + address)evolver_server.EvolverSerialError: Error: Response has incorrect address.
        Expected: stir
        Found:~~~
tempr,1634,1629,1608,1626,1611,1612,1606,1625,1632,1623,1618,1624,1612,1626,1625,1631,_!
tempb,1633,1628,1607,1624,1610,1614,1600,1627,1634,1621,1614,1624,1616,1624,1623,1630,end
tempa,,,,,,,,,,,,,,,,,_!
Broadcasting data
{'data': {'od_135': ['55924', '51498', '55422', '53968', '56325', '55111', '55488', '52903', '55628', '55234', '56564', '56145', '56696', '51868', '55158', '52454'], 'od_90': ['25949', '28702', '28740', '28379', '28139', '27947', '28091', '28407', '29071', '27978', '28697', '26864', '27715', '28569', '29864', '28729'], 'temp': ['1633', '1628', '1607', '1624', '1610', '1614', '1600', '1627', '1634', '1621', '1614', '1624', '1616', '1624', '1623', '1630']}, 'config': {'lxml': {'fields_expected_incoming': 17, 'fields_expected_outgoing': 17, 'recurring': False, 'value': ['4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095']}, 'od_135': {'fields_expected_incoming': 17, 'fields_expected_outgoing': 2, 'recurring': True, 'value': '1000'}, 'od_90': {'fields_expected_incoming': 17, 'fields_expected_outgoing': 2, 'recurring': True, 'value': '1000'}, 'od_led': {'fields_expected_incoming': 17, 'fields_expected_outgoing': 17, 'recurring': True, 'value': ['4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095', '4095']}, 'pump': {'fields_expected_incoming': 49, 'fields_expected_outgoing': 49, 'recurring': True, 'value': ['0.64|60', '0.65|60', '0.66|60', '0.66|60', '0.64|60', '0.62|60', '0.64|60', '0.65|60', '0.65|60', '0.67|60', '0.66|60', '0.62|60', '0.65|60', '0.64|60', '0.64|60', '0.67|60', '1.28|60', '1.30|60', '1.32|60', '1.32|60', '1.29|60', '1.23|60', '1.27|60', '1.31|60', '1.30|60', '1.33|60', '1.32|60', '1.25|60', '1.29|60', '1.28|60', '1.27|60', '1.33|60', '--', '--', '--', '--', '--', '--', '--', '--', '--', '--', '--', '--', '--', '--', '--', '--']}, 'stir': {'fields_expected_incoming': 17, 'fields_expected_outgoing': 17, 'recurring': True, 'value': [10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10, 10]}, 'temp': {'fields_expected_incoming': 17, 'fields_expected_outgoing': 17, 'recurring': True, 'value': ['1634', '1629', '1608', '1626', '1611', '1612', '1606', '1625', '1632', '1623', '1618', '1624', '1612', '1626', '1625', '1631']}}, 'ip': '10.0.0.100', 'timestamp': 1613770031.4352436}
od_135r,1000,_!
od_135b,55916,51497,55439,54106,56321,55123,55490,52865,55629,55236,56581,56168,56629,51858,55170,52440,end
od_135a,,_!
od_90r,1000,_!
od_90b,25988,28683,28735,28375,28143,27915,28103,28394,29093,27843,28648,26888,27731,28632,29871,28763,end
od_90a,,_!
od_ledr,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,_!
od_lede,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,4095,end
od_leda,,,,,,,,,,,,,,,,,_!
pumpr,0.64|60,0.65|60,0.66|60,0.66|60,0.64|60,0.62|60,0.64|60,0.65|60,0.65|60,0.67|60,0.66|60,0.62|60,0.65|60,0.64|60,0.64|60,0.67|60,1.28|60,1.30|60,1.32|60,1.32|60,1.29|60,1.23|60,1.27|60,1.31|60,1.30|60,1.33|60,1.32|60,1.25|60,1.29|60,1.28|60,1.27|60,1.33|60,--,--,--,--,--,--,--,--,--,--,--,--,--,--,--,--,_!

Traceback (most recent call last):
  File "/home/pi/evolver/evolver/evolver_server.py", line 242, in run_commands
    returned_data = serial_communication(command['param'], command['value'], command['type'])
  File "/home/pi/evolver/evolver/evolver_server.py", line 285, in serial_communication
    raise EvolverSerialError('Error: Response has incorrect address.\n\tExpected: ' + param + '\n\tFound:' + address)evolver_server.EvolverSerialError: Error: Response has incorrect address.
        Expected: pump
        Found:
stirr,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,10,_!
{{߽_brŹŹŹŹŹŹŹŹŹŹŹŹŹŹŹŹ
Traceback (most recent call last):
  File "/home/pi/evolver/evolver/evolver_server.py", line 242, in run_commands
    returned_data = serial_communication(command['param'], command['value'], command['type'])
  File "/home/pi/evolver/evolver/evolver_server.py", line 285, in serial_communication
    raise EvolverSerialError('Error: Response has incorrect address.\n\tExpected: ' + param + '\n\tFound:' + address)evolver_server.EvolverSerialError: Error: Response has incorrect address.
        Expected: stir
        Found:{{߽
tempr,1634,1629,1608,1626,1611,1612,1606,1625,1632,1623,1618,1624,1612,1626,1625,1631,_!
tempb,1637,1634,1605,1628,1609,1608,1602,1628,1637,1626,1621,1625,1616,1630,1624,1636,end
tempa,,,,,,,,,,,,,,,,,_!
Broadcasting data

[ericlyall]

It seem like there are two ways to send a pump command, one with “time | period” (used in chemostat code) and one with just “time” (used in turbidostat mode
The command were you just send the pump a time, and no period works. With one with time| period doesn’t work.
I wrote a new chemostat function that sorts out the pumping interval on the python end, and sends only the pumping commands with just a “time” to pump ( similar to the type of command that’s in chemostat mode.
This new function works well ( at least testing with water). I’m running a real trial now. Will share the github soon.

[heinsz]

I wouldn’t recommend that for reasons that I won’t go into right now, I can elaborate later if you’re interested. But that would also just be treating the symptom of the underlying problem, which is the error you see in that log.

Try these two things. Increase the serial_timeout field in conf.yml to 0.7. If that doesn’t work, increase the time.sleep(0.5) commands in evolver_server.py to 0.7 to 1. My suspicion is that because the command is pretty long its it’s not able to get processed in time before the next commands get sent out of the RPi to the microcontrollers.

[heinsz]

I forgot to mention you need to restart the server after making these changes - either do sudo supervisorctl followed by restart evolver or power cycle the RPi.

[ericlyall]

Here’s the github link to the new chemostat function that I was using. I ran a 30 hour chemostat trial & everything seems to have been working as expected. Would you be able to elaborate why this method is a bad idea?
I’ll try the steps you suggested with increasing the timeout and sleep commands.

[heinsz]

The main reason is the the custom script function is only called when data is received from the server, which occurs ~20s by default, but this is a variable that can be altered in conf.yml. So by default, you are limited to checking if you should pump at 20s increments. This means that your function will be slightly inaccurate at best, extremely inaccurate if that variable is changed for other experimental/computational reasons, and in the worst case you will completely miss pumping events if for whatever reason the connection is dropped or there is a delay on the RPi. It also means if you need to pump more rapidly than 20s or whatever the server timing is set to (could be longer) you would be pumping slower than desired. The reason we have special formats for sending chemostat commands to the RPi is to decouple the timing of hitting a desired flow rate from the RPi/DPU timing. By having all that timing occur on the uC, no matter what happens on the RPi/DPU, your pumps will hit the programmed flow rates.

I’m pretty confident your issue is just that the serial communications are getting messed up by the message being a bit long. I’ll make a PR to increase this timing by default in the next release which should be coming out soon.

[ericlyall]

Ok thanks for clearing that up. With a lower dilution rate this is probably less of an issue, but I can see how I could run into problems with a higher dilution rate or when the pi connection is dropped.

[ericlyall]

Another suggestion for you next PR- perhaps add a bokeh plot that has all the vials on the same plot, something like this:

It makes it easy to quickly compare vials / get the big picture of where an experiment is at.

[ericlyall]

Pumps seem to be working with the increased serial timeout on the conf.yml file. I think I’ll also change the sleep commands in the evolver_server.py file just for safety.
The time.sleep commands in the evolver_server.py script are currently at 0.05 seconds. I’m assuming you would like me to try 0.07 and 0.1 next?

[heinsz]

Oh sorry I definitely misread that, but yes, incremental increases would be my suggestion. If the increase to the serial_timeout solved the problem you shouldn’t need to modify the other values.

As for the plots, we’re actually moving towards the electron GUI for real-time data visualization. Most likely your suggestion will not be in the initial release, but I’ll add it on the roadmap for new features we develop.

[bgwong]

Thanks for all the help @heinsz