Pump 6 continuously pumping WITHOUT command

Hi all,

Today I wanted to initiate an experiment, but somehow something went haywire.
I started filling Tstats using the GUI as I always do to prep the machine (we run our own scripts using the terminal thereafter). However, I wanted to stop all pumps in the GUI but then pump 6 kept on going for no reason. I tried restarting everything (PC, router, eVOLVER), even disconnected all cables, etc. but somehow, each time I powered on the pump array again, pump 6 would start. Even if both the PC and eVOLVER base unit were powered off (and the pump rack was on) it actuates pump 6. The only thing from keeping it is to disconnect the 8-pin ribbon from port 1. Interestingly, if ON and I plug in the 8-pin ribbon from port 1 into the array part for pump 8-15, then spontaneously pump 14 starts to actuate…

I have rewritten our custom script to not include pump 6 for our experiments, but in any case I cannot have it disabled while the expeirment is running. Would it be bad for pump 6 to continuously pump for 72 hours (while other pumps do their own discrete thing according to script)?

Thanks for any input!

Kevin

Can you share your config.yml file?

Go to Terminal and type in ssh pi@<evolver ip address>, for example, our evolver is 192.168.1.2 so we would put ssh pi@192.168.1.2 The password is DbLaoXDr

  1. Type the following: nano evolver/evolver/conf.yml
  2. Copy and paste what is listed there to a text file and share

Thanks!

Brandon

Here is the output:

acknowledge_char: a
broadcast_timing: 20
calibration: calibration.json
calibrations_directory: calibrations
data_response_char: b
device: evolver-config.json
echo_response_char: e
evolver_ip: 192.168.1.101
experimental_params:
lxml:
fields_expected_incoming: 17
fields_expected_outgoing: 17
recurring: false
value: [β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’,
β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’]
od_135: {fields_expected_incoming: 17, fields_expected_outgoing: 2, recurring: true,
value: β€˜1000’}
od_90: {fields_expected_incoming: 17, fields_expected_outgoing: 2, recurring: true,
value: β€˜1000’}
od_led:
fields_expected_incoming: 17
fields_expected_outgoing: 17
recurring: true
value: [β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’,
β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’, β€˜4095’]
pump:
fields_expected_incoming: 49
fields_expected_outgoing: 49
recurring: false
value: [–, --, --, --, --, --, --, --, --, --, --, --, --, --, --, --, β€˜20’,
β€˜20’, β€˜20’, β€˜20’, --, β€˜20’, --, β€˜20’, --, --, --, --, --, --, --, --, --, --,
–, --, --, --, --, --, --, --, --, --, --, --, --, –]
stir:
fields_expected_incoming: 17
fields_expected_outgoing: 17
recurring: true
value: [β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’, β€˜0’,
β€˜0’, β€˜0’]
temp:
fields_expected_incoming: 17
fields_expected_outgoing: 17
recurring: true
value: [β€˜2286’, β€˜2283’, β€˜2281’, β€˜2280’, β€˜2280’, β€˜2279’, β€˜2282’, β€˜2276’, β€˜2274’,
β€˜2281’, β€˜2279’, β€˜2280’, β€˜2283’, β€˜2288’, β€˜2285’, β€˜2278’]
fitted_data_directory: fittedCal
immediate_command_char: i
num_sleeves: 16
od_calibration_directory: od
port: 8081
raw_data_directory: rawCal
recurring_command_char: r
serial_baudrate: 9600
serial_delay: 0.5
serial_end_incoming: end
serial_end_outgoing: _!
serial_port: /dev/ttyAMA0
serial_timeout: 1
temp_calibration_directory: temp

Can you output what the log files show on the eVOLVER? To do this, can you ssh into the pi using these instructions and copy and paste what is shown after a fluidic command is sent?

Go to Terminal (on your computer) and type in ssh pi@<evolver ip address>, for example, our evolver is 192.168.1.2 so we would put ssh pi@192.168.1.2 The password is DbLaoXDr

Can you post a screenshot of the server log as you run the fluidic pumps from the GUI? You can access this by ssh’ing to the RPi and running sudo supervisorctl then tail -f evolver

The fluidic command should look something like:
pumpi,2|4,10,–,–,–,–,–,–,–,–,–,–,–,–,–,–,3|4,15,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,–,_!

I ran a command from the GUI to pump 10 seconds for all pumps. When starting up everything again this morning (new workday), pump 6 again spontaneously pumps. It keeps pumpingafter the command as well…

Here’s the screenshot

One more thing to check:

Can you read the serial output from the Arduino? You have to install the Arduino drivers, open the fluidics box and connect to the Arduino via microusb.

This reads the output directly from the Arduino while the program runs. please paste the output here after sending a command. Do not upload a new version of the code yet.

This will help debug if it is a hardware fix or a software bug that I’ve never seen before.

Ok, I will check next week.

We are currently running experiments bypassing pump 6…

Do you think it is OK to have that pump 6 running continuously in the meantime? I have put the in and out of pump 6 in water. If it physically wears faster it would just stop running right?

Yes the pump motor might burn out if left running continuously. Nothing bad should happen to the other pumps.

Just to be sure: if the motor β€˜burns out’, you don’t expect it to catch fire, right?
We plan to leave it over the weekend unsupervised. Or would you advise against it?

Sorry for using colloquial English. Probably not catch fire, just stop working.

1 Like

Small update: we were able to run a 36 hour and subsequent 72 hour expeirment with pump 6 continuously pumping, without failure.

The Arduino troubleshoot is postponed until May 21st…

We’ll get back to you.

Hi Brandon,

We installed the Arduino IDE together with the SAMD Board install 32 bits arm cortex M0 + and we installed the Sparkfun board definition as instructed. The board showed up, but nothing showed up in the serial monitor.
Then to test if our connection was active we uploaded a sketch (see below)

"void setup() {
pinMode(LED_BUILTIN, OUTPUT);

Serial.begin(9600);
delay(5000);

Serial.println(β€œSTARTED”);
}

void loop() {
digitalWrite(LED_BUILTIN, HIGH);
Serial.println(β€œLED ON”);
delay(1000);

digitalWrite(LED_BUILTIN, LOW);
Serial.println(β€œLED OFF”);
delay(1000);
}
"

Still nothing.

Then we uploaded a second sketch (see below)

"
void setup() {
pinMode(LED_BUILTIN, OUTPUT);

SerialUSB.begin(9600);
delay(5000);

SerialUSB.println(β€œSTARTED SerialUSB”);
}

void loop() {
digitalWrite(LED_BUILTIN, HIGH);
SerialUSB.println(β€œLED ON”);
delay(1000);

digitalWrite(LED_BUILTIN, LOW);
SerialUSB.println(β€œLED OFF”);
delay(1000);
}

"

After that.
We saw output - finally we sam messages written to the serial monitor - :

STARTED SerialUSB
LED ON
LED OFF
…

When plugging back in the rpi, all the pumps from that array (0-7) (initial pump was 6 (from before)) started pumping. Only that pump cable was attached.

What are the next instruction? I think we wiped the previous program or does the rpi initiate this again? We tried multiple times to read messages from the serial monitor when everything was still original but nothing got written to the serial monitor.

In that case could you send us the sketch that needs to be uploaded, please?

I apologize for the miscommunication, I did not want you to upload new code to the Arduino only read the serial communication from the Arduino.

You need to upload the sketch from here:

https://github.com/FYNCH-BIO/evolver-arduino/blob/master/SAMD21/RS485_FLUIDICS/RS485_FLUIDICS.ino

In the meantime we were able to reupload the correct sketch, bringing us back to only pump 6 still being erroneous.

Just to summarize what happened:

  outputString += end_mark;

  delay(500); // important to make sure pin 12 flips appropriately

  SerialUSB.println(outputString);
  Serial1.print(outputString);

  delay(500); // important to make sure pin 12 flips appropriately

Thanks again for your patience and the troubleshooting!