Where or how is OD blank info stored?

danigarcia · May 13, 2020, 8:46am

I’m starting a turbidostat experiment with 4 vials and I will be adding the rest every few days. For this, I wanted to be able to blank the OD of those vials just before adding them to the experiment instead of doing it at the very beginning. So this made me think of several questions:

Where or how is the OD blank info stored?
How is the blank applied to the OD values that are logged?
Starting an experiment with -let’s say- 4 vials, is the blank done only for those 4?

Thanks!

heinsz · May 13, 2020, 4:22pm

The initial OD measurement is located here:

github.com

FYNCH-BIO/dpu/blob/master/experiment/template/eVOLVER.py#L83


# update temperatures if needed
data = self.transform_data(data, VIALS, od_cal, temp_cal)
if data is None:
    logger.error('could not tranform raw data, skipping user-'
                 'defined functions')
    return

# should we "blank" the OD?
if self.use_blank and self.OD_initial is None:
    logger.info('setting initial OD reading')
    self.OD_initial = data['transformed']['od']
elif self.OD_initial is None:
    self.OD_initial = np.zeros(len(VIALS))
data['transformed']['od'] = (data['transformed']['od'] -
                                self.OD_initial)
# save data
self.save_data(data['transformed']['od'], elapsed_time,
                VIALS, 'OD')
self.save_data(data['transformed']['temp'], elapsed_time,
                VIALS, 'temp')

It essentially takes the first OD measurement as a zero and will subtract that out from every subsequent measurement. It is only applied to the calculated OD from calibration, not the raw data. The blank is applied across all vials, it just subtracts it out from the array:

github.com

FYNCH-BIO/dpu/blob/master/experiment/template/eVOLVER.py#L86


    logger.error('could not tranform raw data, skipping user-'
                 'defined functions')
    return

# should we "blank" the OD?
if self.use_blank and self.OD_initial is None:
    logger.info('setting initial OD reading')
    self.OD_initial = data['transformed']['od']
elif self.OD_initial is None:
    self.OD_initial = np.zeros(len(VIALS))
data['transformed']['od'] = (data['transformed']['od'] -
                                self.OD_initial)
# save data
self.save_data(data['transformed']['od'], elapsed_time,
                VIALS, 'OD')
self.save_data(data['transformed']['temp'], elapsed_time,
                VIALS, 'temp')

for param in od_cal['params']:
    self.save_data(data['data'].get(param, []), elapsed_time,
                VIALS, param + '_raw')

danigarcia · May 15, 2020, 9:04am

Thanks for your answer Zach.

I also see that the blank info is saved afterwards using pickle in case we need to restart the experiment afterwards:

github.com

FYNCH-BIO/dpu/blob/299d97f137a54e535640672f5984115cd645236b/experiment/template/eVOLVER.py#L438


def save_data(self, data, elapsed_time, vials, parameter):
    if len(data) == 0:
        return
    for x in vials:
        file_name =  "vial{0}_{1}.txt".format(x, parameter)
        file_path = os.path.join(EXP_DIR, parameter, file_name)
        text_file = open(file_path, "a+")
        text_file.write("{0},{1}\n".format(elapsed_time, data[x]))
        text_file.close()

def save_variables(self, start_time, OD_initial):
    # save variables needed for restarting experiment later
    save_path = os.path.dirname(os.path.realpath(__file__))
    pickle_name = "{0}.pickle".format(EXP_NAME)
    pickle_path = os.path.join(EXP_DIR, pickle_name)
    logger.debug('saving all variables: %s' % pickle_path)
    with open(pickle_path, 'wb') as f:
        pickle.dump([start_time, OD_initial], f)

def get_flow_rate(self):
    file_path = os.path.join(SAVE_PATH, PUMP_CAL_FILE)

I understand then that the blank is set for all 16 vials and we don’t have the option to blank again only a small set of vials. Do you think it would be a good idea to implement an option to load/save/overwrite the blank for only a set of chosen vials?

heinsz · May 18, 2020, 3:15pm

Could you elaborate on the use case for this functionality? Is it to handle the situation where you innoculate before starting the experiment and want to get a blank by inserting an empty media vial? I worry a little bit about adding a bunch of runtime command prompt menus. Maybe just having the values stored in a file that the user can manually set if desired would be fine, perhaps as a header of the OD files.

danigarcia · May 22, 2020, 10:28am

One of the possible use cases would be the one you mention, yes, but we always inoculate after blank. The use case I was thinking of was the following:

Imagine that you are going to run a 16 vials experiment, but for whatever reason you start first 4 vial row one week, next row next week etc. If you want to have all the vials in the same experiment folder (and therefore have all the output data and graphing together), you would need to stop the current experiment, add the new vials, restart, blank and inoculate. However, I believe that in the current setup, that would blank again the first vials that are already running. In this scenario, we would like to conserve the blank of all running vials and be able to blank the new ones.

So maybe logging the initial OD used as a blank on a human readable file (OD files seem like a good idea) would be good enough. We could add a small utility script to get the current OD of the vials in text format so we can copy that and modify the OD blank log file.

heinsz · June 2, 2020, 9:18pm

Could you not in this case just have a blank present at the start of the experiment?

danigarcia · June 20, 2020, 10:16am

Yes, that’s an option. However, if you want to have a good blank, you would need to use the vial that you will use for the experiment.

Of course is not something essential, but it may be useful to be able to log / modify the OD blank for some use cases.

heinsz · June 22, 2020, 8:08pm

Maybe @bgwong can provide some input here, but I think that having a blank with any vial with the media in should be sufficient. Also, take a look at this thread Preventing wobbling of the glass vial for better OD measurements. I think using the 3D printed aligner + having a blank of a vial with media present at the start should be fine even if the vial is later switched out. We routinely switch vials out mid experiment to combat things like biofilm, and OD seems to be fine afterwards. I have not actually tested out how much effect changing the vial will have on a zero though.

Another thing to keep in mind is that temperature changes can effect the OD zero measurement (the diodes are temperature sensitive), so having the vials be at temperature before the zero is taken will also help to ensure a good measurement.

bgwong · June 22, 2020, 8:30pm

@danigarcia This can easily be solved by just running multiple versions of the dpu script in parallel. The blank files are stored in the .pickle file, and not stored on the evolver itself. If you have parallel scripts running without cross triggering vials, it should properly blank the vial. All the data will still stream to all programs running, but I don’t think they should interfere.

As a more concrete example, let’s use your example of 4 vials running during week 1. For experiment 1, the data for vials 0-3 trigger pump commands but vials 4-15 don’t. Vials 4-15’s data is still being logged but they are essentially OFF because there is no feedback control (e.g. no pump commands are triggered). If you start second batch (experiment 2, with a totally different experimental folder and data record) a week later, all sixteen vials are still logged, but the blanks will be whatever it is when the experiment started, if you run another instance of the script. The logged blanks will be totally independent of experiment 1, found in a different pickle file. The blanked vial is saved on the level of the lab computer python script and not the Raspberry Pi.

One caveat is that you have to be careful not to program any redundant feedback control between experiments. Additionally, I would make sure all the temperatures are set to the same value. You don’t want experiment 1 to set temps for vial 0-4 at 30 and experiment 2 to accidentally change temps at 37 when the experiment starts.

Having it be human readable is a good idea! But for the meantime, pickle files are easy to open also.

https://wiki.python.org/moin/UsingPickle

Hope this helps!

Brandon

danigarcia · June 23, 2020, 11:08am

Thanks for all the comments and suggestions. Yeah, running different experiments will work, although the data would be “separated”. And it would also be easy to write a script to modify the data inside the pickle file.

Anyway, thanks for all the input. We will be evolving soon and we want to understand the machine as much as possible!

Best,

Dani

Topic		Replies	Views
Can I apply a new OD calibration to old data? Software	5	429	December 1, 2023
Initial OD Calibration Experiments	1	714	August 6, 2020
Regarding ODset in Turbidostat Data Processing Unit	2	340	January 6, 2021
Explanation of data folders Data Processing Unit	7	906	January 14, 2021
General improvements using eVOLVER as a turbidostat	9	571	July 26, 2023

Where or how is OD blank info stored?

Related topics