Hi!
Does anyone have recommendations for data smoothing scripts for eVOLVER OD data?
I am thinking over how to polish up OD curves from eVOLVER, and was wondering what other folks on the forum use. I am thinking of adapting something like this: https://towardsdatascience.com/removing-spikes-from-raman-spectra-8a9fdda0ac22
More specifically, I want to remove OD spikes that arise if I pull out a vial to sample or look at it, while the experiment is running.
Best,
Katie
Hi Katie,
I typically pause the experiment (ctrl+c one time on the command prompt) before I lift a vial out if I don’t want to mess up my OD or other data. Are you seeing large spikes even without lifting vials though?
@cmancuso did you ever have this kind of issue in your experiments?
Zack
Agreed on pausing during the vial lifting, if in turbidostat mode, since that also can trigger dilution events. For chemostat it doesn’t really matter much, but it will show up in your data. Spikes can happen for a variety of reasons though, like stirbar randomly jumping or spills so it’s a good question in general.
For smoothing data, I generally recommend doing this in a post-processing step after the data is all collected. Since all the data is saved in text files, you can use whatever software you like to do this. I wrote a set of MATLAB scripts to do my post-processing, but the file names and organization have changed so it needs an update. @mgalardini / @heinsz did you guys ever put up your python post-analysis code?
In general, I set a threshold OD range that I expect data from (say 0-0.8 OD for a turbidostat between 0.2-0.4 OD) and delete datapoints that fall outside that range. This removes the spikes and leaves a gap. I also apply a smoothing function when predicting growth rates or plotting OD over time for figures (reducing down from 10k+ datapoints), but you might choose not to do this.
During the experiment, we use median filters to avoid making dilution decisions based on spikes, but that only works if the spike is shorter than the median filter window.
Hi Zack,
That’s a great idea! Nope, the spikes are purely from my temptation to lift out the vials to peak, and I had not thought about pausing. That sounds much easier than removing them after.
Thank you!
Katie
I never wrote any other python post-processing for OD, last time I was looking at OD for analysis we still were using your scripts @cmancuso. If @mgalardini doesn’t have anything this would be a great small project for a new user or student!
Thank you both, Zack and Chris! I will take a look at Chris’ Matlab code as a starting point, and let you know if I come up with any useful additions/changes.
I have python notebooks that might be helpful, at least for pulling and structuring the OD data. And with Pandas, I have found it relatively easy to put thresholds in place. I’ve attached one you can view/run/copy from here