pysilcam.postprocess module

class pysilcam.postprocess.TimeIntegratedVolumeDist(settings)

Bases: object

class used for summarising recent stats in real-time

Possibly redundant

@todo - re-implement this later

update_from_stats(stats, timestamp)

Update size distribution from stats

pysilcam.postprocess.add_best_guesses_to_stats(stats)

Calculates the most likely tensorflow classification and adds best guesses to stats dataframe.

Parameters

stats (DataFrame) – particle statistics from silcam process

Returns

particle statistics from silcam process

with new columns for best guess and best guess value

Return type

stats (DataFrame)

pysilcam.postprocess.add_depth_to_stats(stats, time, depth)

if you have a depth time-series, use this function to find the depth of each line in stats

Parameters

  • stats – pandas DataFrame of particle statistics

  • time – time stamps associated with depth argument

  • depth – depths associated with the time argument

Returns

pandas DataFrame of particle statistics, now with a depth column

Return type

stats

pysilcam.postprocess.bright_norm(im, brightness=255)

eye-candy function for normalising the image brightness

Parameters

  • im (uint8) – image

  • brightness=255 – median of histogram will be shifted to align with this value

Returns

image with modified brightness

Return type

im (uint8)

pysilcam.postprocess.count_images_in_stats(stats)

count the number of raw images used to generate stats

Parameters

stats – pandas DataFrame of particle statistics

Returns

number of raw images

Return type

n_images

pysilcam.postprocess.d50_from_stats(stats, settings)

Calculate the d50 from the stats and settings

Parameters

  • stats (DataFrame) – particle statistics from silcam process

  • settings (PySilcamSettings) – settings associated with the data, loaded with PySilcamSettings

Returns

the 50th percentile of the cumulative sum of the volume distributon, in microns

Return type

d50 (float)

pysilcam.postprocess.d50_from_vd(vd, dias)

Calculate d50 from a volume distribution

Parameters

  • vd – particle volume distribution calculated from vd_from_stats()

  • dias – mid-points in the size classes corresponding the the volume distribution, returned from get_size_bins()

Returns

the 50th percentile of the cumulative sum of the volume distributon, in microns

Return type

d50 (float)

pysilcam.postprocess.d50_timeseries(stats, settings)

Calculates time series of d50 from stats

Parameters

  • stats – pandas DataFrame of particle statistics

  • settings – PySilCam settings

Returns

time series of median particle diameter (50th percentile of volume distribution) time : timestamps for d50 values

Return type

d50

pysilcam.postprocess.explode_contrast(im)

eye-candy function for exploding the contrast of a particle iamge (roi)

Parameters

im (uint8) – image (normally a particle ROI)

Returns

image following exploded contrast

Return type

im_mod (uint8)

pysilcam.postprocess.export_name2im(exportname, path)

returns an image from the export name string in the -STATS.h5 file

get the exportname like this: exportname = stats[‘export name’].values[0]

Parameters

  • exportname – string containing the name of the exported particle e.g. stats[‘export name’].values[0]

  • path – path to exported h5 files

Returns

particle ROI image

Return type

im

pysilcam.postprocess.extract_latest_stats(stats, window_size)

extracts the stats data from within the last number of seconds specified by window_size.

Parameters

  • stats – pandas DataFrame of particle statistics

  • window_size – number of seconds to extract from the end of the stats data

Returns

stats dataframe (from the last window_size seconds)

pysilcam.postprocess.extract_nth_largest(stats, n=0)

return statistics of the nth largest particle

pysilcam.postprocess.extract_nth_longest(stats, n=0)

return statistics of the nth longest particle

pysilcam.postprocess.filter_stats(stats, crop_stats)

Filters stats file based on whether the particles are within a rectangle specified by crop_stats. A temporary cropping solution due to small window in AUV

Parameters

  • stats (df) – silcam stats file

  • crop_stats (tuple) – 4-tuple of lower-left (row, column) then upper-right (row, column) coord of crop

Returns

cropped silcam stats file

Return type

stats (df)

pysilcam.postprocess.gen_roifiles(stats, auto_scaler=500)

generates a list of filenames suitable for making montages with

Parameters

  • stats (DataFrame) – particle statistics from silcam process

  • auto_scaler=500 – approximate number of particle that are attempted to be pack into montage

Returns

a selection of filenames that can be passed to montage_maker() for making nice montages

Return type

roifiles

pysilcam.postprocess.get_j(dias, nd)

calculates the junge slope from a correctly-scale number distribution (number per micron per litre must be the units of nd)

Parameters

  • dias – mid-point of size bins

  • nd – number distribution in number per micron per litre

Returns

Junge slope from fitting of psd between 150 and 300um

Return type

j

pysilcam.postprocess.get_sample_volume(pix_size, path_length=10, imx=2048, imy=2448)

calculate the sample volume of one image

Parameters

  • pix_size – size of pixels in microns (settings.PostProcess.pixel_size)

  • path_length=10 – path length of sample volume in mm

  • imx=2048 – image x dimention in pixels

  • imy=2448 – image y dimention in pixels

Returns

the volume of the sample volume in litres

Return type

sample_volume_litres

pysilcam.postprocess.get_size_bins()

Retrieve size bins for PSD analysis

Returns

mid-points of size bins bin_limits_um (array) : limits of size bins

Return type

bin_mids_um (array)

pysilcam.postprocess.make_montage(stats_file, pixel_size, roidir, auto_scaler=500, msize=1024, maxlength=100000, oilgas=<outputPartType.all: 1>, crop_stats=None)

wrapper function for montage_maker

Parameters

  • stats_file – location of the stats_csv file that comes from silcam process

  • pixel_size – pixel size of system defined by settings.PostProcess.pix_size

  • roidir – location of roifiles usually defined by settings.ExportParticles.outputpath

  • auto_scaler=500 – approximate number of particle that are attempted to be pack into montage

  • msize=1024 – size of canvas in pixels

  • maxlength=100000 – maximum length in microns of particles to be included in montage

  • oilgas=outputPartType.all – enum defining which type of particle to be selected for use in the montage

  • crop_stats=None – None or 4-tuple of lower-left then upper-right coord of crop

Returns

a nicely-made montage in the form of an image, which can be plotted using plotting.montage_plot(montage, settings.PostProcess.pix_size)

Return type

montage (uint8)

pysilcam.postprocess.make_timeseries_vd(stats, settings)

makes a dataframe of time-series volume distribution and d50

Parameters

  • stats (silcam stats dataframe) – loaded from a *-STATS.h5 file

  • settings (silcam settings) – loaded from PySilCamSettings

Returns

of time series volume concentrations are in uL/L columns with number headings are diameter min-points

Return type

dataframe

pysilcam.postprocess.montage_maker(roifiles, roidir, pixel_size, msize=2048, brightness=255, tightpack=False, eyecandy=True)

makes nice looking matages from a directory of extracted particle images

use make_montage to call this function

Parameters

  • roifiles – list of roi files obtained from gen_roifiles(stats, auto_scaler=auto_scaler)

  • roidir – location of roifiles usually defined by settings.ExportParticles.outputpath

  • pixel_size – pixel size of system defined by settings.PostProcess.pix_size

  • msize=2048 – size of canvas in pixels

  • brightness=255 – brighness of packaged particles

  • tightpack=False – boolean which if True packs particles using segmented alpha-shapes instead of bounding boxes. This is slow, but packs particles more tightly.

  • eyecandy=True – boolean which if True will explode the contrast of packed particles (nice for natural particles, but not so good for oil and gas).

Returns

a nicely-made montage in the form of an image, which can be plotted using plotting.montage_plot(montage, settings.PostProcess.pix_size)

Return type

montageplot

pysilcam.postprocess.nc_from_nd(count, sv)

Calculate the number concentration from the count and sample volume

Parameters

  • count (array) – particle number distribution

  • sv=1 (float) – the volume of the sample which should be used for scaling concentrations

Returns

the total number concentration in #/L

Return type

nc (float)

pysilcam.postprocess.nc_vc_from_stats(stats, settings, oilgas=<outputPartType.all: 1>)

Calculates important summary statistics from a stats DataFrame

Parameters

  • stats (DataFrame) – particle statistics from silcam process

  • settings (PySilcamSettings) – settings associated with the data, loaded with PySilcamSettings

  • oilgas=oc_pp.outputPartType.all – the oilgas enum if you want to just make the figure for oil, or just gas (defaults to all particles)

Returns

the total number concentration in #/L vc (float) : the total volume concentration in uL/L sample_volume (float) : the total volume of water sampled in L junge (float) : the slope of a fitted juge distribution between 150-300um

Return type

nc (float)

pysilcam.postprocess.nd_from_stats(stats, settings)

calcualte number distirbution from stats units are number per bin per sample volume

Parameters

  • stats (DataFrame) – particle statistics from silcam process

  • settings (PySilcamSettings) – settings associated with the data, loaded with PySilcamSettings

Returns

mid-points of size bins necd : number distribution in number/size-bin/sample-volume

Return type

dias

pysilcam.postprocess.nd_from_stats_scaled(stats, settings)

calcualte a scaled number distribution from stats and settings units of nd are in number per micron per litre

Parameters

  • stats (DataFrame) – particle statistics from silcam process

  • settings (PySilcamSettings) – settings associated with the data, loaded with PySilcamSettings

Returns

mid-points of size bins nd : number distribution in number/micron/litre

Return type

dias

pysilcam.postprocess.nd_rescale(dias, nd, sample_volume)
rescale a number distribution from

number per bin per sample volume

to

number per micron per litre

Parameters

  • dias – mid-points of size bins

  • nd – unscaled number distribution

  • sample_volume – sample volume of each image

Returns

scaled number distribution (number per micron per litre)

Return type

nd

class pysilcam.postprocess.outputPartType(value)

Bases: enum.Enum

Enum class for all (1), oil (2) or gas (3)

all = 1
gas = 3
oil = 2
pysilcam.postprocess.show_h5_meta(h5file)

prints metadata from an exported hdf5 file created from silcam process

Parameters

h5file – h5 filename from exported data from silcam process

pysilcam.postprocess.silc_to_bmp(directory)

Convert a directory of silc files to bmp images

Parameters

directory – path of directory to convert

pysilcam.postprocess.stats_to_xls_png(config_file, stats_filename, oilgas=<outputPartType.all: 1>)

summarises stats in two excel sheets of time-series PSD and averaged PSD.

Parameters

  • config_file (string) – Path of the config file for this data

  • stats_filename (string) – Path of the stats csv file

  • oilgas=oc_pp.outputPartType.all – the oilgas enum if you want to just make the figure for oil, or just gas (defaults to all particles)

Returns

of time series files: in the proc folder)

Return type

dataframe

pysilcam.postprocess.statscsv_to_statshdf(stats_file)

Convert old STATS.csv file to a STATS.h5 file

Parameters

stats_file – filename of stats file

pysilcam.postprocess.trim_stats(stats_file, start_time, end_time, write_new=False, stats=[])

Chops a STATS.h5 file given a start and end time

Parameters

  • stats_file – filename of stats file

  • start_time – start time of interesting window

  • end_time – end time of interesting window

  • write_new=False – boolean if True will write a new stats csv file to disc

  • stats=[] – pass stats DataFrame into here if you don’t want to load the data from the stats_file given. In this case the stats_file string is only used for creating the new output datafilename.

Returns

pandas DataFram of particle statistics outname : name of new stats csv file written to disc

Return type

trimmed_stats

pysilcam.postprocess.vd_from_nd(count, psize, sv=1)

Calculate volume concentration from particle count

sv = sample volume size (litres)

e.g: sample_vol_size=25*1e-3*(1200*4.4e-6*1600*4.4e-6); %size of sample volume in m^3 sv=sample_vol_size*1e3; %size of sample volume in litres

Parameters

  • count (array) – particle number distribution

  • psize (float) – pixel size of the SilCam contained in settings.PostProcess.pix_size from the config ini file

  • sv=1 (float) – the volume of the sample which should be used for scaling concentrations

Returns

the particle volume distribution

Return type

vd (array)

pysilcam.postprocess.vd_from_stats(stats, settings)

calculate volume distribution from stats units of miro-litres per sample volume

Parameters

  • stats (DataFrame) – particle statistics from silcam process

  • settings (PySilcamSettings) – settings associated with the data, loaded with PySilcamSettings

Returns

mid-points of size bins vd : volume distribution in micro-litres/sample-volume

Return type

dias

pysilcam.postprocess.vd_to_nc(vd, dias)

calculate number concentration from volume distribution

Parameters

  • vd (array) – particle volume distribution calculated from vd_from_stats()

  • dias (array) – mid-points in the size classes corresponding the the volume distribution, returned from get_size_bins()

Returns

number concentration (scaling is the same unit as the input vd).

If vd is a 2d array [time, vd_bins], nc will be the concentration for row

Return type

nn (float)

pysilcam.postprocess.vd_to_nd(vd, dias)

convert volume distribution to number distribution

Parameters

  • vd (array) – particle volume distribution calculated from vd_from_stats()

  • dias (array) – mid-points in the size classes corresponding the the volume distribution, returned from get_size_bins()

Returns

number distribution as number per micron per bin (scaling is the same unit as the input vd)

Return type

nd (array)