Database Management - David Court

Current Tasks

News & Updates:
OUTLINE OF THE NEW GLOBAL DATABASE

I. Import ALL radiometric, pigment, and absorption data from SeaBASS

II. Use PERL script to extract the data from its SeaBASS format.

1. Save as hyperspectral values
   1. Lu412.2 saved as Lu412
   2. Es682.6 saved as Es683
   3. Channels then averaged into 2 nm bins (evenly numbered)
      1. Es684 = average of Es683, Es684, Es685
3. second level A

02.01.2012: Absorption Figures for Our Global Dataset

01.30.2012: The Global Database has been updated Global_Data_2012_01_28.xls
1) Plumes & Blooms chlorophyll values corrected (cruises pb168 and pb213)
2) New data added - NSF-BWZ (chlorophyll, reflectance and absorption)
3) Recommend omitting data from AMT19. Reflectance values appear to be too high for wavelengths gt 490.

12.16.2011: BBOP data processed. See notes below in BBOP section

12.14.2011: The global database has been updated. /home/dcourt/SeaBASS_Data/Global_Data/Global_Data_2011_12_14.xls

PLEASE NOTE that I will be making some major improvements over the winter break. I will be re-doing the database so that it includes hyper-spectral measurements for each parameter. This will allow the user to choose a much wider range of options and it will also allow for better updates in the future.

11.30.2011: I've reprocessed the BBOP MicroPro data with the calibrations listed below - as specified in the timeline.
I then compared the newly reprocessed profiles (where different calibration files were used) with the former data.

BBOP RE-PROCESSING RESULTS: a comparison of the data - REPROCESSED/ORIGINAL (2009 - 2010)

UV channels - ED [306, 325, 340, 380]

Visible channels - ED [412, 443, 490, 510]

NIR channels - ED [555, 664, 683]

UV channels - LU [306, 325, 340, 380]

Visible channels - LU [412, 443, 490, 510]

NIR channels - LU [555, 664, 683]

BBOP CALIBRATION HISTORY: a closer look at the changes that have occurred with the MicroPro (2008 - 2011)
- Click on the links below:

- Calibration History Figures: MicroPro-113 Irradiance

- Calibration History Figures: MicroPro-113 Radiance

- Calibration History: Reference Irradiance

- Calibration History Spreadsheet

The figure below illustrates how each calibration was done in correlation with the cruise schedule:

10.01.2011: I will be in Switzerland from Dec 20th - Jan 7th.
I will be working some days while I am away and I will almost always be available via email

09.14.2011: DATA EDITS AND SUBMISSIONS
- The sample times for all the MicroPro data have been edited. All times are GMT.
- All MicroPro data have been RE-submitted to SeaBASS to ensure correct sample times. Jermey has been notified
- PAR values have been calculated from all in-water SPMR & MicroPro irradiance profile data 18.10.2012
- I've asked for a Microsoft operating system to be installed - along with VMware Fusion 02.10.2012
- I've found the "underway" irradiance data associated with the MER.
- I've extracted (and formatted) these Es data and have calculated daily PAR values based on the sampling interval of 1.4 seconds.
- These daily PAR values are now available on-line: ftp://ftp.eri.ucsb.edu/pub/org/oceancolor/BBOP/BBOP_PAR/BBOP_daily_PAR_1992_2011.txt

Ongoing:

Global Database of In-Situ Matchups
- Adding new data as it becomes available from SeaBASS
- Applying QC/QA measures to updates
- Most recent update: September 01, 2012
- Need to redevelop MySQL tools for on-line access

BBOP
- WEBPAGE: http://www.icess.ucsb.edu/bbop/
- Processing BBOP MicroPro data
- Formatting data for SeaBASS submissions
- Latest cruise: B276
- Processed: YES:
- Latest data submitted to SeaBASS: B276 & BVAL46 (01.08.2012)
- Data available on-line: YES

Plumes & Blooms
- WEBPAGE: http://www.icess.ucsb.edu/PnB/PnB.html
- Formatting processed CTD for SeaBASS submissions
- Processing and formatting PRR data for SeaBASS submissions
- Latest cruise: PB234
- Latest data submitted to SeaBASS:

 CTD: PB234

 PRR: PB225 *waiting for new processing scripts

- Data available on-line: YES

CLIVAR
- Process MicroPro radiometric data as it becomes available
- Matchup data products (AOPs, IOPs & pigments)
- Last cruise: P6 - April 2010
- Submitted to SeaBASS: YES
- Additional cruises: A20 A22 AMMA I08S I09N P16S P16N P18

Global Matchups (In-Situ Data + Remotely Sensed Data Products)
- WEBPAGE: http://www.icess.ucsb.edu/~dcourt/MATCHUPS/

MEaSUREs
- Plan to build some web-tools per Stéphane's needs

Current Projects:

Re-organization of the BBOP database
Purpose:
- To increase the performance of the (in-situ) matchups
Methods:
- Imported all pigment measurements from SeaBASS
- Organized all radiometric data (MER, SPMR, & MicrPro)
- Organized all absorption data (ag, ad, ap & aph)
- Cross-referencing all measurements with BATS log sheets
Issues:
- The chlorophyll values in the original database were incorrect. The chlorophyll values were taken from the IOP bottle-files that had been submitted to SeaBASS. These were labeled as "chl_a", which is defined by SeaBASS as an HPLC product. However, these are Flourometric measurements. At some point, a script needs to be written to re-format these files and a they must then be re-submitted to SeaBASS.
- Many of the sample times are incorrect. This creates inaccuracies when trying to match-up the different measurements. Most of the problems are a result of submitting the data in local (Bermuda) time rather than GMT - as required by SeaBASS. Other issues exist as a result of the technician's computer clock being incorrectly set - sometimes by an order of years. Again, these data will ultimately have to edited and re-submitted to SeaBASS.