About IOTC SS3 VRE About IOTC SS3 VRE

Summary

 

The SS3 VRE is a tool that hosts codes necessary to run the Stock Synthesis version 3 (SS3) stock assessment model (a widely-used statistical catch-at-age model), parallelize, visualize and store the standardized data outputs.

 

Stock assessment softwares are complex and advanced technical skills are required to develop the models. Producing output becomes time-intensive and even more complex as thousands of simulations must be run on supercomputers in order to include the multiple sources of uncertainty in assessment results. As few stock assessment participants have the specific technical skills required to reproduce these outputs, IRD, IFREMER, the IOTC and FAO have partnered with the H2020 BlueBRIDGE project to develop a simple online tool in the form of a Virtual Research Environment (VRE) to facilitate the use of SS3 to users with varying levels of expertise.

 

The VRE enables any user to easily parameterize, execute and edit online various steps of the stock assessment work flow. It transforms the outputs of the models into a standard NetCDF data format, embeds metadata about the model run in the data file, and stores these data on the VRE infrastructure. The metadata for each run can then be searched on metadata catalogs hosted by the VRE, thus enabling traceability and reproducibility of past assessments.

 

The VRE also provides several collaborative web services, including: (i) a workspace to share documents, codes or data, (ii) an RStudio server to process data online, (iii) R Shiny applications to quickly and easily visualize the output of model runs, and (iv) Sharelatex to provide a platform for an automated reporting service to dynamically generate documents to package the results.

 

 

Currently, the IOTC_SS3 VRE has replicated and made available the results of past IOTC stock assessments using Stock Synthesis version 3.24z (ss3.24z) of swordfish (SWO, Xiphias gladius), bigeye tuna (BET; Thunnus obesus), yellowfin tuna (YFT; Thunnus albacares), skipjack tuna (SKJ, Katsuwonus pelamis) provided by the IOTC and their consultants (Table 1). We have recently added the 2015 stock assessment of greater amberjack (GAJ, Seriola dumerili) and a new SS3 model of Atlantic bluefin tuna (BFT, Thunnus thynnus) provided by our partners at NOAA. Theoretically, any species model can be similarly replicated.
 

Table 1: The species models that are currently available on the VRE.

RFMO

Common name

Scientific name

FAO code

Assess

year

Model name

Modeler

IOTC

Swordfish

Xiphias gladius

SWO

2017

SWO_2017

Dan Fu, IOTC

IOTC

Yellowfin tuna

Thunnus albacares

YFT

2016

YFT_2016

Adam Langley, Trophia

IOTC

Bigeye tuna

Thunnus obesus

BET

2016

BET_2016_h70

Adam Langley, Trophia

IOTC

Bigeye tuna

Thunnus obesus

BET

2016

BET_2016_h80

Adam Langley, Trophia

IOTC

Bigeye tuna

Thunnus obesus

BET

2016

BET_2016_h90

Adam Langley, Trophia

IOTC

Skipjack tuna

Katsuwonus pelamis

SKJ

2017

SKJ_2017

Dan Fun, IOTC

WECAFC

Greater amberjack

Seriola dumerili

GAJ

2015

GAJ_2015

Nancie Cummings, NOAA

ICCAT

Atlantic bluefin tuna

Thunnus thynnus

BFT

2017

BFT_2017

Rishi Sharma, NOAA

 

Background

Several different types of stock assessment models are used to provide scientific advice to managers about exploited populations. Stock Synthesis 3 (SS3) is a statistical catch-at-age model that is used widely (Methot and Wetzel, 2013), including assessments for several stocks under the management of the Indian Ocean Tuna Commission (IOTC). SS3 is flexible in terms of data inputs and complexity, making it possible to run with data-poor stocks. It can use a diverse array of fishery and survey data, including both age and size structure of the population.

 

SS3 is based on ADMB C++ software that maximizes the goodness-of-fit of a set of parameter values, and then calculates the variance of these parameters using inverse Hessian and MCMC methods. This software is complex and advanced technical skills are required to develop the models. The production of outputs can be time-intensive and complex when thousands of simulations are needed to include the multiple sources of uncertainty in the assessment results. Interactions with the results also necessitate skilled language programming.

 

As few stock assessment participants have the specific technical skills required to reproduce these outputs, we developed this VRE to facilitate the parameterization, parallelization, and execution of various steps and the visualization of the results of SS3 to users with varying levels of expertise.

Benefits

A collaborative environment such as the VRE uses simple tools to enable the storage and access of the data and source codes necessary to replicate past results or to try new parameterizations of the model. Increasing access to this complex model will bring more transparency and collaboration within working groups by providing “non-modelers” with a possibility to test hypotheses for the stock assessment. This will also increase the number of users of various levels of expertise: from experts, to managers, to even wider audiences with the potential applications of these tools to serious games.  Technical performance, document production, and harmonization of content will also be enhanced.

 

Main Services & functionalities:

 

 

  • Collaborative Website: exchange and collaborate between users
    • Data (inputs) stored online,
    • Source codes are now accessible that were not before: access in the VRE through URLs
    • Model outputs can be browsed / visualized with graphical interfaces (Shiny)
  • Run models from Graphical User Interface (Web Forms)
    • Focus on parameterization only : more users able to run the models
  • Processing server: no need for local compilation environment & computing power:
    • Compilation environment for R with Rstudio online
    • Parallelization : much more runs / sensitivity analyses
    • Latex with Sharelatex (including knitr)
  • Data server: data access online (not to be downloaded on local PC) through different protocols
    • Archives / backup of the stock assessment ensured
    • Selected runs (specifications and data) can be stored and replicated:
      • Generic (meta)data formats and outputs (figures, table)
        • workflows and results properly described
    • Download service:  using the NetCDF data formats from workspace URLs and Thredds,
    • OPeNDAP protocol: Thredds
  • Metadata catalogs (CKAN and Geonetwork), allow data discovery, traceability and reproducibility of past assessments.

Target users and terms of use

 

Target users are scientists / experts, NGOs, managers, and policy makers involved with stock assessment. Currently, SS3 is available to researchers with an NOAA VLab account. We have confirmed that it is acceptable to the developers and maintainers of SS3 that we make the software available to users in the format of the VRE. All users would require an account to access the VRE. The registration to the SS3 VRE is open to anybody but moderated by the VRE manager. Click on "Request Access".

 

Scenarios of use

 

We tested the run times of several stock assessment models, provided by the IOTC and their consultants, including SWO, BET, and YFT. On the Bluebridge infrastructure Rstudio online, these models take between 1.5-20 minutes. Based on these run times, we identified various scenarios of use, and calculated the CPU resources they require (Table 1).

 

Summary

Iterations

Time required

CPUs required:

Expert use (e.g. consultants)

Developing a model and running sensitivity analyses before the stock assessment.

10,000

1 day * 3 major modifications

(0,5*10000)/

(3*24*60) =

~2 CPUs

Modifications during the stock assessment.

1,000

1 hr (within 1

day of meeting)

(0,5*1000)/60 =

~8 CPUs

Interested use (e.g. meeting participants)

One simulation per user

1 (with e.g. 30 users)

full duration of the

meeting, e.g. 5 days

30 CPUs

throughout meeting

Each user allowed 10 iterations. Results required immediately (i.e., duration of single run).

10

Specified period

within 1 day

30*10 =

300 CPUs

 

Workflow

We tested the feasibility of reproducing past IOTC SS3 stock assessment models of tropical tunas and billfish on the BlueBridge infrastructure. We repackage the SS3 codes so that they can be parametrized, executed and edited online from a simple web page, with standardized data outputs.

 

 

By doing so, the workflow will be managed in different steps:

  1. Builds and parameterizes the model (using data inputs), launches the model run(s), reads the outputs and transforms them into NetCDF, embedding standard metadata.
  2. Visualizations (shiny) and selection of runs,
  3. Automated reports (pdf or html from Sharelatex / knitr /Rmarkdown
 

We encourage suggestions from users on the parameters that are tested and changed most frequently during model selection, and the specific outputs that the user would like to visualize to investigate the model.

 

Keywords

Stock Synthesis, SS3, large pelagics, tuna, billfish, reproducible science, collaboration

 

Related presentations and reports

IOTC papers describing the VRE

  • IOTC Working Party on Billfish (September 2017, AZTI, San Sebastian, Spain), IOTC-2017- WPB15-26.
  • IOTC Working Party on Methods (October 2017, IOTC, Victoria, Seychelles), IOTC-2017-WPM08-12.
  • IOTC Working Party on Tropical Tuna (October 2017, IOTC, Victoria, Seychelles), IOTC-2017-WPTT19-39.

IOTC papers describing the standardization of outputs

  • IOTC Working Party on Data Collection & Statistics (November 2017, IOTC Victoria, Seychelles), IOTC-2017-WPDCS13-38_Rev2.
  • IOTC Working Party on Data Collection & Statistics (November 2017, IOTC Victoria, Seychelles), IOTC-2017-WPDCS13-33.
 

References

R.D. Methot Jr., C.R. Wetzel, Stock synthesis: A biological and statistical framework for fish stock assessment and fishery management, Fisheries Research 142 (2013) 86– 99.

 

The IOTC_SS3 VRE  has been made possible by  funding  from  the  European  Union’s  Horizon  2020  research and innovation program under the BlueBRIDGE project (Grant agreement No 675680).

Enter IOTC SS3 VRE Enter IOTC SS3 VRE

Access the  IOTC SS3 VRE with your BlueBRIDGE Gateway credentials.