Think of admiral as a toolbox of modular blocks (R
functions) to create analysis derivations:
each block has a stand alone purpose (each function provides a
specific functionality)
Data Scientists can create their own blocks (create own R
functions)
Constructing ADaM dataset should become like building out of blocks
that are based on admiral modular functions and user
created modular functions.
Why did we decide to start
admiral?
Data analysis challenges in clinical trials vary depending on
scientific goals, therapeutic areas, indications, data sources and data
quality. We all face the same challenge so why limit ourselves only to
company-level adoption and crowd-sourcing to create ADaM datasets?
Build ADaMs via collaboration and co-creation
Early engagement with other like-minded companies moving towards R
could lead to our solution being shared open source as a framework for
contribution across-industry
Building ADaMs like a modular building blocks, everyone can
contribute and each module has a clear input and output to enable
re-usable solutions
Users can “slot in” their own modules to address specific
company/TA/Molecule/Study requirements
TA specific requirements can be open sourced again and transformed
into a common ADaM approach for such analysis
the long-term gain of a consistent way of producing ADaM and a wider
community of across-industry developers contributing to grow the
codebase to cover the infinite array of possibilities
Contributors: An option to make a name for yourself in the Pharma
open-source community & an avenue to collaborate with other
like-minded people across the industry
Imagine if ADaMs are built in a consistent manner with the same code
from openly maintained functions and its impact on the Health
Authorities, readable code, QC, talent flow
Why did we use R as a programming language?
R is not an isolated software product, everyone can contribute (open
source principal)
People from University/Statistical talent pipeline more likely to
come through with R skills rather than a proprietary language
There seems to be a strong data science/analytics R community
FDA open to accepting R submissions and are heavy users
themselves
Top of the line visualization/graphics - R-Shiny for interactive
data displays and also R Markdown offers great report writing
functionality
R is very popular among statisticians so new statistical methods are
likely implemented in R before any other language
There might be equally suited programming languages out there -
however at some stage we had to make a decision :)
Why do we use a certain R version and package
versions for development?
The choice of R Version is not set in stone. However, a common
development environment is important to establish when working across
multiple companies and multiple developers. We currently work in the
earliest of the three latest R Versions. This need for a common
development environment also carries over for our choice of package
versions.
GitHub allows us through the Actions/Workflows to test
admiral under several versions of R as well as several
versions of dependent R packages needed for admiral.
Currently we test admiral against the three latest R
Versions and the closest snapshots of packages to those R versions. You
can view this workflow and others on our admiralci GitHub
Repository.
This common development allows us to easily re-create bugs and
provide solutions to each other issues that developers will
encounter.
Reviewers of Pull Requests when running code will know that their
environment is identical to the initiator of the Pull Request. This
ensures faster review times and higher quality Pull Request
reviews.
We achieve this common development environment by using a
lockfile created from the renv package.
New developers will encounter a suggested renv::restore()
in the console to revert or move forward your R version and package
versions.
Admiral offers a toolbox of functions to facilitate
ADaM. What does that mean?
Functions are usually not necessarily specific but parameter driven:
e.g. the derive_vars_aage has a parameterized start and
end-date and a unit.
Depending on the parameters results may vary as does the
specification.
Functions serve as a toolbox so the user can create their ADaM
according to the requirements.
The principles, programming strategy and documentation of
admiral are considered as a framework for users to
contribute.
How does a user know what a function does
exactly?
Function details and its purpose, the requirements, parameters,
dependencies and examples are documented in the header of each
function.
Complex functions potentially have a vignette on the
admiral homepage to provide more details.
admiral does not provide a link to an explicit
specification in the define.xml.
Would {admiral}create a whole ADaM
dataset?
admiral is meant as a toolbox to enable Data
Scientists to build ADaMs according to their varying analysis needs
admiral is not meant as a “click a button, out comes
your ADaM” tool
on the admiral webpage, example scripts are provided
which can be used as a starting point to create an ADaM (see at the end
of a vignette)
In which order does a user need to execute the
functions?
Guidance will be provided for ADSL, BDS and OCCDS ADaM structure
including template scripts.
Test cases for each function will be part of the R package.
Users can add to the tests or provide additional feedback.
The testing the admiral team will do for each
function does not replace the QC and validation process at each
company.
A GitHub
action (using open source packages) exists to generate a validation
report for an R package, which would be an option for any company to
use. An example report using an earlier version of
admiral exists here
as an illustration.
What will be provided around function testing?
Unit tests for reliability of each function - available as part of
open source release
Some integration testing will be done to ensure functions can be
called together to create ADaM (e.g. even via the internal testing
teams)
Guidance for testing and documentation expectations of community
contribution functions. Then it is for each company to cover the
following:
validation to be able to use the package on company-specific SCE for
GxP purposes and associated audit evidence
strategy of how the use of admiral fits into
company-specific quality assurance process (double programming
comparison versus your company-specific legacy ADaM solution could be
appropriate until confidence builds)
The users are responsible to make sure they use the functions and
their parameters in the right way to ensure alignment with their
define.xml
Will {admiral} provide ADaM IG CDISC
compliant datasets?
Although admiral follows CDISC standards it does not
claim that the dataset resulting from calling admiral
functions is ADaM compliant. This has to be ensured by the user.
How much of the ADaM IG is covered by admiral?
ADaM IG is a standard framework without a specific number of
datasets or variables, so it cannot be used as a specific baseline to
answer that question.
How has {admiral} been tested
externally to Roche/GSK?
During Sept/Oct 2021, a limited release testing was conducted with
18 other companies (and >50 individuals) in order to assess
compatibility of the admiral toolkit with different
company standards implementations and to test the usability of the
functions, e.g. clarity, reliability, robustness, and flexibility.
This foundational version of admiral achieved a 7.9 /
10 average score from all the survey respondents and >75% said they’d
advocate using admiral for ADaM transformations in
R.
Some tester quotes:
“Extremely easy to learn and get into, well thought and planned.
Plenty of minor functions instead of aiming to create a large”jack of
all trades” framework. The toolkit does not attempt to become a large
one-button ADaM generator (which is fantastic).”
“It is a huge advantage for all Pharma companies that we have
common functions for common stuff we develop. It will be easier for the
authorities when it is the same foundation in the ADaM programs. The
development goes faster for every one when we develop across companies,
and bug-fixing is faster as many are using same package and will most
likely find potential bugs.”
“I am a huge proponent of shared solutions within Pharma.
Overall I was VERY impressed with the admiral project –
both the design, development, documentation, and validation details are
available for teams to readily adopt.”
