HubPub 1.8.1
HubPub provides users with functionality to help with the Bioconductor Hub
structures. The package provides the ability to create a skeleton of a Hub
style package that the user can then populate with the necessary information.
There are also functions to help add resources to the Hub pacakge metadata
files as well as publish data to the Bioconductor S3 bucket.
Install the most recent version from Bioconductor:
if(!requireNamespace("BiocManager", quietly = TRUE))
install.packages("BiocManager")
BiocManager::install("HubPub")Then load HubPub:
library(HubPub)The create_pkg() function creates the skeleton of a package that follows the
guidelines for a Bioconductor Hub type package. More information about what
are the requirements and content for a Hub style package the developer can look
at the “Creating A Hub Package” vignette from this package.
create_pkg() requires a path to where the packages is to be created and the type
of package that should be created (“AnnotationHub” or “ExperimentHub”). There is
also a variable use_git that indicates if the package should be set up with
git (default is TRUE).
NOTE: This function is intended for a developer that has not created the package yet. If the package has already been created, then this function will not benefit the developer. There are a couple other functions in this package that deal with resources that might be helpful, more on these later in the vignette.
fl <- tempdir()
create_pkg(file.path(fl, "examplePkg"), "ExperimentHub")
#> ✔ Creating '/tmp/Rtmp8rQxmQ/examplePkg/'
#> ✔ Setting active project to '/tmp/Rtmp8rQxmQ/examplePkg'
#> ✔ Creating 'R/'
#> ✔ Writing 'DESCRIPTION'
#> Package: examplePkg
#> Title: What the Package Does (One Line, Title Case)
#> Version: 0.99.0
#> Date: 2023-08-10
#> Authors@R (parsed):
#> * First Last <first.last@example.com> [aut, cre] (YOUR-ORCID-ID)
#> Description: What the package does (one paragraph).
#> License: Artistic-2.0
#> BugReports: https://support.bioconductor.org/t/examplePkg
#> Imports:
#> ExperimentHub
#> Suggests:
#> ExperimentHubData
#> Encoding: UTF-8
#> Roxygen: list(markdown = TRUE)
#> RoxygenNote: 7.2.3
#> biocViews: ExperimentHub
#> ✔ Writing 'NAMESPACE'
#> ✔ Setting active project to '<no active project>'
#> ✔ Setting active project to '/tmp/Rtmp8rQxmQ/examplePkg'
#> ✔ Initialising Git repo
#> ✔ Adding '.Rproj.user', '.Rhistory', '.Rdata', '.httr-oauth', '.DS_Store', '.quarto' to '.gitignore'
#> ✔ Writing 'R/examplePkg-package.R'
#> ✔ Writing 'NEWS.md'
#> ✔ Creating 'man/'
#> ✔ Creating 'inst/scripts/'
#> ✔ Writing 'inst/scripts/make-data.R'
#> ✔ Writing 'inst/scripts/make-metadata.R'
#> ✔ Writing 'R/zzz.R'
#> ✔ Creating 'inst/extdata/'
#> ✔ Adding 'testthat' to Suggests field in DESCRIPTION
#> ✔ Adding '3' to Config/testthat/edition
#> ✔ Creating 'tests/testthat/'
#> ✔ Writing 'tests/testthat.R'
#> • Call `use_test()` to initialize a basic test file and open it for editing.
#> ✔ Writing 'tests/testthat/test_metadata.R'
#> [1] "/tmp/Rtmp8rQxmQ/examplePkg"Once the package is created the developer can go through and make any changes to the package. For example, the DESCRIPTON file contains very basic requirements but the developer should go back and fill in the ‘Title:’ and ‘Description:’ fields.
Another useful function in HubPub is add_resource(). This function can be
useful for developers who are creating a new Hub related package or for
developers who want to add a new resource to an existing Hub package. The
purpose of this function is to add a hub resource to the package metadata.csv
file. The function requires the name of the package (or the path to the newly
created package) and a named list with the data to be added to the resource. To
get the elements and content for this list look at ?hub_metadata. There is
also information in the “Creating A Hub Package” vignette from this package.
metadata <- hub_metadata(
Title = "ENCODE",
Description = "a test entry",
BiocVersion = "4.1",
Genome = NA_character_,
SourceType = "JSON",
SourceUrl = "http://www.encodeproject.org",
SourceVersion = "x.y.z",
Species = NA_character_,
TaxonomyId = as.integer(9606),
Coordinate_1_based = NA,
DataProvider = "ENCODE Project",
Maintainer = "tst person <tst@email.com>",
RDataClass = "Rda",
DispatchClass = "Rda",
Location_Prefix = "s3://experimenthub/",
RDataPath = "ENCODExplorerData/encode_df_lite.rda",
Tags = "ENCODE:Homo sapiens"
)
add_resource(file.path(fl, "examplePkg"), metadata)
#> Warning: replacing previous import 'utils::findMatches' by
#> 'S4Vectors::findMatches' when loading 'ExperimentHubData'
#> [1] "/tmp/Rtmp8rQxmQ/examplePkg/inst/extdata/metadata.csv"Then if you want to see what the metadata file looks like you can read in the csv file like the following.
resource <- file.path(fl, "examplePkg", "inst", "extdata", "metadata.csv")
tst <- read.csv(resource)
tst
#> Title Description BiocVersion Genome SourceType
#> 1 ENCODE a test entry 4.1 NA JSON
#> SourceUrl SourceVersion Species TaxonomyId
#> 1 http://www.encodeproject.org x.y.z NA 9606
#> Coordinate_1_based DataProvider Maintainer RDataClass
#> 1 NA ENCODE Project tst person <tst@email.com> Rda
#> DispatchClass Location_Prefix RDataPath
#> 1 Rda s3://experimenthub/ ENCODExplorerData/encode_df_lite.rda
#> Tags
#> 1 ENCODE:Homo sapiensThe final function in HubPub helps the developer with publishing data resources
to an Bioconductor AWS S3. The function utilizes functions for the aws.s3
package to place files or directories on S3. The developer should have already
contacted the Bioconductor hubs maintainers to get the necessary credentials to
access the bucket. Once the credentials are received the developer should
declare them in the system environment before running this function. The
function requires a path to the file or name of the directory to be added to the
bucket and a name for how the object should be named on the bucket. If adding a
directory be sure there are no nested directories and only files.
The below code chunk demonstrates the use of the function using a dummy dataset. It will only work if the necessary global environments have been declared with the hub credentials.
## For publishing directories with multiple files
fl <- tempdir()
utils::write.csv(mtcars, file = file.path(fl, "mtcars1.csv"))
utils::write.csv(mtcars, file = file.path(fl, "mtcars2.csv"))
publish_resource(fl, "test_dir")
#> Warning in publish_resource(fl, "test_dir"): Not all system environment
#> variables are set, do so and rerun function.
#> copy '/tmp/Rtmp8rQxmQ/BiocStyle' to 's3://annotation-contributor/test_dir/BiocStyle'
#> copy '/tmp/Rtmp8rQxmQ/examplePkg' to 's3://annotation-contributor/test_dir/examplePkg'
#> copy '/tmp/Rtmp8rQxmQ/mtcars1.csv' to 's3://annotation-contributor/test_dir/mtcars1.csv'
#> copy '/tmp/Rtmp8rQxmQ/mtcars2.csv' to 's3://annotation-contributor/test_dir/mtcars2.csv'
#> copy '/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fbioc%2Fsrc%2Fcontrib.rds' to 's3://annotation-contributor/test_dir/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fbioc%2Fsrc%2Fcontrib.rds'
#> copy '/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fbooks%2Fsrc%2Fcontrib.rds' to 's3://annotation-contributor/test_dir/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fbooks%2Fsrc%2Fcontrib.rds'
#> copy '/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fdata%2Fannotation%2Fsrc%2Fcontrib.rds' to 's3://annotation-contributor/test_dir/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fdata%2Fannotation%2Fsrc%2Fcontrib.rds'
#> copy '/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fdata%2Fexperiment%2Fsrc%2Fcontrib.rds' to 's3://annotation-contributor/test_dir/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fdata%2Fexperiment%2Fsrc%2Fcontrib.rds'
#> copy '/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fworkflows%2Fsrc%2Fcontrib.rds' to 's3://annotation-contributor/test_dir/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fworkflows%2Fsrc%2Fcontrib.rds'
#> copy '/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fcloud.r-project.org%2Fsrc%2Fcontrib.rds' to 's3://annotation-contributor/test_dir/repos_https%3A%2F%2Fcloud.r-project.org%2Fsrc%2Fcontrib.rds'
#> $`/tmp/Rtmp8rQxmQ/BiocStyle`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/examplePkg`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/mtcars1.csv`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/mtcars2.csv`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fbioc%2Fsrc%2Fcontrib.rds`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fbooks%2Fsrc%2Fcontrib.rds`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fdata%2Fannotation%2Fsrc%2Fcontrib.rds`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fdata%2Fexperiment%2Fsrc%2Fcontrib.rds`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fbioconductor.org%2Fpackages%2F3.17%2Fworkflows%2Fsrc%2Fcontrib.rds`
#> NULL
#>
#> $`/tmp/Rtmp8rQxmQ/repos_https%3A%2F%2Fcloud.r-project.org%2Fsrc%2Fcontrib.rds`
#> NULL
## For publishing a single file
utils::write.csv(mtcars, file = file.path(fl, "mtcars3.csv"))
publish_resource(file.path(fl, "mtcars3.csv"), "test_dir")
#> Warning in publish_resource(file.path(fl, "mtcars3.csv"), "test_dir"): Not all
#> system environment variables are set, do so and rerun function.
#> copy '/tmp/Rtmp8rQxmQ/mtcars3.csv' to 's3://annotation-contributor/test_dir/mtcars3.csv'
#> $`/tmp/Rtmp8rQxmQ/mtcars3.csv`
#> NULLsessionInfo()
#> R version 4.3.1 (2023-06-16)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Ubuntu 22.04.2 LTS
#>
#> Matrix products: default
#> BLAS: /home/biocbuild/bbs-3.17-bioc/R/lib/libRblas.so
#> LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0
#>
#> locale:
#> [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
#> [3] LC_TIME=en_GB LC_COLLATE=C
#> [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
#> [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
#> [9] LC_ADDRESS=C LC_TELEPHONE=C
#> [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
#>
#> time zone: America/New_York
#> tzcode source: system (glibc)
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] futile.logger_1.4.3 HubPub_1.8.1 BiocStyle_2.28.0
#>
#> loaded via a namespace (and not attached):
#> [1] sys_3.4.2 rstudioapi_0.15.0
#> [3] jsonlite_1.8.7 magrittr_2.0.3
#> [5] GenomicFeatures_1.52.1 rmarkdown_2.23
#> [7] fs_1.6.3 BiocIO_1.10.0
#> [9] zlibbioc_1.46.0 vctrs_0.6.3
#> [11] memoise_2.0.1 Rsamtools_2.16.0
#> [13] RCurl_1.98-1.12 askpass_1.1
#> [15] base64enc_0.1-3 htmltools_0.5.5
#> [17] S4Arrays_1.0.5 usethis_2.2.2
#> [19] progress_1.2.2 lambda.r_1.2.4
#> [21] AnnotationHub_3.8.0 curl_5.0.1
#> [23] sass_0.4.7 bslib_0.5.0
#> [25] desc_1.4.2 testthat_3.1.10
#> [27] futile.options_1.0.1 cachem_1.0.8
#> [29] available_1.1.0 GenomicAlignments_1.36.0
#> [31] whisker_0.4.1 mime_0.12
#> [33] lifecycle_1.0.3 pkgconfig_2.0.3
#> [35] Matrix_1.6-0 R6_2.5.1
#> [37] fastmap_1.1.1 BiocCheck_1.36.1
#> [39] GenomeInfoDbData_1.2.10 MatrixGenerics_1.12.3
#> [41] shiny_1.7.4.1 digest_0.6.33
#> [43] AnnotationDbi_1.62.2 S4Vectors_0.38.1
#> [45] OrganismDbi_1.42.0 rprojroot_2.0.3
#> [47] ExperimentHub_2.8.1 aws.signature_0.6.0
#> [49] GenomicRanges_1.52.0 RSQLite_2.3.1
#> [51] filelock_1.0.2 fansi_1.0.4
#> [53] httr_1.4.6 abind_1.4-5
#> [55] compiler_4.3.1 bit64_4.0.5
#> [57] withr_2.5.0 biocViews_1.68.1
#> [59] BiocParallel_1.34.2 DBI_1.1.3
#> [61] R.utils_2.12.2 biomaRt_2.56.1
#> [63] openssl_2.1.0 rappdirs_0.3.3
#> [65] DelayedArray_0.26.7 rjson_0.2.21
#> [67] tools_4.3.1 interactiveDisplayBase_1.38.0
#> [69] httpuv_1.6.11 R.oo_1.25.0
#> [71] glue_1.6.2 restfulr_0.0.15
#> [73] R.cache_0.16.0 promises_1.2.1
#> [75] grid_4.3.1 stringdist_0.9.10
#> [77] generics_0.1.3 R.methodsS3_1.8.2
#> [79] hms_1.1.3 xml2_1.3.5
#> [81] utf8_1.2.3 XVector_0.40.0
#> [83] BiocGenerics_0.46.0 BiocVersion_3.17.1
#> [85] pillar_1.9.0 stringr_1.5.0
#> [87] later_1.3.1 dplyr_1.1.2
#> [89] BiocFileCache_2.8.0 lattice_0.21-8
#> [91] AnnotationHubData_1.30.0 rtracklayer_1.60.0
#> [93] bit_4.0.5 RBGL_1.76.0
#> [95] tidyselect_1.2.0 Biostrings_2.68.1
#> [97] knitr_1.43 biocthis_1.10.3
#> [99] bookdown_0.35 IRanges_2.34.1
#> [101] SummarizedExperiment_1.30.2 stats4_4.3.1
#> [103] xfun_0.40 Biobase_2.60.0
#> [105] credentials_1.3.2 brio_1.1.3
#> [107] matrixStats_1.0.0 stringi_1.7.12
#> [109] yaml_2.3.7 evaluate_0.21
#> [111] codetools_0.2-19 tibble_3.2.1
#> [113] BiocManager_1.30.22 graph_1.78.0
#> [115] cli_3.6.1 xtable_1.8-4
#> [117] jquerylib_0.1.4 styler_1.10.1
#> [119] roxygen2_7.2.3 Rcpp_1.0.11
#> [121] GenomeInfoDb_1.36.1 gert_1.9.3
#> [123] dbplyr_2.3.3 png_0.1-8
#> [125] RUnit_0.4.32 XML_3.99-0.14
#> [127] parallel_4.3.1 ellipsis_0.3.2
#> [129] blob_1.2.4 prettyunits_1.1.1
#> [131] aws.s3_0.3.21 AnnotationForge_1.42.2
#> [133] bitops_1.0-7 ExperimentHubData_1.26.0
#> [135] purrr_1.0.2 crayon_1.5.2
#> [137] rlang_1.1.1 formatR_1.14
#> [139] KEGGREST_1.40.0