R Packages available for CRMDA cluster members

This is the 20170425 update, which includes an updated module set and reports of success with Java and TkTcl-based R packages. In other words, an almost complete victory is achieved. Special thanks to Wes Mason of ITTC.

To use R, here is a set of commands I run to set the environment. This is necessary every time I want to use R with Emacs. Let's call this the magic 5 line stanza, for sake of discussion.

module purge
module load legacy
module load emacs  
module use /panfs/pfs.local/work/crmda/tools/modules
module load Rstats/3.3

I agree if you say "it is a pain in the rump to have to remember to do that every time I log in." In the old cluster, I was in a position to place those startup commands into all of the CRMDA user environments. That is no longer the case.

I'm checking on ways you can automate this within your own account. Details are posted at the end of this article.

Consider (strongly) obliterating your $HOME/R package folder

When you want to work with R on the CRC cluster, please consider using the R packages we install within the $WORK folder for CRMDA group members. These packages have some special features and if you try to install them in your user folder (under $HOME/R, as R invites you to do if you run "install.packages()" in a session), then they may not compile correctly.

Recently, we have had runtime errors because the R we are recommending, as described below, is not compatible with packages that users build and install with other versions of R (or the same version of R in a different build environment). In particular, if

  1. You have packages built on the old ACF cluster, or
  2. You have recently installed packages without loading the modules listed below

then you should delete the packages you have under $HOME/R. I think it is best if you let us try to install what you need, but if you install R packages in your own home folder, please do so only AFTER loading the modules listed below. Please DO NOT load the CRC-provided module "R/3.3". It does not provide the services we need.

Background information

The module Rstats/3.3 is built by Wes Mason of ITTC and it is installed into the $WORK folder for CRMDA (hence the module use command above). We work together to make sure the OpenMPI layer is compiled correctly, so it is possible to use Rmpi and the R package parallel. The compiler used is GCC-6.3, which is quite a bit newer than the standard GCC which is provided with the cluster node operating system. This is the principal reason why the CRC-provided "R/3.3" is not acceptable. It does not make sure that the OpenMPI and GCC components are kept in lock-step with R itself. Observe, if we start with an empty session and run

module purge
module use /panfs/pfs.local/work/crmda/tools/modules
module load Rstats/3.3

we find that we actually load several modules:

$ module list

Currently Loaded Modules:
1) compiler/gcc/6.3   
2) openmpi/2.0
3) java/1.8.0_131
4) xz/5.2.3
5) icu/59.1
6) tcltk/8.6.6
7) Rstats/3.3

The openmpi version must be kept in lock-step with R and the packages we have installed in the past. gcc-6.3 is the compiler version we use for all of the packages. It is necessary to have that new version because of demands by packages like Rstan and OpenMX. The java and tcltk modules are needed by various R packages, such as rJava and tkrplot. The xz module is a decompression suite, needed to interact with source code itself. The Rstats module itself is, for the most part, a "holding company" that keeps all of this together. It simply loads the requirements of gcc, openmpi, java, xz, icu, and tcltk, and then it accesses the R provided by the CRC system maintainers from /panfs/pfs.local/software/install/MRO/3.3. The R packages provided by the base R install are found in the directory /panfs/pfs.local/work/crmda/tools/mro/3.3/site-library.

The R packages in our collection are, in most cases, going to be updates & replacements of those packages because we are building with the different compiler. Users who load Rstats/3.3 should notice that our directory comes into the user path before the system-wide folder. Inside R, we see:

> .libPaths()
[1] "/panfs/pfs.local/work/crmda/tools/mro/3.3/site-library"
[2] "/panfs/pfs.local/software/install/MRO/3.3/microsoft-r/3.3/lib64/R/library"

In my basic 5 line session starter sequence, I also have modules named legacy and emacs. In my opinion, that is a little dangerous because I'm jumbling together modules from the old and new cluster. That's necessary because I need an IDE with which to interact with R. Emacs was configured with ESS by Wes Mason, and it is in the legacy module set. If you prefer, legacy also provides RStudio version 9.98.978. That is, unfortunately, outdated and unmaintained. I've filed a request with CRC to get a new version of Rstudio.

What do you get if you load our environment?

After the magic 5 line stanza above, within your R session, you have access to these packages (Run "library()" to see all folders your path, and all packages within):

Packages in library '/panfs/pfs.local/work/crmda/tools/mro/3.3/site-library':

ADGofTest               Anderson-Darling GoF test
AER                     Applied Econometrics with R
Amelia                  A Program for Missing Data
BH                      Boost C++ Header Files
BMA                     Bayesian Model Averaging
BradleyTerry2           Bradley-Terry Models
Cairo                   R graphics device using cairo graphics library
                        for creating high-quality bitmap (PNG, JPEG,
                        TIFF), vector (PDF, SVG, PostScript) and
                        display (X11 and Win32) output
Cubist                  Rule- And Instance-Based Regression Modeling
DBI                     R Database Interface
DCluster                Functions for the Detection of Spatial Clusters
                        of Diseases
DEoptimR                Differential Evolution Optimization in Pure R
Devore7                 Data sets from Devore's "Prob and Stat for Eng
                        (7th ed)"
DiagrammeR              Create Graph Diagrams and Flowcharts Using R
ENmisc                  Neuwirth miscellaneous
Ecdat                   Data Sets for Econometrics
Ecfun                   Functions for Ecdat
Formula                 Extended Model Formulas
GPArotation             GPA Factor Rotation
HistData                Data Sets from the History of Statistics and
                        Data Visualization
Hmisc                   Harrell Miscellaneous
HyperbolicDist          The hyperbolic distribution
ISwR                    Introductory Statistics with R
Iso                     Functions to Perform Isotonic Regression
JGR                     JGR - Java GUI for R
JM                      Joint Modeling of Longitudinal and Survival
JMdesign                Joint Modeling of Longitudinal and Survival
                        Data - Power Calculation
JavaGD                  Java Graphics Device
Kendall                 Kendall rank correlation and Mann-Kendall trend
LearnBayes              Functions for Learning Bayesian Inference
MCMCglmm                MCMC Generalised Linear Mixed Models
MCMCpack                Markov Chain Monte Carlo (MCMC) Package
MCPAN                   Multiple Comparisons Using Normal Approximation
MEMSS                   Data sets from Mixed-effects Models in S
MNP                     R Package for Fitting the Multinomial Probit
MPV                     Data Sets from Montgomery, Peck and Vining's
MatchIt                 Nonparametric Preprocessing for Parametric
                        Casual Inference
Matching                Multivariate and Propensity Score Matching with
                        Balance Optimization
MatrixModels            Modelling with Sparse And Dense Matrices
ModelMetrics            Rapid Calculation of Model Metrics
MplusAutomation         Automating Mplus Model Estimation and
NMF                     Algorithms and Framework for Nonnegative Matrix
                        Factorization (NMF)
OpenMx                  Extended Structural Equation Modelling
PBSmapping              Mapping Fisheries Data and Spatial Analysis
PolynomF                Polynomials in R
R2HTML                  HTML Exportation for R Objects
R2OpenBUGS              Running OpenBUGS from R
R6                      Classes with Reference Semantics
RColorBrewer            ColorBrewer Palettes
RCurl                   General Network (HTTP/FTP/...) Client Interface
                        for R
RGtk2                   R bindings for Gtk 2.8.0 and above
RSvgDevice              An R SVG graphics device.
RandomFields            Simulation and Analysis of Random Fields
RandomFieldsUtils       Utilities for the Simulation and Analysis of
                        Random Fields
Rcmdr                   R Commander
RcmdrMisc               R Commander Miscellaneous Functions
Rcpp                    Seamless R and C++ Integration
RcppArmadillo           'Rcpp' Integration for the 'Armadillo'
                        Templated Linear Algebra Library
RcppEigen               'Rcpp' Integration for the 'Eigen' Templated
                        Linear Algebra Library
Rd2roxygen              Convert Rd to 'Roxygen' Documentation
Rmpi                    Interface (Wrapper) to MPI (Message-Passing
Rook                    Rook - a web server interface for R
SAScii                  Import ASCII files directly into R using only a
                        SAS input script
SASmixed                Data sets from "SAS System for Mixed Models"
SemiPar                 Semiparametic Regression
SoDA                    Functions and Examples for "Software for Data
SparseM                 Sparse Linear Algebra
StanHeaders             C++ Header Files for Stan
StatDataML              Read and Write StatDataML Files
SweaveListingUtils      Utilities for Sweave Together with TeX
                        'listings' Package
TH.data                 TH's Data Archive
TeachingDemos           Demonstrations for Teaching and Learning
UsingR                  Data Sets, Etc. for the Text "Using R for
                        Introductory Statistics", Second Edition
VGAM                    Vector Generalized Linear and Additive Models
VIM                     Visualization and Imputation of Missing Values
XML                     Tools for Parsing and Generating XML Within R
                        and S-Plus
Zelig                   Everyone's Statistical Software
abind                   Combine Multidimensional Arrays
acepack                 ACE and AVAS for Selecting Multiple Regression
actuar                  Actuarial Functions and Heavy Tailed
ada                     The R Package Ada for Stochastic Boosting
ade4                    Analysis of Ecological Data : Exploratory and
                        Euclidean Methods in Environmental Sciences
adehabitat              Analysis of Habitat Selection by Animals
akima                   Interpolation of Irregularly and Regularly
                        Spaced Data
alr3                    Data to accompany Applied Linear Regression 3rd
amap                    Another Multidimensional Analysis Package
aod                     Analysis of Overdispersed Data
ape                     Analyses of Phylogenetics and Evolution
aplpack                 Another Plot PACKage: stem.leaf, bagplot,
                        faces, spin3R, plotsummary, plothulls, and some
                        slider functions
arm                     Data Analysis Using Regression and
                        Multilevel/Hierarchical Models
arules                  Mining Association Rules and Frequent Itemsets
assertthat              Easy Pre and Post Assertions
backports               Reimplementations of Functions Introduced Since
base64enc               Tools for base64 encoding
bayesm                  Bayesian Inference for
bcp                     Bayesian Analysis of Change Point Problems
bdsmatrix               Routines for Block Diagonal Symmetric matrices
bestglm                 Best Subset GLM
betareg                 Beta Regression
biglm                   bounded memory linear and generalized linear
bit                     A class for vectors of 1-bit booleans
bit64                   A S3 Class for Vectors of 64bit Integers
bitops                  Bitwise Operations
bnlearn                 Bayesian Network Structure Learning, Parameter
                        Learning and Inference
brew                    Templating Framework for Report Generation
brglm                   Bias reduction in binomial-response generalized
                        linear models.
broom                   Convert Statistical Analysis Objects into Tidy
                        Data Frames
caTools                 Tools: moving window statistics, GIF, Base64,
                        ROC AUC, etc.
cairoDevice             Embeddable Cairo Graphics Device Driver
car                     Companion to Applied Regression
caret                   Classification and Regression Training
cellranger              Translate Spreadsheet Cell Ranges to Rows and
censReg                 Censored Regression (Tobit) Models
checkmate               Fast and Versatile Argument Checks
chron                   Chronological Objects which can Handle Dates
                        and Times
clue                    Cluster Ensembles
clv                     Cluster Validation Techniques
cocorresp               Co-Correspondence Analysis Methods
coda                    Output Analysis and Diagnostics for MCMC
coin                    Conditional Inference Procedures in a
                        Permutation Test Framework
colorspace              Color Space Manipulation
combinat                combinatorics utilities
commonmark              High Performance CommonMark and Github Markdown
                        Rendering in R
copula                  Multivariate Dependence with Copulas
corpcor                 Efficient Estimation of Covariance and
                        (Partial) Correlation
crayon                  Colored Terminal Output
cslogistic              Conditionally Specified Logistic Regression
cubature                Adaptive Multivariate Integration over
data.table              Extension of `data.frame`
deldir                  Delaunay Triangulation and Dirichlet (Voronoi)
desc                    Manipulate DESCRIPTION Files
descr                   Descriptive Statistics
dichromat               Color Schemes for Dichromats
digest                  Create Compact Hash Digests of R Objects
diptest                 Hartigan's Dip Test Statistic for Unimodality -
distr                   Object Oriented Implementation of Distributions
dlm                     Bayesian and Likelihood Analysis of Dynamic
                        Linear Models
doBy                    Groupwise Statistics, LSmeans, Linear
                        Contrasts, Utilities
doMC                    Foreach Parallel Adaptor for 'parallel'
doMPI                   Foreach parallel adaptor for the Rmpi package
doSNOW                  Foreach Parallel Adaptor for the 'snow' Package
dplyr                   A Grammar of Data Manipulation
dse                     Dynamic Systems Estimation (Time Series
e1071                   Misc Functions of the Department of Statistics,
                        Probability Theory Group (Formerly: E1071), TU
earth                   Multivariate Adaptive Regression Splines
ecodist                 Dissimilarity-based functions for ecological
effects                 Effect Displays for Linear, Generalized Linear,
                        and Other Models
eha                     Event History Analysis
eiPack                  eiPack: Ecological Inference and
                        Higher-Dimension Data Management
emplik                  Empirical Likelihood Ratio for
                        Censored/Truncated Data
evaluate                Parsing and Evaluation Tools that Provide More
                        Details than the Default
expint                  Exponential Integral and Incomplete Gamma
expm                    Matrix Exponential, Log, 'etc'
faraway                 Functions and Datasets for Books by Julian
fastICA                 FastICA Algorithms to perform ICA and
                        Projection Pursuit
fastmatch               Fast match() function
fda                     Functional Data Analysis
ffmanova                Fifty-fifty MANOVA
fields                  Tools for Spatial Data
flexmix                 Flexible Mixture Modeling
forcats                 Tools for Working with Categorical Variables
formatR                 Format R Code Automatically
forward                 Forward search
gam                     Generalized Additive Models
gamlss                  Generalised Additive Models for Location Scale
                        and Shape
gamlss.data             GAMLSS Data
gamlss.dist             Distributions to be Used for GAMLSS Modelling
gamm4                   Generalized Additive Mixed Models using 'mgcv'
                        and 'lme4'
gbm                     Generalized Boosted Regression Models
gclus                   Clustering Graphics
gdata                   Various R Programming Tools for Data
gee                     Generalized Estimation Equation Solver
geepack                 Generalized Estimating Equation Package
geoR                    Analysis of Geostatistical Data
geoRglm                 A Package for Generalised Linear Spatial Models
ggm                     Functions for graphical Markov models
ggplot2                 Create Elegant Data Visualisations Using the
                        Grammar of Graphics
glmc                    Fitting Generalized Linear Models Subject to
glmmBUGS                Generalised Linear Mixed Models with BUGS and
glmmML                  Generalized Linear Models with Clustering
glmnet                  Lasso and Elastic-Net Regularized Generalized
                        Linear Models
glmpath                 L1 Regularization Path for Generalized Linear
                        Models and Cox Proportional Hazards Model
gmodels                 Various R Programming Tools for Model Fitting
gmp                     Multiple Precision Arithmetic
gpclib                  General Polygon Clipping Library for R
gridBase                Integration of base and grid graphics
gridExtra               Miscellaneous Functions for "Grid" Graphics
grpreg                  Regularization Paths for Regression Models with
                        Grouped Covariates
gsl                     Wrapper for the Gnu Scientific Library
gsubfn                  Utilities for strings and function arguments.
gtable                  Arrange 'Grobs' in Tables
gtools                  Various R Programming Tools
haven                   Import and Export 'SPSS', 'Stata' and 'SAS'
hexbin                  Hexagonal Binning Routines
highr                   Syntax Highlighting for R Source Code
hms                     Pretty Time of Day
htmlTable               Advanced Tables for Markdown/HTML
htmltools               Tools for HTML
htmlwidgets             HTML Widgets for R
httpuv                  HTTP and WebSocket Server Library
httr                    Tools for Working with URLs and HTTP
igraph                  Network Analysis and Visualization
ineq                    Measuring Inequality, Concentration, and
influence.ME            Tools for Detecting Influential Data in Mixed
                        Effects Models
influenceR              Software Tools to Quantify Structural
                        Importance of Nodes in a Network
inline                  Functions to Inline C, C++, Fortran Function
                        Calls from R
iplots                  iPlots - interactive graphics for R
irlba                   Fast Truncated SVD, PCA and Symmetric
                        Eigendecomposition for Large Dense and Sparse
itertools               Iterator Tools
jpeg                    Read and write JPEG images
kernlab                 Kernel-Based Machine Learning Lab
knitr                   A General-Purpose Package for Dynamic Report
                        Generation in R
kutils                  Project Management Tools
labeling                Axis Labeling
laeken                  Estimation of indicators on social exclusion
                        and poverty
languageR               Data sets and functions with "Analyzing
                        Linguistic Data: A practical introduction to
lars                    Least Angle Regression, Lasso and Forward
latticeExtra            Extra Graphical Utilities Based on Lattice
lava                    Latent Variable Models
lavaan                  Latent Variable Analysis
lavaan.survey           Complex Survey Structural Equation Modeling
lazyeval                Lazy (Non-Standard) Evaluation
leaps                   Regression Subset Selection
lme4                    Linear Mixed-Effects Models using 'Eigen' and
lmeSplines              Add smoothing spline modelling capability to
lmec                    Linear Mixed-Effects Models with Censored
lmerTest                Tests in Linear Mixed Effects Models
lmm                     Linear Mixed Models
lmtest                  Testing Linear Regression Models
locfit                  Local Regression, Likelihood and Density
logspline               Logspline Density Estimation Routines
longitudinal            Analysis of Multiple Time Course Data
longitudinalData        Longitudinal Data
lpSolve                 Interface to 'Lp_solve' v. 5.5 to Solve
                        Linear/Integer Programs
ltm                     Latent Trait Models under IRT
lubridate               Make Dealing with Dates a Little Easier
magic                   create and investigate magic squares
magrittr                A Forward-Pipe Operator for R
manipulate              Interactive Plots for RStudio
maps                    Draw Geographical Maps
maptools                Tools for Reading and Handling Spatial Objects
markdown                'Markdown' Rendering for R
matrixcalc              Collection of functions for matrix calculations
maxLik                  Maximum Likelihood Estimation and Related Tools
mboost                  Model-Based Boosting
mcgibbsit               Warnes and Raftery's MCGibbsit MCMC diagnostic
mclust                  Gaussian Mixture Modelling for Model-Based
                        Clustering, Classification, and Density
mcmc                    Markov Chain Monte Carlo
mda                     Mixture and Flexible Discriminant Analysis
mediation               Causal Mediation Analysis
memisc                  Tools for Management of Survey Data and the
                        Presentation of Analysis Results
memoise                 Memoisation of Functions
mi                      Missing Data Imputation and Model Checking
micEcon                 Microeconomic Analysis and Modelling
mice                    Multivariate Imputation by Chained Equations
microbenchmark          Accurate Timing Functions
mime                    Map Filenames to MIME Types
minqa                   Derivative-free optimization algorithms by
                        quadratic approximation
misc3d                  Miscellaneous 3D Plots
miscTools               Miscellaneous Tools and Utilities
mitools                 Tools for multiple imputation of missing data
mix                     Estimation/Multiple Imputation for Mixed
                        Categorical and Continuous Data
mixtools                Tools for Analyzing Finite Mixture Models
mlbench                 Machine Learning Benchmark Problems
mnormt                  The Multivariate Normal and t Distributions
modelr                  Modelling Functions that Work with the Pipe
modeltools              Tools and Classes for Statistical Models
msm                     Multi-State Markov and Hidden Markov Models in
                        Continuous Time
multcomp                Simultaneous Inference in General Parametric
munsell                 Utilities for Using Munsell Colours
mvProbit                Multivariate Probit Models
mvbutils                Workspace organization, code and documentation
                        editing, package prep and editing, etc.
mvtnorm                 Multivariate Normal and t Distributions
neighbr                 Classification, Regression, Clustering with K
                        Nearest Neighbors
network                 Classes for Relational Data
nloptr                  R interface to NLopt
nnls                    The Lawson-Hanson algorithm for non-negative
                        least squares (NNLS)
nor1mix                 Normal (1-d) Mixture Models (S3 Classes and
norm                    Analysis of multivariate normal datasets with
                        missing values
nortest                 Tests for Normality
np                      Nonparametric kernel smoothing methods for
                        mixed data types
numDeriv                Accurate Numerical Derivatives
nws                     R functions for NetWorkSpaces and Sleigh
openssl                 Toolkit for Encryption, Signatures and
                        Certificates Based on OpenSSL
openxlsx                Read, Write and Edit XLSX Files
ordinal                 Regression Models for Ordinal Data
orthopolynom            Collection of functions for orthogonal and
                        orthonormal polynomials
pan                     Multiple Imputation for Multivariate Panel or
                        Clustered Data
pander                  An R Pandoc Writer
partDSA                 Partitioning Using Deletion, Substitution, and
                        Addition Moves
party                   A Laboratory for Recursive Partytioning
pbivnorm                Vectorized Bivariate Normal CDF
pbkrtest                Parametric Bootstrap and Kenward Roger Based
                        Methods for Mixed Model Comparison
pcaPP                   Robust PCA by Projection Pursuit
permute                 Functions for Generating Restricted
                        Permutations of Data
pixmap                  Bitmap Images (``Pixel Maps'')
pkgKitten               Create Simple Packages Which Do not Upset R
                        Package Checks
pkgmaker                Package development utilities
plm                     Linear Models for Panel Data
plotmo                  Plot a Model's Response and Residuals
plotrix                 Various Plotting Functions
pls                     Partial Least Squares and Principal Component
plyr                    Tools for Splitting, Applying and Combining
pmml                    Generate PMML for Various Models
pmmlTransformations     Transforms Input Data from a PMML Perspective
polspline               Polynomial Spline Routines
polycor                 Polychoric and Polyserial Correlations
polynom                 A Collection of Functions to Implement a Class
                        for Univariate Polynomial Manipulations
portableParallelSeeds   Allow Replication of Simulations on Parallel
                        and Serial Computers
ppcor                   Partial and Semi-Partial (Part) Correlation
praise                  Praise Users
profileModel            Tools for profiling inference functions for
                        various model classes
proto                   Prototype Object-Based Programming
proxy                   Distance and Similarity Measures
pscl                    Political Science Computational Laboratory,
                        Stanford University
psidR                   Build Panel Data Sets from PSID Raw Data
pspline                 Penalized Smoothing Splines
psych                   Procedures for Psychological, Psychometric, and
                        Personality Research
purrr                   Functional Programming Tools
quadprog                Functions to solve Quadratic Programming
quantreg                Quantile Regression
rJava                   Low-Level R to Java Interface
randomForest            Breiman and Cutler's Random Forests for
                        Classification and Regression
randomForestSRC         Random Forests for Survival, Regression and
                        Classification (RF-SRC)
rattle                  Graphical User Interface for Data Mining in R
rbenchmark              Benchmarking routine for R
rbugs                   Fusing R and OpenBugs and Beyond
rda                     Shrunken Centroids Regularized Discriminant
readr                   Read Rectangular Text Data
readxl                  Read Excel Files
registry                Infrastructure for R Package Registries
relimp                  Relative Contribution of Effects in a
                        Regression Model
rematch                 Match Regular Expressions with a Nicer 'API'
reshape                 Flexibly Reshape Data
reshape2                Flexibly Reshape Data: A Reboot of the Reshape
rgenoud                 R Version of GENetic Optimization Using
rgexf                   Build, Import and Export GEXF Graph Files
rgl                     3D Visualization Using OpenGL
rlecuyer                R Interface to RNG with Multiple Streams
rmarkdown               Dynamic Documents for R
rms                     Regression Modeling Strategies
rngtools                Utility functions for working with Random
                        Number Generators
robustbase              Basic Robust Statistics
rockchalk               Regression Estimation and Presentation
roxygen2                In-Line Documentation for R
rpart.plot              Plot 'rpart' Models: An Enhanced Version of
rpf                     Response Probability Functions
rprojroot               Finding Files in Project Subdirectories
rrcov                   Scalable Robust Estimators with High Breakdown
rstan                   R Interface to Stan
rstudio                 Tools and Utilities for RStudio
rstudioapi              Safely Access the RStudio API
rvest                   Easily Harvest (Scrape) Web Pages
sandwich                Robust Covariance Matrix Estimators
scales                  Scale Functions for Visualization
scatterplot3d           3D Scatter Plot
segmented               Regression Models with Breakpoints/Changepoints
selectr                 Translate CSS Selectors to XPath Expressions
sem                     Structural Equation Models
semTools                Useful Tools for Structural Equation Modeling
setRNG                  Set (Normal) Random Number Generator and Seed
sets                    Sets, Generalized Sets, Customizable Sets and
sfsmisc                 Utilities from "Seminar fuer Statistik" ETH
shapefiles              Read and Write ESRI Shapefiles
shiny                   Web Application Framework for R
simsem                  SIMulated Structural Equation Modeling
sm                      Smoothing methods for nonparametric regression
                        and density estimation
smoothSurv              Survival Regression with Smoothed Error
sna                     Tools for Social Network Analysis
snow                    Simple Network of Workstations
snowFT                  Fault Tolerant Simple Network of Workstations
sourcetools             Tools for Reading, Tokenizing and Parsing R
sp                      Classes and Methods for Spatial Data
spam                    SPArse Matrix
spatialCovariance       Computation of Spatial Covariance Matrices for
                        Data on Rectangles
spatialkernel           Nonparameteric estimation of spatial
                        segregation in a multivariate point process
spdep                   Spatial Dependence: Weighting Schemes,
                        Statistics and Models
splancs                 Spatial and Space-Time Point Pattern Analysis
stabledist              Stable Distribution Functions
stabs                   Stability Selection with Error Control
startupmsg              Utilities for Start-Up Messages
statmod                 Statistical Modeling
statnet.common          Common R Scripts and Utilities Used by the
                        Statnet Project Software
stepwise                Stepwise detection of recombination breakpoints
stringi                 Character String Processing Facilities
stringr                 Simple, Consistent Wrappers for Common String
strucchange             Testing, Monitoring, and Dating Structural
subselect               Selecting Variable Subsets
survey                  Analysis of Complex Survey Samples
survival                Survival Analysis
systemfit               Estimating Systems of Simultaneous Equations
tables                  Formula-Driven Table Generation
tcltk2                  Tcl/Tk Additions
tensorA                 Advanced tensors arithmetic with named indices
testthat                Unit Testing for R
texreg                  Conversion of R Regression Output to LaTeX or
                        HTML Tables
tfplot                  Time Frame User Utilities
tframe                  Time Frame Coding Kernel
tibble                  Simple Data Frames
tidyr                   Easily Tidy Data with 'spread()' and 'gather()'
tidyverse               Easily Install and Load 'Tidyverse' Packages
timeDate                Rmetrics - Chronological and Calendar Objects
tis                     Time Indexes and Time Indexed Series
tkrplot                 TK Rplot
tree                    Classification and Regression Trees
triangle                Provides the Standard Distribution Functions
                        for the Triangle Distribution
trimcluster             Cluster analysis with trimming
trust                   Trust Region Optimization
ucminf                  General-Purpose Unconstrained Non-Linear
urca                    Unit Root and Cointegration Tests for Time
                        Series Data
vcd                     Visualizing Categorical Data
vegan                   Community Ecology Package
viridis                 Default Color Maps from 'matplotlib'
viridisLite             Default Color Maps from 'matplotlib' (Lite
visNetwork              Network Visualization using 'vis.js' Library
waveslim                Basic wavelet routines for one-, two- and
                        three-dimensional signal processing
wnominate               Roll Call Analysis Software
xgboost                 Extreme Gradient Boosting
xml2                    Parse XML
xtable                  Export Tables to LaTeX or HTML
xts                     eXtensible Time Series
yaml                    Methods to Convert R Data to YAML and Back
zipfR                   Statistical models for word frequency
zoo                     S3 Infrastructure for Regular and Irregular
                        Time Series (Z's Ordered Observations)

Packages in library '/panfs/pfs.local/software/install/MRO/3.3/microsoft-r/3.3/lib64/R/library':

KernSmooth              Functions for Kernel Smoothing Supporting Wand
                        & Jones (1995)
MASS                    Support Functions and Datasets for Venables and
                        Ripley's MASS
Matrix                  Sparse and Dense Matrix Classes and Methods
MicrosoftR              Microsoft R umbrella package
R6                      Classes with Reference Semantics
RUnit                   R Unit test framework
RevoIOQ                 Microsoft R Services Test Suite
RevoMods                R Functions Modified For Revolution R
RevoUtils               Microsoft R Utility Package
RevoUtilsMath           Microsoft R Services Math Utilities Package
base                    The R Base Package
boot                    Bootstrap Functions (Originally by Angelo Canty
                        for S)
checkpoint              Install Packages from Snapshots on the
                        Checkpoint Server for Reproducibility
class                   Functions for Classification
cluster                 "Finding Groups in Data": Cluster Analysis
                        Extended Rousseeuw et al.
codetools               Code Analysis Tools for R
compiler                The R Compiler Package
curl                    A Modern and Flexible Web Client for R
datasets                The R Datasets Package
deployrRserve           Binary R server
doParallel              Foreach Parallel Adaptor for the 'parallel'
foreach                 Provides Foreach Looping Construct for R
foreign                 Read Data Stored by Minitab, S, SAS, SPSS,
                        Stata, Systat, Weka, dBase, ...
grDevices               The R Graphics Devices and Support for Colours
                        and Fonts
graphics                The R Graphics Package
grid                    The Grid Graphics Package
iterators               Provides Iterator Construct for R
jsonlite                A Robust, High Performance JSON Parser and
                        Generator for R
lattice                 Trellis Graphics for R
methods                 Formal Methods and Classes
mgcv                    Mixed GAM Computation Vehicle with GCV/AIC/REML
                        Smoothness Estimation
nlme                    Linear and Nonlinear Mixed Effects Models
nnet                    Feed-Forward Neural Networks and Multinomial
                        Log-Linear Models
parallel                Support for Parallel computation in R
png                     Read and write PNG images
rpart                   Recursive Partitioning and Regression Trees
spatial                 Functions for Kriging and Point Pattern
splines                 Regression Spline Functions and Classes
stats                   The R Stats Package
stats4                  Statistical Functions using S4 Classes
survival                Survival Analysis
tcltk                   Tcl/Tk Interface
tools                   Tools for Package Development
utils                   The R Utils Package

As usual, if these don't work right, its something I got wrong and will fix. Email me.

As of 2017-04-25, we have solved the problems of compiling Java and tk-based R packages. In other words, we find ourselves roughly back in the place where we were in October, 2016, or perhaps a little bit ahead of that. Now that the gcc issues have been addressed, we are able to stay up to date with changes in the cutting edge packages like Rcpp, Rstan and OpenMx.

If you need other packages, I'll install them if you email me .

If you launch R and you don't find packages (in the output of library(), for example), it probably means you forgot the module magic.

If you are having trouble with Rstan, the likely sources of trouble are 1) errors in your ~/.R/Makevars file, or 2) old packages in your home folder ~/R/ that do not cooperate with the new R and the other packages we make available.

Make a module script file

I have another more lesson. Instead of re-typing that stanza whenever it is needed, put those lines in a file. I just tested this. I put module stanza rstats.sh. I saved that in $HOME/bin and made it executable ("chmod +x rstats.sh"). It seems to succeed then to run

source rstats.sh

Building your own packages?

If we don't build packages for you, you have to build your own. Here is a lesson from the school of hard knocks. In the new CRC cluster, the memory limits of your sessions are strictly enforced. The compiler will often use more than 2GB memory. As a result, when you try to build a package inside R with "install.packages", you may get a vague message of failure. To protect yourself against that, it is wise to ask for an interactive session with more memory. I do this, for example:

$ msub -I -X -l nodes=1:ppn=1,pmem=6144m

That is sufficient to compile Rstan, which is the most intensive package I have tried to build.

This entry was posted in Data Analysis. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *