When searching modules through cache files resolve files and directories
marked as limited access. First check such entries match the search
query, then test if these files and directories can be accessed by
current user.
The access test relies on the findModulesFromDirsAndFiles procedure, as
if module search was not performed with cache file, but just for the
limited-access entries.
Split code from findModules procedure that walk through a list of files
and directories to find modules. Create a new procedure named
findModulesFromDirsAndFiles for that purpose.
This new procedure will be helpful to resolve limited access files and
directories spotted in cache file.
When building cache, check all files and directory within modulepath
that have limited access. Limited access is when user building cache or
others cannot read file or cannot either read or search directory.
Limited access file or directory are recorded in cache file with
respectively limited-access-file and limited-access-directory cachefile
commands.
Limited access commands replace modulefile/modulerc definitions in the
cache if files or their parent directory has limited access.
Add cache_expiry_secs configuration option to control after how many
seconds (after last modification) to consider cache file expired and
ignore it. Set to 0 by default which means cache file never expires.
Value can vary from 0 to 31536000 which corresponds to the number of
seconds during one year.
When configuration option is set, it defies the
MODULES_CACHE_EXPIRY_SECS environment variable.
Add cache_buffer_bytes configuration option to control the buffer size
used when reading or writing cache files. A 32 kbytes buffer is set by
default. It can vary from 4 kbytes to 1 MB. It is useful to have a large
buffer when reading or writing large files to reduce the number or read
or write system calls.
When configuration option is set, it defies the
MODULES_CACHE_BUFFER_BYTES environment variable.
Introduce readCacheContent procedure to specifically read cache file
rather using read procedure for modulefile. Avoid cache file to be
cached in memory. Produce more specific error message.
Do not check or evaluate cache file if ignore_cache option is enabled.
Use this new option in cachebuild sub-command to ignore cache when
building cache files.
Add ignore_cache configuration option that controls whether or not
module cache file should be considered if available. Disabled by
default. When configuration is modified it sets the MODULES_IGNORE_CACHE
environment variable.
--ignore-cache command line switch can be used to enable ignore_cache
for the current execution.
--ignore_cache is also supported for compatibility with Lmod.
Enable both mcookie_check and mcookie_version_check configuration
options when recording cache. These two options are not honored when
building or using cache.
As the improvement obtained by either of these optimization methods was
not perceptible, they are withdrawn to keep filtering code simpler.
Gain was also not perceptible since the additional cost taken by cache
was not perceptible in the first place.
As first try is not 100% accurate since it does not compare element root
names. This second attempt first compute the list of element root names
to compare, same lsearch -glob mechanism for comparison. Then unset
elements whose root name has been returned by comparison.
Increase filtering performance in findModulesInCacheFile: instead of
testing for match each module in cache through modEqStatic, gets all
module to exclude from result at once with lsearch command.
Update structure of result returned by findModulesInCacheFile to
indicate if cache load was successful or not. It is possible this way to
distinguish between a *there is no cache* result and a *no match in
cache* or *empty cache* results.
Add code in findModulesInCacheFile to filter fetched cache content to
tailor it to the search query. Proceed following what is done in
findModules procedure but in a reverse order (we remove elements from a
full structure, rather we add elements to an empty structure).
Avoid evaluating full message string to replace time pattern by
execution time value. Message could contain string that could be
misinterpreted by format command as time pattern.
Add modulerc-content cachefile command used to record a modulerc
available in modulepath. Record modulerc information in memory cache
variables: g_modfileContent.
Add modulefile-content cachefile command used to record a modulefile
available in modulepath. Record modulefile information in memory cache
variables: g_modfileContent and g_fileMtime.
Add core code for execute-cachefile procedure. Initialize an interpreter
to evaluate the cache file. Reuse the same mechanisms than for
modulefile or modulerc evaluation.
Add call to findModulesInCacheFile procedure in findModules to retrieve
available modulefile and modulerc information from modulepath cache
file.
findModulesInCacheFile tests a cache file is available and evaluate it
if not yet done with execute-cachefile procedure. Cache file information
will then be parsed to return the information matching the search query.
Initial code structure is inserted with this commit. Code for cache file
evaluation and cache information tailoring will come next.
Append major.minor version number to the #%Module header prefix of
generated cache files. Cache file can be used only if generated for the current
release of Modules used.
Change cache.tcl into cache.tcl.in to get Modules release number
resolved at build time.
Implement formatModuleCacheContent procedure. findModules procedure is
first called to get all files from designated modulepath directory. A
cache entry is made for each result obtained from findModules with
cache file commands: modulefile-content, modulerc-content and
modulefile-invalid.
Goal is to record all information produced by findModules to be able to
replace findModules processing by cache file evaluation. Thus modulefile
modification time and validity check issues are recorded in cache file.
Modulerc and modulefile content is read and recorded as-is in cache
file. Error occurring during file read is raised and ends cache file
creation.
Implement cachebuild sub-command. For each modulepath enabled or passed
as argument, if modulepath is writable for user build cache content for
this modulepath and record it in cache file.
Report message block is used to report file creation. Reports are output
in normal verbosity level or higher. A warning message is emitted if
modulepath is not writable for user, or if cache content to create is
empty.
Provide empty skeleton for formatModuleCacheContent procedure.
Implement cacheclear sub-command. For each modulepath enabled, check
module cache file exists. If yes and if modulepath is writable for user,
try to delete the cache file.
Report message block is used to report file deletion. Reports are output
in normal verbosity level or higher. A warning message is emitted if
modulepath is not writable for user.
Add tcl/cache.tcl source file to store procedure related to cache
mechanism. Update Makefile to properly generate modulecmd.tcl with this
new tcl file.
Introduce cacheclear sub-command to delete cache file under modulepaths.
This sub-command cannot be called from a modulefile evaluation context.
Add sub-command empty skeleton: cmdModuleCacheclear.
Sub-command accepts no option nor argument. Cache files are deleted in
every enabled modulepaths.
Introduce cachebuild sub-command to create cache file under modulepaths.
This sub-command cannot be called from a modulefile evaluation context.
Add sub-command empty skeleton: cmdModuleCachebuild.
Sub-command accepts no option. It may take optional arguments: list of
modulepath directory where to build cache. When no argument is
specified, cache is built for all enabled modulepaths.
Introduce hooks to define specific variables and commands for modulerc
interpreter context. It relies on variables defined in siteconfig.tcl
configuration file:
* modulerc_extra_vars is a list of variable name and value pairs
* modulerc_extra_cmds is a list of command name and procedure pairs
Fixes#286
Introduce hooks to define specific variables and commands for modulefile
interpreter context. It relies on variables defined in siteconfig.tcl
configuration file:
* modulefile_extra_vars is a list of variable name and value pairs
* modulefile_extra_cmds is a list of command name and procedure pairs
Fixes#286
Track loaded modules that are qualified for a refresh evaluation.
Refresh evaluation is only useful if module defines volatile environment
changes: shell completion, alias or function.
An internal state, named refresh_qualified, marks loading modules
defining such volatile environment changes. Such modules are then added
to the __MODULES_LMREFRESH environment variable to track this property
once module is loaded.
Code is also added to remove module from __MODULES_LMREFRESH when it is
unloaded.
This reverts commit 59ac24e243.
FPATH variable cannot be handled within autoinit process as this is not
an environment variable. Thus when modulecmd.tcl is executed, it does
not know the current value of FPATH. So if it sets FPATH during
autoinit, previous value of this variable is lost. As a result FPATH
must stay defined in initialization script.
Implement 'stashlist' sub-command. Rely on cmdModuleSavelist but adapt
this procedure to make several behavior changes when called by
cmdModuleStashlist:
* elements are sorted in reverse order
* element index starts at 0
* when no collection target set, only return collection without a target
set
Introduce stashlist sub-command to list stash collection file.
This sub-command cannot be called from a modulefile evaluation context.
Add sub-command empty skeleton: cmdModuleStashlist.
Sub-command accepts no argument. It accepts --terse, --long and --json
options.
Implement 'stashclear' sub-command. Fetch all existing stash collection
for current collection target. Then call 'saverm' sub-command over all
of these collection obtained.
Introduce stashclear sub-command to delete all stash collection file.
This sub-command cannot be called from a modulefile evaluation context.
Add sub-command empty skeleton: cmdModuleStashclear.
Sub-command accepts no argument nor option.
Introduce stashshow sub-command to display stash collection file.
This sub-command cannot be called from a modulefile evaluation context.
Add sub-command empty skeleton: cmdModuleStashshow.
Sub-command accepts no option. It may take an optional argument: the
index of the stash collection to display or the name of the stash
collection. If no argument is provided, stash collection 0 is assumed.
Introduce stashrm sub-command to delete stash collection file.
This sub-command cannot be called from a modulefile evaluation context.
Add sub-command empty skeleton: cmdModuleStashrm.
Sub-command accepts no option. It may take an optional argument: the
index of the stash collection to delete or the name of the stash
collection. If no argument is provided, stash collection 0 is assumed.
Implement 'stashpop' sub-command. First parse stash argument to
determine corresponding stash collection name or raise an error if stash
value is invalid. Then restore stash collection. Afterward delete stash
collection.
