| Title: | Tictoc-style memory usage tracking |
|---|---|
| Description: | Simple start/stop memory tracking functions 'tic_mem()' and 'toc_mem()' that can be nested, inspired by the 'tictoc' package. Track RAM usage during code execution with support for logging, custom messages, nested tracking blocks, and parallel worker monitoring. Features continuous background polling to capture true peak memory usage across main process and workers. Integrates with the 'future' package ecosystem for automatic worker detection. Designed for monitoring memory consumption in parallel workflows. |
| Authors: | Juan Ocampo [aut, cre] (ORCID: <https://orcid.org/0000-0002-9353-7581>) |
| Maintainer: | Juan Ocampo <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.1.0 |
| Built: | 2026-09-30 07:05:59 UTC |
| Source: | https://github.com/jcoa05/memtoc |
The memtoc package provides simple start/stop memory tracking functions that can be nested, inspired by the tictoc package for timing. Track RAM usage during code execution with support for logging, custom messages, background polling for true peak detection, and parallel worker monitoring.
tic_mem: Start memory tracking
toc_mem: Stop memory tracking and report
mem_log: Retrieve logged memory measurements
mem_clearlog: Clear the memory log
mem_clear: Clear the tracking stack
mem_recover: Recover data from crashed sessions
mem_capabilities: Check available features
mem_diagnose: Detailed diagnostics
mem_parallel_info: Check parallel backend status
tic_mem("data loading")
data <- read.csv("large_file.csv")
toc_mem()
#> data loading: 142.3 MB peak | 89.1 MB current | 2.34 sec | 3 samples
By default, memtoc spawns a background process to continuously sample memory, capturing the true peak even for short-lived allocations:
tic_mem("computation", interval = 0.5) # Sample every 0.5 seconds
x <- matrix(rnorm(1e8), ncol = 1000) # Large temporary allocation
result <- toc_mem()
result$trajectory # Full memory trajectory
When using the future package, memtoc can monitor memory across workers:
library(future)
plan(multisession, workers = 4)
tic_mem("parallel job", workers = "auto")
# ... parallel computation ...
result <- toc_mem()
result$worker_stats # Per-worker memory usage
tic_mem("full pipeline")
tic_mem("preprocessing")
# ... code ...
toc_mem()
tic_mem("modeling")
# ... code ...
toc_mem()
toc_mem()
Maintainer: Your Name <[email protected]>
Useful links:
Tests which features are available on the current system. This is useful for understanding why certain features might be disabled.
mem_capabilities()mem_capabilities()
A named logical vector with elements:
Can query process memory via ps package
Can spawn background processes via callr
mem_capabilities()mem_capabilities()
Removes all entries from the tracking stack. This is useful if an error
occurred before toc_mem() could be called, leaving orphaned entries
on the stack.
mem_clear()mem_clear()
Invisibly returns NULL.
tic_mem("will be cleared") mem_clear() # Stack is now emptytic_mem("will be cleared") mem_clear() # Stack is now empty
Removes all entries from the memtoc log. This does not affect the tracking stack (active tic_mem/toc_mem blocks).
mem_clearlog()mem_clearlog()
Invisibly returns NULL.
mem_log to retrieve the log before clearing
tic_mem("test") toc_mem(log = TRUE, quiet = TRUE) nrow(mem_log()) #> 1 mem_clearlog() nrow(mem_log()) #> 0tic_mem("test") toc_mem(log = TRUE, quiet = TRUE) nrow(mem_log()) #> 1 mem_clearlog() nrow(mem_log()) #> 0
Runs detailed diagnostics to identify issues with background process
spawning. Useful for troubleshooting when mem_capabilities() shows
background_polling = FALSE.
mem_diagnose()mem_diagnose()
This function tests each component of memtoc's functionality and provides detailed error messages and suggestions for fixing issues, particularly on Windows where Rtools may be required for background process spawning.
Invisibly returns a list with diagnostic results including:
Whether ps memory queries work
Whether process listing works
Whether system memory queries work
Whether callr package is installed
Whether background processes can be spawned
Whether all checks passed
mem_capabilities for a quick capability check
mem_diagnose()mem_diagnose()
Returns a data frame containing all logged memory tracking results.
Results are added to the log when toc_mem(log = TRUE) is called.
mem_log(format = c("data.frame", "list"))mem_log(format = c("data.frame", "list"))
format |
Character string specifying the output format.
|
A data frame (default) or list containing logged results. The data frame has columns:
Label from tic_mem(), or NA if none provided
Memory (RSS) in bytes at start
Memory (RSS) in bytes at end
Peak memory in bytes
Change in memory (bytes)
Elapsed time in seconds
When tic_mem() was called
When toc_mem() was called
mem_clearlog to clear the log, toc_mem with log = TRUE
mem_clearlog() tic_mem("step 1") Sys.sleep(0.1) toc_mem(log = TRUE, quiet = TRUE) tic_mem("step 2") Sys.sleep(0.1) toc_mem(log = TRUE, quiet = TRUE) mem_log()mem_clearlog() tic_mem("step 1") Sys.sleep(0.1) toc_mem(log = TRUE, quiet = TRUE) tic_mem("step 2") Sys.sleep(0.1) toc_mem(log = TRUE, quiet = TRUE) mem_log()
Returns diagnostic information about the detected parallel backend and any workers that can be monitored. Useful for verifying that memtoc can detect your parallel workers before running a job.
mem_parallel_info()mem_parallel_info()
Invisibly returns a list with parallel backend information including:
Whether the future package is installed
Whether the parallelly package is installed
Current future plan (e.g., "multisession", "sequential")
Number of configured workers
Integer vector of detected worker PIDs
Integer vector of child process PIDs
PID of the main R process
tic_mem for the workers parameter,
mem_capabilities for general capability check
mem_parallel_info() # With future workers active: # library(future) # plan(multisession, workers = 2) # mem_parallel_info()mem_parallel_info() # With future workers active: # library(future) # plan(multisession, workers = 2) # mem_parallel_info()
Displays the memory tracking log in a human-readable format, similar to how tictoc's tic.log() output can be printed with writeLines().
mem_print_log()mem_print_log()
Invisibly returns a character vector of formatted log lines.
mem_clearlog() tic_mem("step 1") Sys.sleep(0.1) toc_mem(log = TRUE, quiet = TRUE) tic_mem("step 2") Sys.sleep(0.2) toc_mem(log = TRUE, quiet = TRUE) mem_print_log()mem_clearlog() tic_mem("step 1") Sys.sleep(0.1) toc_mem(log = TRUE, quiet = TRUE) tic_mem("step 2") Sys.sleep(0.2) toc_mem(log = TRUE, quiet = TRUE) mem_print_log()
Attempts to recover memory samples from a previous session that crashed or was interrupted before toc_mem() was called. The monitor periodically saves checkpoints to disk, so some data may be recoverable.
mem_recover(pid = NULL, path = NULL)mem_recover(pid = NULL, path = NULL)
pid |
Process ID of the crashed R session. If NULL, lists all available recovery files. |
path |
Direct path to a recovery file. Overrides pid if provided. |
If pid or path is provided, returns a data frame of recovered samples or NULL if not found. If both are NULL, returns a list of available recovery files with their metadata.
tic_mem for starting memory tracking with persistence enabled
# List available recovery files mem_recover() # Recover from a specific PID # mem_recover(pid = 12345) # Recover from a specific file # mem_recover(path = "/path/to/memtoc_12345.rds")# List available recovery files mem_recover() # Recover from a specific PID # mem_recover(pid = 12345) # Recover from a specific file # mem_recover(path = "/path/to/memtoc_12345.rds")
Begins a memory tracking block. Call toc_mem to end the block and
see the results. Multiple calls to tic_mem() can be nested, and each
toc_mem() will match with the most recent unmatched tic_mem().
tic_mem(msg = NULL, quiet = TRUE, interval = 1, persist = TRUE, workers = "auto")tic_mem(msg = NULL, quiet = TRUE, interval = 1, persist = TRUE, workers = "auto")
msg |
Optional character string label for this tracking block.
This label is displayed in the output from |
quiet |
Logical. If |
interval |
Numeric. Polling interval in seconds for background
monitoring. Set to |
persist |
Logical. If |
workers |
Worker specification for parallel monitoring:
|
When interval is set (default), a background R process is spawned to
continuously sample memory usage. This captures the true peak memory
even for short-lived allocations. The background process adds minimal
overhead (typically less than 1% CPU).
For very short operations (under 1 second), consider using interval = NULL
to avoid the background process startup overhead (~200ms).
When using future for parallel processing, memtoc can automatically
detect and monitor worker processes. Set workers = "auto" to enable
this feature. The trajectory will include memory samples from all
workers, and the result will show aggregate statistics.
Invisibly returns the timestamp when tracking started.
toc_mem to stop tracking, mem_log to retrieve logged results,
mem_recover to recover data after a crash,
mem_parallel_info to check parallel backend status
# Basic usage with background polling (default) tic_mem("my operation") Sys.sleep(0.5) x <- numeric(1e6) toc_mem() # Snapshot-only mode (no background process) tic_mem("quick op", interval = NULL) y <- 1:100 toc_mem() # Custom polling interval tic_mem("long operation", interval = 0.5) Sys.sleep(2) toc_mem() # Monitor with parallel workers (if future is set up) # future::plan(future::multisession, workers = 2) # tic_mem("parallel job", workers = "auto") # result <- future.apply::future_lapply(1:10, function(x) rnorm(1e6)) # toc_mem() # Nested tracking tic_mem("outer") tic_mem("inner", interval = NULL) Sys.sleep(0.1) toc_mem() toc_mem()# Basic usage with background polling (default) tic_mem("my operation") Sys.sleep(0.5) x <- numeric(1e6) toc_mem() # Snapshot-only mode (no background process) tic_mem("quick op", interval = NULL) y <- 1:100 toc_mem() # Custom polling interval tic_mem("long operation", interval = 0.5) Sys.sleep(2) toc_mem() # Monitor with parallel workers (if future is set up) # future::plan(future::multisession, workers = 2) # tic_mem("parallel job", workers = "auto") # result <- future.apply::future_lapply(1:10, function(x) rnorm(1e6)) # toc_mem() # Nested tracking tic_mem("outer") tic_mem("inner", interval = NULL) Sys.sleep(0.1) toc_mem() toc_mem()
Ends a memory tracking block started by tic_mem and reports the results.
By default, prints a summary message showing peak memory, current memory,
and elapsed time.
toc_mem(log = FALSE, quiet = FALSE)toc_mem(log = FALSE, quiet = FALSE)
log |
Logical. If |
quiet |
Logical. If |
Invisibly returns a memtoc_result object (a list) containing:
The label passed to tic_mem(), or NULL
Memory (RSS) in bytes at start (main process)
Memory (RSS) in bytes at end (main process)
Peak memory during the interval (total across all processes)
Change in memory (end - start) in bytes
Elapsed time in seconds
POSIXct timestamp when tic_mem() was called
POSIXct timestamp when toc_mem() was called
Data frame of memory samples (if background polling was used)
Number of samples collected
Number of worker processes monitored (0 if main process only)
Data frame with per-worker peak memory (if workers monitored)
Peak system RAM usage percentage
Total system RAM in bytes
User CPU time consumed (seconds)
System CPU time consumed (seconds)
tic_mem to start tracking, mem_log to retrieve logged results
tic_mem("example") x <- rnorm(1e6) Sys.sleep(1) result <- toc_mem() # Access result components result$elapsed result$mem_peak result$n_samples # View the full trajectory (if available) if (!is.null(result$trajectory)) { head(result$trajectory) } # View per-worker stats (if workers were monitored) if (!is.null(result$worker_stats)) { result$worker_stats }tic_mem("example") x <- rnorm(1e6) Sys.sleep(1) result <- toc_mem() # Access result components result$elapsed result$mem_peak result$n_samples # View the full trajectory (if available) if (!is.null(result$trajectory)) { head(result$trajectory) } # View per-worker stats (if workers were monitored) if (!is.null(result$worker_stats)) { result$worker_stats }