Skip to content

About

High-performance hierarchical logging for Julia modules, with fine-grained runtime control and near-zero filtered-call overhead.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

96 Commits

Folders and files

Repository files navigation

ComponentLogging

Dev Build Status Benchmarks

ComponentLogging.jl is a lightweight, high-performance logging layer for Julia built around explicit component loggers and hierarchical log-level control.

Unlike Julia's standard Logging, which dynamically selects the active logger from the current task with a global fallback, ComponentLogging primarily associates shared logging configuration with a module or software component. This provides stable hierarchical control across tasks and threads while retaining explicit logger passing when execution-specific logging is needed.

Installation

julia>] add ComponentLogging

Quick start

using ComponentLogging

# Keys = groups; Values = minimum enabled level (integer or LogLevel)
rules = Dict(
    :core         => 0,      # Info+
    :io           => 1000,   # Warn+
    (:net, :http) => 2000,   # Error+
    :__default__  => 0       # fallback for unmatched groups (default to Info)
)

logger = ComponentLogger(rules; sink=PlainLogger())

@forward_logger logger

clog(:core, 0, "starting job"; jobid=42)   # 0 = Info
clog(:io, 1000, "retrying I/O"; attempt=3) # 1000 = Warn

To inspect the hierarchical rules inside logger, use display(logger).

Output:

ComponentLogger
  sink:  PlainLogger
  min:   0
  rules: 4
   ├─ :__default__     0
   ├─ :core            0
   ├─ :io              1000
   └─ :net
      └─ :http         2000

What the rules mean

  • Key: a group (Symbol or NTuple{N,Symbol}) such as :core or (:net,:http).
  • Value: the minimum level enabled for that group. Messages below this level are filtered out.
  • Define a catch-all like :__default__ => 0 to control unmatched groups.

Hierarchical runtime switches

The same group hierarchy can be used as a lightweight runtime control plane. With the forwarding helpers created by @forward_logger, set_log_level!(group => true) enables the group's default Info-level check, while set_log_level!(group => false) disables it.

function solve(problem)
    clogenabled((:solver, :presolve)) && run_presolve!(problem)

    if clogenabled((:solver, :heuristics))
        run_heuristics!(problem)
    end

    clogenabled((:solver, :cache)) && update_cache!(problem)
end

# Change behavior globally for this module/component.
set_log_level!(
    (:solver, :presolve) => true,
    (:solver, :heuristics) => false,
    (:solver, :cache) => true,
)
solve(problem)

This lets deeply nested behavior be switched at runtime without passing verbose, enable_*, or other configuration flags through every function in the call chain. The switches can control logging, diagnostics, optional computations, algorithmic branches, caching strategies, or any other behavior you choose to place behind clogenabled.

Without forwarding helpers, use the logger-explicit forms instead:

set_log_level!(logger, (:solver, :heuristics) => false)
clogenabled(logger, (:solver, :heuristics))
# Query the level after hierarchical fallback.
get_log_level(logger, (:solver, :heuristics))

Since v2.0.0, set_log_level! accepts only Pair arguments after the logger (or only pairs in its forwarded form); the old separate group and level arguments are no longer supported.

Core APIs

clog

Emits a log record for a group at a given level.

clog(logger, group, level, msg...; kwargs...)

Arguments:

  • logger::AbstractLogger — any logger instance. Typically you pass a ComponentLogger configured with per-group rules and a sink (e.g. PlainLogger). Many codebases also define forwarding helpers to avoid threading the logger explicitly (see below).
  • group::Union{Symbol,NTuple{N,Symbol}} — a Symbol or a tuple of symbols, e.g. :core or (:net, :http).
  • level::Union{Integer,LogLevel} — prefer integers (no need to import Logging). We immediately convert with LogLevel(level).
    • Mapping (integers first): -1000 (Debug), 0 (Info), 1000 (Warn), 2000 (Error).

Example:

clog(logger, :core,  0, "starting job"; jobid=42)   # 0 = Info
clog(logger, :io, 1000, "retrying I/O"; attempt=3)  # 1000 = Warn

clogenabled

Checks whether records at level are enabled for group.

Example:

if clogenabled(logger, :core, 1000)  # guard expensive work
    stats = compute_expensive_stats()
    clog(logger, :core, 1000, "stats ready"; stats)
end

@clog

What if generating the log message itself incurs a computational cost? @clog provides for the lazy construction of messages: it evaluates expressions only when the corresponding log level is enabled.

@clog logger group level msg...

Example:

arr = randn(1000)
sumsq = 0.0
for i in eachindex(arr)
    x = arr[i]
    sumsq += x^2
    @clog :core 1 begin
        meanval = mean(arr[1:i])
        stdval = std(arr[1:i])
        # The return value will be used as the log message
        "i=$i, x=$x, mean=$(meanval), std=$(stdval), sumsq=$(sumsq)"
    end
end

If the final message expression evaluates to nothing, no log record is emitted.

@forward_logger

@forward_logger logger

This creates module-local forwarding methods for clog, cdebug, cinfo, cwarn, cerror, clogenabled, set_log_level!, get_log_level, with_min_level, @clog, @cdebug, @cinfo, @cwarn, and @cerror macros. The logger expression is bound where @forward_logger is declared, including when the generated macro is imported by another module.

For example, the generated clog is conceptually equivalent to the following module-local forwarding method: clog(args...; kwargs...) = ComponentLogging.clog(logger, args...; kwargs...)

with_min_level

Temporarily sets the logger’s global minimum level and restores it on exit.

For example, to benchmark compute_sumsq() without any logging‑related work:

with_min_level(logger, 2000) do
    @benchmark compute_sumsq()
end

PlainLogger

PlainLogger is roughly a Base.CoreLogging.SimpleLogger without the [Info:-style prefixes. Its output looks like print, subject to its configured min_level filter. When wrapped by ComponentLogger, a record is enabled only when both the component rules and the sink's own filtering allow it.

PlainLogger and ComponentLogger are independent. You can also include("src/PlainLogger.jl") to use PlainLogger on its own.

Example:

using ComponentLogging, Logging

logger = PlainLogger()
with_logger(logger) do
    @info "Hello, Julia!"
end

Output:

Hello, Julia!
@ README.md:183

PlainLogger uses show with MIME"text/plain" to display 2D and 3D matrices, as it improves matrix readability. For other values, it prints them directly using print.

with_logger(logger) do
    @warn rand(1:9, 3, 3)
end

Output:

3×3 Matrix{Int64}:
 8  5  6
 3  4  9
 7  8  5
@ README.md:196

Benchmarking

ComponentLogging is continuously benchmarked over time. Current and historical performance results are available on the benchmark dashboard.

Similar Packages

Memento.jl is a flexible, hierarchical logging framework that brings its own ecosystem of loggers, handlers, formatters, records, and IO backends. Loggers are named (e.g., "Foo.bar"), form a hierarchy with propagation to a root logger, and are configured via config!, setlevel!, and by attaching handlers (file, custom formatters, etc.).

HierarchicalLogging.jl defines a Base.Logging-compatible HierarchicalLogger that associates loggers with hierarchically-related objects (e.g., module → submodule). Each node has a LogLevel that can be set with min_enabled_level!, which also recursively updates children; you can attach different underlying loggers (e.g., ConsoleLogger) to different parts of the tree.

ComponentLogging.jl takes a different approach: it is a lightweight, performance-oriented layer over Base.CoreLogging built around explicit component loggers and hierarchical group rules. It routes accepted records to any AbstractLogger sink, keeps filtered paths extremely cheap, and also exposes the same hierarchy as a lightweight runtime control plane through clogenabled and Boolean level switches.

  • Choose Memento.jl for a self-contained logging framework with hierarchical named loggers and a rich handler/formatter system.
  • Choose HierarchicalLogging.jl for stdlib-compatible hierarchical control over hierarchically related objects with recursive level management.
  • Choose ComponentLogging.jl for explicit hierarchical control, very low filtered-call overhead, direct composition with AbstractLogger sinks, and a hierarchy that can double as a runtime control plane.

About

High-performance hierarchical logging for Julia modules, with fine-grained runtime control and near-zero filtered-call overhead.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages