How to pass the invisible

洪 民憙 (Hong Minhee) @hongminhee@hackers.pub
One of the enduring challenges in software programming is this: “How do we pass the invisible?” Loggers, HTTP request contexts, current locales, I/O handles—these pieces of information are needed throughout our programs, yet threading them explicitly through every function parameter would be unbearably verbose.
Throughout history, various approaches have emerged to tackle this problem. Dynamic scoping, aspect-oriented programming, context variables, and the latest effect systems… Some represent evolutionary steps in a continuous progression, while others arose independently. Yet we can view all these concepts through a unified lens.
Dynamic scoping
Dynamic scoping, which originated in 1960s Lisp, offered the purest form of solution. “A variable's value is determined not by where it's defined, but by where it's called.” Simple and powerful, yet it fell out of favor in mainstream programming languages after Common Lisp and Perl due to its unpredictability. Though we can still trace its lineage in JavaScript's this
binding.
;; Common Lisp example - logger bound dynamically
(defvar *logger* nil)
(defun log-message (message)
(when *logger*
(funcall *logger* message)))
(defun process-user-data (data)
(log-message (format nil "Processing user: ~a" data))
;; actual processing logic…
)
(defun main ()
(let ((*logger* (lambda (msg) (format t "[INFO] ~a~%" msg))))
(process-user-data "john@example.com"))) ; logger passed implicitly
Aspect-oriented programming
AOP structured the core idea of “modularizing cross-cutting concerns.” The philosophy: “Inject context, but with rules.” By separating cross-cutting concerns like logging and transactions into aspects, it maintained dynamic scoping's flexibility while pursuing more predictable behavior. However, debugging difficulties and performance overhead limited its spread beyond Java and .NET ecosystems.
// Spring AOP example - logging separated as cross-cutting concern
@Aspect
public class LoggingAspect {
private Logger logger = LoggerFactory.getLogger(LoggingAspect.class);
@Around("@annotation(Loggable)")
public Object logMethodCall(ProceedingJoinPoint joinPoint) throws Throwable {
String methodName = joinPoint.getSignature().getName();
logger.info("Entering method: " + methodName);
Object result = joinPoint.proceed();
logger.info("Exiting method: " + methodName);
return result;
}
}
@Service
public class UserService {
@Loggable // logger implicitly injected through aspect
public User processUser(String userData) {
// actual processing logic…
return new User(userData);
}
}
Context variables
Context variables represent dynamic scoping redesigned for modern requirements—asynchronous and parallel programming. Python's contextvars
and Java's ThreadLocal
exemplify this approach. Yet they still suffer from runtime dependency and the fact that API context requirements are only discoverable through documentation.
Another manifestation of context variables appears in React's contexts and similar concepts in other UI frameworks. While their usage varies, they all solve the same problem: prop drilling. Implicit propagation through component trees mirrors propagation through function call stacks.
# Python contextvars example - custom logger propagated through context
from contextvars import ContextVar
# Define custom logger function as context variable
logger_func = ContextVar('logger_func')
def log_info(message):
log_fn = logger_func.get()
if log_fn:
log_fn(f"[INFO] {message}")
def process_user_data(data):
log_info(f"Processing user: {data}")
validate_user_data(data)
def validate_user_data(data):
log_info(f"Validating user: {data}") # logger implicitly propagated
def main():
# Set specific logger function in context
def my_logger(msg):
print(f"CustomLogger: {msg}")
logger_func.set(my_logger)
process_user_data("john@example.com")
Monads
Monads approach this from a different starting point. Rather than implicit context passing, monads attempt to encode effects in the type system—addressing a more fundamental problem. The Reader
monad specifically corresponds to context variables. However, when combining multiple effects through monad transformers, complexity exploded. Developers had to wrestle with unwieldy types like ReaderT Config (StateT AppState (ExceptT Error IO))
. Layer ordering mattered, each layer required explicit lifting, and usability suffered. Consequently, monadic ideas remained largely confined to serious functional programming languages like Haskell, Scala, and F#.
-- Haskell Logger monad example - custom Logger monad definition
newtype Logger a = Logger (IO a)
instance Functor Logger where
fmap f (Logger io) = Logger (fmap f io)
instance Applicative Logger where
pure = Logger . pure
Logger f <*> Logger x = Logger (f <*> x)
instance Monad Logger where
Logger io >>= f = Logger $ do
a <- io
let Logger io' = f a
io'
-- Logging functions
logInfo :: String -> Logger ()
logInfo msg = Logger $ putStrLn $ "[INFO] " ++ msg
processUserData :: String -> Logger ()
processUserData userData = do
logInfo $ "Processing user: " ++ userData
validateUserData userData
validateUserData :: String -> Logger ()
validateUserData userData = do
logInfo $ "Validating user: " ++ userData -- logger passed through monad
runLogger :: Logger a -> IO a
runLogger (Logger io) = io
main :: IO ()
main = runLogger $ processUserData "john@example.com"
Effect systems
Effect systems emerged to solve the compositional complexity of monads. Implemented in languages like Koka and Eff, they operate through algebraic effects and handlers. Multiple effect layers compose without ordering constraints. Multiple overlapping layers require no explicit lifting. Effect handlers aren't fixed—they can be dynamically replaced, offering significant flexibility.
However, compiler optimizations remain immature, interoperability with existing ecosystems poses challenges, and the complexity of effect inference and its impact on type systems present ongoing research questions. Effect systems represent the newest approach discussed here, and their limitations will be explored as they gain wider adoption.
// Koka effect system example - logging effects flexibly propagated
effect logger
fun log-info(message: string): ()
fun log-error(message: string): ()
fun process-user-data(user-data: string): logger ()
log-info("Processing user: " ++ user-data)
validate-user-data(user-data)
fun validate-user-data(user-data: string): logger ()
log-info("Validating user: " ++ user-data) // logger effect implicitly propagated
if user-data == "" then
log-error("Invalid user data: empty string")
fun main()
// Different logger implementations can be chosen dynamically
with handler
fun log-info(msg) println("[INFO] " ++ msg)
fun log-error(msg) println("[ERROR] " ++ msg)
process-user-data("john@example.com")
The art of passing the invisible—this is the essence shared by all the concepts discussed here, and it will continue to evolve in new forms as an eternal theme in software programming.