How to pass the invisible

洪 民憙 (Hong Minhee) @hongminhee@hackers.pub

One of the enduring challenges in software programming is this: “How do we pass the invisible?” Loggers, HTTP request contexts, current locales, I/O handles—these pieces of information are needed throughout our programs, yet threading them explicitly through every function parameter would be unbearably verbose.

Throughout history, various approaches have emerged to tackle this problem. Dynamic scoping, aspect-oriented programming, context variables, and the latest effect systems… Some represent evolutionary steps in a continuous progression, while others arose independently. Yet we can view all these concepts through a unified lens.

Dynamic scoping

Dynamic scoping, which originated in 1960s Lisp, offered the purest form of solution. “A variable's value is determined not by where it's defined, but by where it's called.” Simple and powerful, yet it fell out of favor in mainstream programming languages after Common Lisp and Perl due to its unpredictability. Though we can still trace its lineage in JavaScript's this binding.

;; Common Lisp example - logger bound dynamically
(defvar *logger* nil)

(defun log-message (message)
  (when *logger*
    (funcall *logger* message)))

(defun process-user-data (data)
  (log-message (format nil "Processing user: ~a" data))
  ;; actual processing logic…
  )

(defun main ()
  (let ((*logger* (lambda (msg) (format t "[INFO] ~a~%" msg))))
    (process-user-data "john@example.com"))) ; logger passed implicitly

Aspect-oriented programming

AOP structured the core idea of “modularizing cross-cutting concerns.” The philosophy: “Inject context, but with rules.” By separating cross-cutting concerns like logging and transactions into aspects, it maintained dynamic scoping's flexibility while pursuing more predictable behavior. However, debugging difficulties and performance overhead limited its spread beyond Java and .NET ecosystems.

// Spring AOP example - logging separated as cross-cutting concern
@Aspect
public class LoggingAspect {
    private Logger logger = LoggerFactory.getLogger(LoggingAspect.class);
    
    @Around("@annotation(Loggable)")
    public Object logMethodCall(ProceedingJoinPoint joinPoint) throws Throwable {
        String methodName = joinPoint.getSignature().getName();
        logger.info("Entering method: " + methodName);
        Object result = joinPoint.proceed();
        logger.info("Exiting method: " + methodName);
        return result;
    }
}

@Service
public class UserService {
    @Loggable  // logger implicitly injected through aspect
    public User processUser(String userData) {
        // actual processing logic…
        return new User(userData);
    }
}

Context variables

Context variables represent dynamic scoping redesigned for modern requirements—asynchronous and parallel programming. Python's contextvars and Java's ThreadLocal exemplify this approach. Yet they still suffer from runtime dependency and the fact that API context requirements are only discoverable through documentation.

Another manifestation of context variables appears in React's contexts and similar concepts in other UI frameworks. While their usage varies, they all solve the same problem: prop drilling. Implicit propagation through component trees mirrors propagation through function call stacks.

# Python contextvars example - custom logger propagated through context
from contextvars import ContextVar

# Define custom logger function as context variable
logger_func = ContextVar('logger_func')

def log_info(message):
    log_fn = logger_func.get()
    if log_fn:
        log_fn(f"[INFO] {message}")

def process_user_data(data):
    log_info(f"Processing user: {data}")
    validate_user_data(data)

def validate_user_data(data):
    log_info(f"Validating user: {data}")  # logger implicitly propagated

def main():
    # Set specific logger function in context
    def my_logger(msg):
        print(f"CustomLogger: {msg}")
    
    logger_func.set(my_logger)
    process_user_data("john@example.com")

Monads

Monads approach this from a different starting point. Rather than implicit context passing, monads attempt to encode effects in the type system—addressing a more fundamental problem. The Reader monad specifically corresponds to context variables. However, when combining multiple effects through monad transformers, complexity exploded. Developers had to wrestle with unwieldy types like ReaderT Config (StateT AppState (ExceptT Error IO)). Layer ordering mattered, each layer required explicit lifting, and usability suffered. Consequently, monadic ideas remained largely confined to serious functional programming languages like Haskell, Scala, and F#.

-- Haskell Logger monad example - custom Logger monad definition
newtype Logger a = Logger (IO a)

instance Functor Logger where
    fmap f (Logger io) = Logger (fmap f io)

instance Applicative Logger where
    pure = Logger . pure
    Logger f <*> Logger x = Logger (f <*> x)

instance Monad Logger where
    Logger io >>= f = Logger $ do
        a <- io
        let Logger io' = f a
        io'

-- Logging functions
logInfo :: String -> Logger ()
logInfo msg = Logger $ putStrLn $ "[INFO] " ++ msg

processUserData :: String -> Logger ()
processUserData userData = do
    logInfo $ "Processing user: " ++ userData
    validateUserData userData

validateUserData :: String -> Logger ()
validateUserData userData = do
    logInfo $ "Validating user: " ++ userData  -- logger passed through monad

runLogger :: Logger a -> IO a
runLogger (Logger io) = io

main :: IO ()
main = runLogger $ processUserData "john@example.com"

Effect systems

Effect systems emerged to solve the compositional complexity of monads. Implemented in languages like Koka and Eff, they operate through algebraic effects and handlers. Multiple effect layers compose without ordering constraints. Multiple overlapping layers require no explicit lifting. Effect handlers aren't fixed—they can be dynamically replaced, offering significant flexibility.

However, compiler optimizations remain immature, interoperability with existing ecosystems poses challenges, and the complexity of effect inference and its impact on type systems present ongoing research questions. Effect systems represent the newest approach discussed here, and their limitations will be explored as they gain wider adoption.

// Koka effect system example - logging effects flexibly propagated
effect logger
  fun log-info(message: string): ()
  fun log-error(message: string): ()

fun process-user-data(user-data: string): logger ()
  log-info("Processing user: " ++ user-data)
  validate-user-data(user-data)

fun validate-user-data(user-data: string): logger ()
  log-info("Validating user: " ++ user-data)  // logger effect implicitly propagated
  if user-data == "" then
    log-error("Invalid user data: empty string")

fun main()
  // Different logger implementations can be chosen dynamically
  with handler
    fun log-info(msg) println("[INFO] " ++ msg)
    fun log-error(msg) println("[ERROR] " ++ msg)
  process-user-data("john@example.com")

The art of passing the invisible—this is the essence shared by all the concepts discussed here, and it will continue to evolve in new forms as an eternal theme in software programming.

11
1
0

1 comment

If you have a fediverse account, you can comment on this article from your own instance. Search https://hackers.pub/ap/articles/0197a310-52f3-7c96-8440-da4f6000074e on your instance and reply to it.

1