The DQN agent learns to play Atari video games purely by watching the screen — just like a human sitting in front of a TV. It starts by pressing buttons randomly and observing what happens to the score. Over millions of attempts it gradually works out which button to press in each situation to make the score go up as much as possible over time. It does this entirely through trial and error: no one tells it the rules of the game or which moves are good. Its only signal is the game's score. Using a deep neural network to recognise patterns in what it sees on screen, it builds up an internal 'map' of which actions tend to lead to higher scores from any given situation, and over time this map becomes good enough to play many Atari games at human-level or above — all without any game-specific programming.

The raw data is the pixel output of the Atari 2600 emulator — the game screen as it appears at each time step. Specifically: • Dimensions: 210 × 160 pixels • Colour: RGB (three colour channels — red, green, blue) • Bit depth: each channel has integer values in the range 0–255 • Frame rate: one frame is captured at each time step (the emulator runs at roughly 60 fps, but the agent acts at a lower frequency, typically every 4 frames) The raw input is therefore a 210 × 160 × 3 array of integers per frame — purely the visual information the game renders to screen, with no additional game-state information such as internal variables, sprite positions, or RAM contents. The agent must infer everything it needs (object positions, velocities, score trends) from these pixels alone.

Four preprocessing steps are applied to each raw frame: 1. Greyscale conversion — the three RGB colour channels are collapsed into a single luminance channel, producing a 210 × 160 × 1 image. 2. Downsampling (resizing) — the greyscale image is rescaled to 84 × 84 pixels using interpolation, reducing the spatial resolution while retaining the key visual features. 3. Frame stacking — four consecutive preprocessed frames are stacked together along the channel dimension to form an 84 × 84 × 4 tensor. This becomes the state input to the network. 4. Pixel value normalisation — raw pixel values (0–255) are divided by 255 so they lie in the range [0, 1], giving the network inputs with a consistent scale.

1. Greyscale conversion — colour provides very little additional information for deciding which action to take in most Atari games; the shapes and positions of objects matter far more than their hue. Removing colour reduces the input size by a factor of three, cutting both memory usage and computational cost substantially. 2. Downsampling to 84 × 84 — a 210 × 160 image still contains far more pixels than the network needs to identify objects and make decisions. Reducing to 84 × 84 dramatically reduces the number of parameters in the first convolutional layer, making training feasible on the hardware of the time, while preserving enough detail to distinguish game elements such as the ball, paddles, and enemies. 3. Frame stacking — a single static frame gives no information about motion: the agent cannot tell whether a ball is moving left or right, or how fast. Stacking four consecutive frames provides the network with implicit temporal information — by comparing the position of an object across frames the convolutional layers can infer velocity and direction, which are essential for anticipating where objects will be next. 4. Normalisation — neural networks train more stably when inputs have small, consistent magnitudes. Raw pixel values up to 255 can cause large, erratic gradient updates. Scaling to [0, 1] keeps activations in a well-behaved range, prevents saturation of activation functions, and makes learning rate tuning more predictable.

Input: The network receives the preprocessed state — a stack of four 84 × 84 greyscale frames (84 × 84 × 4 tensor) representing the current and three most recent game screens. Output: The network produces one Q-value estimate per possible action. For a game with n legal button combinations, there are n output neurons. Each output Q(s, a) estimates the expected cumulative discounted reward of taking action a from state s and thereafter following the optimal policy. Training loop: At each step the agent selects an action using an ε-greedy policy (random with probability ε, otherwise the action with the highest Q-value). The resulting transition (s, a, r, s') is stored in a replay buffer. A random mini-batch is then sampled from the buffer (experience replay), and for each transition a target value is computed using a separate, periodically-updated target network: y = r + γ · max_a' Q_target(s', a') Error metric: The loss is the Huber loss (smooth L1) between the predicted Q-value and the target y: L = HuberLoss(y − Q(s, a; θ)) Huber loss is used instead of plain MSE because it is less sensitive to large TD errors (outliers), which are common early in training. Gradients of this loss are backpropagated to update the online network weights θ, while the target network weights are held fixed and only copied from θ every C steps.

In a typical supervised learning pipeline the training dataset is fixed before training begins: all inputs and their correct labels exist upfront, are shuffled, and are presented to the model in epochs. The distribution of examples does not change during training. DQN differs in several important ways: 1. Data is generated online — there is no pre-existing labelled dataset. Training examples (s, a, r, s') are produced by the agent as it plays the game. Early in training the agent plays randomly, so early data reflects random behaviour; as the policy improves, the data distribution shifts to reflect better play. The training set is constantly evolving. 2. Non-stationary distribution — because the data depends on the current policy, and the policy is changing, the distribution of states the agent encounters changes throughout training. This violates the i.i.d. assumption of standard supervised learning and can cause instability. Experience replay partially mitigates this by sampling from a buffer of past transitions, breaking temporal correlations and introducing older data. 3. Labels are not given — in supervised learning each input has a ground-truth label provided by a human. In DQN the 'label' (the TD target y = r + γ max Q_target) is itself estimated by a second neural network and changes as training progresses. This bootstrapping means the network is partially chasing a moving target — a known source of instability that the target network (updated only periodically) is designed to reduce. 4. Reward sparsity — supervised labels are available for every input. In DQN the reward signal is often sparse (many transitions have r = 0), so credit must be assigned backwards in time through the discount factor — a much harder credit-assignment problem than label-based learning.

A generative system is a computer program that can produce new, original content — such as images, music, text, or video — rather than simply analysing or classifying things that already exist. Instead of being given a question with a right or wrong answer, it is given a goal ('create a painting in the style of Van Gogh' or 'compose a jazz melody') and it produces something new that fits that goal. The output did not exist before the system created it, and in many cases the system can produce an effectively unlimited variety of different outputs from the same starting goal.

Machine learning systems generate creative artefacts by learning a statistical model of the structure of existing examples, then sampling new examples from that model. For images — a Generative Adversarial Network (GAN) trains two networks jointly: a generator that maps random noise vectors to images, and a discriminator that learns to distinguish real images from generated ones. Through adversarial training the generator learns to produce images indistinguishable from the training set. Alternatively a Variational Autoencoder (VAE) learns a continuous latent space of images, and new images are produced by sampling or interpolating points in that space and passing them through the decoder. For music — similar approaches apply: a VAE or RNN-based model (e.g. MusicVAE) learns the structure of MIDI sequences (melody, harmony, rhythm), and new sequences are sampled from its latent space. Diffusion models are increasingly used, learning to iteratively denoise random noise into coherent audio spectrograms. In all cases the key insight is the same: train on large corpora to model the underlying distribution of human-made artefacts, then sample from that distribution to generate novel but plausible new examples.

Margaret Boden proposed a highly influential taxonomy of computational creativity in: Boden, M. A. (2004). The Creative Mind: Myths and Mechanisms (2nd ed.). Routledge. Boden identifies three types of creativity: 1. Exploratory creativity — generating novel ideas or artefacts by searching within an existing conceptual space (e.g. composing a new melody within the rules of tonal harmony). Most current generative AI systems operate at this level. 2. Combinational creativity — producing novel combinations of familiar ideas or styles (e.g. blending the brushwork of Monet with the colour palette of Van Gogh). Style-transfer networks and interpolation in latent space fall here. 3. Transformational creativity — the most radical type: changing the rules of the conceptual space itself, opening up possibilities that were previously inconceivable (e.g. inventing atonality in music). Very few if any current AI systems achieve this level autonomously.

AARON, created by artist and computer scientist Harold Cohen from the 1970s onwards, is a robotic system that autonomously generates and physically paints original drawings and colour paintings using a robotic arm and brushes — it was never given images to copy. In Boden's taxonomy AARON operates primarily at the exploratory creativity level: Cohen encoded a rich set of rules about composition, figure placement, colour relationships, and mark-making, and AARON explores the space of possible paintings defined by those rules, producing genuinely novel images each time. It does not exhibit transformational creativity because it cannot modify its own rule system — the conceptual space is defined and fixed by Cohen.

Convolutional Neural Networks (CNNs) are the standard architecture for image data. Why CNNs are suited to images: 1. Translation invariance / equivariance — a convolutional filter learns to detect a feature (e.g. an edge, a curve) wherever it appears in the image. The same filter is applied across the whole image (weight sharing), so the network detects the feature in the top-left corner just as well as in the bottom-right, without needing separate weights for each position. A fully-connected layer has no such inductive bias. 2. Local connectivity — pixels that are spatially close are more likely to be related than distant ones (edges, textures, objects are spatially coherent). Convolutional layers only connect each neuron to a small local receptive field, matching this structure and dramatically reducing the number of parameters compared to a fully connected layer over the same input. 3. Hierarchical feature learning — stacking convolutional layers allows the network to build up features hierarchically: early layers detect low-level features (edges, colours), middle layers combine these into parts (corners, textures), and deeper layers represent high-level concepts (faces, objects) — matching how visual processing works in the brain.

FOR: 1. Novelty and surprise — computers regularly produce outputs that surprise even their creators. AlphaGo's Move 37 in Game 2 against Lee Sedol was described by professional Go players as a move no human would have played — genuinely unexpected and later recognised as brilliant. Generative AI systems produce images, music, and text that humans cannot reliably distinguish from human-made work. If we define creativity by the novelty and value of outputs, computers appear to qualify. 2. Boden's transformational creativity has computational analogues — NEAT and other neuroevolution systems can change their own architecture (the structure of the 'conceptual space') as they evolve, not just search within a fixed space. Evolutionary systems have produced circuit designs, antenna shapes, and walking gaits that human engineers had not conceived, suggesting a form of space-transforming creativity in practice. AGAINST: 1. No intentionality or understanding — human creativity is inseparable from meaning, intention, and experience. A composer writes a requiem because they have experienced grief; a poet chooses a metaphor because they understand what it feels like to be human. A language model produces text by predicting statistically likely tokens — it has no understanding of what it says, no intention, and no inner life. Output that looks creative is not the same as a creative process. 2. Creativity requires evaluative judgement — truly creative humans not only generate ideas but select among them based on aesthetic, ethical, and contextual values. Current AI systems either rely on human evaluators (discriminators, RLHF raters) or optimise proxy metrics. They cannot autonomously judge that something is beautiful, meaningful, or timely in the way a human creator can — their 'taste' is borrowed from the humans whose data or feedback trained them.

Positive effect — democratisation of music creation: AI composition tools lower the barrier to musical expression for people without formal training. A person who has a melody in their head but cannot notate, harmonise, or arrange it can use an AI system to realise their creative vision. Platforms such as Google's Magenta and commercial tools like Suno or Udio allow non-musicians to produce complete, polished musical pieces from a text prompt or hummed melody. This expands who gets to participate in musical culture — historically gated by years of formal training and expensive instruments — and enables new forms of collaborative human–AI creativity. Negative effect — economic displacement of human musicians: If AI systems can produce commercially acceptable background music, film scores, jingles, and stock audio at near-zero marginal cost, the market for human composers in these genres collapses. Streaming platforms already use AI-generated tracks to fill mood playlists without paying royalties to human artists. This is not hypothetical: in 2023 Universal Music Group, Sony, and Warner all issued takedown requests for AI-generated tracks that mimicked their artists' styles. Session musicians, soundtrack composers, and music library contributors — often not the most highly paid people in the industry — face direct income loss. This harm is concentrated on a specific professional group while the economic benefits accrue primarily to technology companies.

A genetic algorithm finds good solutions to difficult problems by mimicking how evolution works in nature. Imagine you want to design the best possible car bumper but there are billions of possible shapes — far too many to try one by one. The algorithm starts with a large group of random designs. It tests each one (how well does it protect passengers in a crash?), keeps the better ones, and throws away the worst. The surviving designs then 'reproduce' — parts of two good designs are combined to create new designs (like a child inheriting traits from both parents), and occasionally a small random tweak is introduced (like a genetic mutation). This new generation is tested, the best survive and reproduce again, and the process repeats hundreds or thousands of times. Over many generations the designs gradually improve, just as species become better adapted to their environment over evolutionary time — without anyone having to work out manually what the best design looks like.

Genetically encoding a problem means representing candidate solutions as strings of data (the 'genome' or 'chromosome') that the genetic algorithm can manipulate — just as DNA encodes the blueprint for a biological organism. The encoding must satisfy three requirements: 1. Completeness — every possible solution must be representable by some genome. The encoding must cover the full design space. 2. Validity — ideally every genome should correspond to a valid (testable) solution, though in practice some encodings produce invalid configurations that must be repaired or penalised. 3. Locality — small changes to the genome should produce small changes in the solution (phenotype). If a tiny mutation radically changes the solution, the algorithm cannot make incremental improvements and degenerates into random search. Common encoding types include binary strings (each bit encodes a feature), real-valued vectors (each number encodes a parameter such as a dimension or angle), permutations (for ordering problems like TSP), and tree structures (for programs in Genetic Programming).

The crash protection system is a 3D metal structure placed at the front of the car. Several encoding strategies could be used: Option 1 — Parametric (real-valued) encoding: Define a fixed set of geometric parameters that describe the structure: e.g. the number, position, length, cross-sectional profile, and wall thickness of each structural beam or crumple-zone element. The genome is a real-valued vector [x₁, y₁, z₁, l₁, t₁, x₂, …] where each value is a dimension. This is compact and easy to apply standard crossover and mutation to (swap sub-vectors, add Gaussian noise). However it constrains the topology — the overall arrangement of beams is fixed by the parametrisation, so it can only explore variations within a predefined structural template. Option 2 — Voxel / direct encoding: Divide the volume at the front of the car into a 3D grid of small cubic cells. Each cell is encoded as a binary value: 1 if the cell contains metal, 0 if it is empty. The genome is a binary string or matrix of cell values. This encoding can represent any structure that fits the grid, giving it high expressiveness and the ability to discover unexpected topologies. The downside is that the genome is very large (many cells), many combinations produce physically invalid or unmanufacturable structures, and locality is poor — flipping one bit can disconnect a critical load path. Option 3 — Graph / beam-network encoding: Represent the structure as a graph where nodes are connection joints (with 3D coordinates) and edges are metal beams (with material and cross-section attributes). The genome encodes the node positions and the adjacency matrix. Crossover can swap sub-graphs between parents; mutation can add/remove a node or beam, or perturb a joint position. This representation naturally handles variable-topology structures and is closer to how an engineer would think about the design. Recommended choice: A hybrid approach — use a parametric encoding with enough parameters to allow meaningful shape variation (beam layout, angles, thicknesses, material grades) while constraining the genome to always produce a valid, manufacturable structure. This balances expressiveness with locality and avoids the combinatorial explosion of the voxel approach.

The fitness function is a mathematical measure that assigns a scalar score to each candidate solution in the population, quantifying how well it solves the problem. It is the only mechanism through which the genetic algorithm 'knows' what it is trying to achieve — the algorithm optimises whatever the fitness function measures, nothing more. Individuals with higher fitness are more likely to be selected as parents for the next generation. The fitness function therefore acts as the selection pressure that drives the population toward better solutions over time. Designing a fitness function that accurately captures the true objective (rather than a proxy that can be 'gamed') is one of the most critical and difficult aspects of applying a genetic algorithm to a real problem.

The fitness function must capture the real-world objective: minimise passenger injury in a frontal crash, subject to practical constraints on cost, weight, and manufacturability. A suitable function would be: Step 1 — Define the primary metric: run a finite-element simulation (e.g. using LS-DYNA or Abaqus) of the structure under a standardised crash test (e.g. Euro NCAP 40% offset frontal impact at 64 km/h). Extract the peak deceleration experienced by a crash-test dummy in the passenger cabin — this is the key predictor of injury (chest compression, head acceleration, etc.). Lower peak deceleration = higher fitness. Step 2 — Add secondary objectives as penalties or constraints: • Mass penalty: heavier structures reduce fuel efficiency. Add a term −α × mass to the fitness score so the algorithm balances protection against weight. • Intrusion penalty: if the structure collapses into the passenger cabin beyond a maximum allowable limit, apply a large penalty or set fitness to zero. • Manufacturability: penalise geometries that cannot be stamped, bent, or welded within standard manufacturing tolerances. Step 3 — Composite fitness: fitness = −peak_deceleration − α·mass − β·intrusion − γ·manufacture_penalty The weighting coefficients (α, β, γ) encode engineering priorities and must be chosen carefully with domain experts — if mass is over-weighted, the algorithm will evolve an extremely light but dangerous structure. Challenge: Each fitness evaluation requires a crash simulation, which can take minutes to hours. With a population of hundreds of individuals over many generations, total computation is very expensive — surrogate models (e.g. neural networks trained on previous simulation results) are often used to approximate the fitness cheaply.

Comparison with Topology Optimisation (specifically the SIMP method — Solid Isotropic Material with Penalisation), a gradient-based automated structural design method widely used in mechanical engineering. DIFFERENCES: 1. Population vs single solution — a GA maintains a population of hundreds of candidate designs evolving in parallel, which gives it broad coverage of the search space and resilience against local optima. Topology optimisation works with a single design that is iteratively refined by computing gradients of the objective (e.g. compliance) with respect to the density of each element. It converges quickly but can get trapped in local optima depending on the initialisation. 2. Gradient requirement — topology optimisation requires the fitness/objective function to be differentiable with respect to the design variables, so gradients can be computed via adjoint methods. GAs are gradient-free (black-box): they only need to evaluate fitness, not differentiate it. This makes GAs applicable to non-differentiable, noisy, or simulation-based objectives (such as crash FEA) where gradients are unavailable. SIMILARITIES: 1. Both are automated — neither method requires a human to specify what the structure should look like. Given a design space, an objective, and constraints, both methods search autonomously and can discover non-intuitive designs that a human engineer would not have conceived. 2. Both can handle multiple constraints — both can incorporate constraints (maximum weight, minimum stiffness, manufacturing limits) either as hard constraints or penalty terms added to the objective, allowing them to navigate complex multi-criteria engineering trade-offs automatically.