Correct answer: A. Overlapping progress of tasks

Concurrency permits multiple tasks to make progress during overlapping time intervals. Their instructions may interleave even on a single processor.

Correct answer: B. Actual simultaneous execution

Parallelism requires work to execute at the same time, for example on separate CPU cores. Concurrency alone does not necessarily imply simultaneous execution.

Correct answer: C. Different activities execute concurrently

Task parallelism assigns different operations or activities to concurrent workers. The activities can contribute to a shared overall result.

Correct answer: D. Same computation over different data subsets

Data parallelism applies the same operation to multiple parts of a dataset. The separate portions can be processed simultaneously.

Correct answer: A. Single Instruction Multiple Data

SIMD means Single Instruction, Multiple Data. One instruction operates on multiple data elements using multiple processing lanes.

Correct answer: B. Multiple Instruction Multiple Data

MIMD means Multiple Instruction, Multiple Data. Different processors may execute different instruction streams on different data.

Correct answer: C. Single Program Multiple Data

SPMD means Single Program, Multiple Data. Several workers execute the same program but typically process distinct input partitions.

Correct answer: D. Executing program instance with resource context

A process is a program in execution together with its execution state and allocated resources. Operating systems normally isolate processes from one another.

Correct answer: A. Execution stream within a process

A thread is an independently schedulable execution stream inside a process. Threads in the same process usually share its address space.

Correct answer: B. Chip with multiple processing cores

A multicore processor contains more than one processing core on a chip. Different cores can execute independent instructions concurrently.

Correct answer: C. Workers access common address space

In a shared-memory model, workers can read and write a common address space. Synchronization may be needed to coordinate their accesses.

Correct answer: D. Nodes have separate address spaces

In distributed memory, each node or process has its own address space. Data on another node must normally be obtained through communication.

Correct answer: A. Communicates via send and receive

Message passing exchanges information using explicit sends and receives. It is especially useful when workers do not share an address space.

Correct answer: B. Tasks need little interworker coordination

Embarrassingly parallel tasks can run mostly independently. Little communication or coordination is required between workers.

Correct answer: C. Shows precedence among tasks

A dependency graph describes precedence constraints between tasks. A directed edge typically means one task must finish before another can proceed.

Correct answer: D. Longest weighted dependent path

The critical path is the longest path when task durations are used as weights. Its length limits how quickly the whole dependency graph can finish.

Correct answer: A. Total computational operations

Total work W represents the aggregate computation required across all tasks. It approximates the time taken by a single ideal worker.

Correct answer: B. Length of longest dependent chain

The span S is the duration of the longest dependency chain. Even unlimited processors cannot finish faster than this chain permits.

Correct answer: C. Runtime on p workers is at least W divided by p

With p identical processors performing at most one unit of work per unit time, Tₚ ≥ W/p. This follows because all W units of work must be completed.

Correct answer: D. Parallel runtime is at least S

Parallel execution cannot bypass required dependencies, so Tₚ ≥ S. This bound is independent of how many processors are available.

Correct answer: A. Sequential time divided by parallel time

Speedup is T₁/Tₚ, where T₁ is sequential execution time and Tₚ is time using p processors. A larger value indicates a greater runtime reduction.

Correct answer: B. Speedup divided by processor count

Parallel efficiency is Eₚ = (T₁/Tₚ)/p = speedup/p. It expresses the achieved speedup relative to the processor count.

Correct answer: C. 4

Speedup equals sequential time divided by parallel time: 100/25 = 4. Thus the parallel run is four times as fast in elapsed time.

Correct answer: D. 75 percent

Efficiency is speedup divided by processor count: 6/8 = 0.75, or 75%. This is the fraction of ideal linear speedup achieved.

Correct answer: A. Fixed workload with increasing workers

Strong scaling measures how runtime changes as processor count rises while the total problem size remains fixed.

Correct answer: B. Fixed work per worker while increasing total workload and workers

Weak scaling increases the overall problem size as more workers are added so that the workload per worker stays approximately constant.

Correct answer: C. Serial fraction limits fixed-size speedup

Amdahl’s law shows how the part of a fixed workload that cannot be parallelized limits overall speedup.

Correct answer: D. 10

With serial fraction f = 0.10, the infinite-processor limit of Amdahl’s speedup is 1/f = 10.

Correct answer: A. Scaled-size speedup perspective

Gustafson’s law considers scaled workloads: with additional processors, more total work may be completed in approximately the same time.

Correct answer: B. Additional communication and coordination cost

Parallel overhead is extra work or time due to communication, synchronization, scheduling and related coordination. It may prevent ideal scaling.

Correct answer: C. Distribute work to reduce idle resources

Load balancing distributes work so that some workers do not remain idle while others are overloaded. Effective balancing improves resource use.

Correct answer: D. Work assignment chosen in advance

Static scheduling assigns tasks to workers before execution, usually according to a predetermined plan. It has low decision overhead but can be less adaptive.

Correct answer: A. Assign work as execution proceeds

Dynamic scheduling assigns work while execution is underway. It can respond to irregular task durations and changing loads.

Correct answer: B. Idle worker obtains tasks from another queue

In work stealing, an idle worker takes available work from another worker’s task queue. This can balance irregular parallel workloads.

Correct answer: C. Amount of work per task

Task granularity refers to how much computation each task contains. Very small tasks may incur proportionally more scheduling overhead.

Correct answer: D. Relatively large work unit

A coarse-grained task contains relatively substantial work. Fewer, larger tasks can reduce scheduling overhead but may limit flexibility.

Correct answer: A. Relatively small work unit

A fine-grained task represents a smaller unit of work. Many small tasks can improve balancing but add coordination overhead.

Correct answer: B. Combine partial results using aggregation

A reduction combines partial outputs into an aggregate result, such as a sum or maximum. A tree-shaped reduction can combine independent partial results in parallel.

Correct answer: C. Allows regrouping mathematical operands

Associativity allows operands to be regrouped without changing the mathematical result, for example (a+b)+c = a+(b+c). This enables tree-based grouping.

Correct answer: D. May change rounding results

Floating-point arithmetic introduces rounding at intermediate steps. Because floating-point addition is not exactly associative, regrouping can change the final rounded value.

Correct answer: A. Workers meet at a synchronization point

A barrier requires participating workers to reach a synchronization point before any of them proceeds into the next phase.

Correct answer: B. Result depends on uncontrolled concurrent timing

A race condition occurs when the result depends on the timing or ordering of concurrent operations. Unsynchronized access to shared data is a common cause.

Correct answer: C. Code using shared state requiring coordinated access

A critical section is code that accesses shared state requiring coordination. A mutex or another synchronization mechanism may protect it.

Correct answer: D. Mutual-exclusion synchronization primitive

A mutex provides mutual exclusion: at most one holder may enter the protected critical region at a time.

Correct answer: A. Indivisible relative to specified concurrent observers

An atomic operation appears indivisible to the specified concurrent observers. They do not observe an intermediate, partially completed state.

Correct answer: B. Counter-based synchronization primitive

A semaphore uses a counter with synchronization operations to coordinate threads or limit access to a number of resources.

Correct answer: C. Cyclic waiting prevents progress

Deadlock arises when participants are permanently blocked waiting for events that cannot occur, often because each holds a resource needed by another.

Correct answer: D. A task waits indefinitely while others make progress

Starvation occurs when a worker is repeatedly denied the resources or scheduling opportunities it needs, even while other workers continue making progress.

Correct answer: A. Tasks remain active but make no useful progress

In livelock, threads or processes remain active and keep reacting to one another, but fail to make useful progress.

Correct answer: B. Allows threads to wait for a state predicate with mutex coordination

A condition variable lets threads wait until a shared-state predicate may be true. Correct use normally involves holding a mutex and rechecking the predicate after waking.

Correct answer: C. Message Passing Interface

MPI is the Message Passing Interface, a standardized API specification for processes exchanging data. Different implementations support parallel programs on clusters and other distributed-memory platforms.

Correct answer: D. Defines a group and communication context

A communicator defines both a set of participating processes and a communication context. Messages in different communicators are kept distinct even when their source and tag values coincide.

Correct answer: A. Identifies a process within a communicator

Rank is a process’s identifier within a particular communicator, normally numbered from zero. The same process can have different ranks in different communicators.

Correct answer: B. Number of processes in a communicator

Communicator size is the number of processes belonging to it. Programs can query this count to partition workloads and organize collective operations.

Correct answer: C. Initiates standard point-to-point send

MPI_Send performs a standard-mode point-to-point send. It returns when the send buffer can safely be reused; this does not necessarily mean the receiving process has completed its receive.

Correct answer: D. Receives a matching message

MPI_Recv receives a matching point-to-point message into the supplied buffer. Its completion indicates that the received data are available for use.

Correct answer: A. Helps distinguish communication types

A tag helps the receiver distinguish message categories when matching sends and receives. Source, communicator and tag collectively help specify eligible messages.

Correct answer: B. Returns after the receive is complete

A blocking receive returns only after the matching message has been received into the destination buffer. It may wait if no eligible send is available.

Correct answer: C. Initiates a nonblocking send request

MPI_Isend starts a nonblocking send and returns a request handle. The application must complete that request before reusing the send buffer in a way that changes its contents.

Correct answer: D. Waits for completion of a nonblocking MPI request

MPI_Wait blocks until the specified outstanding request reaches completion. For a nonblocking send, completion establishes that its send buffer can be safely reused.

Correct answer: A. Distributes data from a root to members of communicator

MPI_Bcast broadcasts data from one designated root process to all members of the communicator. All participants call the collective with compatible arguments.

Correct answer: B. Combines values from processes at a selected root

MPI_Reduce combines contributions, such as sums or maxima, across participating processes and places the reduced result at a chosen root process.

Correct answer: C. Combines values and makes result available to all participating processes

MPI_Allreduce computes a collective reduction and supplies the result to every participant. It is useful when all processes need a shared global value.

Correct answer: D. Distributes different chunks from root to processes

MPI_Scatter distributes distinct portions of a root process’s data to participants. It is often used to divide an array among workers.

Correct answer: A. Collects contributions from processes at root

MPI_Gather collects a contribution from each participating process and assembles them at the root. It is the complementary gathering pattern to scatter.

Correct answer: B. Makes gathered contributions available to all processes

MPI_Allgather collects all participants’ contributions and makes the combined results available to every process, rather than only the root.

Correct answer: C. Synchronizes participating processes at a collective point

MPI_Barrier is a collective synchronization call. A process returns only after all relevant members of the communicator have entered that barrier.

Correct answer: D. Must be compatible among participating processes

Processes in a communicator must invoke collective operations in a compatible order. Mismatched operations can lead to deadlock or undefined communication behavior.

Correct answer: A. Describes transmitted element representation

An MPI datatype specifies how communicated values are represented and arranged. It lets MPI handle element counts and matching data layouts.

Correct answer: B. Represents structured or noncontiguous memory layouts

Derived datatypes describe arrays, strided regions or structures without manually copying every element into a contiguous temporary buffer.

Correct answer: C. Connects specific sending and receiving processes

Point-to-point communication connects a particular sender with a particular receiver, with matching rules for source, destination, tag and communicator.

Correct answer: D. Involves a defined group of participating processes

A collective involves a participating group defined by a communicator. Examples include broadcasts, reductions and barriers; all required members must participate correctly.

Correct answer: A. Time until message transfer or response becomes available under a model

Latency is communication delay, often modeled as a fixed startup cost before data transfer dominates. It matters especially for short messages.

Correct answer: B. Data volume transferred per unit time

Bandwidth is the amount of data transferred per unit time, commonly bytes per second. Large-message performance is often limited by available bandwidth.

Correct answer: C. Message cost approximates startup latency plus size-dependent transfer time

A common model is T(n) = α + βn, where α is startup latency, β is transfer time per byte and n is message size. Both fixed and size-dependent costs matter.

Correct answer: D. Can incur high total startup overhead

Each small message incurs a startup cost. Grouping data into fewer, larger messages can reduce aggregate overhead when application requirements permit.

Correct answer: A. Progresses useful work while transfers are in flight

Useful computations proceed while data communication is in progress. This can conceal some transfer latency, provided the MPI implementation and application actually make progress.

Correct answer: B. Can occur if processes wait for messages that none can send

Deadlock is possible when processes block waiting for receives whose corresponding sends cannot execute. Careful matching and nonblocking patterns may avoid such cycles.

Correct answer: C. Can accept data from more than one eligible source under specified matching rules

A wildcard receive can match more than one eligible sending process, for example by using MPI_ANY_SOURCE. Tags and communicator still constrain matching.

Correct answer: D. Preserves ordering for messages meeting relevant ordering conditions between matching endpoints

MPI applies nonovertaking rules under specified matching conditions for ordered sends. It does not guarantee a single global order across unrelated senders.

Correct answer: A. Transfers information through common addressable locations

Threads in a shared address space communicate by reading and writing shared variables. Appropriate synchronization is needed to avoid races and visibility errors.

Correct answer: B. Gives each thread its own instance of specified data

Thread-local storage provides a distinct instance of specified data for each thread. Changes to one thread’s instance do not modify another thread’s instance.

Correct answer: C. Restricts observable memory ordering according to memory model

A fence constrains the ordering or visibility of memory operations under the relevant memory model. Its guarantees depend on how it is combined with atomic synchronization.

Correct answer: D. Prevents specified later accesses from being reordered before an acquire

An acquire operation restricts later operations from moving before it and can observe values published by a compatible release operation.

Correct answer: A. Ensures specified earlier effects precede a release operation

A release operation orders preceding accesses before publication through that atomic action. Paired with an appropriate acquire, it can establish synchronization.

Correct answer: B. Atomically tests and conditionally updates a location

Compare-and-swap atomically compares a memory location with an expected value and updates it when they match. It is a building block for nonblocking structures.

Correct answer: C. Atomically returns and updates a numeric value

Fetch-and-add performs an atomic addition and returns the prior value in the common API convention. This supports safe counters shared by multiple threads.

Correct answer: D. Waits by repeatedly checking lock availability

A spinlock repeatedly polls until it can acquire ownership instead of sleeping. It may suit very short waits but wastes CPU time under long contention.

Correct answer: A. Allows concurrent readers while controlling writers

A read-write lock usually admits multiple readers simultaneously but requires writers to hold exclusive access. Fairness and starvation properties depend on the implementation.

Correct answer: B. Multiple workers compete for a synchronization resource

Contention occurs when multiple workers request the same lock or resource. Increasing contention forces additional waiting and can limit parallel speedup.

Correct answer: C. Affects how long other workers may wait on a shared lock

Long critical sections keep shared locks occupied for longer intervals. Other workers may be forced to wait, reducing concurrency and throughput.

Correct answer: D. May occur when required participants never reach the barrier

A barrier can block indefinitely when one required participant never arrives, such as when control flow skips the barrier or a participant has failed.

Correct answer: A. Lower-priority holder delays higher-priority worker through a shared resource

Priority inversion happens when a high-priority task waits for a lower-priority lock holder. Priority inheritance is one possible mitigation.

Correct answer: B. Releases associated mutex while waiting and reacquires it before return under common APIs

A condition-variable wait typically releases the associated mutex atomically with entering the wait, then reacquires it before returning. This lets another thread update the predicate.

Correct answer: C. A waiting thread resumes without the awaited predicate necessarily becoming true

Threads can wake without the desired condition being true, including spurious wakeups. Therefore a condition-variable wait should usually be enclosed in a loop that rechecks its predicate.

Correct answer: D. Coordinates data production and consumption through a shared buffer or channel

Producers place items into a buffer or channel, while consumers remove and process them. Synchronization coordinates empty and full conditions.

Correct answer: A. Has finite capacity requiring backpressure or waiting when full

A bounded buffer has finite capacity. Producers must block, drop work or apply backpressure when the buffer is full, depending on system design.

Correct answer: B. Slows upstream producers when downstream capacity is limited

Backpressure propagates reduced downstream capacity toward upstream producers. It prevents uncontrolled queue growth and resource exhaustion.

Correct answer: C. Guarantees system-wide progress despite individual thread delays under its model

Lock-freedom guarantees that some operation across the system completes in a finite number of steps despite delays of individual threads. It does not promise each thread will succeed promptly.

Correct answer: D. Guarantees each operation completes within a bounded number of its own steps

Wait-freedom guarantees that every individual operation completes after a bounded number of its own steps under the assumed model. This is stronger than lock-freedom.

Correct answer: A. Independent computers coordinating over communication links

A distributed system comprises independent computers that coordinate by exchanging messages. Each machine has its own state, so network delays and failures must be considered.

Correct answer: B. Some components fail while others remain operational

A partial failure leaves some nodes or links unavailable while other components continue running. This makes a global view of system health difficult.

Correct answer: C. Disrupt communication between groups of nodes

A partition prevents some groups of nodes from communicating reliably with others. The nodes may still be functioning locally despite losing contact.

Correct answer: D. Delay can resemble a failed or unreachable peer

Timeouts detect lack of a timely reply, not the reason for it. Congestion, scheduling delays and node failures can look identical to an observer.

Correct answer: A. Periodic signal indicating a node or service is responsive

Heartbeats are periodic messages or signals used to check responsiveness. Missing heartbeats raise suspicion but alone cannot definitively prove a crash.

Correct answer: B. An ordering mechanism based on events rather than exact physical time

Logical clocks assign event-order information without depending on synchronized wall clocks. They help reason about causality in asynchronous systems.

Correct answer: C. Program order and message send-before-receive relations and their transitive consequences

The happened-before relation includes event order within a process, sending a message before its reception, and transitivity. It defines a partial order.

Correct answer: D. Timestamp of a is less than timestamp of b

Lamport clocks satisfy a → b implies L(a) < L(b), provided clock-update rules are followed. The implication does not generally hold in reverse.

Correct answer: A. No

An inequality L(a) < L(b) is not proof that event a caused event b. Concurrent events can receive different Lamport timestamps.

Correct answer: B. They can identify causal order and concurrency for represented processes

Vector timestamps store information about events observed from multiple processes. Componentwise comparison can distinguish causal precedence from concurrency.

Correct answer: C. Neither is componentwise less than or equal to the other

Two vector timestamps are incomparable when neither vector is componentwise at most the other. Under the standard vector-clock model, the corresponding events are concurrent.

Correct answer: D. A consistent global state of nodes and communication channels

A distributed snapshot records process states along with relevant messages in transit. It aims to describe a consistent global cut rather than a single perfectly synchronized instant.

Correct answer: A. Include a receive event without its causally preceding send event

If a snapshot contains a message receipt, it must also contain the preceding send. Otherwise the recorded global state violates causal consistency.

Correct answer: B. Reliable FIFO channels and a suitable marker protocol

The classical Chandy–Lamport algorithm uses marker messages on reliable FIFO channels. These assumptions allow each process to record in-transit messages consistently.

Correct answer: C. Neither happened before the other

Two events are concurrent when neither is causally before the other. Their ordering in physical time does not necessarily establish a causal relation.

Correct answer: D. Agreement on a decision despite specified failure conditions

Consensus requires processes to select an agreed decision while satisfying defined safety and liveness properties. The guarantees depend on the failure model.

Correct answer: A. Decided values must satisfy the proposal validity condition

Validity restricts the values that may be chosen, typically linking decisions to permitted proposals. Exact formulations depend on the consensus protocol.

Correct answer: B. Correct participants must not decide different values

Agreement ensures that correct participants do not decide conflicting values. It is a safety requirement that prevents diverging decisions.

Correct answer: C. Every correct participant eventually decides under the model’s assumptions

Termination is a liveness condition: under the protocol’s assumptions, each correct participant eventually reaches a decision.

Correct answer: D. The FLP impossibility result

The FLP result shows that deterministic consensus cannot guarantee termination in a fully asynchronous message-passing system if even one process may crash. It does not prohibit consensus with extra assumptions.

Correct answer: A. Fault-tolerant distributed consensus

Paxos is a family of protocols for agreeing on values despite crash faults under appropriate assumptions. Its safety relies on intersecting quorums.

Correct answer: B. Leader-based replicated logs and elections

Raft uses elected leaders to coordinate replicated log entries. Terms, elections and commitment rules help replicas agree on a consistent log.

Correct answer: C. At least three voters

More than half of five voters means at least three. Requiring three votes prevents two disjoint groups from each forming a majority.

Correct answer: D. Any two majorities share at least one member

Any two strict majorities overlap: each contains more than half the group. Shared members help carry information between decisions, subject to protocol rules.

Correct answer: A. Multiple disconnected groups act as authoritative leaders

Split brain occurs when competing partitions or nodes each act as primary. Without safeguards, both sides might accept incompatible updates.

Correct answer: B. A resource can reject writes from an outdated former leader

An increasing fencing token distinguishes a current leader from an obsolete one. Storage can reject requests carrying an older token.

Correct answer: C. Selecting a node to coordinate a replicated group

Leader election selects a coordinator for a period or term. Correct protocols also constrain when a leader may act to prevent conflicting authority.

Correct answer: D. Copying ordered state-changing entries among replicas

Log replication copies ordered state-changing commands to multiple nodes. Applying the same committed sequence lets replicas maintain consistent state.

Correct answer: A. Deterministic processing of the same ordered commands under its model

Replicas of a state machine must execute the same ordered commands deterministically, assuming equivalent initial states. Otherwise their states may diverge.

Correct answer: B. A component may behave arbitrarily including sending conflicting information

Byzantine faults allow arbitrary or inconsistent behavior, including sending different messages to different peers. They are stronger than simple crash faults.

Correct answer: C. A process halts and takes no further steps

In the crash-stop model, a failed process halts permanently and performs no later steps. Protocols must tolerate missing responses.

Correct answer: D. A failed process to restart potentially using durable state

Crash-recovery protocols allow a node to restart after failing. Durable state is often necessary to avoid violating safety after recovery.

Correct answer: A. Some state must survive crashes for safety or recovery

Persistent storage preserves essential protocol state across crashes, such as terms or acknowledged log entries. Without it, a reboot may erase commitments.

Correct answer: B. Maintaining copies of data or services across nodes

Replication keeps additional copies of data or services on separate machines. It may improve availability and fault tolerance but introduces consistency challenges.

Correct answer: C. Delay before a follower reflects leader updates

Replication lag is the delay before a follower has incorporated updates already accepted elsewhere. Reads from lagging followers can return older values.

Correct answer: D. Acknowledges after required copies confirm the update

Under synchronous replication, a write is acknowledged only after the required replicas confirm the operation. The required replica count depends on the policy.

Correct answer: A. May acknowledge a write before followers have copied it

Asynchronous replication can acknowledge a write before copies have reached all followers. This can improve latency but may allow stale replicas or lost recent updates on failure.

Correct answer: B. Replicas converge after updates stop under suitable conditions

Eventual consistency permits temporary disagreement among replicas. If updates stop and communication and reconciliation succeed, replicas are expected to converge.

Correct answer: C. Operations appear atomic respecting real-time precedence

Linearizable operations appear to take effect atomically at points between invocation and response, while respecting observed real-time precedence.

Correct answer: D. Concurrent transactions are equivalent to some serial transaction order

Serializability means concurrent transactions have an outcome equivalent to a serial ordering. It does not by itself impose real-time ordering.

Correct answer: A. Yes

Yes. A schedule may be equivalent to a serial transaction order without respecting real-time precedence. Thus serializability alone does not imply linearizability.

Correct answer: B. Requests to nonfailed nodes eventually receive responses

CAP availability requires that each request received by a nonfailed node eventually gets a response. It does not mean that the response contains the newest global value.

Correct answer: C. Linearizable consistency and complete availability

During a network partition, a system cannot guarantee both linearizable consistency and full availability for all relevant requests. CAP’s tradeoff is conditional on partitions.

Correct answer: D. Refuse or delay requests to preserve consistency

A CP design may reject or delay some operations when partitions prevent safe coordination. This preserves specified consistency guarantees at the cost of availability.

Correct answer: A. Serve requests with temporary replica divergence

An AP design may continue serving operations in separated partitions and reconcile changes later. Temporary replica differences may be visible.

Correct answer: B. Required number of participating replicas for an operation

A quorum is the minimum number of replicas required to participate in an operation. Its size affects fault tolerance and the possibility of intersecting read/write sets.

Correct answer: C. R plus W greater than N

For N replicas, read quorum R and write quorum W have unavoidable overlap when R + W > N. Overlap is helpful but does not alone ensure correct version ordering.

Correct answer: D. No

No. Intersecting quorums do not automatically establish linearizability. A protocol also needs suitable rules for version selection, concurrency and failures.

Correct answer: A. Moving service responsibility to a healthy component after a failure

Failover transfers responsibility from a failed or unavailable component to another healthy component. Detection and state recovery influence how quickly service resumes.

Correct answer: B. Continuing specified service behavior despite supported faults

Fault tolerance means meeting specified service guarantees despite faults covered by a system’s failure model. The guarantee is never independent of assumptions about what can fail.

Correct answer: C. Many similar operations over large datasets

GPUs are designed to execute many similar operations across large datasets with high throughput. Their many execution lanes are particularly useful when sufficient parallel work exists.

Correct answer: D. A platform and programming model for NVIDIA GPU computing

CUDA is NVIDIA’s GPU programming platform and model. It lets programmers launch parallel kernels and manage device memory using supported languages and libraries.

Correct answer: A. A function executed over many device threads

A GPU kernel is device code launched over a grid of threads. Different threads generally process different elements of the workload.

Correct answer: B. A group of GPU threads with defined coordination facilities

A CUDA thread block groups threads that can coordinate through block-local synchronization and shared memory. Blocks can be scheduled independently onto suitable processing units.

Correct answer: C. A group of threads scheduled together under SIMT execution

A warp is a hardware scheduling group of GPU threads operating under a SIMT model. An instruction can apply to active lanes according to execution masks.

Correct answer: D. Different lanes take different control paths causing masked execution

Warp divergence arises when lanes in one warp follow different control paths. The hardware may need to execute the paths separately, reducing efficiency.

Correct answer: A. Combining nearby lane accesses into efficient memory transactions

Coalescing combines suitable memory requests from neighboring lanes into efficient global-memory transactions. Adjacent, appropriately aligned accesses often need fewer transactions.

Correct answer: B. Fast on-chip data reuse and cooperation within a block

On-chip shared memory allows threads within a block to reuse data with less global-memory traffic. The program must synchronize accesses where needed.

Correct answer: C. Ensuring participating threads reach a point before proceeding

A block-level barrier prevents participating threads from proceeding until the required threads in that block have arrived. It is commonly used before reading data other threads wrote.

Correct answer: D. Computations per byte of data transfer

Arithmetic intensity is the number of arithmetic operations per byte moved through a specified memory level. High intensity often favors compute-bound execution.

Correct answer: A. Memory service rate or latency

A memory-bound kernel is limited mainly by memory bandwidth or latency rather than arithmetic throughput. Reducing data movement can help more than adding arithmetic units.

Correct answer: B. Arithmetic execution throughput

A compute-bound kernel is mainly limited by the arithmetic processing capacity. Optimizing instruction throughput and utilization may improve its speed.

Correct answer: C. Compute peak memory bandwidth and arithmetic intensity

The roofline model estimates attainable performance using peak compute throughput, memory bandwidth and arithmetic intensity. Performance lies below the limiting roof.

Correct answer: D. Resident active warps or threads relative to hardware capacity limits

Occupancy describes how many warps or threads are resident relative to the hardware limit. It can help hide latency, although maximum occupancy does not guarantee maximum speed.

Correct answer: A. Combining different processor types such as CPUs and accelerators

Heterogeneous systems combine processors with different capabilities, such as CPUs and GPUs. Tasks can be placed on the device best suited to each workload.

Correct answer: B. A group of networked computers cooperating on workloads or services

A cluster consists of connected computers that cooperate on tasks or services. It can pool processing, storage and networking resources.

Correct answer: C. A participating machine or execution host in a cluster

A compute node supplies CPUs, memory and possibly accelerators to execute cluster jobs. Multiple nodes communicate over the cluster network.

Correct answer: D. A service allocating queued workloads to available resources

An HPC job scheduler queues submitted workloads and allocates available processors, memory and nodes. Scheduling policies also address fairness and resource constraints.

Correct answer: A. Work submitted for scheduled execution without continuous user interaction

A batch job runs under a scheduling system without requiring continuous interactive control. Users submit requests and collect results after execution.

Correct answer: B. A saved execution state supporting restart after interruption

A checkpoint stores sufficient execution state to restart an interrupted computation. Regular checkpoints trade storage overhead for less lost work after failure.

Correct answer: C. Ability to increase or decrease provisioned resources in response to demand

Elasticity means provisioning additional resources when demand rises and reducing them when it falls. It can improve responsiveness and control unnecessary capacity costs.

Correct answer: D. Adding more processing instances or nodes

Horizontal scaling adds more service instances or machines. It distributes demand across nodes rather than making one machine larger.

Correct answer: A. Adding capacity to an existing node or instance

Vertical scaling increases the capacity of an existing machine, for example by adding CPU or memory. Hardware and service limits may restrict how far it can scale.

Correct answer: B. A software-defined execution environment emulating or exposing machine resources

A virtual machine provides an isolated guest environment with virtualized computing resources. A hypervisor typically manages underlying hardware sharing.

Correct answer: C. An isolated application environment sharing the host kernel under common OS container models

A container packages an application and its dependencies into an isolated process environment. Most common OS containers share the host kernel rather than running a separate guest kernel.

Correct answer: D. Automating deployment placement scaling and recovery

Orchestration automates placing, starting, restarting and scaling containerized services. It helps maintain the desired running state despite changes or failures.

Correct answer: A. A separately deployable service focused on a bounded function

A microservice is an independently deployable service focused on a bounded responsibility. Services communicate through APIs or messages and require coordination across the network.

Correct answer: B. Locating current network endpoints for services

Service discovery identifies the current address of a service instance. It is important when instances are frequently created, moved or replaced.

Correct answer: C. A component distributing incoming work among service instances

A load balancer directs incoming requests among service instances according to a policy. This can reduce hotspots and improve resilience when healthy instances are available.

Correct answer: D. Automatically changing resource counts or capacity based on configured signals

Autoscaling changes the number or size of resources automatically based on predefined signals such as CPU use, queue depth or request rate.

Correct answer: A. Produces intermediate key-value pairs from input records

In MapReduce, map workers transform input records into intermediate key-value pairs. Independent records can often be mapped in parallel.

Correct answer: B. Groups and transfers intermediate values by key to reducers

Shuffle groups intermediate values by key and transfers them to the relevant reducers. Network traffic and skew during this phase can become major bottlenecks.

Correct answer: C. Aggregates values for each key or key group

Reduce workers process grouped intermediate values to produce aggregated outputs. For example, a word-count reducer sums counts for one word.

Correct answer: D. Processing near data can reduce costly network transfers

Processing data near its storage location can avoid transferring large datasets across the network. The benefit depends on workload and resource availability.

Correct answer: A. A file-storage system spanning nodes while offering coordinated access

A distributed file system coordinates storage across several machines while presenting a file-oriented interface. Data blocks or chunks may be spread and replicated for resilience.

Correct answer: B. Maintaining multiple copies of data for availability or access

Replication stores several copies of data on different nodes. Copies improve redundancy and availability but require consistency and update-management policies.

Correct answer: C. Encoding data into fragments so loss of some fragments can be tolerated

Erasure codes transform data into redundant encoded fragments so the original can be recovered despite some fragments being missing. They can use less space than full replication.

Correct answer: D. Partitioning records across different storage nodes

Sharding divides a dataset into partitions held by different nodes. Effective shard keys distribute load and support efficient queries.

Correct answer: A. A partition receiving disproportionately high load

A hot partition receives substantially more requests or data than others. Its overloaded resources may determine the overall system throughput.

Correct answer: B. Replicas converge after updates cease under suitable reliability assumptions

Eventual consistency permits replicas to differ temporarily. If writes cease and updates propagate under appropriate conditions, their states converge.

Correct answer: C. Repeating it has the same intended effect as applying it once

An idempotent operation has the same intended effect when repeated as when applied once. This is useful when network retries may duplicate requests.

Correct answer: D. Duplicate effects from repeated logical requests

An idempotency key identifies repeated attempts of one logical request. A service can use the key to avoid performing a payment or other side effect twice.

Correct answer: A. Following a request across multiple services using correlated trace context

Distributed tracing propagates correlated trace identifiers across service calls. The recorded spans reveal how a single request flows through a multi-service system.

Correct answer: B. A high-percentile response time showing slowest portions of request distribution

Tail latency describes slow high-percentile responses, such as the 99th percentile. It highlights delays that averages can conceal.

Correct answer: C. An unusually slow task delaying overall parallel completion

A straggler is a task that runs much slower than its peers. A distributed job may have to wait for that one task to finish before completing.

Correct answer: D. Running a backup copy of slow work and taking an acceptable result

Speculative execution launches another copy of unusually slow work. The system can accept a valid successful result and cancel unnecessary copies, trading resources for lower tail latency.

Correct answer: A. Scheduling more runnable threads than available hardware execution slots

CPU oversubscription occurs when runnable threads exceed available execution slots. Competition can cause context switching and scheduling delays.

Correct answer: B. Multiple workers or devices perform storage operations concurrently

Parallel I/O lets multiple workers access storage concurrently. When the storage system supports it, aggregate throughput can exceed that of serial transfers.

Correct answer: C. A tool measuring where execution time or resources are consumed

A profiler measures where a program spends execution time or resources, such as CPU cycles or memory. Profiling evidence helps prioritize meaningful optimizations.

Correct answer: D. Improve total workload outcome rather than one isolated component metric

End-to-end optimization targets overall application outcomes instead of isolated component metrics. Improving one stage is not useful if it worsens total latency or throughput.