Graduate Seminar: Compact Objects in Dense Stellar Systems
Omega Centauri Society Problem Sets

Problem Set 3 — Bayesian Null-Result Analysis

Estimated time: 3–4 hours Total marks: 40 Tools: workflow-null-result omegacentauri.me/pset-3-null-result.html
Learning objectives
  1. Derive single-transmitter detection probability from the Civiletti (2025) geometric model, extended with OCS survey-fraction factors
  2. Compute the Bayes factor K and posterior update from a null SETI result
  3. Convert a detection probability to a null-result significance in Gaussian σ units using the probit function
  4. Compare the inferential strength of different survey architectures (VLA/COSMIC vs. SKA projection)
Adopted values (used throughout PS 1–4)
  1. Distance to OC: d = 5.49 kpc = 17,900 ly (oMEGACat VI, arXiv:2503.04903). The survey depths δ below are quoted in light years and are measured from that distance.
  2. Milky Way scale in the Civiletti model: RMW = 52,000 ly.
  3. Prior on a transmitter existing, where not otherwise stated: P₀ = 0.5 (log₁₀ P₀ = −0.301).

Background

A null result in a SETI survey carries information. Its strength depends on how likely a detection would have been if a signal existed. Civiletti (2025) provides a geometric model for the single-transmitter detection probability: a survey of depth δ out of a Galaxy of radius RMW catches a passing signal with probability ≈ 0.6·δ/RMW (averaged over galactic geometries). Source ruling (2026-09-25): the published paper (M. Civiletti, "Quantifying the Fermi paradox via passive SETI: a general framework", Open J. Astrophysics, DOI 10.33232/001c.154771, arXiv:2505.00062) contributes the 0.6·δ/R core and its pairing with the Drake-style N = 𝒩δ/c; the survey-fraction factors fsky and ffreq used throughout this problem set are OCS extensions, not part of the published model, and are labelled as such below.

Single-transmitter detection probability (Civiletti 2025 core 0.6·δ/R; f_sky, f_freq = OCS extensions): P_single = f_sky · f_freq · min(1, 0.6 · δ / R_MW) where: f_sky = fraction of sky covered by the survey beam f_freq = fraction of monitored frequency space δ = survey depth (maximum distance at which a signal at the assumed EIRP would be detectable), ly R_MW = 52,000 ly (Milky Way half-light radius) For N independent transmitters, the probability of ≥ 1 detection: P_detect = 1 − (1 − P_single)^N Bayesian update from a non-detection: K = 1 − P_detect (Bayes factor for H₀: no signal) P_post = K · P₀ / (K · P₀ + (1 − P₀)) where P₀ is the prior probability that a transmitter exists. Null-result significance in Gaussian σ: σ_null = Φ⁻¹(P_detect) (probit of P_detect)

Problems

1. Hypothetical VLA/COSMIC-style null result: detection probability

Hypothetical exercise. No published 2025 VLA/COSMIC survey of Omega Centauri with these parameters exists; the numbers below define a worked example (chosen to resemble a commensal VLA + COSMIC-style observation of the cluster) so you can practise the machinery. Parameters:

  • fsky = 0.75 (fraction of OC's angular area within primary beams)
  • ffreq = 0.10 (L and S band combined, relative to full radio spectrum)
  • δ = 1,000 ly (sensitivity limit for EIRP = 1014 W at 17,900 ly distance)
  • N = 10 (number of assumed transmitters in OC)
  1. Compute Psingle, Pdetect, and σnull by hand for these parameters. Show your work. [8 marks]
  2. Now use the OCS Null-Result workflow (VLA/COSMIC preset) to verify your answer. Do the numbers agree? If they differ by more than 5%, explain why. [3 marks]
  3. Is a σnull < 1 result strong evidence against the existence of an EIRP = 1014 W transmitter in OC? Why or why not? [4 marks]
↗ Null-Result Workflow — VLA/COSMIC preset (N=10, δ=1000 ly, f_sky=0.75, f_freq=0.10)
Show solution
Part (a): P_single = f_sky · f_freq · min(1, 0.6 · δ / R_MW) = 0.75 × 0.10 × min(1, 0.6 × 1000 / 52000) = 0.075 × min(1, 0.01154) = 0.075 × 0.01154 = 8.65 × 10⁻⁴ P_detect = 1 − (1 − 8.654×10⁻⁴)¹⁰ = 1 − exp(10 × ln(0.9991346)) = 1 − exp(−8.658×10⁻³) = 1 − 0.991380 = 0.00862 or 0.86% σ_null = Φ⁻¹(P_detect) = Φ⁻¹(0.00862) = −2.38 Sign convention: σ_null is the probit of P_detect, not of 1 − P_detect, so a survey with a low chance of detection returns a negative value. A σ_null of −2.38 means P_detect sits 2.38 Gaussian σ below the median, that is, far below the coverage at which a non-detection would carry weight. A survey that would have detected the signal half the time returns σ_null = 0, and only surveys with P_detect > 0.5 return positive values. This is a weak null result: with a transmitter present, the survey had under a 1% chance of seeing it.
Part (b): The workflow should agree to within rounding. Any discrepancy >5% likely reflects a different convention for the probit function or a rounding of logN = log₁₀(10) = 1.0 exactly.

Part (c): No. A P_detect of <1% means that even if a 10¹⁴ W transmitter exists in OC, this hypothetical survey had only a 0.87% chance of detecting it. Not detecting it therefore provides very little Bayesian evidence against the transmitter hypothesis. The posterior probability barely changes from the prior. Only surveys that would have high P_detect (say >50%) if a transmitter existed can strongly constrain the hypothesis.
2. SKA projection and observational design

The Square Kilometre Array (SKA1-Mid) is projected to reach EIRP = 1013 W sensitivity at 17,900 ly with the following parameters:

  • fsky = 0.95 (near-complete sky coverage with commensal mode)
  • ffreq = 0.40 (broad 350 MHz – 14 GHz coverage)
  • δ = 10,000 ly (improved EIRP sensitivity). Note: the deeplink below sets log₁₀ δ = 4 explicitly. The tool'"'"'s built-in SKA Projection preset button uses δ = 1,000 ly and will not reproduce these answers.
  • N = 100 (log₁₀ N = 2; optimistic Drake estimate)
  1. Compute Pdetect for the SKA projection. How many σ does this correspond to? [6 marks]
  2. Compare the Bayes factor K for the SKA projection vs. VLA/COSMIC. By what factor does the SKA non-detection more strongly update the prior? [4 marks]
  3. What survey ffreq would be needed to achieve σnull ≥ 2.0, holding all other SKA parameters fixed? Solve analytically. [5 marks]
↗ Null-Result Workflow — SKA projection (N=100, δ=10,000 ly)
Show solution
Part (a): P_single = 0.95 × 0.40 × min(1, 0.6 × 10000 / 52000) = 0.38 × min(1, 0.1154) = 0.38 × 0.1154 = 0.04385 P_detect = 1 − (1 − 0.04385)¹⁰⁰ = 1 − (0.95615)¹⁰⁰ (0.95615)¹⁰⁰ = exp(100 × ln(0.95615)) = exp(100 × (−0.04482)) = exp(−4.482) = 0.01127 P_detect = 1 − 0.01127 = 0.9887 σ_null = Φ⁻¹(0.9887) ≈ 2.28σ The SKA null result would correspond to ~2.3σ significance.

Part (b): K_SKA = 1 − 0.9887 = 0.0113; K_VLA ≈ 1 − 0.0087 ≈ 0.9913. The Bayes factor ratio is K_SKA/K_VLA = 0.0113/0.9913 ≈ 0.0114. The SKA non-detection is ~88× more powerful evidence against the transmitter hypothesis (posterior updates 88× more strongly per unit prior).

Part (c): We want P_detect = Φ(2.0) = 0.9772, so (1 − P_single)¹⁰⁰ = 0.0228. Taking ln: 100 × ln(1 − P_single) = ln(0.0228) = −3.78. So ln(1 − P_single) = −0.0378, giving 1 − P_single = 0.9629, P_single = 0.0371. Then f_freq = P_single / (f_sky × min(1, 0.6 δ/R_MW)) = 0.0371 / (0.95 × 0.1154) = 0.0371 / 0.1096 ≈ 0.338 — so f_freq ≥ 0.34 (34% frequency coverage) is needed.
3. Bayesian prior sensitivity

The posterior update depends critically on the prior P₀. Use the SKA projection (P_detect ≈ 0.989) and compute the posterior Ppost for three priors, by hand and then verify with the tool.

  1. Compute Ppost for P₀ = 0.5 (agnostic), P₀ = 0.1 (skeptical), and P₀ = 0.01 (very skeptical). Show how dramatically the posterior changes with the prior. [5 marks]
  2. For P₀ = 0.01, a skeptic notes: "Even after the SKA null result, the posterior is still close to the prior — it barely changes." Explain mathematically why this happens when P₀ is very small. [5 marks]
Show solution
Part (a): K = 1 − P_detect = 0.0113. P_post = K · P₀ / (K · P₀ + (1 − P₀)) P₀ = 0.5: P_post = 0.0113 × 0.5 / (0.0113 × 0.5 + 0.5) = 0.00565 / (0.00565 + 0.5) = 0.00565 / 0.5057 ≈ 0.0112 → dropped from 50% to 1.1% P₀ = 0.1: P_post = 0.0113 × 0.1 / (0.0113 × 0.1 + 0.9) = 0.00113 / (0.00113 + 0.9) = 0.00113 / 0.9011 ≈ 0.00125 → dropped from 10% to 0.125% P₀ = 0.01: P_post = 0.0113 × 0.01 / (0.0113 × 0.01 + 0.99) = 1.13×10⁻⁴ / (1.13×10⁻⁴ + 0.99) = 1.13×10⁻⁴ / 0.99011 ≈ 1.14×10⁻⁴ → barely changed from 1%
Part (b): When P₀ ≪ 1, the denominator is dominated by (1 − P₀) ≈ 1. So P_post ≈ K · P₀ / 1 = K · P₀. This means the posterior is simply K times the prior. With K = 0.0113 and P₀ = 0.01, P_post ≈ 0.0113 × 0.01 = 1.13 × 10⁻⁴, a factor of 88 reduction. The skeptic's complaint measures the change in absolute probability, which is 0.0100 − 0.00011 = 0.0099, or one percentage point, and by that measure nothing much happened. The ratio tells a different story: P_post/P₀ = K = 0.0113, an 88-fold reduction, and that ratio is independent of P₀. The two descriptions are of the same update, and which one sounds impressive depends only on whether the prior was large enough for a large ratio to move a large absolute quantity. A Bayesian analysis cannot make a hypothesis plausible when the prior is very small; it can only multiply whatever probability was already there.

References