feat: US-015 - Starter tasks: Category E onboarding comprehension (2 tasks)

- CCX-onboard-041: scipy.stats API audit in pandas-dev/pandas (python-ml-stack fixture)
  - Oracle: 4 files with 'from scipy.stats import' — nanops.py, plotting/misc.py,
    plotting/hist.py, tests/groupby/test_reductions.py
  - eval.sh: file_set_match + provenance checks
  - Validity gate: VALID (gold=1.0, empty=0.0)

- CCX-onboard-050-ds: End-to-end Deployment creation flow (kubernetes-ecosystem fixture)
  - Deep Search variant — open-ended cross-repo synthesis question
  - Oracle chain: kubernetes-client-go/deployment.go (Create) →
    kubernetes/pkg/registry/apps/deployment/strategy.go (PrepareForCreate) →
    etcd/server/storage/mvcc/kvstore_txn.go (Put)
  - eval.sh: dependency_chain + provenance checks
  - criteria.json: 4 AAA quality rubric criteria for narrative quality
  - deepsearch_relevant=true in selection file
  - Validity gate: VALID (gold=1.0, empty=0.0)

- Both tasks registered in configs/selected_mcp_unique_tasks.json (8 total tasks)
- Both tasks in benchmarks/ccb_mcp_onboarding/

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: sjarmak

benchmarks/ccb_mcp_onboarding/ccx-onboard-041/environment/Dockerfile

@@ -0,0 +1,27 @@
+FROM ubuntu:22.04
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# Base tools
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    git \
+    ca-certificates \
+    curl \
+    python3 \
+    python3-pip \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /workspace
+
+# Clone local checkout repos (baseline config: agent has local access to these)
+RUN git clone --depth 1 --branch 1.6.1 https://github.com/scikit-learn/scikit-learn /workspace/scikit-learn
+
+# Initialize git identity for agent commits
+RUN git config --global user.email "agent@example.com" && \
+    git config --global user.name "Agent" && \
+    git config --global safe.directory '*'
+
+# Create log directories
+RUN mkdir -p /logs/agent /logs/verifier
+
+ENTRYPOINT []

benchmarks/ccb_mcp_onboarding/ccx-onboard-041/environment/Dockerfile.sg_only

@@ -0,0 +1,28 @@
+FROM ubuntu:22.04
+
+ENV DEBIAN_FRONTEND=noninteractive
+
+# Base tools
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    git \
+    ca-certificates \
+    curl \
+    python3 \
+    python3-pip \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /workspace
+
+# sg_only mode: no repo clones — agent must use Sourcegraph MCP to access all repos
+# Mark sg_only mode so eval.sh can detect it
+RUN touch /tmp/.sg_only_mode
+
+# Initialize git identity for agent commits
+RUN git config --global user.email "agent@example.com" && \
+    git config --global user.name "Agent" && \
+    git config --global safe.directory '*'
+
+# Create log directories
+RUN mkdir -p /logs/agent /logs/verifier
+
+ENTRYPOINT []

benchmarks/ccb_mcp_onboarding/ccx-onboard-041/instruction.md

@@ -0,0 +1,47 @@
+# Onboarding Audit: scipy.stats API Call Sites in pandas
+
+## Your Task
+
+You are a new engineer joining the pandas-dev team. As part of onboarding, you've been asked
+to audit which pandas source files have runtime dependencies on `scipy.stats`. These call sites
+are important to document because they determine where pandas degrades gracefully when scipy
+is not installed.
+
+**Specific question**: Which Python source files in `pandas-dev/pandas` contain a
+`from scipy.stats import` statement (i.e., directly import functions or classes from
+`scipy.stats` at runtime)?
+
+Include files in any part of the `pandas-dev/pandas` codebase — production code **and** test
+files. Do not include files that only mention `scipy.stats` in docstrings or comments.
+
+## Context
+
+You are onboarding to a polyrepo Python scientific stack. The local `/workspace/` contains
+`scikit-learn/scikit-learn` as a reference implementation of a well-maintained scipy consumer.
+
+**Note:** The `pandas-dev/pandas` repository is accessible via Sourcegraph MCP tools:
+- `pandas-dev/pandas` (dataframe-library)
+- `numpy/numpy` (array-computing)
+- `scipy/scipy` (scientific-computing)
+
+## Output Format
+
+Create a file at `/workspace/answer.json` with your findings:
+
+```json
+{
+  "files": [
+    {"repo": "pandas-dev/pandas", "path": "relative/path/to/file.py"}
+  ],
+  "text": "Narrative summary of your findings, citing the repos and file paths."
+}
+```
+
+List all files that contain `from scipy.stats import`. Your answer is evaluated against
+a closed-world oracle — completeness matters.
+
+## Evaluation
+
+Your answer will be scored on:
+- **File recall and precision**: Did you find all pandas files that `from scipy.stats import`?
+- **Provenance**: Does your narrative cite the repo and file paths found?

benchmarks/ccb_mcp_onboarding/ccx-onboard-041/task.toml

@@ -0,0 +1,28 @@
+version = "1.0"
+
+[metadata]
+name = "CCX-onboard-041"
+description = "Audit scipy.stats API call sites in pandas"
+license = "Apache-2.0"
+
+[task]
+id = "CCX-onboard-041"
+repo = "scikit-learn/scikit-learn"
+category = "onboarding-comprehension"
+language = "python"
+difficulty = "medium"
+time_limit_sec = 900
+mcp_suite = "ccb_mcp_onboarding"
+use_case_id = 41
+repo_set_id = "python-ml-stack"
+mcp_unique = true
+
+[verification]
+type = "eval"
+command = "bash /tests/eval.sh"
+
+reward_type = "score"
+description = "Audit scipy.stats API call sites in pandas"
+
+[environment]
+build_timeout_sec = 600.0

benchmarks/ccb_mcp_onboarding/ccx-onboard-041/tests/eval.sh

@@ -0,0 +1,68 @@
+#!/bin/bash
+# eval.sh — MCP-unique benchmark evaluator for CCX-onboard-041
+# Exit-code-first (SWE-Factory pattern):
+#   exit 0 — agent produced useful output (composite score > 0)
+#   exit 1 — total failure (composite score == 0 or missing answer)
+#
+# Writes /logs/verifier/reward.txt with the composite score [0.0, 1.0]
+
+set -euo pipefail
+
+TASK_ID="CCX-onboard-041"
+ANSWER_PATH="/workspace/answer.json"
+TASK_SPEC_PATH="/tests/task_spec.json"
+ORACLE_CHECKS="/tests/oracle_checks.py"
+REWARD_PATH="/logs/verifier/reward.txt"
+
+mkdir -p /logs/verifier
+
+echo "=== CCX-onboard-041 evaluator ==="
+echo "Task spec: $TASK_SPEC_PATH"
+echo "Answer:    $ANSWER_PATH"
+echo ""
+
+# sg_only mode guard: restore full repo if verifier wrapper exists
+if [ -f /tmp/.sg_only_mode ] && [ -f /tests/sgonly_verifier_wrapper.sh ]; then
+    echo "sg_only mode: sourcing verifier wrapper..."
+    source /tests/sgonly_verifier_wrapper.sh
+fi
+
+# Verify answer file exists
+if [ ! -f "$ANSWER_PATH" ]; then
+    echo "ERROR: answer.json not found at $ANSWER_PATH"
+    echo "0.0" > "$REWARD_PATH"
+    exit 1
+fi
+
+# Validate answer is valid JSON
+if ! python3 -c "import json; json.load(open('$ANSWER_PATH'))" 2>/dev/null; then
+    echo "ERROR: answer.json is not valid JSON"
+    echo "0.0" > "$REWARD_PATH"
+    exit 1
+fi
+
+echo "answer.json found and valid JSON"
+
+# Run oracle checks
+if [ ! -f "$ORACLE_CHECKS" ]; then
+    echo "ERROR: oracle_checks.py not found at $ORACLE_CHECKS"
+    echo "0.0" > "$REWARD_PATH"
+    exit 1
+fi
+
+echo "Running oracle checks..."
+SCORE=$(python3 "$ORACLE_CHECKS" --answer "$ANSWER_PATH" --spec "$TASK_SPEC_PATH" --verbose 2>&1 | tee /dev/stderr | tail -1)
+
+# Validate score is a number
+if ! echo "$SCORE" | python3 -c "import sys; float(sys.stdin.read().strip())" 2>/dev/null; then
+    echo "ERROR: oracle_checks.py did not return a valid score: $SCORE"
+    echo "0.0" > "$REWARD_PATH"
+    exit 1
+fi
+
+echo ""
+echo "Composite score: $SCORE"
+echo "$SCORE" > "$REWARD_PATH"
+
+# Exit based on score (SWE-Factory exit-code-first pattern)
+python3 -c "import sys; sys.exit(0 if float('$SCORE') > 0 else 1)"

benchmarks/ccb_mcp_onboarding/ccx-onboard-041/tests/oracle_answer.json

@@ -0,0 +1,14 @@
+{
+  "files": [
+    {"repo": "pandas-dev/pandas", "path": "pandas/core/nanops.py"},
+    {"repo": "pandas-dev/pandas", "path": "pandas/plotting/_matplotlib/misc.py"},
+    {"repo": "pandas-dev/pandas", "path": "pandas/plotting/_matplotlib/hist.py"},
+    {"repo": "pandas-dev/pandas", "path": "pandas/tests/groupby/test_reductions.py"}
+  ],
+  "text": "Found 4 files in pandas-dev/pandas that contain 'from scipy.stats import': pandas/core/nanops.py (imports kendalltau and spearmanr for correlation methods), pandas/plotting/_matplotlib/misc.py (imports gaussian_kde for KDE plots), pandas/plotting/_matplotlib/hist.py (imports gaussian_kde for histogram density plots), and pandas/tests/groupby/test_reductions.py (imports sem for test assertions). These files represent all runtime call sites where pandas depends on scipy.stats functions.",
+  "_metadata": {
+    "oracle_type": "file_set_match",
+    "discovery_method": "sourcegraph_keyword_search",
+    "query": "repo:^github.com/pandas-dev/pandas$ \"from scipy.stats import\" file:pandas/"
+  }
+}