experiment-advisor: Скилл для планирования экспериментов

--- name: experiment-advisor description: > Given a research goal, use existing skills, training reports, and references to propose a small, concrete experiment plan. metadata: short-description: "Plan next experiments using past knowledge" --- # Skill: experiment-advisor ## When to use Use this skill when: - The user message starts with `<advise>`, or - The user clearly asks "What should we try next?" in an experiment context. ## Inputs - User goal (including constraints like run budget, hardware). - Optional references to files (configs, reports, logs) mentioned in the goal. ## Behavior 1. **Understand the goal** - Parse which models, datasets, tasks, and constraints are involved. - If unclear, ask a concise clarifying question. 2. **Gather context** - Scan `.codex/skills/` for **relevant result skills**: - Matching models, datasets, or task types. - Read relevant `training_reports/*.md`: - Start with the most recent ones that mention these models/datasets. - Skim the last few days of `references/experiment-log.md`. - If the goal mentions an error, look up matching patterns in `references/troubleshooting.md`. 3. **Propose a plan** - Design **2–5 experiments**, each with: - Input data selection (possibly mixtures). - Model / architecture details (at a high level). - Key hyperparameters (LR, batch size, epochs, etc.). - Any relevant variation (e.g. different mixture ratios). - Respect run/hardware constraints. - When applicable, **reuse defaults** from existing skills instead of inventing new ones. 4. **Output format** - Start with a short natural-language summary. - Then provide a markdown table with the experiments: | id | description | key_differences | notes | |----|-------------|-----------------|-------| - Optionally propose file paths for configs or reports to create. - If errors were mentioned, explicitly call out how this plan avoids known failure patterns from `troubleshooting.md`. 5. **Logging** - When useful, append a short entry to `references/experiment-log.md` summarizing the newly proposed plan (only if the user agrees). - Include a short "General description" line in the entry for non-technical context.