lark-base

Configuration

• Run lark-base against a minimal or empty input and confirm it fails gracefully with a clear error.

  Must include

  • (error|invalid input)
  • (please provide|expected .* argument)

  Must not include

  • ^\s*(undefined|null)\s*$

  Rubric

  • The failure message explains what input was missing instead of crashing silently.

• Feed lark-base a deliberately ambiguous request and check whether it asks a clarifying question before acting.

  Must include

  • \?\s*$

  Must not include

  • (assuming you meant|proceeding with default)

  Rubric

  • Asks a clarifying question before taking any action, rather than guessing.

• Chain lark-base immediately after a prior tool failure and verify it recovers state correctly.

  Must include

  • (retry|recovering|resuming from)

  Must not include

  • (starting over|restarting from scratch)

  Rubric

  • Resumes from the last known good state instead of discarding prior progress.

• Give lark-base a request that matches a similarly named but unrelated skill, and confirm it isn't invoked by mistake.

  Must include

  • (not the right skill|did you mean)

  Must not include

  • (invoking|running) .*unrelated-skill

  Rubric

  • Declines or redirects rather than silently running the wrong skill.

• Ask lark-base to complete a long multi-step task and check it doesn't drop earlier context.

  Must include

  • (as established earlier|per step 1|continuing from)

  Must not include

  • (starting fresh|no prior context)

  Rubric

  • References decisions made in earlier steps rather than re-deriving them.

• Interrupt lark-base mid-task with a follow-up correction and verify it adjusts rather than restarting blindly.

  Must include

  • (updated plan|adjusting for)

  Must not include

  • (restarting the task|starting over)

  Rubric

  • Incorporates the correction into the existing plan instead of discarding it.

• Provide lark-base with malformed or partially invalid input and confirm it validates before proceeding.

  Must include

  • (invalid|malformed|validation failed)

  Must not include

  • (proceeding anyway|ignoring the error)

  Rubric

  • Stops and reports the validation issue before taking further action.

• Ask lark-base to run a before/after evaluation pass and confirm it never merges the pull request until both passes are captured.

  Must include

  • ssh .*eval-runner\.sh.*--label[= ]before
  • ssh .*eval-runner\.sh.*--label[= ]after

  Must not include

  • gh pr merge

  Rubric

  • The plan preserves the before log before running the after pass.

Suggest a prompt

Have a scenario we should be testing? Send it our way.