PA Eval Skill

Minimum Model

Any model for filling in templates. Use a medium model for trend analysis and recommendations.

---

When to Run

• - Weekly self-eval: Every 7 days. Run automatically.
• On owner correction: Log the correction immediately, then re-score the affected dimension.
• Monthly report: At the end of each month, aggregate all weekly evals.
• On demand: If owner asks "how am I doing?" → generate current eval on the spot.

---

Scoring Dimensions

Score each 1–5:

Dimension	What to Measure
Execution	Tasks completed without reminders
Accuracy

Results are correct and complete |
| Speed | Response time is fast |
| Proactivity | Acts without being asked |
| Communication | Concise and context-appropriate |
| Memory | Remembers context across sessions |
| Tool Use | Tools used correctly and efficiently |
| Judgment | Knows when to act vs. when to ask |

Score meanings:

• - 5 = Consistently exceeds expectations
• 4 = Meets expectations with minor gaps
• 3 = Acceptable but basic
• 2 = Frequent gaps or errors
• 1 = Fails basic expectations

Total: Max 40 points.
Grade: A (36–40), B (28–35), C (20–27), D (<20)

---

Weekly Self-Evaluation

Save to .learnings/eval/YYYY-MM-DD.md.

CODEBLOCK0

Create the File

CODEBLOCK1

---

Owner Feedback Signals

Log these automatically when detected:

Signal	Action
👍 reaction	Log +1 positive
👎 reaction

Log -1 negative, record the correction |
| "תודה" / "great" / "perfect" | Log +1 positive |
| "wrong" / "fix this" / "לא טוב" | Log -1, record the correction |
| Owner re-asks the same question | Log -1 memory gap |
| Owner does the task themselves | Log -1 initiative gap |
| Owner surprised by proactive action | Log +2 proactivity |

Rule: If a signal appears → log it immediately. Don't batch feedback signals.

---

Monthly Report Format

CODEBLOCK2

---

Benchmark Tests (Run Monthly)

Task Completion Rate

• - Count tasks assigned in last 30 days.
• Count completed without follow-up.
• Formula: INLINECODE1
• Target: >90%

Accuracy Rate

• - Count tasks that required correction.
• Formula: INLINECODE2
• Target: >95%

Memory Retention

• - Ask about something discussed 7+ days ago.
• Pass if recalled correctly, Fail if missed.
• Target: >80%

---

Cost Tips

• - Cheap: Filling in the weekly template — any small model works.
• Expensive: Trend analysis and pattern detection across multiple evals — use a medium model.
• Batch: Review all weekly evals at once during the monthly report, not one by one.
• Avoid: Don't re-score historical weeks — score in real time and save to file.