EdStem
Fetch and organize EdStem discussion threads from any course or institution with automatic staff/student differentiation.
Quick Start
Fetch recent threads for any course:
CODEBLOCK0
Examples:
CODEBLOCK1
Finding Your Course ID
To find your EdStem course ID:
- 1. Log into EdStem and navigate to your course
- Look at the URL: INLINECODE0
- The number in the URL is your course ID
Alternatively, use the API to list your courses:
CODEBLOCK2
What Gets Fetched
For each course:
- - threads.json - Full thread list with metadata
- thread-XXX.md - Individual threads formatted as markdown
- Thread title, category, timestamps
- Original post content
- All answers and comments
-
[STAFF] or
[STUDENT] tags on every post
Features
- - Institution-agnostic: Works with any school using EdStem
- Staff differentiation: Clearly marks instructor/TA posts vs student posts
- Structured output: Markdown format for easy reading and searching
- API-based: Uses EdStem's official API (no scraping)
- Flexible output: Choose your own output directory and organization scheme
Authentication
The skill uses a bearer token stored in the Python script. To use with your own account:
- 1. Log into EdStem in your browser
- Open Developer Tools → Network tab
- Reload any EdStem page
- Find an API request and copy the
Authorization: Bearer ... token - Update
ED_TOKEN in INLINECODE3
Current token location: Line 20 in INLINECODE4
If API calls fail (401 Unauthorized), your token likely expired and needs refresh.
Scripts
fetch-edstem.py (recommended)
Full-featured Python script with markdown formatting and staff/student differentiation.
Usage:
CODEBLOCK3
Options:
- -
output_dir - Where to save threads (default: ./edstem-<course_id>) - INLINECODE7 - Display name for the course
- INLINECODE8 - Number of threads to fetch (default: 10)
Features:
- - Fetches thread metadata and full details
- Full markdown formatting with answers and comments
- Automatic staff role detection
- JSON cache of thread list
- Auto-creates output directory
fetch-edstem.sh (lightweight alternative)
Bash/curl version for raw JSON fetching without dependencies.
Usage:
CODEBLOCK4
Outputs:
- - Raw JSON files for each thread
- Requires manual formatting or post-processing
Common Workflows
Check for new posts
CODEBLOCK5
Sync multiple courses
CODEBLOCK6
Review recent activity
After fetching, check the markdown files:
CODEBLOCK7
Search across threads
CODEBLOCK8
Output Structure
CODEBLOCK9
Each markdown file contains:
- - Thread metadata (number, title, category, timestamps)
- Original post with author role
- All answers (sorted, with role tags)
- All comments (with role tags)
Integration Examples
With LLM agents
CODEBLOCK10
Automated monitoring
CODEBLOCK11
Custom organization
CODEBLOCK12
Troubleshooting
401 Unauthorized: Token expired. Re-authenticate and update ED_TOKEN in the script.
Course not found: Verify the course ID and that your account has access.
Empty threads: Check that the course has discussion posts and you're enrolled.
Rate limiting: EdStem may rate-limit API requests. Add delays between fetches if needed.
Contributing
This skill is open-source and institution-agnostic by design. Improvements welcome:
- - Better content parsing (EdStem uses XML-based document format)
- Support for filtering by category or date range
- Incremental sync (only fetch new threads)
- Export to other formats (JSON, HTML, etc.)
Version History
- - 1.1.0 - Made institution-agnostic with flexible parameters
- 1.0.0 - Initial release
EdStem
从任何课程或机构获取并整理EdStem讨论线程,自动区分教职工/学生身份。
快速开始
获取任意课程的近期线程:
bash
cd /home/axel/.openclaw/workspace/skills/edstem/scripts
python3 fetch-edstem.py <课程ID> [输出目录] [--course-name 课程名称]
示例:
bash
获取到默认目录 (./edstem-<课程ID>)
python3 fetch-edstem.py 92041
获取到指定目录
python3 fetch-edstem.py 92041 ./machine-learning
指定课程名称以获得更清晰的输出
python3 fetch-edstem.py 92041 --course-name Machine Learning
同时指定目录和课程名称
python3 fetch-edstem.py 92041 ./ml-course --course-name Machine Learning
获取更多线程(默认为10个)
python3 fetch-edstem.py 92041 --limit 25
查找课程ID
要找到你的EdStem课程ID:
- 1. 登录EdStem并导航到你的课程
- 查看URL:https://edstem.org/us/courses/<课程ID>/
- URL中的数字即为你的课程ID
或者,使用API列出你的课程:
bash
curl -H Authorization: Bearer 你的令牌 https://us.edstem.org/api/user | jq .courses[] | {id: .course.id, name: .course.name}
获取内容
每个课程获取的内容:
- - threads.json - 包含元数据的完整线程列表
- thread-XXX.md - 格式化为Markdown的单个线程
- 线程标题、分类、时间戳
- 原始帖子内容
- 所有回答和评论
- 每个帖子上的
[STAFF] 或
[STUDENT] 标签
功能特点
- - 机构无关性:适用于任何使用EdStem的学校
- 教职工区分:清晰标记教师/助教帖子与学生帖子
- 结构化输出:Markdown格式,便于阅读和搜索
- 基于API:使用EdStem官方API(无需爬取)
- 灵活输出:可选择自己的输出目录和组织方案
身份验证
该技能使用存储在Python脚本中的Bearer令牌。要使用你自己的账户:
- 1. 在浏览器中登录EdStem
- 打开开发者工具 → 网络标签
- 重新加载任意EdStem页面
- 找到API请求并复制 Authorization: Bearer ... 令牌
- 更新 scripts/fetch-edstem.py 中的 ED_TOKEN
当前令牌位置: scripts/fetch-edstem.py 第20行
如果API调用失败(401未授权),你的令牌可能已过期,需要刷新。
脚本
fetch-edstem.py(推荐)
功能完整的Python脚本,支持Markdown格式化和教职工/学生区分。
用法:
bash
python3 scripts/fetch-edstem.py <课程ID> [输出目录] [选项]
选项:
- - output_dir - 保存线程的目录(默认:./edstem-<课程ID>)
- --course-name NAME - 课程的显示名称
- --limit N - 获取的线程数量(默认:10)
功能特点:
- - 获取线程元数据和完整详情
- 完整的Markdown格式化,包含回答和评论
- 自动检测教职工角色
- 线程列表的JSON缓存
- 自动创建输出目录
fetch-edstem.sh(轻量级替代方案)
Bash/curl版本,用于无依赖的原始JSON获取。
用法:
bash
bash scripts/fetch-edstem.sh <课程ID> [输出目录]
输出:
- - 每个线程的原始JSON文件
- 需要手动格式化或后处理
常见工作流程
检查新帖子
bash
python3 scripts/fetch-edstem.py 92041 ~/courses/ml-spring-2025
同步多个课程
bash
创建简单的同步脚本
for course in 92041:machine-learning 94832:advanced-rl; do
IFS=: read -r id name <<< $course
python3 scripts/fetch-edstem.py $id ~/courses/$name --course-name $name
done
查看近期活动
获取后,检查Markdown文件:
bash
ls -lt ./edstem-92041/*.md | head
cat ./edstem-92041/thread-001.md
跨线程搜索
bash
grep -r 梯度下降 ./edstem-92041/*.md
输出结构
<输出目录>/
├── threads.json # 线程元数据
├── thread-001.md # 单个线程
├── thread-002.md
└── ...
每个Markdown文件包含:
- - 线程元数据(编号、标题、分类、时间戳)
- 原始帖子及作者角色
- 所有回答(排序,带角色标签)
- 所有评论(带角色标签)
集成示例
与LLM代理集成
bash
获取线程并用你的代理进行分析
python3 fetch-edstem.py 92041 ./course-data
然后:总结 ./course-data/ 中最常见的问题
自动监控
bash
添加到cron进行每日同步
0 9
* cd /path/to/skills/edstem/scripts && python3 fetch-edstem.py 92041 ~/courses/ml
自定义组织
bash
按学期和机构组织
python3 fetch-edstem.py 92041 ~/school/stanford/2025-spring/cs229
python3 fetch-edstem.py 94832 ~/school/mit/2025-spring/6.7920
故障排除
401未授权: 令牌已过期。重新认证并更新脚本中的 ED_TOKEN。
未找到课程: 验证课程ID以及你的账户是否有访问权限。
空线程: 检查课程是否有讨论帖子以及你是否已注册。
速率限制: EdStem可能对API请求进行速率限制。如有需要,在获取之间添加延迟。
贡献
该技能是开源的,设计为机构无关。欢迎改进:
- - 更好的内容解析(EdStem使用基于XML的文档格式)
- 支持按分类或日期范围过滤
- 增量同步(仅获取新线程)
- 导出为其他格式(JSON、HTML等)
版本历史
- - 1.1.0 - 通过灵活参数实现机构无关性
- 1.0.0 - 初始版本