Commit Graph

4 Commits

Author SHA1 Message Date
zbc
455489cc89 1. 修改上报队列机制,确保上报失败有可靠的重试机制,不会静默失败;2.修改conversation上报遗漏的问题; 2026-05-27 09:48:38 +08:00
zbc
c8586ee6ee 1. 修复匿名上报url错误; 2.增加注释; 3.增加异常场景处理提升获取对话diff的健壮性“ 2026-05-26 16:39:19 +08:00
林凯90331
12caa6706b perf(rawDump): reduce CPU usage and fix worker spawn reliability
Add session messages and repo info caching to avoid repeated JSONL
parsing and git subprocess spawning across tasks.

Share state across all tasks in a single batch to minimize file lock
contention and JSON parse overhead.

Convert queue operations from sync to async to prevent event loop
blocking in the main process.

Restore inline batch worker fallback when external spawn fails,
and increase worker interval from 30s to 2min to reduce CPU impact.

Add 5s debounce per session+messageID to prevent duplicate enqueues.

Improve binary mode spawn reliability by falling back to node runtime
when bun is not available.

Signed-off-by: 林凯90331 <90331@sangfor.com>
Co-authored-by: CoStrict <zgsm@sangfor.com.cn>
2026-05-14 11:07:02 +08:00
林凯90331
f3ece2f469 refactor(rawDump): prevent 429 with queue + batch worker
Replace per-turn detached workers with a file-backed queue consumed by
a single long-running batch worker (30s interval, 0-10s jitter).

- queue.ts, batchWorker.ts: file queue with pid-based lock for worker
  singleton; tasks deduped by sessionID:messageID before processing
- worker.ts: retry with backoff (5s, 10s) for 429 and network errors;
  update commit state per upload so partial failures resume cleanly;
  export auth, loadSessionMessages, getSessionDirectory and upload*
  helpers for batchWorker reuse
- git.ts: cap commit log at 50 entries within last 7 days; pause 500ms
  every 10 commits to spread load
- worker.ts: resolve session jsonl via ~/.claude/projects/{normalized}
  with fallbacks, scanning for the file containing the target session
- logger.ts: file + stderr logger gated by CSC_RAW_DUMP_DEBUG, default
  silent
- sessionDataUploader.ts: implement createSessionTurnUploader to pick
  the last assistant message; query.ts fires uploadSessionTurn after
  query_api_streaming_end (non-blocking)

Signed-off-by: 林凯90331 <90331@sangfor.com>
Co-authored-by: CoStrict <zgsm@sangfor.com.cn>
2026-05-07 21:45:20 +08:00