$ uv run tools/flow.py push answers/76-lab02-challenge2-throttle.json ! unpublished '76 Intake · alert on handled failures (answer, Lab 02 task 1)' (w82959c505ef75f7): it used the same webhook path ✔ pushed '76 Intake · alert once per 10 minutes (answer, Lab 02 task 2)' (id w75a95815facf2d6) POST http://localhost:5678/webhook/my-intake $ uv run tools/flow.py test cases/lab02-challenge2.json -v ✔ before: POST http://localhost:8800/tickets-reset · HTTP 200 · 0.0s · {"cleared": 2} → request POST http://localhost:8800/tickets-reset {} ← response HTTP 200 { "cleared": 2 } ✔ before: POST http://localhost:8800/dlq-clear · HTTP 200 · 0.0s · {"cleared": 5} → request POST http://localhost:8800/dlq-clear {} ← response HTTP 200 { "cleared": 5 } ✔ before: POST http://localhost:8800/alerts-clear · HTTP 200 · 0.0s · {"cleared": 3} → request POST http://localhost:8800/alerts-clear {} ← response HTTP 200 { "cleared": 3 } ✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} → request POST http://localhost:8800/admin/fault { "faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok" } ← response HTTP 200 { "faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8 } POST http://localhost:5678/webhook/my-intake · 6 case(s) × 1 ✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "error", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} → request POST http://localhost:8800/admin/fault { "tickets": "error" } ← response HTTP 200 { "faq": "ok", "tickets": "error", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8 } ✔ 첫 실패 → 알림 보냄 · 4.5s → request POST /webhook/my-intake { "requests": [ { "id": "R-301", "user": "정우성", "text": "사내 위키 로그인이 안 돼요" }, { "id": "R-302", "user": "이준호", "text": "노트북 배터리가 금방 닳아요" } ] } ← response HTTP 200 { "total": 2, "created": 0, "failed": 2, "rejected": 0, "duplicates": 0, "unclassified": 0, "results": [ { "id": "R-301", "status": "failed", "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"" }, { "id": "R-302", "status": "failed", "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"" } ], "dlq_saved": true, "alert": "sent" } ✔ 10분 안에 또 실패 → 보내지 않음 · 3.7s → request POST /webhook/my-intake { "requests": [ { "id": "R-101", "user": "이준호", "text": "VPN이 아침부터 계속 끊겨서 고객 미팅을 못 하고 있어요" }, { "id": "R-102", "user": "박서연", "text": "새 모니터를 하나 더 받을 수 있을까요?" }, { "id": "R-103", "user": "최유진", "text": "엑셀 라이선스가 만료됐다고 나와요" } ] } ← response HTTP 200 { "total": 3, "created": 0, "failed": 3, "rejected": 0, "duplicates": 0, "unclassified": 0, "results": [ { "id": "R-101", "status": "failed", "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"" }, { "id": "R-102", "status": "failed", "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"" }, { "id": "R-103", "status": "failed", "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"" } ], "dlq_saved": true, "alert": "suppressed" } ✔ dlq 알림은 1개 · 0.0s → request POST http://localhost:8800/alerts/list { "reason": "dlq" } ← response HTTP 200 { "count": 1, "alerts": [ { "workflow": "intake", "node": "Send to DLQ", "severity": "high", "reason": "dlq", "count": 2, "message": "2 request(s) could not be created and wait in the DLQ", "execution_id": "10135", "at": 1793251017.2530532 } ] } ✔ 그래도 DLQ에는 5건 모두 · 0.0s → request POST http://localhost:8800/dlq/list {} ← response HTTP 200 { "size": 5, "parked": 0, "items": [ { "dlq_id": "D-1", "attempts": 1, "parked": false, "item": { "id": "R-301", "user": "정우성", "text": "사내 위키 로그인이 안 돼요" }, "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"", "workflow": "intake", "at": 1793251017.2315063 }, { "dlq_id": "D-2", "attempts": 1, "parked": false, "item": { "id": "R-302", "user": "이준호", "text": "노트북 배터리가 금방 닳아요" }, "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"", "workflow": "intake", "at": 1793251017.2315085 }, { "dlq_id": "D-3", "attempts": 1, "parked": false, "item": { "id": "R-101", "user": "이준호", "text": "VPN이 아침부터 계속 끊겨서 고객 미팅을 못 하고 있어요" }, "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"", "workflow": "intake", "at": 1793251020.9757996 }, { "dlq_id": "D-4", "attempts": 1, "parked": false, "item": { "id": "R-102", "user": "박서연", "text": "새 모니터를 하나 더 받을 수 있을까요?" }, "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"", "workflow": "intake", "at": 1793251020.9758017 }, { "dlq_id": "D-5", "attempts": 1, "parked": false, "item": { "id": "R-103", "user": "최유진", "text": "엑셀 라이선스가 만료됐다고 나와요" }, "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"", "workflow": "intake", "at": 1793251020.975803 } ] } ✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "error", "prompts": "ok", "usage": "ok", "dlq": "error", "slow_seconds": 8} → request POST http://localhost:8800/admin/fault { "tickets": "error", "dlq": "error" } ← response HTTP 200 { "faq": "ok", "tickets": "error", "prompts": "ok", "usage": "ok", "dlq": "error", "slow_seconds": 8 } ✔ DLQ가 멈춘 것은 늘 보냄 (critical) · 2.4s → request POST /webhook/my-intake { "requests": [ { "id": "R-201", "user": "한지훈", "text": "프린터 용지가 걸려요" }, { "id": "R-202", "user": "한지훈", "text": "" }, { "id": "R-203", "user": "김민지", "text": "메일 첨부가 안 열려요" } ] } ← response HTTP 200 { "total": 3, "created": 0, "failed": 2, "rejected": 1, "duplicates": 0, "unclassified": 0, "results": [ { "id": "R-201", "status": "failed", "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"" }, { "id": "R-203", "status": "failed", "reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"" }, { "id": "R-202", "status": "rejected", "reason": "text is required" } ], "dlq_saved": false, "alert": "sent" } ✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} → request POST http://localhost:8800/admin/fault { "faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok" } ← response HTTP 200 { "faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8 } ✔ 고친 뒤 · 실패 없음 · 알림 없음 · 2.8s → request POST /webhook/my-intake { "requests": [ { "id": "R-401", "user": "김민지", "text": "회의실 화면 공유가 안 돼요" }, { "id": "R-401", "user": "김민지", "text": "회의실 화면 공유가 안 돼요" }, { "id": "R-402", "user": "최유진", "text": "비밀번호를 바꾸라는 창이 계속 떠요" } ] } ← response HTTP 200 { "total": 3, "created": 2, "failed": 0, "rejected": 1, "duplicates": 0, "unclassified": 0, "results": [ { "id": "R-401", "status": "created", "ticket_id": "T-2001", "duplicate": false, "category": "네트워크", "priority": "보통" }, { "id": "R-402", "status": "created", "ticket_id": "T-2002", "duplicate": false, "category": "계정", "priority": "보통" }, { "id": "R-401", "status": "rejected", "reason": "duplicate id in this request" } ], "alert": "none" } Passed 6/6 report: runs/lab02-challenge2-20261029-141706.json ✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} → request POST http://localhost:8800/admin/fault { "faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok" } ← response HTTP 200 { "faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8 } $ uv run tools/flow.py test cases/lab02-challenge.json ✔ before: POST http://localhost:8800/tickets-reset · HTTP 200 · 0.0s · {"cleared": 2} ✔ before: POST http://localhost:8800/dlq-clear · HTTP 200 · 0.0s · {"cleared": 5} ✔ before: POST http://localhost:8800/alerts-clear · HTTP 200 · 0.0s · {"cleared": 2} ✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} POST http://localhost:5678/webhook/my-intake · 6 case(s) × 1 ✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "error", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} ✔ API가 멈춤 → DLQ · 2.7s ✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} ✔ 처리한 실패도 알림 (high · 개수) · 0.0s ✔ 실패가 없으면 알림 없음 · 2.9s ✔ 알림은 그대로 1개 · 0.0s ✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "error", "prompts": "ok", "usage": "ok", "dlq": "error", "slow_seconds": 8} ✔ DLQ도 멈추면 critical · 2.3s ✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} ✔ 잃어버린 id를 알림에 · 0.0s Passed 6/6 report: runs/lab02-challenge-20261029-141714.json ✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} $ uv run tools/flow.py test cases/lab01-challenge2.json ✔ before: POST http://localhost:8800/tickets-reset · HTTP 200 · 0.0s · {"cleared": 3} ✔ before: POST http://localhost:8800/dlq-clear · HTTP 200 · 0.0s · {"cleared": 2} ✔ before: POST http://localhost:8800/alerts-clear · HTTP 200 · 0.0s · {"cleared": 2} ✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} POST http://localhost:5678/webhook/my-intake · 4 case(s) × 1 ✔ 같은 id가 두 번이면 두 번째는 거절 · 2.1s ✔ 11건은 400 (한 번에 10건까지) · 0.0s ✔ 400이면 하나도 만들지 않음 · 0.0s ✔ 평소 요청은 그대로 · 2.7s Passed 4/4 report: runs/lab01-challenge2-20261029-141719.json ✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8} $ uv run tools/flow.py ops fault switch: all ok created tickets: 5 T-2001 key R-401 네트워크/높음 회의실 화면 공유가 안 돼요 T-2002 key R-402 계정/보통 비밀번호를 바꾸라는 창이 계속 떠요 T-2003 key R-101 네트워크/높음 VPN이 아침부터 계속 끊겨서 고객 미팅을 못 하고 있 T-2004 key R-102 장비/보통 새 모니터를 하나 더 받을 수 있을까요? T-2005 key R-103 소프트웨어/높음 엑셀 라이선스가 만료됐다고 나와요 DLQ: 0 (parked 0) alerts: 0