222 lines
7.8 KiB
Plaintext
222 lines
7.8 KiB
Plaintext
$ uv run tools/flow.py push reference/lab02-example2-error-handler.json reference/lab02-example2-intake.json
|
||
! unpublished '76 Intake · DLQ (reference, Lab 02 example 1, includes Lab 01 tasks)' (w92de92194fc1657): it used the same webhook path
|
||
✔ pushed '76 Error handler (reference)' (id errHandler76001)
|
||
✔ pushed '76 Intake · DLQ + error workflow (reference, Lab 02 example 2)' (id w01376519d41c1ce)
|
||
POST http://localhost:5678/webhook/my-intake
|
||
|
||
$ uv run tools/flow.py test cases/lab02.json -v
|
||
✔ before: POST http://localhost:8800/tickets-reset · HTTP 200 · 0.0s · {"cleared": 0}
|
||
→ request POST http://localhost:8800/tickets-reset
|
||
{}
|
||
← response HTTP 200
|
||
{
|
||
"cleared": 0
|
||
}
|
||
✔ before: POST http://localhost:8800/dlq-clear · HTTP 200 · 0.0s · {"cleared": 2}
|
||
→ request POST http://localhost:8800/dlq-clear
|
||
{}
|
||
← response HTTP 200
|
||
{
|
||
"cleared": 2
|
||
}
|
||
✔ before: POST http://localhost:8800/alerts-clear · HTTP 200 · 0.0s · {"cleared": 0}
|
||
→ request POST http://localhost:8800/alerts-clear
|
||
{}
|
||
← response HTTP 200
|
||
{
|
||
"cleared": 0
|
||
}
|
||
✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8}
|
||
→ request POST http://localhost:8800/admin/fault
|
||
{
|
||
"faq": "ok",
|
||
"tickets": "ok",
|
||
"prompts": "ok",
|
||
"usage": "ok",
|
||
"dlq": "ok"
|
||
}
|
||
← response HTTP 200
|
||
{
|
||
"faq": "ok",
|
||
"tickets": "ok",
|
||
"prompts": "ok",
|
||
"usage": "ok",
|
||
"dlq": "ok",
|
||
"slow_seconds": 8
|
||
}
|
||
POST http://localhost:5678/webhook/my-intake · 4 case(s) × 1
|
||
✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "error", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8}
|
||
→ request POST http://localhost:8800/admin/fault
|
||
{
|
||
"tickets": "error"
|
||
}
|
||
← response HTTP 200
|
||
{
|
||
"faq": "ok",
|
||
"tickets": "error",
|
||
"prompts": "ok",
|
||
"usage": "ok",
|
||
"dlq": "ok",
|
||
"slow_seconds": 8
|
||
}
|
||
✔ 예제 1 · API가 멈추면 실패로 표시 · 200 · 4.8s
|
||
→ request POST /webhook/my-intake
|
||
{
|
||
"requests": [
|
||
{
|
||
"id": "R-301",
|
||
"user": "정우성",
|
||
"text": "사내 위키 로그인이 안 돼요"
|
||
},
|
||
{
|
||
"id": "R-302",
|
||
"user": "이준호",
|
||
"text": "노트북 배터리가 금방 닳아요"
|
||
}
|
||
]
|
||
}
|
||
← response HTTP 200
|
||
{
|
||
"total": 2,
|
||
"created": 0,
|
||
"failed": 2,
|
||
"rejected": 0,
|
||
"duplicates": 0,
|
||
"unclassified": 0,
|
||
"results": [
|
||
{
|
||
"id": "R-301",
|
||
"status": "failed",
|
||
"reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\""
|
||
},
|
||
{
|
||
"id": "R-302",
|
||
"status": "failed",
|
||
"reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\""
|
||
}
|
||
]
|
||
}
|
||
✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8}
|
||
→ request POST http://localhost:8800/admin/fault
|
||
{
|
||
"faq": "ok",
|
||
"tickets": "ok",
|
||
"prompts": "ok",
|
||
"usage": "ok",
|
||
"dlq": "ok"
|
||
}
|
||
← response HTTP 200
|
||
{
|
||
"faq": "ok",
|
||
"tickets": "ok",
|
||
"prompts": "ok",
|
||
"usage": "ok",
|
||
"dlq": "ok",
|
||
"slow_seconds": 8
|
||
}
|
||
✔ 예제 1 · 실패한 2건은 DLQ에 · 0.0s
|
||
→ request POST http://localhost:8800/dlq/list
|
||
{}
|
||
← response HTTP 200
|
||
{
|
||
"size": 2,
|
||
"parked": 0,
|
||
"items": [
|
||
{
|
||
"dlq_id": "D-1",
|
||
"attempts": 1,
|
||
"parked": false,
|
||
"item": {
|
||
"id": "R-301",
|
||
"user": "정우성",
|
||
"text": "사내 위키 로그인이 안 돼요"
|
||
},
|
||
"reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"",
|
||
"workflow": "intake",
|
||
"at": 1793250914.161773
|
||
},
|
||
{
|
||
"dlq_id": "D-2",
|
||
"attempts": 1,
|
||
"parked": false,
|
||
"item": {
|
||
"id": "R-302",
|
||
"user": "이준호",
|
||
"text": "노트북 배터리가 금방 닳아요"
|
||
},
|
||
"reason": "500 - \"{\\\"error\\\": \\\"tickets service is failing (fault switch)\\\"}\"",
|
||
"workflow": "intake",
|
||
"at": 1793250914.161775
|
||
}
|
||
]
|
||
}
|
||
✔ 예제 2 · 예상 못 한 오류 (nightly-report) · 0.1s
|
||
→ request POST /webhook/nightly-report
|
||
{}
|
||
← response HTTP 500
|
||
{
|
||
"message": "Error in workflow"
|
||
}
|
||
… before: wait 3s
|
||
✔ 예제 2 · Error Workflow가 알림을 남김 · 0.0s
|
||
→ request POST http://localhost:8800/alerts/latest
|
||
{}
|
||
← response HTTP 200
|
||
{
|
||
"workflow": "76 Nightly report (provided, has a bug)",
|
||
"workflow_id": "wb9ce32c18d7df56",
|
||
"node": "Count by category",
|
||
"message": "rows is not iterable [line 4]",
|
||
"execution_id": "10104",
|
||
"severity": "critical",
|
||
"reason": "unexpected",
|
||
"at": 1793250914.256605
|
||
}
|
||
Passed 4/4
|
||
report: runs/lab02-20261029-141517.json
|
||
✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8}
|
||
→ request POST http://localhost:8800/admin/fault
|
||
{
|
||
"faq": "ok",
|
||
"tickets": "ok",
|
||
"prompts": "ok",
|
||
"usage": "ok",
|
||
"dlq": "ok"
|
||
}
|
||
← response HTTP 200
|
||
{
|
||
"faq": "ok",
|
||
"tickets": "ok",
|
||
"prompts": "ok",
|
||
"usage": "ok",
|
||
"dlq": "ok",
|
||
"slow_seconds": 8
|
||
}
|
||
|
||
$ uv run tools/flow.py ops
|
||
fault switch: all ok
|
||
created tickets: 0
|
||
DLQ: 2 (parked 0)
|
||
D-1 R-301 attempts 1 parked False 500 - "{\"error\": \"tickets service is failing (f
|
||
D-2 R-302 attempts 1 parked False 500 - "{\"error\": \"tickets service is failing (f
|
||
alerts: 1
|
||
[critical] 76 Nightly report (provided, has a bug) · unexpected · Count by category · rows is not iterable [line 4]
|
||
|
||
$ uv run tools/flow.py test cases/lab01.json
|
||
✔ before: POST http://localhost:8800/tickets-reset · HTTP 200 · 0.0s · {"cleared": 0}
|
||
✔ before: POST http://localhost:8800/dlq-clear · HTTP 200 · 0.0s · {"cleared": 2}
|
||
✔ before: POST http://localhost:8800/alerts-clear · HTTP 200 · 0.0s · {"cleared": 1}
|
||
✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8}
|
||
POST http://localhost:5678/webhook/my-intake · 6 case(s) × 1
|
||
✔ 정상 3건 · 2.9s
|
||
✔ 같은 요청을 다시 보내도 중복 없음 · 2.6s
|
||
✔ 만들어진 접수 건은 3개 · 0.0s
|
||
✔ 잘못된 항목은 그 항목만 거절 · 1.9s
|
||
✔ 목록이 아니면 400 · 0.0s
|
||
✔ before: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "flaky", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8}
|
||
✔ 가끔 실패하는 API · 다시 해 보기로 성공 · 2.5s
|
||
✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8}
|
||
Passed 6/6
|
||
report: runs/lab01-20261029-141527.json
|
||
✔ after: POST http://localhost:8800/admin/fault · HTTP 200 · 0.0s · {"faq": "ok", "tickets": "ok", "prompts": "ok", "usage": "ok", "dlq": "ok", "slow_seconds": 8}
|