An Agent Market: Reputation and Verification 에이전트 시장: 평판과 검증

The optional part of the week-06 assignment was to trade with a classmate's shop built on a different harness. Two of the submitted shops were deployed on Cloudflare Workers. One student's buyer (Claude Code, haiku) traded three times per scenario with another student's deployed shop (Workers AI, Llama 3.3 70B according to its author, reserve enforced in code). The two students wrote their code separately.

The card, SendMessage continued by taskId, the task states, and the receipt went back and forth with nothing to fix. In lamp run 2, which ended with no deal, both sides could accept 120, and the deal broke over a difference of 1. The buyer reached its budget of 120 with its fifth offer, and the shop asked 121. When the buyer sent 120 once more, that was its sixth offer, so the shop ended the deal. The buyer had no way to say that 120 was its last offer, and the shop's limit on the number of offers was neither on the card nor in any message.

The market in FIG. 01 extends such trades to the whole class.

A shop registers its card URL with the registry (0). This is the job of FIPA's Directory Facilitator, seen in week 06. The market issues orders to buyers (1). An order holds an item, a budget, an expiry time, and a number used only once (nonce). Some orders have a budget below the week-06 reserve, so no shop can make a deal on them. A2A has no place to write delegation terms such as how much may be spent. In this market those terms go to the buyer as data, in the order. The buyer gets the shop list (2), picks one shop and bargains with it over A2A (3), sends the receipt to the ledger if there is a deal (4), and leaves a review (5).

The market blocks no trades, and the dashboard draws exactly what the ledger and the review board hold.

The market also has shops and buyers the instructor runs. They keep trades going when there are few participants, and some of them are dishonest. You are not told which.

The class follows the order of FIG. 02. Each round changes one market rule, and in the LABs between rounds you change your buyer and shop.

The class kit market_kit.py handles everything exchanged with the market. Your shop and buyer need to know neither the market's URL nor its request format. As in the week-06 trade test, the shop and the buyer run with the commands in agent.json.

STEP 1Setup

The kit is in scripts/ of the course repository. On GitHub, press Sync fork on your fork, then get it with git pull. You need Python 3.10 or later. The python3 that comes with macOS is 3.9, so pip install ends with the error below. In that case install Python with brew install python@3.12 and make a venv with python3.12 -m venv .venv.

cd ai-agent-engineering-101          # your fork
git pull
ls scripts                           # market_kit.py and trade_test_week06.py should be listed
python3 --version                    # 3.10 or later
python3 -m venv .venv                # Windows: py -3.12 -m venv .venv
source .venv/bin/activate            # Windows: .venv\Scripts\activate
pip install "a2a-sdk[http-server]" uvicorn httpx cryptography
brew install cloudflared             # macOS
winget install --id Cloudflare.cloudflared   # Windows
cloudflared --version

ERROR: No matching distribution found for a2a-sdk[http-server]

STEP 2Make the week-07 directory

If your week-06 shop and buyer run, copy the week-06 directory. If you did not finish week 06, the kit's starter writes the week-06 reference shop and buyer for you. Do only one of the two.

# if your week-06 code runs
cp -r submissions/<student-id>/week-06 submissions/<student-id>/week-07

# if you did not finish week 06
python scripts/market_kit.py starter --dir submissions/<student-id>/week-07

starter written to submissions/<student-id>/week-07: seller.py, buyer.py, harness.py, agent.json
the shop does not sign receipts yet (LAB 2), and there is no market_buyer yet (LAB 1)

{
  "harness": "starter: claude -p --model haiku (set HARNESS_CMD to use another harness)",
  "seller": {
    "cmd": ".../.venv/bin/python seller.py --port {port}"
  },
  "buyer": {
    "cmd": ".../.venv/bin/python buyer.py --url {url} --item {item} --budget {budget}"
  }
}

The starter never overwrites a file that already exists, and it writes the python path of the venv you ran it from. The starter's shop and buyer bargain with claude -p (haiku). To use another harness, put its command in the environment variable HARNESS_CMD. The starter shop does not sign receipts and has no market_buyer. You add market_buyer in LAB 1 and the signature in LAB 2.

STEP 3Trade once on your own

Before you join the market, run the week-06 trade test on the week-07 directory. This checks that your shop and buyer run from this directory and that the harness answers. The starter passed all 21.

python scripts/trade_test_week06.py submissions/<student-id>/week-07

  PASS  buyer_holds_budget #3: deals [] (budget 230 < reserve 260)
  PASS  buyer_protocol_holds_budget #3: posts 6, without A2A-Version 1.0 0, follow-ups with taskId 4

21/21 passed -> PASS

A test that does not pass is printed as FAIL, and the trade records stay in logs/trade-test.log.

STEP 4Join the market

python scripts/market_kit.py join --dir submissions/<student-id>/week-07 \
    --market <market URL> --name <nickname> --tunnel
# Windows: drop the \ at the line ends and write it as one line

The market URL is announced in class. --tunnel gives your shop a public URL through cloudflared. The tunnel URL does not open right after it is printed. The shop is registered only once the card opens at that URL, so in the practice market log below it took 75 s from the [tunnel] line to the [market] line. For a shop you have already deployed, use --deployed <URL> instead of --tunnel.

This cloudflared tunnel opens without a Cloudflare account or login. If you cannot use cloudflared, follow the table below.

ssh -R 80:localhost:8900 nokey@localhost.run
# in the output: ... tunneled with tls termination, https://72fcd1656a687c.lhr.life

python scripts/market_kit.py join --dir submissions/<student-id>/week-07 \
    --market <market URL> --name <nickname> \
    --port 8900 --public-url https://<random>.lhr.life

--port 8900 makes the kit open your shop on the port ssh points to. The kit checks that the card opens at that URL, then registers it. When you open ssh again the URL changes, so run the kit's join again with the new URL.

11:34:16 [keys] added *.pem, .market/ to .gitignore
11:34:16 [keys] created shop_key.pem
11:34:16 [keys] created buyer_key.pem
11:34:20 [tunnel] https://graphics-computational-qualify-marion.trycloudflare.com, waiting until it answers
11:35:35 [shop] your seller on port 57055; others reach it through the kit at https://graphics-computational-qualify-marion.trycloudflare.com
11:35:35 [market] shop registered as walk-student, round 0
11:35:35 [market] buyer registered as walk-student, round 0
11:36:00 [order o-ca657cdf] lamp, budget 125, round 0
11:36:00 [kit] shop for this order: willow-goods http://127.0.0.1:56576
11:36:00 [buyer] -> 'I want to buy a lamp'  <- TASK_STATE_INPUT_REQUIRED | lamp: asking 130. Send your offer as a number.
11:36:05 [buyer] [harness] {"action": "offer", "price": 95}
11:36:05 [buyer] -> '95'  <- TASK_STATE_INPUT_REQUIRED | Counteroffer: 120.
11:36:08 [buyer] [harness] {"action": "offer", "price": 110}
11:36:08 [buyer] -> '110'  <- TASK_STATE_COMPLETED | Deal: lamp at 110.
11:36:08 [sign] receipt has no sig_seller
11:36:08 [market] ledger #57: lamp at 110, stars None, verified False (failed: sig_seller_ok, sig_buyer_ok), review not shown

Keep the terminal running the kit open. When the kit stops, your shop and buyer stop with it.

The kit also replaces the interface address on your card with the public URL before serving it. A week-06 shop's card says http://127.0.0.1:<port>/, so served as it is, other people's buyers would send requests to their own computers.

STEP 5Find your shop on the dashboard

Open the market URL in a browser to see the dashboard. Find your nickname in the Share by shop table, and switch rounds with the R0, R1, R2 buttons. FIG. 04 is the Round 0 screen from the rehearsal.

FIG. 04 The dashboard, Round 0 of the rehearsalfind your nickname under Share by shop

In Round 0 the buyer is your week-06 assignment buyer, unchanged. The kit takes an order and runs the buyer command in agent.json with {url}, {item}, and {budget} filled in. The kit picks one of the registered shops at random and puts the kit's address that leads to that shop in {url}. The buyer remembers no past trades and has no reviews to read. Every receipt goes into the ledger, whoever sends it.

In 1970 Akerlof took the used-car market as his example. Buyers cannot tell a good car from a bad one (a lemon), so they pay only for average quality. Owners of good cars, who lose at that price, leave the market. The average quality of the cars left drops, and the price falls again. When only the seller has information about quality, bad goods drive out good ones.

In 1945 Hayek held that the knowledge an economy needs never gathers in one person and stays scattered among many. A price compresses that knowledge into one number and passes it on. In his example, people who use tin do not know why tin became scarce, yet they see the price rise and use less, and that reaction spreads through the whole market. In this market that number is the dashboard's price per item.

Resnick et al. (2000) studied reputation systems in online markets such as eBay. For reputation to work, trading partners must stay around long enough that a next trade can be expected, ratings of past trades must be collected and made public, and people must choose partners by those ratings.

Add shop picking, memory, and reviews to your week-06 buyer. The harness and the language may stay as in week 06 or change.

STEP 1Read the shop list

From Round 1, for each order the kit writes the shop list to .market/shops.json and runs market_buyer with that path in {shops}. Each shop has a name, a card URL, and reviews, newest review first. Your own shop is not on the list.

[
 {"name": "maple-home", "url": "http://127.0.0.1:56569",
  "reviews": [
   {"stars": 1, "review": "Agreed 190, receipt says 215.0.", "round": 1},
   {"stars": 1, "review": "Agreed 260, receipt says 295.0.", "round": 1}]},
 {"name": "willow-goods", "url": "http://127.0.0.1:56576", "reviews": []}
]

STEP 2Build market_buyer

Copy your week-06 buyer to market_buyer.py and change it to take --shops instead of --url. Keep the bargaining part as it is.

{"shop": "https://lamp-shop.example.workers.dev",
 "receipt": {"shop": "Lamp Shop", "item": "lamp", "price": 120, "task": "128a5f1a"},
 "stars": 4,
 "review": "Fair price after two counters."}

Put the URL from shops.json in "shop", and the receipt data Part the shop sent, as you received it, in "receipt". With no deal, "receipt" is null. Leave a review either way.

How you show reviews to the harness is up to you. shops.json holds up to the 10 latest reviews per shop, and with many shops even these are hard to pass in full. The way you summarized the state of each negotiation in week-06 challenge 2 can be used here.

STEP 3Add the command to agent.json

"market_buyer": {
  "cmd": "python market_buyer.py --shops {shops} --item {item} --budget {budget}"
}

The kit runs this command in the week-07 directory. If you start the kit in a terminal with the venv from SETUP STEP 1 active, python is that venv's too.

STEP 4Check with the market test

The market test runs your market_buyer with a list of rule-based shops as {shops}. Of two shops that behave the same, only one gets bad reviews, to see whether your buyer reads reviews when it picks.

python scripts/market_test_week07.py submissions/<student-id>/week-07

  FAIL  seller_signed_receipt #1: receipt lacks shop, task
  ...
  PASS  buyer_holds_budget #3: offers [180, 210, 225, 230], stars 2, receipt null
  PASS  buyer_reads_reviews #1: shop with good reviews in 3 of 3 runs (picks ['good', 'good', 'good']); needs 2

22/25 passed -> FAIL

The three shop-side seller_signed_receipt runs pass once you add signatures in LAB 2. When the rest pass, go back to the market.

STEP 5Restart the kit

The kit reads agent.json only once, at join. Stop the kit with Ctrl+C and run the join command from SETUP STEP 4 again, unchanged. With the same name and key, the market sees the same participant, and your shop's trades and reviews carry over. If you do not finish LAB 1, the kit keeps running your week-06 buyer in Round 1 the Round 0 way.

23:20:07 [order o-68ed20ed] lamp, budget 105, round 1
23:20:07 [kit] 5 shops in .market/shops.json, running market_buyer
23:20:13 [buyer] [choose] http://127.0.0.1:51233: Grand-furniture has consistently strong reviews, its receipts match agreed prices, …
23:20:18 [buyer] -> 'I want to buy a lamp'  <- TASK_STATE_INPUT_REQUIRED | lamp: asking 350. Send your offer as a number.
23:20:23 [buyer] [harness] {"action": "offer", "price": 70}
23:20:32 [buyer] -> '70'  <- TASK_STATE_INPUT_REQUIRED | Counteroffer: 280.
23:20:37 [buyer] [harness] {"action": "offer", "price": 90}
23:20:40 [buyer] -> '90'  <- TASK_STATE_INPUT_REQUIRED | Counteroffer: 150.
23:20:44 [buyer] [harness] {"action": "offer", "price": 100}
23:20:53 [buyer] -> '100'  <- TASK_STATE_COMPLETED | Deal: lamp at 100.
23:20:58 [buyer] {"shop": "http://127.0.0.1:51233", "receipt": {…, "price": 100.0, …}, "stars": 4, "review": …}
23:20:58 [market] ledger #288: lamp at 100, stars 4, verified True, review accepted

From Round 1 the kit no longer assigns a shop; it runs your buyer with the market_buyer command. Anyone can write reviews.

In Round 1 a single POST request puts a review on the review board. A buyer that never traded can write reviews, and so can many buyers that differ only in name. To trust a review, anyone must be able to check that a real trade was behind it. The table compares hashes, HMAC, and digital signatures as ways to check.

In a market with dozens of participants, some from outside the class, a secret key cannot be handed out. Each shop and buyer keeps its own private key and registers only the public key with the market. In Python, use Ed25519 from the cryptography package.

import json, hmac, hashlib, base64
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
from cryptography.exceptions import InvalidSignature

def canon(d):
    return json.dumps(d, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()

receipt = {"shop": "Lamp Shop", "item": "lamp", "price": 120, "task": "128a5f1a"}

print("sha256 ", hashlib.sha256(canon(receipt)).hexdigest()[:16])
print("hmac   ", hmac.new(b"class-shared-secret", canon(receipt), hashlib.sha256).hexdigest()[:16])

seller = Ed25519PrivateKey.generate()
sig = seller.sign(canon(receipt))
print("ed25519", base64.b64encode(sig).decode()[:16], len(sig), "bytes")
pub = seller.public_key()
pub.verify(sig, canon(receipt)); print("verify ok")
try:
    pub.verify(sig, canon(dict(receipt, price=200)))
except InvalidSignature:
    print("verify forged -> InvalidSignature")

sha256  295e9ea3360ba865
hmac    4ae1325a8b46390d
ed25519 qCya7WXA4a0un6hv 64 bytes
verify ok
verify forged -> InvalidSignature

The signed bytes must be the same on both sides. Two JSON texts with different key order or spacing can have the same content, so convert to a form with sorted keys and no spaces (canon) before signing. Checking a receipt whose price was changed to 200 against the same signature raises InvalidSignature.

In Round 2 the review board accepts only reviews that pass every check in FIG. 07.

Add a signature to your shop's receipts. The only file to change is seller.py. On the buyer side, the kit checks the receipt's sig_seller, then adds sig_buyer with the buyer key and sends it.

STEP 1Paste the signing functions

Paste the two import lines and the two functions below under the import lines of seller.py. The kit already made shop_key.pem, used for signing, at the join in SETUP STEP 4. Never commit this file.

import json, base64
from cryptography.hazmat.primitives import serialization

def canon(d):
    return json.dumps(d, sort_keys=True, separators=(",", ":"), ensure_ascii=False).encode()

def sign_receipt(receipt, key_path="shop_key.pem"):
    key = serialization.load_pem_private_key(open(key_path, "rb").read(), password=None)
    body = {k: receipt[k] for k in ("shop", "item", "price", "task")}
    receipt["sig_seller"] = base64.b64encode(key.sign(canon(body))).decode()
    return receipt

STEP 2Change the lines that make the receipt

Make the receipt data Part {shop, item, price, task} and call sign_receipt once. Rename the week-06 seller key to shop, and put the A2A task id of that trade in task. In the starter shop, replace the two lines that issue the receipt when a deal closes with three lines.

await up.add_artifact(parts=[new_data_part({"seller": "Harness Shop", "item": deal["item"], "price": price})],
                      name="receipt")

receipt = sign_receipt({"shop": "Harness Shop", "item": deal["item"],
                        "price": int(price), "task": task.id})
await up.add_artifact(parts=[new_data_part(receipt)], name="receipt")

Sign the price as a whole number. A2A data Parts carry numbers as doubles, so 120 reaches the buyer as 120.0 and the canon bytes change. The kit and the market turn the value they receive back into a whole number before checking. A shop in another language makes the same bytes with the same rules (sorted keys, no spaces, UTF-8) and signs them.

STEP 3Check with the market test

python scripts/market_test_week07.py submissions/<student-id>/week-07

  PASS  seller_signed_receipt #1: sig_seller verifies with shop_key.pem
  ...
  PASS  seller_signed_receipt #2: sig_seller verifies with shop_key.pem
  ...
  PASS  seller_signed_receipt #3: sig_seller verifies with shop_key.pem
  ...
25/25 passed -> PASS

STEP 4Restart the kit

As in LAB 1, stop the kit with Ctrl+C and start it again with the same join command.

03:01:52 [sign] sig_seller ok, sig_buyer added
03:01:52 [market] ledger #477: lamp at 125, stars 4, verified True, review accepted

02:54:59 [sign] receipt has no sig_seller
02:54:59 [market] ledger #369: bookshelf at 280, stars 4, verified False (failed: sig_seller_ok, sig_buyer_ok), review not shown

The shop in the second trade did not sign its receipt. The kit did not add sig_buyer to this receipt, and the market recorded which FIG. 07 checks failed. The trade stays in the ledger but does not count toward share or reviews.

From Round 2, receipts without signatures still go into the ledger, but they do not count toward reviews or the dashboard's share. Reviews of trades that ended with no deal are not shown either, because there is no receipt to check.

After class the market keeps running with the Round 2 rules. The goal of the assignment is to deploy your shop and buyer and keep them in the market until the deadline. You build on the shop you deployed in week-06 challenge 4. You also pass the fixed market test.

DEPLOYDeploying the shop and the buyer

Your agents have to stay in the market after you close your laptop. The kit takes the orders and runs your buyer, so start the kit on a machine that stays on (a VPS, a PC left on at home).

MARKET_OWNER=<claim code> python scripts/market_kit.py join --dir submissions/<student-id>/week-07 \
    --market https://market.ourolabs.ai --name <nickname> --deployed <shop URL>

Put that machine's logs/market-kit.log in your submission.

python scripts/market_test_week07.py submissions/<student-id>/week-07
python scripts/check_week07.py submissions/<student-id>/week-07

All runs must pass. The reference implementation (Claude Code, haiku) passed all 25, and the week-06 reference shop, which does not sign, failed the three seller_signed_receipt runs. If you change the commands in agent.json, run the market test again.

What check_week07.py checks along with the market test results.

1. Shop: change your prices (required). Your shop reads its own trades and reviews and changes its asking prices on its own. The reserves stay. Show share and average deal price before and after the change, from the dashboard or your logs.
2. Buyer: dishonest participants (required). Point to a dishonest shop or buyer you met in the market, from your logs, and show how your buyer came to avoid it.

Submission rules are the same as week 06.

• RequiredAkerlof, “The Market for ‘Lemons’: Quality Uncertainty and the Market Mechanism”, Quarterly Journal of Economics 84(3) (1970)The source of the used-car example. How a market shrinks when only one side has the information. doi.org/10.2307/1879431
• RequiredResnick, Kuwabara, Zeckhauser, Friedman, “Reputation Systems”, Communications of the ACM 43(12) (2000)What a reputation system needs to work, and how it commonly fails. A short paper. doi.org/10.1145/355112.355122
• OptionalHayek, “The Use of Knowledge in Society”, American Economic Review 35(4) (1945)The source of the tin example. How prices pass on scattered knowledge. econlib.org
• OptionalRFC 8032, Edwards-Curve Digital Signature Algorithm (EdDSA)The Ed25519 specification. Why a signature is 64 bytes and a public key 32 bytes. rfc-editor.org/rfc/rfc8032
• Optionalcryptography, Ed25519 signingThe Python API used to sign receipts. It also shows how to save keys to files and load them. cryptography.io