faust
faust copied to clipboard
@app.table_route points to the wrong worker
Checklist
- [x] I have included information about relevant versions
- [x] I have verified that the issue persists when using the
master
branch of Faust.
Steps to reproduce
Trying to use faust table view with 3 faust workers and 3 topics partitions with 1 Redpanda broker. Please check my sandbox repository to view clearly what happen and how to precisely reproduce it with Docker!
In a nutshell, here is a dummy app, executed with TOPIC_PARTITIONS: 3
itself in 3 docker containers.
import faust
from faust.web import Request, Response, View
import os
app = faust.App( 'test',
broker=os.getenv("FAUST_BROKER_URL"),
topic_partitions=3
)
table = app.Table('info', key_type=str, value_type=str)
@app.agent(key_type=str, value_type=str, concurrency=1)
async def write_info(stream):
async for key, value in stream.items():
table[key] = value
yield
@app.page('/info/{key}/')
@app.table_route(table=table, match_info='key')
async def get_info(web, request: Request, key) -> Response:
try:
value = table[key]
return web.json(value)
except KeyError:
raise View.NotFound(f"Cannot find this symbol among [{table.keys()}]")
@app.page("/info")
class info(View):
async def get(self, request: Request) -> Response:
# Downloads infos
return self.json({symbol: value for symbol, value in table.items()})
async def post(self, request: Request) -> Response:
json_data = await request.json()
await write_info.cast(key=json_data['key'], value=json_data['value'])
return self.json({"message": f"Post {json_data['key']}"})
After populating the table info
with A,B,C,D
elements, the faust router not to seem working as expected.
$ curl localhost:6066/info/D/
$ {"error":"Cannot find this symbol among [dict_keys(['A', 'B', 'C'])]"}
Expected behavior
The routing points to the correct worker faust-2
which hold the active partition with the element D
.
Actual behavior
The routing points to the worker faust-1
which hold the stand-by but correct partition 0.
Full traceback
Versions
- Python version : 3.11
- Faust version : 0.10.13
- Operating system : Linux
- Redpanda version : 23.3.11