I need help, I can't figure out how to process such a data structure asynchronously?
markets = {
'AAAA': [
[100, 500, 300], # chunk 1
[101, 501, 301], # chunk 2
[102, 502, 302], # chunk 3
[103, 503, 303], # chunk 4
],
'BBBB': [
[110, 510, 310], # chunk 1
[111, 511, 311], # chunk 2
[112, 512, 312], # chunk 3
[113, 513, 313], # chunk 4
],
'CCCC': [
[120, 520, 320], # chunk 1
[121, 521, 321], # chunk 2
[122, 522, 322], # chunk 3
[114, 514, 314], # chunk 4
],
}
What is the meaning of the example, I have stores, and I need to update the product data via the API, I need to do this as quickly as possible, but I can't make too many requests to the store. I solved this problem by splitting this task into parts of 3 pieces (numbers symbolize each individual task) and an array of numbers symbolizes a chunk
- I want to create a coroutine for all markets
- In each market, the chunk brothers sequentially, and send it for processing
- Each chunk value must be processed in a coroutine
- Return the processed data in the same form
If you imagine it in the form of a diagram, it looks something like this
My complete code that I'm trying to revive, however, there are a lot of errors in it, and I'm starting to think that I'm doing everything wrong at all
import asyncio
markets = {
'AAAA': [
[100, 500, 300],
[101, 501, 301],
[102, 502, 302],
[103, 503, 303],
],
'BBBB': [
[110, 510, 310],
[111, 511, 311],
[112, 512, 312],
[113, 513, 313],
],
'CCCC': [
[120, 520, 320],
[121, 521, 321],
[122, 522, 322],
[114, 514, 314],
],
}
async def plus(number):
return number + 1000
async def load(chunks):
calculated = []
for number in chunks:
calculated += [await plus(number)]
return calculated
async def store(data):
response = []
for chunks in data:
response += [await load(chunks)]
return await asyncio.gather(*response)
async def main():
coroutines_markets = {}
for markets_id, data in markets.items():
coroutines_markets[markets_id] = [await store(data)]
processed = await asyncio.gather(*coroutines_markets)
print(processed)
if __name__ == '__main__':
loop = asyncio.new_event_loop()
asyncio.run(main())
Perhaps you can help me make the logic of processing this thing?
source https://stackoverflow.com/questions/76443034/how-to-process-asynchronously-such-a-data-structure
Comments
Post a Comment