# Downloading multiple fields assynchronously with astroquery

**URL:** <https://community.openastronomy.org/t/downloading-multiple-fields-assynchronously-with-astroquery/729>\
**Category:** Astropy\
**Tags:** question\
**Created:** [August 23, 2023, 10:12pm UTC](https://community.openastronomy.org/t/downloading-multiple-fields-assynchronously-with-astroquery/729 "2023-08-23T22:12:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jpssrocha](https://dub1.discourse-cdn.com/flex005/user_avatar/community.openastronomy.org/jpssrocha/32/410_2.png) [@jpssrocha](https://community.openastronomy.org/u/jpssrocha)\
**Post date:** [August 23, 2023, 10:12pm UTC](https://community.openastronomy.org/t/downloading-multiple-fields-assynchronously-with-astroquery/729/1 "2023-08-23T22:12:51Z")

</div>

Hi,

i’ve been trying to download multiple fields from Gaia using the cone\_search\_async method, by starting the jobs on a loop and them getting the results on another one. Like on the example bellow.

```auto
fields = [
      (250, -46, 2),
      (250, -47, 2),
      (250, -48, 2),
      (250, -49, 2),
      (250, -50, 2),
      (250, -51, 2),
  ]

  fields = [helper(*field) for field in fields]

  gaia_credentials_file = "gaia_credentials.txt"

  with gaia_credentials(gaia_credentials_file):
      results = [get_gaia_catalog(*inputs) for inputs in fields]
      results = [res.get_data() for res in results]

```

the helper() and get\_gaia\_catalog() functions are just helpers to help format the arguments, and the gaia\_credentials context manager just help with logging in and out. (get\_gaia\_catalog basically loads coordinates into skycoords and call Gaia.cone\_search\_async with the proper parameters)

I’ve also tried to use the async/await syntax but it haven’t worked either (with asyncio.gather)… By watching the logs i can see that the jobs are being started and finished before the next one start, therefore it’s running sequentially (the loop above is taking 5 secs for 1 element and 30 secs for the 6 queries … therefore no gain in speed).

I think i’m just not using it properly. But i can’t find an example that’s fetching more than 1 field at a time.

Someone can give an example of how to fetch multiple fields asynchronously?

---

<div class="post-metadata">

**Author:** ![jpssrocha](https://dub1.discourse-cdn.com/flex005/user_avatar/community.openastronomy.org/jpssrocha/32/410_2.png) [@jpssrocha](https://community.openastronomy.org/u/jpssrocha)\
**Post date:** [August 24, 2023, 12:06pm UTC](https://community.openastronomy.org/t/downloading-multiple-fields-assynchronously-with-astroquery/729/2 "2023-08-24T12:06:35Z")

</div>

Solved the problem by using a ThreadPool on top of astroquery:

```auto
fields = [helper*((250, -46 + i, 2)) for i in range(6)] 

gaia_credentials_file = "gaia_credentials.txt"

with gaia_credentials(gaia_credentials_file):
    with ThreadPool() as p:
        res = p.starmap_async(get_gaia_catalog, fields)
        res = [r.get() for r in res]

```

Don’t know what is happening with the astroquery api, but the code snnipet above gave me a 30x speed gain, and i can see clearly the jobs being initialized one after the other, them downloaded in parallel.