Using SAMP for integrating TOPCAT and Jupyter

Astropy
, ,
1

Hi,

I am trying to write a piece of python code that allows me to send tables back and forth between a jupyter notebook and TOPCAT. Reading the Astropy documentation, I found out that the best way to do so is using the SAMP protocol (SAMP (Simple Application Messaging Protocol) (astropy.samp) — Astropy v5.3.3).
Following this specific tutorial: Sending and Receiving Tables and Images over SAMP — Astropy v5.3.3
I managed to write a piece of code to send an astropy.table.Table to TOPCAT as below:

from astropy.table import Table
from astropy.samp import SAMPIntegratedClient
def send_to_TOPCAT(astropy_table, table_name='table_from_jupyter', 
                    directoty='some_directory/'):
    """
    """
    #makes a local copy for the table
    astropy_table.write(directoty + table_name + '.fits', format='fits', 
                        overwrite=True)
    client = SAMPIntegratedClient()
    client.connect()
    #define params
    params = {} 
    params["url"] = 'file:' + directoty + table_name + '.fits'
    params["name"] = table_name
    #write the message to be sent
    message = {}
    message["samp.mtype"] = "table.load.fits"
    message["samp.params"] = params
    print(client.get_registered_clients())
    client.notify_all(message)
    client.disconnect()

This currently works but only in specific cases as it has two main issues:

  1. I have to save the table locally as a fits file before defining a URL to a local file and sending it to TOPCAT.

  2. This only works for some tables. For example, some tables retrieved directly from ViZier using astroquery will return the error when I try to save it locally:

ValueError: The header keyword ‘description’ with its value is too long

To go around the issue number 2., I tried to change the table format to csv by saving it as such and changing message["samp.mtype"] = "table.load.fits" to “table.load.csv”, but this does neither work nor returns an error.
I did some googling but I couldn’t find anywhere in the astropy documentation or elsewhere a list of what are the “samp.mtype” allowed.

My questions are: Does anyone know how to access the possible “samp.mtype”?
Also: Does anyone know if it would be possible to pass a params["url"] that instead of pointing to a local file with a table, points instead to an astropy.table.Table in the active jupyter session?

Thanks a lot in advance!

2

The mtype values that topcat responds to (as there are other messages that are not relevant to topcat) can be found at Messages Received

The way that SAMP works is that data is sent around via URLs, so some code on your end needs to run a simple webserver that will provide access to the data you want to send (I don’t think this has changed, but I haven’t looked at the SAMP spec in a while).

The AstroPy SAMP documentation includes an example for how you can set up something that will send a file when needed at Sending and Receiving Tables and Images over SAMP — Astropy v5.0.2 although it may need tweaking if you want to send multiple files or you don’t want your code sitting around waiting for topcat to reply.

3

The mtype values that topcat responds to (as there are other messages that are not relevant to topcat) can be found at Messages Received

Thanks a lot! This first link is really helpful.

The way that SAMP works is that data is sent around via URLs, so some code on your end needs to run a simple webserver that will provide access to the data you want to send (I don’t think this has changed, but I haven’t looked at the SAMP spec in a while).

Does this mean that the table must be hosted somewhere and not only on jupyter’s memory?
Or would the jupyter “localhost:…” count as a local webserver?

The AstroPy SAMP documentation includes an example for how you can set up something that will send a file when needed at Sending and Receiving Tables and Images over SAMP — Astropy v5.0.2 although it may need tweaking if you want to send multiple files or you don’t want your code sitting around waiting for topcat to reply.

The second link was precisely the example I used to build the piece of code on my original post, but thanks anyway!

4

So jupyter (whether the lab or notebook interface) has some support for data access, but I admit to being really hazy on the details. For instance, I am pretty sure there’s security implications that may need knobs to be turned or buttons to be pressed.

If you’re using jupyter lab - rather than the older notebook interface - then you can try using the URL for the file, as long as it’s accessible to the notebook - e.g. JupyterLab URLs — JupyterLab 3.3.2 documentation. However, I haven’t tried this so I could just be wasting your time.