Extract unmasked elements from Astropy Table

Astropy
, ,
1

Let’s kick the tires of this forum!

I’m trying to extract the non-masked rows from an Astropy Table. From the Astropy Table documentation, I can get the indices of the masked, but how do I get the inverse?

To get the indices of masked elements, use an expression like:
>>> t['a'].mask.nonzero()

I’ve tried
nans = t['a'].mask.nonzero()
unmasked_t = t['a'][-nans]

But that didn’t work.
bad operand type for unary -: 'tuple'

Any pointers?

Thanks!
Kelle

1 Like
2

I think (without testing it) you could use np.logical_not on the mask before the nonzero call.

I.e.

unmasked = np.logical_not(t['a'].mask).nonzero()
3

it works!

any thoughts on this approach vs using Pandas .notna() ?

4

Isn’t the mask a NumPy boolean array? then the non-masked elements are just notmasked = t['a'][~t['a'].mask]
Or do you want the indices of the rows?

1 Like
5

Addendum: ~ can sometimes be finicky. I’ve taken to using not instead, even if it’s a Boolean array.

6

I think what you want is t[~t['a'].mask] to get a new table where rows with masked elements in 'a' have been removed:

In [29]: t
Out[29]: 
<Table masked=True length=2>
  a     b  
int64 int64
----- -----
    1    --
   --     4

In [30]: t[~t['a'].mask]
Out[30]: 
<Table masked=True length=1>
  a     b  
int64 int64
----- -----
    1    --

Pandas has powerful ways to do selection on columns, but I’m not sure there is really a point in reimplementing that in Astropy. In Astropy columns are simple Numpy arrays, so most simple things can be in a slightly more verbose way (compared to Pandas) with simple Numpy selections. And for more advanced usage it’s easy to convert the table to Pandas (t.to_pandas()).

1 Like