Release 1.2.0

Uh oh, it looks like we got a little out-of-control again and changed… well, we changed a lot in SatDump. But never fear, this update contains a lot of great features that many of our users will appreciate - whether you are an amateur, or work in the professional space.

Please note, this is a major update from 1.1.4 - many internal systems have been reworked, and new products may not load in older versions. Saved products might be arranged differently, and there are compatibility problems when mixing/matching versions of the SDR Server and SatDump GUI/CLI as well. As a general rule, it’s a good idea to update all your systems to SatDump 1.2.0 at the same time.

Additionally, we’ve had some reports of issues updating to the latest build on Linux (self-built) and Windows (portable only). Windows Portable users should not simply extract the .zip into their current program directory; instead, they should extract it into a new directory, and copy the config folder over. Linux users should uninstall before pulling and re-building, or delete their /usr/lib/satdump folder and re-install.

With that out of the way, let’s get to the good stuff!

New Features

Geostationary Satellite Products

One of the core features of SatDump is its product system - along with calibration for accurate, useful information. For a long time, you could load imagery from LEO satellites into the viewer and make all kinds of composites and projections - but the same could not be said for any Geostationary Satellites. Raw data would be saved and some basic false-color composites would be auto-generated, but… that was about it. And that changes with this release.

In SatDump 1.2.0, Geostationary Satellites now support the product system similar to LEOs. To facilitate this, imagery from select Geostationary satellites is now organized in dated subfolders, with an associated product.cbor file, like this:

Just like its LEO counterparts, the geo cbor file contains data such as calibration, projection, and timestamp info. If enabled, SatDump will automatically use this data to generate RGB composite imagery. You can also generate imagery manually by loading the product.cbor file in the viewer, and creating a custom-made composite.

This change opens the door to many possibilities when processing Geostationary satellite data with SatDump. Instead of simply saving raw data, we can now make composites such as the standard “Enhanced IR” and “Water vapor” composites found in other programs. On GOES HRIT, we have made them match the products available on NOAA STAR

However, we can do better than these basic LUT-based composites. SatDump’s composite system can create advanced equation-based composites, enabling us to produce incredible composites such as Nighttime Microphysics and Split Window Difference, which again match the output of NOAA STAR/CIRA:

Some additional composites are provided with this release that are not generated by NOAA/CIRA, but are specified in the GOES-R Quick Guides, such as Fog Brightness Temperature Difference. Here you can see fog over Cuba:

Its not all GOES - many of these composites work on Himawari, GK-2A, and Elektro as well. However, there is more work to do on this front.

Further, we can now make more appealing day/night composites, such as this False Color/IR merge that is available with several Geostationary satellites:

Finally, a helper function has been added to the composite system that can remove the background from behind your imagery. This can be especially appealing for products where the background is not already black.

Himawari Day Cloud Convection

Geo Products support has been added for the following satellites, with potentially more to come:

• Elektro-L LRIT/HRIT
• GOES-R HRIT
• GK-2A LRIT/HRIT
• HimawariCast

METEOR KMSS Support

All METEOR-M satellites have several downlinks. LRPT and HRPT are fully supported in SatDump, but the X-Band links have until now been more evasive. However, recent discoveries thanks to monitoring by Fred Jansen and Andrew (@MrFentazis) have revealed METEORs, at least M2-2 and M2-4 operate KMSS at 60m/px resolution in direct broadcast mode while in range of most of Europe & Russia. After a lot of reverse-engineering work, initial support to dump the imagery was implemented. Support for QPSK replay modes is also implemented, but this will require larger dishes in comparison to the direct broadcast mode current being received quite well even on 120cm.
Do note this is not as simple of an instrument to process as many others, so proper channel alignement, projections and so on will most likely be refined in later releases.

As of now, exclusing Proba-1 and TUBIN which both only dump select areas when commanded - this is the highest resolution decodable by SatDump at the moment with such a large swath! :-)

Example images

Sample KMSS projection, extremely experimental!

New Projection System and GeoTIFF

As we said earlier, things changed. A lot. But why projections?

One feature that was being requested very often was the ability to export the data SatDump can generate into formats other software could work with, in order to use it in GIS programs and so on. Unfortunately, most standard file formats for this purpose are messy, to say the least and pretty much non-starters just due to their complexity and poor documentation. That is (somewhat) except for GeoTIFF, which is at the very least mostly standard (though… PROJ likes to take liberties in writing custom strings that are NOT in the official standard).

However, this raised a major problem : the projections implemented in SatDump until now were not exactly like the accepted “standard” used in most situations. This made writing/reading geotiffs and converting them to/from SatDump’s formats very complicated, and it would have been a poor approach either way.

Therefore, rather all of SatDump’s internals were re-wrote in order to match the usual standards, which made implementing GeoTIFF a whole lot easier. Hence now it is possible to export or import GeoTIFF by simple saving a Stereo/Equirectangular/UTM projection as .tif in the viewer.

Example reprojecting GeoTIFF data from the BPP (Meteor products)

This major rework initially for GeoTIFF also brought an initial “automatic” mode, where SatDump will automatically determine the projection required to cover whatever you are trying to re-project. (Do note this is still somewhat experimental, especially on Stereo)

Additionally, the projection layer system has also been changed in its operation. Before, each instrument loaded could be “enabled” or “disabled” as a layer, but any changes to the composite would reflect in the projected layer instantly. This limited possible composites when it was about using the same instrument in different ways. Now, you can instead add the current state of a composite as a layer, which will be stored and not affected by further changes, even letting you add more than one.

Finally, a CLI tool is now available in order to reproduce anything you can achieve using the GUI. The documentation is not yet available for this command, but it will be soon. See below for an example of how to use the command, but do note all parameters from composites and map overlays.

satdump project -layer --type product --file http://192.168.1.71/2024-03-12_20-38_noaa_apt_137.1%20MHz/product.cbor --equation "ch4" --equalize -target --type stereo --center_lat 55 --center_lon 25 --scale 2400 --width 2048 --height 2048 --file test_proj_cli.png --draw_map_overlay

Fill Missing Lines

An often-requested feature is the ability to “heal” or “fill in” missing data caused by interference or signal drop-outs. Starting with SatDump 1.2.0, “Fill Missing Data” (or --fill_missing from the command line) is now available for GOES-R HRIT and METEOR LRPT pipelines! This feature will interpolate good data into missing/bad sections to “heal” the image. It must be explictly enabled at decode time as an acknowledgement that you are modifying decoded satellite data.

METEOR was a natural fit for fill missing, as we already had a good way to track which segments were good, and which were bad. Both the original image and the filled-in version are saved when using this feature. The results are pretty good with mild-moderate data loss:

GOES HRIT, on the other hand, required a little bit more work. Previous versions of SatDump would not track which lines in a segment were bad - meaning some lines would be saved in the wrong location if data was missing. After fixing this bug, adding the ability to fill in missing lines works much the same way as it does on METEOR. Only the filled-in version is saved.

As a note: we’ve already been asked if this feature will be seen with other satellite decoders. For right now, the answer is probably not - but maybe. Filling in missing data is a tricky problem to solve in many situations. Sometimes, you don’t even have a way to know what data is bad, what is good, or if any lines were missed. If any situations arise where it is feasible/makes sense to implement it, we can certainly look into it, but no promises.

Orbcomm STX Plotter

Have you ever wanted to measure what parts of the sky you can see well with your 137 MHz antenna? Starting with SatDump 1.2.0, we’ve added a way to do this using Orbcomm satellites!

Orbcomm STX is the narrow yet strong signals you often see alongside NOAA APT and METEOR LRPT. There’s no publicly accessible data in it, other than some basic positioning data - and we can use this to our advantage. By mapping out which Orbcomm satellites can be heard over time and their locations, you can draw a plot that shows which parts the your sky your antenna can “see”. This information can be used to tune your antenna - or, just for fun!

METEOR Telemetry

In SatDump 1.1.3, support was added to decode telemetry from NOAA DSB, HRPT, and GAC. In this release, similar support has been added for METEOR M2-x LRPT and HRPT! On LRPT, telemetry is provided on what MSU-MR unit is in operation and the temperature/current of various parts of the instrument. The same information is available on HRPT, along with additional digital telemetry.

Added Composites

Nearly every team member pitched in this time around to add many great new composites - and improvements to old ones. Note many Geostationary Satellite composites have been added that are not listed here.

• AIRS HD: Re-added a basic composite for the AIRS instrument.
• KMSS: Added CIR, NDVI, OSAVI, and Vegetation Classification composites.
• MERSI-LL: Added an AVHRR-equivalent 543b IR False Color composite.
• MERSI-RM: Added Day Microphysics.
• MCIR, MCIR Rain, and MSA: These composites have been equalized to prevent being too dull or too bright, resulting in a more appealing product each time.
• SEVIRI: Added 221, 321, Natural Color, Microphysics, Water Vapor, and several AVHRR-Equivalent composites
• VIIRS: Cloud Type RGB, Cloud Phase RGB, Dust/Ash RGB, Snow RGB, and Fire Temperature RGB composites have been added.

From left to right: SEVIRI processed by JA6UAR, AIRS processed by lego11, KMSS received by Andrew, and VIIRS processed by g5juergenm

Image System Rewrite

This is more of an under-the-scenes change for most if not all users, but the implications are pretty significant so it’s worth explaining in a bit more detail.

SatDump needs to do a lot of image manipulation, and unavoidably these images need to be stored into RAM at some point. However, unlike your average picture, satellite imagery has to be stored with more than 8-bits of precision to take full advantage of it. It is very inefficient to, for example, store 12-bits data in RAM - so whenever more than 8 is needed, you have to store 16-bits instead - which twice as large already. The way things were written in SatDump until now meant that this choice of precision had to be done beforehand, and we had to chose between either writing code that could only handle 8-bits, and be more efficient memory-wise, or code that could 8 or 16 at the cost of 2x the memory utilization. This meant that very large parts of the code were stuck to 16-bits, especially the viewer.

Until now, this hadn’t been a real problem as, in most cases, SatDump was being used on PCs with enough RAM for this to not even be noticed. But with the GEO products being re-written, this became an obvious problem when attempting to run on low-end hardware, such as a Raspberry Pi. This triggered a major rewrite affecting pretty much everything in SatDump to make image handling dynamic. Products can now load only load 8-bits if this was all that was needed, resulting in halved RAM utilization for GOES products and many others.

Considering the scale of these changes however, it’s not unlikely some bugs have slipped through in the image system or other things as a result. If you find anything weird, please report!

Demonstration of memory reduction in GOES HRIT processing

Other Features

Some new features are small - but still important! Minor features added in this release include:

• macOS: The macOS builds now come with support for many more SDRs, including BladeRF, LimeSDR, PlutoSDR, SDRPlay, and USRP! This is on top of all the SDRs supported previously. SDRPlay support requires the installation of SDRPlay API 3.15 from here.
• macOS: The build process has changed to add flexibility, and to ensure compatibility with all versions of macOS 10.15 and newer. If you build from source on macOS, please review the build documentation before updating.
• Added support for METEOR M2-4.
• NOAA APT: Improved noise detection and auto-crop to reduce aggressively cropped images, and improve data reliability.
• NOAA APT: SatDump will now correct pass time errors within ~15 seconds to ensure proper map overlay/projection alignment when your system clock is incorrect.
• METEOR LRPT: Added GUI option to disable Reed-Solomon checks, to allow for imagery reception when the satellite transmits bad codes.
• METOP AHRPT: Added checkbox to ignore transmitted TLEs as they can sometimes be incorrect.
• Inmarsat: UDP packets now contain the SatDump version if the station id is set.
• Added/improved packet pipelines for IM-I, FORMOSAT 5/7, SAOCOM 1A/1B TLM, Queqiao-2, and WSF-M S-Band DB.
• SDRPlay: Added support for RSP1B and RSPdx-R2.
• UDP Source has been renamed to Network Source. It can now accept samples over nng pub/sub as well.
• Added support for saving/loading images in the QOI format, for fast, efficient storage of images
• The viewer “Load Dataset” and “Load Product” options have been merged into a single “Open” option.
• Added a decimal precision config for az/el values when sending data to your rotator to prevent issues.
• Added easy-to-use DVB-S/S2 pipelines, which can be used to demodulate signals such as the upcoming HamTV module on the ISS.
• Added the ability to set a minimum elevation to start/stop processing or recording per satellite in autotrack.
• Added Herobrine.

Photo courtesy of Scott Chapman

Performance Improvements

There was a big focus this time around on performance and memory optimization. Here is a sampling of optimizations we have done throughout the program:

• In the past, simple composites have been made with an equation, and complex ones in Lua. Starting with 1.2.0, complex composite code can now be written in C++ and loaded as a plugin! C++ composites are significantly faster than their Lua counterparts, and we use this to our advantage with several of the new Geo composites. Additionally, our cloud underlay/MCIR/MSA composites have been moved to C++ to gain this performance boost.
• General performance boosts for all composites
• Improved map overlay performance by almost 2x
• Improved LRPT decoder performance

Bugs Fixes

Spring has arrived, and with it all the bugs that love to start crawling around this time of year. In our software too - and we removed as many as we could. Here are the bugs we’ve fixed in SatDump 1.2.0.

• METEOR LRPT: Fixed minor channel alignment bugs.
• Fixed incorrect map overlays near the poles.
• Fixed parameter conflicts with the --port flag in the CLI. If you used this flag anywhere, please verify the current usage with the docs.
• Fixed a bug that could result in METEOR HRPT data becoming unprojectable
• Fix DST bug with SatDump basebands.
• Many performance/stability updates to DVB-S/S2 demodulation.
• Improved SatDump UI/SDR Server recoverability when communication between the programs has been interrupted.
• Fixed some minor USRP bugs.
• Fixed bug where AMSU always saved as png.
• Fixed incorrectly breaking up Himawari images from the GOES-West HRIT relay.
• Fixed a crash on Windows when UDP ports 4567 or 7890 are bound, in some situations.
• Fixed many, many other crashes throughout the program.

Wrapping Up

As always, if you come across any bugs, feel free to open an issue on GitHub or reach out to us on Matrix!