--- url: /2015/07/iugg-2015.md --- Daniel Santillan, EOX, presenting virtual science workstation for Swarm mission data to Geomagnetists meeting at IUGG 2015 in Prague [(download VirES poster)](/content/VirES_IUGG_Poster.pdf) --- --- url: /2015/11/3d-video-awesomeness-continued.md --- We are looking into creating new globe video visualizations with different datasets. We have another really nice example provided by [NASA](http://svs.gsfc.nasa.gov/cgi-bin/details.cgi?aid=30709) (Credit: NASA’s Goddard Space Flight Center) which shows a 12 month cycle of plant life on earth. To get additional information on what colors represent and to have a closer look at the globe click on the fullscreen button (on the top right)! It was easily possible to create this animated widget thanks to [mapalupa](http://mapalupa.com)! We have written another post with some nice imagery as video on the globe in a [previous blog post](/2015/11/videos-in-3d/). --- --- url: /impressum.md --- Hier finden sich alle Informationen nach der (allgemeinen) Informationspflicht gemäß § 5 E-Commerce-Gesetz (ECG), § 14 Unternehmensgesetzbuch (UGB), § 63 Gewerbeordnung (GewO) und § 25 Medienesetz. Diese Vorschriften können auf der kostenlos zugänglichen Website [www.ris.bka.gv.at](https://www.ris.bka.gv.at) nachgelesen werden. ## Medieninhaber und Eigentümer Anschrift:\ Thurngasse 8/4\ 1090 Wien\ Österreich / Austria Kontakt:\ Firma: EOX IT Services GmbH\ Firmenbuchnummer: FN 312068z\ Firmensitz: Wien\ Handelsgericht: Wien\ UID ATU64265057 Gewerbe: Dienstleistungen in der automatischen Datenverarbeitung und Informationstechnik\ Fachgruppe: Unternehmensberatung und Informationstechnologie\ Berufsgruppe: IT-Dienstleistungen Unternehmensgegenstand:\ Dienstleistungen im IT-Bereich einschließlich Entwicklung von Software, Erstellung von Studien und Gutachten, Koordination, Management und Durchführung von Projekten, Handel mit Waren aller Art. [Informationspflicht lt. §5 E-Commerce Gesetz, §14 Unternehmensgesetzbuch bzw. §63 Gewerbeordnung und Offenlegungspflicht lt. §25 Mediengesetz](https://www.wkoecg.at/Ecg.aspx?FirmaID=d9e35889-7005-46a1-91b4-f51efdabe7a5) ## Datenschutzhinweis Durch die Nutzung unseres Internetangebotes, werden von Ihnen keine persönlichen Daten gespeichert. Es werden nur allgemeine Daten wie der Name Ihres Internet-Service-Providers, die Web-Adresse (URL), von der Sie auf unsere Seiten kommen und die Internetseiten, die Sie bei uns besuchen gespeichert und ausschließlich für statistische Zwecke verwendet. Persönliche Daten werden nur bei der Verwendung bestimmter Services erhoben. Diese Daten entsprechen ausschließlich den Angaben, die Sie selbst machen. Persönliche Daten, die Sie uns über die Website oder per E-Mail zusenden, werden nicht an Dritte weitergegeben, nur für den vorgesehenen Zweck verwendet und gemäß den jeweils geltenden datenschutzrechtlichen Bestimmungen behandelt. Datenschutzbeauftragter gemäß DSGVO:\ Dr. Christian Schiller\ EOX IT Services GmbH\ Thurngasse 8/4, 1090 Wien\ christian.schiller@eox.at +43 664 8157841 [Privacy Notice](/privacy-notice/) ## General Terms and Conditions [General Terms and Conditions](/terms-conditions/) ## Disclaimer [Disclaimer](/disclaimer/) EOX website v{{ theme.version }} --- --- url: /about.md --- #### Unlocking the Power of Earth Observation Data in the Cloud At EOX, we make vast amounts of satellite data accessible, easy to manage, and ready to use directly on the web. Based in Austria, we build open-source tools, cloud-native infrastructure, and high-resolution global imagery to help users extract maximum value from our changing planet. #### Who We Are EOX was founded when we recognized that tedious geospatial data processing on traditional desktop software was becoming outdated. In the digital era, people need seamless data access in the cloud and on the web. Moving away from monolithic software packages that required installation, we pioneered browser-based technologies for direct data visualization and manipulation. This approach has made Earth observation data significantly more accessible to the public. Today, we support a growing community of geospatial developers, remote sensing experts, cartographers, and data scientists. #### Our Mission Our focus is on extracting the most value from the vast amounts of data collected by Earth observation satellites. We are deeply committed to supporting open-source initiatives and developing software that enables open access to data consumption online. #### How We Work & What Sets Us Apart As Earth observation data volumes and processing demands have grown exponentially, we adapted our solutions to be fully cloud-native. This improves resource allocation, lowers infrastructure costs, and increases deployment speed. * **Open-Source Data Management.** Our EOxServer software has evolved alongside EOX since our inception. Alongside our versatile View Server suite, it registers Earth observation products in cloud environments and exposes them via open interfaces. * **Cloud Infrastructure Setup.** To solve the complexity of configuring large-scale environments, we developed [EOxHub](https://hub.eox.at/), our comprehensive cloud management and setup service. * **High-Resolution Global Imagery.** Recognizing the need for high-quality base maps, we developed software libraries for large-scale cloud processing campaigns. This powers [EOxCloudless](https://cloudless.eox.at/), providing off-the-shelf global satellite composites alongside support for bespoke processing. Customers choose us for high-resolution cloudless imagery at competitive prices, tailored to their specific needs. #### Responsibility & Governance As a supplier to ESA, we fully adhere to [ESA's Corporate Social Responsibility (CSR) Code of Conduct](https://www.esa.int/About_Us/Climate_and_Sustainability/ESA_CSR_Code_of_Conduct_Strengthening_ESA_corporate_responsibility) and require our suppliers to do the same. Our core commitments include: * **Legal Compliance.** We comply with all applicable laws and regulations in Austria, where we are based, and in the countries where we conduct business. * **Human Rights & Fair Treatment.** We uphold internationally recognized human rights, treat our employees fairly, and provide dignified working conditions in accordance with Austrian labor laws. * **Safe Workplaces & Fair Compensation.** We ensure a safe and healthy work environment, offering measures and financial support for home working as per Austrian labor law. Employee wages, benefits, and working hours are fair and compliant with labor guidelines. * **Environmental Protection.** We prioritize environmental protection, managing and mitigating environmental risks and impacts across our operations, products, and supply chain - particularly as all our products are virtual and paperless. * **Ethical Business Conduct.** We conduct business ethically, refraining from fraud and fairly competing for business opportunities. Any gifts or giveaways are strictly for marketing purposes and consistent with reasonable market customs. * **Accurate Record-Keeping.** We maintain accurate records of business transactions and refrain from altering any record to misrepresent or conceal underlying transactions. * **Timely Payments.** We practice fair and timely payment to suppliers and employees in accordance with agreed guidelines. * **Information Security.** We protect information and ensure confidentiality, maintaining GDPR compliance and ISO certification. * **Governance & Reporting.** Our company has established proper governance, ethical guidelines, and compliance processes, with clear communication and reporting channels. --- --- url: /_team/behairia.md description: >- Software Engineer with a passion for writing pristine code to create innovative, and impactful products. Loves travelling and exploring the world. --- --- --- url: /algorithm-provider-terms-and-conditions.md --- (version as of September 12th 2023) Download pdf version [here](/content/algorithm_provider_terms_and_conditions.pdf) to save or print this algorithm provider terms and conditions. ## Section 1 - These EOX Algorithm Provider Terms EOX IT Services GmbH ("EOX") develops, operates, and provides various online services ("EOX Services"), for example hosted JupyterLab instances an online service to provide an interactive computational environment to help exploit Earth Observation data and also run "Algorithms" in a headless manner. The term "Service Provider" used below refers to EOX. Access to, and use of, any EOX Service related to providing algorithms to third parties constitutes acceptance of the following EOX Algorithm Provider Terms. ## Section 2 - Use of and Providing an Algorithm for use by Third Parties via an EOX Service When accessing and using any EOX Service related to providing algorithms to third parties the user ("Algorithm Provider") is required to sign an Algorithm Provider agreement with EOX and shall be obliged to observe the following: a. The Algorithm Provider shall comply to the [EOX Contributions Terms](/contributions-terms-and-conditions/) in its latest version in its entirety whereas any provided algorithm shall be considered as Contribution as defined and used in these terms. b. The Algorithm Provider shall be solely responsible to obtain the required Intellectual Property (IP) rights inherent to the algorithm to publish the algorithm on the EDC marketplace and offer it to Third Parties for execution. c. The Algorithm Provider shall be solely responsible for the correct functioning of provided algorithms. d. The Service Provider reserves the right - though without accepting any obligation - to refuse or remove any provided Algorithm that, in the Service Provider’s sole discretion, violate these Terms or otherwise threaten the security, integrity, or availability of the EOX Services. e. The Algorithm Provider shall provide a textual as well as a programmatic description of the algorithm and its input parameters and output as part of the on-boarding process. f. The Algorithm Provider shall be solely responsible for any third party API integration in provided algorithms. In particular this includes obliging to the third party API Terms and Conditions, Service Level Agreements (SLA), and usage restrictions. g. The Service Provider makes available a regular report including details on algorithm(s) consumption and generated revenue. The generated revenue is stated subtracting the agreed EDC EOxHub execution costs per started hour and subtracting the agreed revenue share. h. The Algorithm Provider shall provide an automatic way to retrieve cost estimates for algorithm runs based on concrete values for the defined and agreed input parameters considering all algorithm inherent costs (e.g. subscriptions for Third Party APIs), costs for execution on the selected target environment, as well as the agreed revenue share based EDC marketplace fee. The Algorithm Provider shall acknowledge that these cost estimates are binding. The Algorithm Provider shall be solely responsible to cover any expenses incurred and exceeding the cost estimate when running the algorithm using the exact same concrete input parameter values. i. The Service Provider reserves the right to suspend any algorithm run where the current EDC EOxHub execution costs are exceeding the provided cost estimate. j. In order to receive payouts the Algorithm Provider shall present invoices to the Service Provider according to the provided revenue report and complying with Austrian and European Union laws and regulations. k. The Algorithm Provider agreement shall define the * target environment * input parameters including defaults * applicable delivery methods * interval of the provided report (quarterly, monthly, etc.) * percentage of the revenue share * minimum consumer price per algorithm run (execution) * maximum duration per algorithm run (execution) l. The Algorithm Provider shall maintain capacity to reproduce consumer executions any time including an active EDC EOxHub Workspace according to the defined target environment subscription in order to mitigate any problems e.g. based on specific input parameters. --- --- url: /_team/miskovica.md description: >- User interface designer with a focus on enhancing user experiences across digital platforms. --- --- --- url: /_team/gruboeka.md description: >- 15+ years in market & data intelligence. Passionate about turning fragmented farm and satellite data into something a bank or a farmer can actually use. Lived abroad for 20 years. Loves collaborative teamwork, travelling, music, dancing. --- --- --- url: /applications.md --- --- --- url: /_team/zz_guest_smitha.md --- --- --- url: /author/santilland.md --- --- --- url: /author/davise.md --- --- --- url: /author/meissls.md --- --- --- url: /author/smitha.md --- --- --- url: /author/achtsnits.md --- --- --- url: /author/schindlerf.md --- --- --- url: /author/dolezalovat.md --- --- --- url: /author/ungarj.md --- --- --- url: /author/brands.md --- --- --- url: /author/sevcikp.md --- --- --- url: /author/lehotskak.md --- --- --- url: /author/csonkab.md --- --- --- url: /author/paris.md --- --- --- url: /author/pasparugar.md --- --- --- url: /author/triebnigg.md --- --- --- url: /author/mabeyam.md --- --- --- url: /author/schillerc.md --- --- --- url: /author/raubergj.md --- --- --- url: /author/bucekl.md --- --- --- url: /author/pacesm.md --- --- --- url: /author/behairia.md --- --- --- url: /author/cebullad.md --- --- --- url: /author/moldenhauerh.md --- --- --- url: /2021/08/average-colors-of-the-world.md --- *Couple of weeks ago we have noticed the [article](https://erdavis.com/2021/06/26/average-colors-of-the-world/?ref=sidebar) getting attention around [twitter](https://twitter.com/simongerman600/status/1416128010906963970?s=20). As it is using our [WMS](https://tiles.maps.eox.at/wms?service=wms\&request=getcapabilities) EOxMaps endpoint we have asked [Erin Davis](https://erdavis.com/) if we could host her idea here. At EOX we endorse brilliant and unique ideas that utilize our tools and data. Thank you for this most interesting contribution. Now follows the original article.* *** I wrote that I wanted to expand creatively beyond maps in 2021, but here we are, halfway in, and half my posts are maps. In my defense, I made these last year and just haven’t gotten around to posting them yet! They’ve been sitting in a folder since December, mocking me. It’s time to get these guys off my computer and out into the world so I can move on to other things. Since it’s been so long, I really don’t remember what inspired me to make these. I’m sure memories of this amazing [NYT Upshot article](https://www.nytimes.com/interactive/2020/09/02/upshot/america-political-spectrum.html) were involved. The difference here is I didn’t even try to tell a story with these maps… I just made them as something pretty to look at. ## Average colors of the USA A question: do you think it’s best practice to label US states? I usually haven’t in my personal work, but I do in my day job… the labeled look might be growing on me. ## Average colors of the world ## How I did it To do this, you’ll need a few things: 1. A shapefile of the areas you want the colors for (from e. g.: ) 2. [QGIS](https://www.qgis.org/en/site/) installed (or GDAL/Python, but installing QGIS will set it all up for you) 3. R and [RStudio](https://www.rstudio.com/) installed In short, my process was the following: 1. Wrote a script (in R) to find the bounding boxes of all the areas I was interested in 2. In R, downloaded pictures of those areas from Sentinel-2 3. In R, wrote a script that created a series of [GDAL](https://gdal.org/) commands to: 1. Georeference the downloaded image to create a geotiff 2. Crop the geotiff to the borders of the area 3. Re-project the geotiff to a sensible projection 4. Ran that GDAL script 5. In R, converted the geotiffs back to pngs, and found the average color of the png Basically, I did everything in R, including creating the GDAL script. I just briefly stepped outside of R to run that script then jumped back in to finish the project. What the GDAL script does is possible within R using the [qgisprocess](https://github.com/paleolimbot/qgisprocess) package, but I’m pretty sure (based on 0 testing tbh) that GDAL is faster. Side note: I feel really embarrassed lifting the curtain and showing my wacky processes to get viz done. I’ve often avoided sharing, especially in professional situations, for fear that better-informed people will laugh at me. But I think it’s good to have more people sharing their code so we can all improve. All my GIS knowledge (and quite a lot of my general coding knowledge) is self-taught, and I owe a huge debt to people online who bothered to explain how they did things. My point? I don’t know… I guess if you know of a better way to do this, please let me know! But also I was really proud I figured out this much on my own. ```r # install the necessary packages. only need to run this once # install.packages(c('sf', 'magick', 'dplyr')) # load in the packages. you need to do this every time you open a fresh R session library(magick) library(sf) library(dplyr) ``` Get your shapefile loaded and prepped: ```r # set your working directory, the folder where all your files will live setwd("C:/Users/Erin/Documents/DataViz/AvgColor/Temp/") # create the subfolders that you'll need dir.create(file.path(getwd(), "satellite"), showWarnings = FALSE) dir.create(file.path(getwd(), "shapes"), showWarnings = FALSE) # load in your shapefile. I'm using the US states one from here: # https://www.census.gov/geographies/mapping-files/time-series/geo/carto-boundary-file.html shapes <- read_sf("cb_2018_us_state_20m.shp") # we need to separate out the shapefile into individual files, one per shape for (i in 1:nrow(shapes)) { shape <- shapes[i, ] write_sf(shape, paste0("./shapes/", shape$NAME, ".shp"), delete_datasource = TRUE) } ``` Download the satellite imagery ```r # load in your shapefile. I'm using the US states one from here: # https://www.census.gov/geographies/mapping-files/time-series/geo/carto-boundary-file.html shapes <- read_sf("cb_2018_us_state_20m.shp") # for each shape in the shapefile, download the covering satellite imagery for (i in 1:nrow(shapes)) { shape <- shapes[i, ] %>% st_transform(4326) box <- st_bbox(shape) url <- paste0("https://tiles.maps.eox.at/wms?service=wms&request=getmap&version=1.1.1&layers=s2cloudless-2019&", "bbox=", box$xmin-.05, ",", box$ymin-.05, ",", box$xmax+.05, ",", box$ymax+.05, "&width=4096&height=4096&srs=epsg:4326") # note: this assumes your shapefile has a column called NAME. you'll need to change this to a # different col if it doesn't download.file(url, paste0("./satellite/", gsub("/", "", shape$NAME), ".jpg"), mode = "wb") } ``` Create a GDAL script to transform those square satellite images into cropped, reprojected geotifs. Note: for some reason, GDAL refuses to deal with any folders other than my temp directory. I couldn’t figure out how to get it to successfully write to my working directory, so I just did stuff in the temp one. Minor annoyance, but what can you do. ```r # time to create the GDAL script if (file.exists("script.txt")) { file.remove("script.txt")} # first, georeference the images - convert them from flat jpgs to tifs that know where on the globe the image is for (i in 1:nrow(shapes)) { shape <- shapes[i, ] box <- st_bbox(shape) file_origin <- paste0(getwd(), '/satellite/', shape$NAME, '.jpg') file_temp <- paste0(tempdir(), "/", shape$NAME, '.jpg') file_final <- paste0(tempdir(), "/", shape$NAME, '_ref.tif') cmd1 <- paste0("gdal_translate -of GTiff", " -gcp ", 0, " ", 0, " ", box$xmin, " ", box$ymax, " -gcp ", 4096, " ", 0, " ", box$xmax, " ", box$ymax, " -gcp ", 0, " ", 4096, " ", box$xmin, " ", box$ymin, " -gcp ", 4096, " ", 4096, " ", box$xmax, " ", box$ymin, ' "', file_origin, '"', ' "', file_temp, '"') cmd2 <- paste0("gdalwarp -r near -tps -co COMPRESS=NONE -t_srs EPSG:4326", ' "', file_temp, '"', ' "', file_final, '"') write(cmd1,file="script.txt",append=TRUE) write(cmd2,file="script.txt",append=TRUE) } # next, crop the georeferenced tifs to the outlines in the individual shapefiles for (i in 1:nrow(shapes)) { shape <- shapes[i, ] file_shp <- paste0(getwd(), '/shapes/', shape$NAME, '.shp') file_orig <- paste0(tempdir(), "/", shape$NAME, '_ref.tif') file_crop <- paste0(tempdir(), "/", shape$NAME, '_crop.tif') cmd <- paste0("gdalwarp -cutline", ' "', file_shp, '"', " -crop_to_cutline -of GTiff -dstnodata 255", ' "', file_orig, '"', ' "', file_crop, '"', " -co COMPRESS=LZW -co TILED=YES --config GDAL_CACHEMAX 2048 -multi") write(cmd,file="script.txt",append=TRUE) } # reproject the shapes to a reasonable projection, so they're not as warped and stretched for (i in 1:nrow(shapes)) { shape <- shapes[i, ] center <- st_centroid(shape) %>% st_coordinates utm <- as.character(floor((center[1] + 180) / 6) + 1) if (nchar(utm) == 1) {utm <- paste0("0", utm)} if (center[2] >=0) { epsg <- paste0("326", utm) } else { epsg <- paste0("327", utm) } file_orig <- paste0(tempdir(), "/", shape$NAME, '_crop.tif') file_mod <- paste0(tempdir(), "/", shape$NAME, '_reproj.tif') cmd <- paste0("gdalwarp -t_srs EPSG:", epsg, ' "', file_orig, '"', ' "', file_mod, '"') write(cmd,file="script.txt",append=TRUE) } ``` **Now’s the time to step out of R and into GDAL** 1. In the start menu, search for OSGeo4W Shell and open it. You should see a command-line style interface. 2. In your working directory, find and open script.txt. It should have a bunch of commands in it. 3. Copy all the commands in script.txt and paste them right into the OSGeo4W Shell. Press enter to run the commands. 4. If you have many shapefiles, this might take a while. Wait until everything has finished running before moving to the next step. **Now, hop back into R.** Move the files we just created back into the working directory. ```r # move the files from temp back to the wd files <- list.files(tempdir(), pattern = "_reproj.tif") for (i in 1:length(files)) { file.rename(from=paste0(tempdir(), "/", files[i]),to= paste0("./reproj/", files[i])) } ``` Now get the average color of each image. Here I use a slightly different method than I did for the plots shown above. While writing up this code, I found a neat trick (resizing the image down to 1 px) that works faster than what I did before (averaging the R, G, and B values of each pixel). ```r # get the average color of each file files <- list.files("./reproj/") colors <- NULL for (i in 1:length(files)) { # remove the white background of the image img <- image_read(paste0("./reproj/", files[i])) img <- image_transparent(img, "#ffffff", fuzz = 0) # convert to a 1x1 pixel as a trick to finding the avg color. ImageMagick will average the # colors out to just 1. img <- image_scale(img,"1x1!") # get the color of the pixel avgcolor <- image_data(img, channels = "rgba")[1:3] %>% paste0(collapse="") # save it in a running list colors <- bind_rows(colors, data.frame(name = gsub("_reproj.tif", "", files[i]), color = paste0("#", avgcolor))) } # save the color file if you need it later write.csv(colors, "colors.csv", row.names = F) ``` Et voilà! A lovely csv with the average color of each of your regions. And if you’d like to plot it all pretty, here’s how I did it in R. For this example, I’m only going to plot the lower 48 states. ```r # install the necessary packages. only need to run this once install.packages(c('sf', 'ggplot2', 'ggthemes', 'dplyr')) # load in the packages. you need to do this every time you open a fresh R session library(sf) library(ggplot2) library(ggthemes) library(dplyr) # set your working directory setwd("C:/Users/Erin/Documents/DataViz/AvgColor/Temp/") # load the data shapes <- read_sf("cb_2018_us_state_20m.shp") colors <- read.csv("colors.csv") # you'll need to change the join column names here if they're different than mine shapes <- inner_join(shapes, colors, by = c("NAME" = "name")) # transform the shapefile to a reasonable projection # if you're unsure here, google "epsg code" + "your region name" to find a good code to use shapes <- st_transform(shapes, 2163) # for example purposes only, I'm removing Alaska, Hawaii, and PR shapes <- subset(shapes, !(STUSPS %in% c("AK", "HI", "PR"))) # plot it ggplot() + theme_map() + geom_sf(data = shapes, aes(fill = I(color)), color = alpha("#fffbf2", .1)) + theme(plot.background = element_rect(fill = '#fffbf2', colour = '#fffbf2')) # save it ggsave("avgcolors.png", width = 7, height = 9, units = "in", dpi = 500) ``` *** Erin Davis Originally posted on the 26th of June 2021 at Link to the article: *Sentinel-2 cloudless - by EOX IT Services GmbH (Contains modified Copernicus Sentinel data 2019)* *** Media coverage and other references: * * * * * --- --- url: /_team/csonkab.md description: >- A missioner of agri-environment, biodiversity and the Copernicus-based Area Monitoring System (CAP). Keen on fine art painting and interior design. --- --- --- url: /2018/01/browse-open-satellite-data-with-eoxc.md --- We made the Open Source tool [EOxC](https://github.com/eoxc) to intuitively browse satellite data. [Try it yourself ](http://aws-catalog.eox.at) and view, search/filter, and download Sentinel-2 and Landsat 8 products directly in your browser. EOxC connects to [OpenSearch](http://opensearch.sentinel-hub.com/resto/api/collections/Sentinel2/describe.xml) and [WMS](http://sentinel-hub.com/develop/documentation/api/ogc_api/wms-parameters) services which in the above link are provided by [Sentinel Hub](http://sentinel-hub.com/). Downloads are provided via the [sentinel-s2-l1c bucket](https://aws.amazon.com/public-datasets/sentinel-2/) and the [landsat-pds bucket](https://aws.amazon.com/public-datasets/landsat/) on Amazon Simple Storage Service (S3). Support has been generously provided by the [European Space Agency (ESA)](http://www.esa.int/ESA). Technical details on how to use the client and supported features are provided in an earlier [blog post](/2017/03/code-de-catalog-client/). We might need to eventually write a new post to detail the functionality added since March last year. Contributions and feedback are always welcome on [GitHub](https://github.com/eoxc)! --- --- url: /2018/10/building-a-python-interface-to-vires.md --- This post is about a new python tool to interact with *VirES for Swarm*. VirES comprises three components: - *VirES server*: provides optimised, flexible access to the original mission data and models - [*VirES web client*](https://vires.services): a state-of-the-art web-based GUI to interact with data through globe and analytics views - [*viresclient Python package*](https://viresclient.readthedocs.io/): a new tool to enable easy programmatic access to the data for mission scientists As for me, I am a PhD student who has been working at EOX as an intern over the summer, developing *viresclient*. My research is within the field of geomagnetism, making heavy use of *Swarm* data, and so this post is written more from the perspective of an end user of VirES. ## ESA's Swarm mission The Swarm satellite mission has been operating in space since late 2013. The mission is actually made up of 3 identical satellites flying in a special constellation in Low Earth Orbit. Its goal is to map and monitor Earth's complex electromagnetic environment which originates in a diverse range of processes, from the movement of material in Earth's core to solar storms energising rings of radiation around Earth. To achieve this, each satellite carries a suite of highly sensitive magnetometers and electric field and plasma instruments, which scientists use to deepen our understanding of processes occurring around our planet. Part of this process involves creating maps of different components of the global magnetic field with unprecedented accuracy. Continued support of Swarm and many other related missions is vital for understanding the Earth system and for protecting society from natural hazards. ## Swarm data access The Swarm mission produces a range of different measurements and models, which are continually evolving and being added to. As with ESA Earth Observation missions, all data is freely available to all ([here](https://earth.esa.int/web/guest/swarm/data-access)). The default way to access data and model parameters is via an FTP server, where different products are presented in a number of (sometimes bespoke) file formats. This means that it is necessary to read through quite a lot of documentation, find or write appropriate software for accessing data files, and sometimes learn how to evaluate different models which may be outside your field of expertise. Accessing Swarm products directly therefore requires particular expertise and a large investment of time. The VirES platform circumvents a lot of this end-user effort and provides more convenient interfaces to the data. ## Overview of VirES VirES is a server-client architecture which acts as a mediating layer between the raw data and the derived data and visualisations that users want. The VirES server ingests the raw data and model parameters and provides optimised evaluation of models and a request-based access to the data. For example, I may want to select data measured by a satellite every time it is passing over the arctic during a whole month. Using the "old" method via the FTP server, I would need to download all the files containing this data, read and parse them, subset out the data I want, and, frequently in the case of Swarm data, run some code to evaluate a model along the orbit to obtain the data-model residuals (which can be computationally intensive). I would also need to figure out a sensible way to organise the files. Via VirES, I ask the server for what I want and it delivers it to me in a format I choose, without me having to know the details of the underlying input data format and the model implementation. This also reduces the effort needed in data and code management on my side, making the whole process cleaner and easier to reproduce. Access to VirES is currently manifested by the VirES web client which allows product selection, filtering, plotting etc. through a GUI with powerful built-in visualisation (see our previous blog post [here](/2015/11/vires-swarm-and-magnetic-model-data-visualization/)). This provides an intuitive way to explore the data and understand the products available, with user-friendly tools to generate figures without the need to write your own software. ## Why viresclient? viresclient is a Python package that provides programmatic access to the VirES server. This provides flexibility and utility in accessing the data. Scripts which make specific requests via viresclient can be built to produce pre-defined outputs, which can be easily tweaked and rerun or made to produce many outputs, in contrast to the point-and-click nature of the web client. This is intended to be complementary to the web client functionality, enabling an additional way to interact with VirES. More importantly, viresclient reduces the gap between the mission products and scientific analysis. This is because the user can request just the data they are interested in, and have it delivered in the format they are ready to work with. They don't need to worry about file formats and the time spent data wrangling is reduced. The Python user and developer community provide a wide range of powerful open-source tools which enable scientists to produce sophisticated analyses with minimal effort, with software engineers, data scientists, and physical scientists collaborating to build the tools we need. viresclient handles the request to and download from the VirES server and converts data directly into the Python objects [pandas.DataFrame](https://pandas.pydata.org/pandas-docs/stable/overview.html), and [xarray.Dataset](http://xarray.pydata.org/en/stable/) (essentially an extension of pandas to handle higher dimensional data), which are great for organising and manipulating data with a lot of useful built-in functionality. From here, it is easy to directly use tools from [SciPy](https://www.scipy.org/) and beyond. For creating map figures, I would like to point out [Cartopy](https://scitools.org.uk/cartopy/docs/latest/gallery/index.html) in particular. To learn more about the viresclient interface, please see the [documentation](https://viresclient.readthedocs.io/en/v0.2.1/notebooks/simple_example.html). Here are a couple of example figures that can be generated using functionality from xarray and cartopy, using data acquired with viresclient. They show the magnetic disturbance created around the auroral regions (i.e. the poles) during one geomagnetically active day (1st January 2016), explored both spatially and temporally. The code to generate this can be found as a Jupyter notebook [here](https://github.com/smithara/viresclient_examples/blob/master/model_residuals_and_cartopy.ipynb). --- --- url: /careers.md --- #### Why Work at Eox * **Provide "real world" services that matter**: We deliver software solutions and operate Web services that are utilized by a worldwide community which is concerned about the physical health of our planet Earth and the evolution of nature surrounding us. Our activities are embedded into major environmental monitoring programs such as the EU's Copernicus and ESA's Earth Explorer satellite missions. * **Help science**: Making scientists get hold of data faster and easier, means helping them in their work, be it research on global warming effects, earth's magnetic field, space weather prediction, etc. * **Flexible hours**: Core hours are from 10:30 to 16:00, compensatory time off. We are productivity oriented. * **Small team**: EOX is a small company with about 30 people. We are following the unfix model with a typical team size of about 8 people. * **Open Source**: Nearly all development is done as Open Source and mostly made available through various GitHub organizations. * **Your own work style**: We will not limit you in the choices you want to make for development, such as underlying frameworks, libraries, package managers, etc, as we think everyone has their own workflow for problem solving. Ideally we will always try to find solutions together to make sure all components can be integrated nicely. * **Location**: Vienna remaining in the first place for the 10th year in the quality of living city rankings. * **Salary**: We pay gross annual salaries, as a minimum: * 38.000 € Software Engineer * 50.000 € Senior Software Engineer with willingness to offer a higher salary depending on your professional experience and qualification. See what this means in net and monthly, as well as in terms of contributions to Austrian social security system [online calculator](https://rechner.cpulohn.at/bmf.gv.at/#bruttoNetto). We allow us bonus payments conditional to very good overall enterprise performance. * **Perks**: We enjoy as additional fringe benefits a monthly contribution to a private retirement fund, public transport ticket (job-ticket), restaurant- and supermarket vouchers, various social and group events like company retreats and family activities. #### Open Positions --- --- url: /certifications-and-memberships.md --- #### ISO-27001 Certificate EOX IT Services GmbH is [ISO-27001:2022 certified](/images/Zertifikat_27001_EOX_e.pdf). Supported by Vienna Business Agency #### Memberships EOX is a proud member of the following organizations: --- --- url: /_team/schillerc.md description: >- From the marine depth to space: accumulating over 25 years of experience in Earth observation information services in academia, international institutions, public applied research and now EOX. Advocating free and open source data, software, scientific publications, and information. --- --- --- url: /2020/10/cloud-workspaces-for-earth-observation-data-processing-from-ideas-to-solutions.md --- The importance of Earth Observation (EO) data for large scale and monitoring tasks has been recognized in many business domains. The way EO data is consumed has become even more demanding, starting with EO data visualization tasks (want to observe [a Sahara dust event in your browser](/2020/06/saharan-dust-event-from-space/)?) to large scale data processing workflows to produce analysis ready data (e.g. the [EOxCloudless product in the context of the Common Agricultural Policy](/2019/05/eoxcloudless-for-common-agricultural-policy-cap/)). The last months brought several initiatives like [RACE](/2020/06/race-dashboard/) (Rapid Action Coronavirus Earth observation), showcasing how satellite data can help to monitor the situation linked with the Corona pandemic, by providing access to various environmental, economic and social indicators to measure the impact of the lockdown. These indicators were provided in an unprecedented collaboration effort between ESA, scientific teams and individual contributors who contributed new unique ideas. These were selected and added from the [Euro Data Cube Custom Script Contest](https://eo4society.esa.int/2020/04/06/euro-data-cube-custom-script-contest-covid-19-edition/) and the subsequent [RACE upscaling competition](https://eo4society.esa.int/2020/04/24/from-the-covid-19-custom-script-contest-to-the-euro-data-cube-european-dashboard/). ![](/images/race-dashboard-trucks.jpg) Europe-wide number of trucks on motorways indicator on [RACE dashboard](https://race.esa.int/?poi=W2-E12c) We at EOX got the opportunity to feature the winner of this competition, [Henrik Fisser](https://twitter.com/fisserhenrik) with his submission "[Sensing trade from space with Sentinel-2](https://github.com/hfisser/Truck_Detection_Sentinel2_Upscaling/blob/master/eox_blog_post.md)" during one of our [Euro Data Cube ESA EO Φ-WEEK 2020 side events.](https://eurodatacube.com/news/phiweek-2020/) In addition, [Sinergise](https://sinergise.com/), our consortium partner in the [Euro Data Cube project](https://eurodatacube.com), published a great [blog post](https://medium.com/sentinel-hub/race-upscaling-competition-results-8a339bb8c942) summarizing the results and achievements of the EO innovations. While the authors behind these ideas got access to (free and commercial) EO data sources and the necessary resources on the Euro Data Cube to process them, they still had to face the same challenges with their EO processing tasks as anyone else working on large scale EO workloads from detailed EO data timeseries data analysis, Machine Learning to Deep Learning and A.I.: * How to handle the huge amount of (not always analysis-ready) data - input EO sources, reference data, intermediate caches, final result output? * How to apply the different analysis and processing methodologies on the data - both interactive for exploration and systematic for large scale? In a [previous blog post](/2020/09/raising-the-bar-for-earth-observation-data-processing/) we explained the need of an elastic and scalable compute layer with various storage options close to EO data for any kind of processing workloads. We also introduced the [EDC EOxHub Workspace](https://eurodatacube.com/marketplace/infra/edc_eoxhub_workspace) services, making this concept of **ready-made cloud infrastructure** real and publicly available to the contest participants for their large scale contributions and also to you, dear reader of this blog, offered in self-service mode on the [EDC marketplace](https://eurodatacube.com/marketplace). ![](/images/eo-workloads-offer.jpg) So what can you use from our offer for your EO workloads: * a [JupyterLab](https://jupyterlab.readthedocs.io/) installation, fully managed for you in your private cloud workspace, as your interactive development and exploration tool, * running your defined EO workloads in the cloud, leveraging your selected flavor of compute (CPU/GPU, memory size) and network resources, * with a prepared base environment (the [EDC kernel](https://github.com/eurodatacube/base-images/blob/master/kernel-edc-environment.yaml)) managed by us with commonly used data analysis and EO processing tools, regularly updated to the latest library versions, versioned itself to foster reproducibility, and [extensible](https://eurodatacube.com/documentation/custom-jupyter-kernels) to support the needs for additional tools and libraries, * with a persistent file system to manage the notebooks and the code defining your EO workloads, * (optional) with mounted object storage, to store huge amounts of data (e.g. staged EO input data or final result data) in a cost effective way and * (optional) with attached block storage, to serve as high-throughput cache layer for intermediate processing results. But as soon as you consider your EO workloads as "ready" and you prepare them for headless invocation (see [preparatory steps here](https://eurodatacube.com/documentation/headless-execution)), you get even more possibilities: * directly [invoke your notebooks via API calls](https://eurodatacube.com/documentation/headless-execution), with the same environment prepared as via interactive invocation through JupyterLab, to ensure the same the behavior (and the same results) for your EO processing tasks, * follow the headless execution path in an automatically generated result notebook, providing the full execution trace and also necessary information for troubleshooting if something went wrong, * automate the invocation based on a time-based job scheduler (not available in self-service mode yet), * understand the behavior of your processes, as various runtime metrics are exposed through your personal dashboard. ![](/images/eo-workloads.jpg) Here at EOX we are already using the underlying technology of EDC EOxHub Workspaces for our [own](https://cloudless.eox.at/) and [customer-driven](https://maps.eox.at/) EO workloads. We are happy that we can foster the processing of several of the indicators as showcased on the [European](https://race.esa.int/) (ESA) and [Worldwide](https://eodashboard.org/) (ESA, NASA, JAXA) dashboard. Offering on-demand functionality on top of powerful cloud infrastructure e.g. with GPU instances for the training and inference of Machine Learning models using [PyTorch](https://pytorch.org/) or [TensorFlow](https://www.tensorflow.org/). Our Cloud Workspace services are constantly evolving and more are being developed as you read this, but we have good news for you, there's no need to wait! You can [start at the Euro Data Cube](https://eurodatacube.com/login) today. --- --- url: /2017/03/code-de-catalog-client.md --- Now that the [CODE-DE](http://code-de.org) (**Co**pernicus **D**ata and **E**xploitation Platform - **De**utschland) has [officially](http://www.d-copernicus.de/code-de-der-nationale-datenzugang-zur-copernicus-welt-ist-online) [launched](http://www.dlr.de/dlr/desktopdefault.aspx/tabid-10081/151_read-21668/#/gallery/26528), we're proud to present our contribution to it: the Catalog Web Client. This component available via the marketplace of the CODE-DE and allows to visually interact with and perform searches on the catalog of Earth Observation image archive. Though it was designed with CODE-DE in mind, its use of open standards and interfaces means that it is easily used with other catalog and viewing systems (such as our own free and open source products: [SxCAT and ngEO-Browse-Server](http://ows.eox.at/code-de/landsat.html) or [EOxServer](http://ows.eox.at/code-de/)). The client can be used to quickly filter satellite scenes that you are actually interested in by setting various filters: time, spatial and additional. ### Time Filter The time is either configured using the date input widget or by drawing on the time-slider. Drawing is an intuitive way to select a time span of interest as the relative number of records can be seen as a histogram. The selected area can be manipulated (resized and moved), to further narrow down the temporal selection. Also, a click on a histogram bucket automatically selects the timespan of that bucket. The buckets always refer to the amount of images in that timespan, and not only to the searched ones. ### Spatial Filter There are several ways to set a spatial filter: either by drawing a point, rectangle, or polygon, or by loading a shapefile into the client and selecting a feature as an area of interest (For example a [shapefile containing country borders](http://thematicmapping.org/downloads/world_borders.php)). When no explicit spatial filter is set, then the current map view extent is used as a filter. ### Additional Filters This widget allows to specify "expert" filters, to further narrow down the search. The list of available additional filters depends on the capabilities of the catalog, but typically includes filters such as the cloud coverage of the image (for optical sensors) or the antenna polarisation channels (for radar based images). Due to the nature of the backend protocols involved, those filters are not reflected by the shown images on the map, so a warning is shown when one of these filters is set. Both the map and the time-slider widgets provide three distinct functionalities: to give a spatial and temporal context, allow to directly edit the spatial/temporal filter, and to show search results. On the map, both the browse images and footprints of the query results are displayed. The footprints are interactive, a click shows a context menu, where the images can be further inspected or added to the download list. The time-slider works similarly: just like the map, it can be zoomed and panned and also shows where on the temporal axes images can be found. When zoomed out, the distribution of the available images is shown as a histogram like plot. But zoomed in, the individual images are shown as small dots or boxes. ## Search Results and Details Whenever a filter changes, a query is performed. The search results are displayed in the right panel as a thumbnails and times the images were taken. The result list, is interactive as-well: clicked images are added to the download list, and additional information for each image can also be viewed. The record details view shows the image itself on an interactive map and further metadata associated with the selected image. ## Download The right panel also houses the Download list: all images that were selected (either directly from the map, from the result list, or the image details) can again be found here. In this list, the images can be inspected once again, and removed if not required. Now, several download options are provided: either directly using the browsers download manager itself or as a metalink file or list of URLs to the selected files. This allows to pass the selected datasets on to an external download manager for a bulk downloading. When a WCS server is configured as a download backend, then additional download options are possible: such as cropping the image or using a specific file format or projection. This assures that the amount of transferred data is reduced to a minimum and that the specialized operations (such as reprojection) are done on the server. ## Backends In the client configuration, each layer is split into three distinct properties: the display, the searching, and the download. For the display part, the layer can be configured to use either WMS or WMTS, two of the most popular viewing protocols for map data. For the search part, either EO-WCS (the extension of WCS for Earth Observation) or OpenSearch is supported. This allows to plug in a wide variety of commercial or free servers and makes the Web Client a very versatile front end for many different systems. The download can either be purely HTTP based or make use of WCS in order to allow the aforementioned benefits. Please feel free to [try out our latest, cool application](https://code-de.org/en/marketplace/view?filter=keyword:CODE-DE\&sort=asc:title) and let us know what you think! --- --- url: /contact.md --- EOX IT Services GmbH\ Thurngasse 8/4\ 1090 Wien\ Österreich / Austria Find us on LinkedIn and GitHub . Feel free to contact us via any of the above means. We’re looking forward to hearing from you. In case you have discovered a bug or any security related issue in one of our services, please contact us via . However, please note that we don't offer any "bug bounty program" nor offer any other kind of monetary or other gratitude in exchange for such information. [Informationspflicht lt. §5 E-Commerce Gesetz, §14 Unternehmensgesetzbuch bzw. §63 Gewerbeordnung und Offenlegungspflicht lt. §25 Mediengesetz](https://www.wkoecg.at/Ecg.aspx?FirmaID=d9e35889-7005-46a1-91b4-f51efdabe7a5) --- --- url: /contributions-terms-and-conditions.md --- (version as of April 9th 2020) Download pdf version [here](/content/contributions_terms_and_conditions.pdf) to save or print this contributions terms and conditions. ## Section 1 - These EOX Contributions Terms EOX IT Services GmbH ("EOX") develops, operates, and provides various online services ("EOX Services"), for example hosted JupyterLab instances an online service to provide an interactive computational environment to help exploit Earth Observation data. The term "Service Provider" used below refers to EOX. Contributing to any EOX Service, for example Jupyter notebooks, constitutes acceptance of the following EOX Contributions Terms. ## Section 2 - Contributing to an EOX Service When contributing to an EOX Service the user shall be obliged to observe the following: a. The user shall comply to the [EOX Service Terms](/service-terms-and-conditions/) in its latest version in its entirety. b. The user shall be solely responsible for the content of, and for any harm resulting from, any contribution posted, uploaded, linked to or otherwise made available ("User-Generated Contributions"). In particular, the user warrants that such user contribution does not contain harmful components, violate any applicable laws or infringe third-party rights. The Service Provider disclaims all responsibility and liability for User-Generated Contributions made available on the EOX Service. c. The Service Provider does not pre-screen User-Generated Contributions but retains the right to access and review such contributions at its sole discretion (e.g. for maintenance purposes or in the case of irregularities). The Service Provider reserves the right - though without accepting any obligation - to refuse or remove any User-Generated Contributions that, in the Service Provider’s sole discretion, violate these Terms or otherwise threaten the security, integrity, or availability of the EOX Service. d. The User-Generated Contributions shall not reveal any user credentials for example by logging or dumping them. Particular care shall be taken when logging HTTP request. e. In case of Jupyter notebooks as User-Generated Contribution the first cell shall describe besides a general description and applicable license what information is persisted to files or uploaded to remote storage. f. In case of Jupyter notebooks as User-Generated Contribution only libraries available in the current environment are supported and shall be used and the installation of further libraries is forbidden. In particular no commands like "pip install" or similar shall be included in the notebook. If the user needs further libraries please contact . A violation of the foregoing, abuse (in particular through unlawful use), excessively frequent requests of, or uploads to, the EOX Service, or any other use of the EOX Service that in the Service Provider's judgment threatens the security, integrity, or availability of the EOX Service may result in the temporary or permanent suspension of the user account, in individual cases, it may also entitle the Service Provider to claim for damages. The Service Provider, in its sole discretion, will determine unlawful, abusive, excessive, or otherwise inappropriate usage. ## Section 3 - License for User-Generated Contributions on the EOX Service The user retains ownership of his/her User-Generated Contributions. The user grants the Service Provider and its legal successors the right to store, parse, and display User-Generated Contributions and make incidental copies as necessary to provide the EOX Service including storing the content in own data bases, creating backups, parsing it into a search index, or otherwise analyzing it on own servers and executing it. The license does not grant the Service Provider the right to sell User-Generated Contributions or otherwise distribute it, unsolicitedly display it, or share it with other users, or use it outside of the provision of the EOX Service. The user will not receive any remuneration for any of the rights granted hereunder and the license will end when the user removes the content from the Service Provider’s servers. The user retains all moral rights to the User-Generated Contributions uploaded, published, or submitted to any part of the EOX Service, including the rights of integrity and attribution. However, the user waives these rights and agrees not to assert them against the Service Provider, to enable the Service Provider to reasonably exercise the rights granted hereunder, but not otherwise. To the extent this agreement is not enforceable by applicable law, the user grants the Service Provider the rights needed to use the User-Generated Contributions without attribution and to make reasonable adaptations of User-Generated Contributions as necessary to provide the EOX Service. By making available User-Generated Contributions on the EOX Service the user grants each user of the EOX Service a nonexclusive, worldwide license to - exclusively for non-commercial research purposes - use, display, and perform the content through the EOX Service and to reproduce the content solely on the EOX Service as permitted through EOX Service's functionality. If the user is uploading content that the user did not create or owns, the user is responsible for ensuring that the content is licensed under terms that grant these permissions to other users of the EOX Service. --- --- url: /2020/04/covid-19-custom-script-contest.md --- The [Euro Data Cube partners](https://eurodatacube.com/) have designed a [custom script contest](https://www.sentinel-hub.com/contest) focused on the COVID-19 crisis to collect ideas on how EO data can help society to mitigate the COVID-19 pandemic and predict its mid- and long-term impacts. The COVID-19 pandemic has caused significant disruption in all our lives. Even when we hopefully will have managed to halt the spread of this disease, we still do not know how long it will take for the world to go back to ‘normal’. We have been thinking for quite a while how an Earth Observation company like ours can contribute to the situation, apart from staying at home and ensuring that the virus does not spread. Looking at the super-human efforts of medical staff worldwide was inspiring and our motivation grew even stronger. Then, one week ago, the Euro Data Cube team was contacted by the European Space Agency (ESA) who got their heads together with the European Commission, and they suggested to try to crowd-source ideas. That was it, we jumped on-board without hesitation! [![](/images/covid-19_contest.jpg)](https://www.sentinel-hub.com/contest) The contest focuses on the COVID-19 crisis and is looking for ideas on how satellite data could help with the situation, both, at this moment as well as in the upcoming months, when the world will try to get back to normal. We are calling for remote sensing experts, machine learning scientists as well as the interested public. Anyone can contribute their ideas. Three focus categories have been selected: * observation of changes in *economic operators* * changes in *human activity* distribution * changes in *agricultural activity* If, however, you have an idea falling out of this scope, please provide it as well. **Any and all ideas are welcome!** There are resources offered for free for participants: Two new data sources ([PlanetScope](https://docs.sentinel-hub.com/api/latest/#/data/PlanetScope) and Airbus [Pleiades](https://docs.sentinel-hub.com/api/latest/#/data/Airbus-Pleiades)) over several locations as well as Euro Data Cube [Sentinel Hub](https://www.sentinel-hub.com/develop/documentation/api/ogc_api) and [EOxHub hosted JupyterLab](https://eurodatacube.com/marketplace/infra/eoxhub) subscriptions. Please [fill the participation form](https://forms.gle/PHeSuw85gj4zAqUA8) if you want to use those resources. ![](/images/nitrogen-dioxide.gif) Fluctuation of nitrogen dioxide emissions across Europe from 1 January 2020 until 11 March 2020, using a 10-day moving average, Sentinel-5P data (source: [Copernicus](https://www.copernicus.eu/en/news/news/coronavirus-copernicus-sentinel-5p-reveals-nitrogen-dioxide-emissions-drop-over-italy)) The contest will run until the end of May. However, due to the importance of the matter we would like to see results as quickly as possible. This is why we will be selecting the best ideas end of every week, every month, and the grand prize at the end of the challenge. It’s not just about prizes - good ideas will be presented to the European Commission and they might become an important tool to improve the situation in the EU. ![](/images/rental-cars.gif) Piling up of rental cars on the field due to reduced demand, just before a fire broke, burning 3500 of them. Source: [James Hyde](https://twitter.com/wxmeddler/status/1246669309839114246) How can you contribute? Here are a few rough examples: * If you are a self-taught remote sensing enthusiast, check the data, play with it, come up with an idea, compile a presentation, and send it in. Note that there does not necessarily need to be a script itself. A simple description of the idea supported by slides and exported images is sufficient. * Earth Observation experts can perform a detailed analysis, using various tools, demonstrating the patterns. * Machine learning experts can model agriculture activities and identify potentially abandoned areas, resulting in reduced yields in the harvest seasons. * Satellite data providers should contribute additional data, so that people can perform even more interesting analyses. Planet, the largest earth observation satellite network, has already stepped in, contributing tens of thousands of km² of their almost daily PlanetScope imagery. One way or another, do participate! While there are prizes, it is not a contest but rather a call for action to collect ideas which may end up helping managing the current situation. The challenge starts today and lasts until May 31st 2020. Submit your idea as soon as possible. **Stay home, stay healthy and contribute to this humankind challenge!** **Signup [here](https://forms.gle/PHeSuw85gj4zAqUA8)!** --- --- url: /2024/04/cubes-and-clouds.md --- In the rapidly evolving field of Earth Observation (EO), the demand for resources to stay in touch with the latest technologies is tangible. To address this, EOX supported the European Space Agency (ESA) in the creation of **Cubes and Clouds Massive Open Online Course (MOOC)**. This course not only furthers our commitment to Open Science but also enhances EOxHub as a cloud platform and its rich educational offerings. Cubes & Clouds - Cloud Native Open Data Sciences for Earth Observation is aiming to bridge the gap between latest trends in bringing users to data as opposed to the traditional approach of downloading data and creating analysis locally. During the course, students have the opportunity to learn about news in cloud optimised data formats, data access and search protocols, allowing users to first explore the data before accessing them fully. Spatio-Temporal Asset Catalog (STAC) is presented as one of the industry standards for EO metadata description. Course is also highlighting the open science principles and **FAIR** data principles. **Cubes & Clouds** is designed to cater to Earth Science students and researchers eager to elevate their technical prowess, as well as Data Scientists aspiring to explore the vast realm of EO. The course stands out by bridging theoretical knowledge with practical application, structured around the powerful concepts of data cubes and EO cloud platforms. Through a carefully crafted curriculum participants delve into the core of EO workflows—from data discovery and actual processing to the important stage of sharing results. What sets this course apart is its very hands-on focused approach, ensuring that learners not only grasp theoretical concepts but also gain practical experience. But the theoretical introduction is not missing - in the first part topics like datacube, platform or data access protocols are introduced. By the course's conclusion, participants are well-equipped to use cloud platforms independently for EO research and confidently adhere to Open Science principles. The learning journey is paved by an array of engaging content, including videos, animations, quizzes, all tailored to ensure a comprehensive understanding of each topic. Most importantly, lessons are accompanied by hands-on coding exercises directly on a cloud platform. The course is designed to be flexible, accommodating the diverse schedules of our global learners while ensuring an effective learning pace with an estimated length of 30 hours. **EOxHub**, a cloud workspace developed by EOX is used as a main platform for the hands-on part of the course. Workspaces for students come with curated environments with all necessary Python libraries and tools installed that students need together with computational resources suitable for coursework. EOxHub workspace is presenting an option to make the first step in the cloud environment without the need of a large and complicated setup. A unique highlight of **Cubes & Clouds** is the interactive map project, where participants collaboratively map snow cover in mountainous regions, showcasing their newfound skills in a real-world application. For showing the results of every successful student we are hosting a publicly available [STAC browser](https://esa.pages.eox.at/cubes-and-clouds-catalog/browser/#/) where all submitted results from coursework are displayed. This feature is not only as a motivation but mainly to conclude the whole process of Open Science principles to actually make data FAIR - findable, accessible, interoperable and reusable. Even though the course theme is snow cover detection, learnt skills are easily transferable to different use cases and different datasets as students are equipped with transferable skills. EOX’s participation in creating and supporting **Cubes & Clouds** reflects our ongoing dedication to promoting Open Science as a foundational element. Open Science not only accelerates scientific discovery and increases transparency but also fosters collaboration and inclusivity across the global scientific community. We invite all interested Earth Science enthusiasts, professionals and Data Scientists to join this transformative learning experience. Whether you are looking to refine your technical skills or embark on a new journey in the EO field, "Cubes & Clouds" offers the tools and knowledge needed to succeed in this dynamic discipline. Enrol today and be a part of shaping the future of Earth Observation. Sign up for **Cubes & Clouds** on [EO-College](https://eo-college.org/courses/cubes-and-clouds/), tag EOX on social media with your submissions and join the community of successful students. --- --- url: /2015/12/curved-labels.md --- Back in my cartography class, in the mid-2000s, we were told to avoid non-horizontal labels in digital maps where possible, in order not to lose the map's legibility. While there is some truth in it, curved labels - carefully placed - can also have a very positive effect in helping the user to understand topographic maps. As our [EOX Maps](http://maps.eox.at) are mainly used to help visualizing Earth Observation (EO) data, we are interested in optimizing our label overlays as good as possible. Therefore we put quite some effort in improving our labeling strategies as well. Apart from [OpenStreetMap](http://www.openstreetmap.org) there is another great resource for map data, called [Natural Earth](http://www.naturalearthdata.com/). It is a collection of helpful datasets to create maps for various scales. Among these datasets, there [are](http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_geography_regions_polys.zip) [two](http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_geography_marine_polys.zip) providing the most important mountain ranges, valleys, regions, island groups and so on. These features are mapped as polygons, but labeling a polygon is quite tricky as most tools use the center point to place the label on. As mountain ranges, for example the Andes, can reach from South to North, we would like the label to be placed vertically along the center line of the source geometry. ![centerline_750](/images/centerline_750.gif) Finding out the center line of a polygon is not as straight forward as it sounds but there are some tricks to get a good approximation. A common method is to calculate a [Voronoi diagram](https://en.wikipedia.org/wiki/Voronoi_diagram) and extract the middle axis. In our approach we refined this method a bit by: * first, segmentizing the geometry to make sure the points are roughly evenly distributed around the geometry, * apply some simplification for performance reasons before creating the Voronoi diagram, * find the "best" centerline, and * finally, smooth the line geometry to get an evenly curved label. We developed a command line tool written in Python and made it [available](https://github.com/ungarj/label_centerlines) for everyone as Open Source under a MIT license. Feel free to use and improve it and don't forget to sent us your feedback. If you have any questions, you are welcome to contact us. --- --- url: /_team/santilland.md description: >- Likes to experiment with visualizations for (geo) data. Tinkerer, 3D printing and gaming! --- --- --- url: /2023/10/data-gravity-source-coop.md --- TL;DR * understand the cost drivers in your "open data" strategy, long-term * don't neglect the ecosystem gravitating around the actual data * you can outsource data promotion but not data governance * allow a neutral cooperative to track uptake and share analytics The term [data gravity](https://datagravitas.com/2010/12/07/data-gravity-in-the-clouds/) describes the observation that data has mass and once it reaches a certain volume, even more data - but also applications and services - are attracted to its location. In the world of Earth Observation (EO) and Machine Learning (ML) that mass may grow quite fast and the greater its gravitational pull, the more difficult (and more expensive) it becomes to move later on. This gravitational pull with its tendency for centralization is not necessarily a bad thing. Having all data, applications and services at one location makes finding and exploiting the data potentially easier. And if we are talking about a public (AWS, Azure, Google, etc.) or a domain specific cloud (like the Copernicus Data Space Ecosystem - CDSE) as data location, the elastic compute close to the data storage layer may enable large-scale workflows and efficient collaboration. Cloud commoditization opened up new possibilities to drive or contribute to open science initiatives, with companies like EOX trying to streamline the processes for scientists to produce new (usually "open") data on one hand and making these data set widely (or even publicly) accessible on the other hand. In this context governmental institutions or agencies like the European Space Agency (ESA) take the mandate to not just act as provider of "open data" by covering the costs of production and maintenance, but also to foster the uptake of these data sets and contribute to the good of society, by enabling a wide audience with data access tools and resources to do something with the data. While this path is laid out quite clearly during the project phase, with dedicated platforms to support the initial project goals, it often manifests to be quite challenging in the long term in regard to sustainability. Contributed solutions tend to be more fragmented as originally envisioned, processes and workflows start spanning multiple data locations, multiple applications and multiple services. And consumer data access suddenly becomes a major cost driver (egress costs are still a fundamental part of many cloud business models, but [there is hope](https://www.cloudflare.com/bandwidth-alliance/)) making it often necessary to reconsider, if all the data should really be open to everyone or just a subset should be made accessible to certain people. > "The data gravity dance is more important than it was before when you were either distributed or centralised, and now you’re both." [Podcast - The Fundamentals Of Data Gravity With Dave McCrory](https://ionir.com/the-fundamentals-of-data-gravity-with-dave-mccrory) What may be a reasonable decision for both data providers as well as data consumers at one point may vary over time, making it necessary to change strategy with the potential conclusion to better migrate and move the data. From a data consumer perspective such a data location change can be largely abstracted and won't necessarily be noticed. But for the data provider such a step is usually expensive or even impossible, considering not just data transfer but also the whole ecosystem built around. This makes the overall data location decision crucial from day one. The competitive landscape in the cloud labelled object storage to be the most flexible and cost effective way to share data, with a largely standardised data access mechanism. By leveraging a data catalogue mechanism on top to search and optionally resolve the linked assets the actual location of the data can be dynamically inferred by applications and services code. Established access token mechanisms (Shared Access Signatures, presigned URLs, etc.) allow access to data with different data governance policies (for a certain time, to a certain group of people, within certain quota limits, etc.), so in federated scenarios with a common mechanism to retrieve an access token for a certain data set this may be fully opaque. The Radiant Earth foundation, widely known in the EO and ML domain with their MLHub for published ML training data, just recently introduced the ["Source Cooperative"](https://radiant.earth/blog/2023/10/what-is-source-cooperative/) as a generic data sharing utility, enabling such a federation over different data sets located in different clouds, providing a user-friendly one-stop-shop for data discovery. The "Source Cooperative" allows data providers to not just seek for outreach through a public listing but also provides all the technical means to govern data access in a unified way. An important point here is that many data providers may want (or need) to retain control over the data, with specific requirements on the data location (e.g. a CDSE hosted bucket). The in the blog post envisioned "Bring your own bucket (BYOB)" approach may enable this scenario by just registering but not actually transferring data. Can this BYOB approach be fully aligned with the general data governance mechanisms and integrated into usage analytics? I hope so, and considering that Source Cooperative is striving to adopt the model of a neutral utility cooperative being fully transparent on usage and costs, I start to see the potential! Does that help with data gravity in our EO and ML domain full of "open data"? No, I would say even the opposite: By streamlining scalable data access with the conversion of data to new cloud native data formats (COG, Zarr, etc.), indexed in new data catalogues (STAC), accessed through newly created platforms bringing elastic compute close to the data (e.g. Dask/Ray-powered JupyterHubs), we are even fostering data gravity and cause further fragmentation. Does it matter? I also say no, if we succeed to **promote the generated "open data"** and become more inclusive. And if we take the chance to get a better understanding of the data consumption behaviour and the induced costs with usage analytics collected through a neutral data sharing utility like the "Source Cooperative", to finally be able to **make the right data governance decisions**: "Shall we deprecate this data set?" "Shall we increase the download quota for this user?" "Shall we still grant access to this user or ask them to pay for their consumption?" "Shall we migrate or even recreate this data set on another cloud?" This will help that budget allocations will be sized right, that the right data will outlive long term and that the right people can access and leverage it. And hopefully that the gravitational pull will start to correlate with the metric of "gained information" instead of just "accessible data". --- --- url: /expertise/data-management.md --- ##### View Server The [View Server](https://vs.pages.eox.at/vs/) is [MIT](https://gitlab.eox.at/vs/vs/-/blob/main/LICENSE) licensed, Docker based, scalable software stack providing external services for searching, viewing, and downloading Earth Observation (EO) data. Services implementations are following OGC Web services standards [OpenSearch](https://github.com/dewitt/opensearch), [WMS](https://www.ogc.org/standards/wms), [WMTS](https://www.ogc.org/standards/wmts), [WCS](https://www.ogc.org/standards/wcs). Initially built with [EOxServer](http://eoxserver.org/) and [Mapcache](https://mapserver.org/mapcache/) as core components, View Server enables EO Data publication in a modular and configurable way. To align with modern cloud-native architectures, the View Server team is actively exploring and integrating [TiTiler](https://developmentseed.org/titiler/) as a core rendering engine. The process starts with data pre-processing and metadata ingestion and ends with serving pre-cached and on-demand rendered images through an attached Web client such as [EOxC](https://github.com/eoxc/eoxc) or on individual service endpoints. [View Server on GitLab](https://vs.pages.eox.at/vs/){.button} ##### eoAPI / TiTiler [eoAPI](https://github.com/developmentseed/eoapi) and [TiTiler](https://developmentseed.org/titiler/) represent the modern standard for cloud-native, fast, and dynamic Earth Observation data rendering and cataloging. EOX maintains a close collaboration with [Development Seed](https://developmentseed.org/) in developing and optimizing these tools. We operationally deploy this advanced software stack through [EOxHub Workspaces](/expertise/managed-cloud-workspace), providing our clients and communities with highly scalable, on-demand visualization and analysis workflows. [Learn more about eoAPI](https://github.com/developmentseed/eoapi){.button} ##### EOxServer [EOxServer](http://eoxserver.org/) is a versatile and extensible server software that allows to register a variety of earth observation products and expose them via open interfaces. While we are evolving our stacks to adopt modern cloud-native engines like eoAPI / TiTiler as core renderers, EOxServer remains a highly reliable component that has historically formed and continues to support the backbone in several of our operational services and research and development projects: * [VirES for Swarm ↗](https://vires.services/) * [VirES for Aeolus ↗](https://aeolus.services/) * [View Server ↗](https://vs.pages.eox.at/vs/) [Visit EOxServer](http://eoxserver.org/){.button} --- --- url: /expertise/data-products.md --- ##### EOxCloudless **Cloud-free Satellite Imagery for Viewing and Analysis** * **[Viewing Products](https://cloudless.eox.at/products/viewing):** Get our prerendered EOxCloudless Viewing products as SQLite map cache, GeoPackage, or GeoTIFFs. * **[Exploitation Products](https://cloudless.eox.at/products/exploitation):** Get off-the-shelf multispectral mosaic data from Sentinel-2 in 8-bit or 16-bit depth. Our products include: * Viewing Ready (GeoPackage or SQLite map cache) * Viewing Basic (GeoTIFF TileDirectory) * Exploitation Starter (8-bit GeoTIFF TileDirectory) * Exploitation Ready (16-bit GeoTIFF TileDirectory) * WMS/WMTS Satellite * Custom products (you define the parameters) [Visit EOxCloudless](https://cloudless.eox.at/){.button} --- --- url: /expertise/data-visualization.md --- ##### eodash The [eodash](https://eodash.org/) is a flexible and customizable ecosystem designed to build, integrate, and ship Earth Observation (EO) data dashboards and stories. It allows you to integrate various Earth Observation data sources and APIs into custom, interactive dashboards, bringing all relevant EO data into a single clear view. * **Client-side rendering:** Fast, responsive, and interactive dashboards rendered directly in the browser, ensuring a smooth user experience. * **Customizable themes:** Offers a high degree of customization for its look and feel through theme definitions. * **Endpoint integration:** Seamlessly connect to various data sources and APIs, bringing all your earth observation data into one place. * **Integration of your APIs:** Define and execute data processing workflows with ease using a structured and powerful configuration. * **Web component:** Fast, responsive, and interactive dashboards rendered directly in the browser, ensuring a smooth user experience. * **Widgets:** Widget-based architecture, making it easy to extend its functionality. [Explore eodash →](https://eodash.org){.button} ##### EOxElements The [EOxElements](https://eox-a.github.io/EOxElements) are a set of reusable, modular and highly configurable web components for building geospatial applications. The goal is to reduce code duplication during development, and offer ready-made solutions for quick bootstrapping and increased long-term stabilty. They are framework-agnostic and can be used in applications built with Vanilla JavaScript, Vue.js, React, Angular etc. [Visit EOxElements](https://eox-a.github.io/EOxElements){.button} ##### EOPF Zarr Explorer The [EOPF Zarr Explorer](https://explorer.eopf.copernicus.eu/) is an open-source tool developed for visualizing and working with cloud-native Sentinel data. [Visit EOPF Zarr Explorer](https://explorer.eopf.copernicus.eu/){.button} ##### geotiff.js The [geotiff.js](https://geotiffjs.github.io/) software library is an implementation of TIFF and several extensions in pure JavaScript. It can be used in both server and browser applications to efficiently read data from potentially huge raster files. It can be used for both visualization and scientific exploitation of various kinds of data. [Visit geotiff.js](https://geotiffjs.github.io/){.button} ##### COG-Explorer The [COG-Explorer](https://geotiffjs.github.io/cog-explorer/) is a proof-of-concept of a browser mapping application to dynamically access and visualize remote TIFF files ideally (but not limited to) following the [Cloud Optimized GeoTIFF (COG)](https://www.cogeo.org/) layout. With this application even non-RGB data files can rendered using customizable visualization pipelines. [Visit COG-Explorer](https://geotiffjs.github.io/cog-explorer/){.button} ##### AgriApp The [EOX AgriApp](/agri) is a flexible dashboard service for continuous remote sensing analysis of agricultural parcels or, more generally, any monitoring areas of interest. In the context of the CAP Area Monitoring System the EOX AgriApp has successfully been in operative use within the Austrian and Irish Paying Agencies since the beginning of 2023, serving near real-time insights for millions of parcels. [Learn about AgriApp](/agri){.button} --- --- url: /_team/zz_guest_cebullad.md --- --- --- url: /disclaimer.md --- Die Inhalte und Informationen der Webseiten, die von EOX IT Services GmbH (nachfolgend kurz EOX genannt) verfasst wurden und werden, egal ob als Text, Graphik, Design, Quellcode oder anderer Form, sind und bleiben das geschützte Eigentum von EOX. Der Inhalt dieser Seiten darf ohne die vorherige schriftliche Erlaubnis von EOX in keiner Art und Weise verändert, vervielfältigt, verteilt, veröffentlicht, verschickt oder übertragen werden. Zur persönlichen, nicht kommerziellen Nutzung, darf der dargestellte Inhalt heruntergeladen, gedruckt und verteilt werden, vorausgesetzt der Inhalt bleibt unverändert. Alle Kopien von Inhalten oder Auszüge aus diesen Seiten müssen alle Eigentumsnachweise von EOX enthalten. Die kommerzielle Verwendung oder Weitergabe ist nur nach schriftlicher Genehmigung durch EOX erlaubt. Die Erstellung und Verbreitung von veränderten, erweiterten, gekürzten oder in Fremdsprachen übersetzten Inhalten ist nur nach schriftlicher Genehmigung der EOX erlaubt. Sie sind berechtigt, Hyperlinks zu den Seiten von EOX zu knüpfen, aber gleichzeitig verpflichtet, die Unterschiede zwischen Ihren Informationen und denen von EOX deutlich erkennbar zu machen. Die Seiten von EOX dürfen nicht als Teil der eigenen Seite erscheinen. Nach den allgemeinen Gesetzen ist der Betreiber einer Seite für deren Inhalt verantwortlich. Die Inhalte dieser Website wurden mit größter Sorgfalt erstellt. Für die Richtigkeit, Vollständigkeit und Aktualität der auf dieser Website dargelegten Informationen wird jedoch keine Haftung übernommen. Die Webseite enthält unter Anderem Links zu Webseiten von Dritten auf deren Inhalt EOX keinen Einfluss nehmen kann und daher für deren Inhalt auch keine Gewähr übernimmt. Die verlinkten Seiten wurden zum Zeitpunkt der Linksetzung auf mögliche Rechtsverstöße geprüft. Eine permanente Kontrolle der verlinkten Websites ist jedoch wegen dem dauernden Wandel der Webseiten und der umfangreichen Linksammlung ohne konkreten Anhaltspunkt nicht zumutbar. Bei bekannt werden von Rechtsverletzungen werden diese Links umgehend von uns entfernt. © EOX IT Services GmbH, All rights reserved. --- --- url: /2016/02/downloading-sentinel-2-data.md --- Sentinel-2 data is online for quite a while and now we would like to have a deeper look into how to get the L1C data. According to the tiling scheme discussed in the [previous post](/2015/12/understanding-sentinel-2-satellite-data/), there are about 22,000 tiles covering the land masses as well as smaller seas. One full coverage of the Earth will consume storage size of about 3 Terabytes. Considering that each of the two Sentinel-2 satellites has a revisiting time of ten days, we will get a full coverage every five days. This is a lot of data and you probably won't need all of it. Hence, there are various tools to find the data you need. ## ScienceHub The [Sentinel Scientific Data Hub](https://scihub.copernicus.eu/) or SciHub is the main public access point maintained by ESA. You will need to register before using it. There are two ways to get data from the ScienceHub via 1. the [DHUS GUI](https://scihub.copernicus.eu/dhus/#/home) and 2. the [API](https://scihub.copernicus.eu/twiki/do/view/SciHubWebPortal/APIHubDescription). ![DHUS interface](/images/dhus.jpg) Using DHUS is quite straight forward, as [other](http://gisgeography.com/how-to-download-sentinel-satellite-data/) blogs already pointed out. Just draw an area of interest, type your search term like "level 1C" in the box and hit enter. Well, maybe it's not that straight forward after all, as "L1C" shows no results and "Level 1 C" selects Sentinel 1 data; so make sure you provide DHUS with the string it can work with (i.e. "level 1C"). You'll see the search results in form of footprints on a map and as a list of products on the left side. You then can inspect the metadata, browse images of the selected products and finally download the data. The search results are sorted by date, listing the most recent data first. The list also shows small thumbnails on the left. We chose to download [this product](https://scihub.copernicus.eu/dhus/odata/v1/Products\('a3f4ac5d-6173-49dd-86a3-1251d8fb9fd0'\)/$value) which covers Vienna on a beautiful day in early February. Data size is 6.4GB so make sure to get a cup of coffee (or two) while downloading. After finishing the coffee you may probably realize the download failed. As ScienceHub is the only "open" access point for the general public maintained by ESA, there are many users accessing the data there and as Sentinel 2 data products are quite high volume, due to the 10m optical resolution, capacities are often exhausted. Apart from DHUS GUI there is also a DHUS-[API](https://scihub.copernicus.eu/twiki/do/view/SciHubWebPortal/APIHubDescription) which allows programmatic access to the API's search and download functions. In addition, there are also already other tools published, `[Sentinel-download](https://github.com/olivierhagolle/Sentinel-download)`, `[esa_sentinel](https://github.com/jonas-eberle/esa_sentinel)`, and `[sentinelsat](https://github.com/ibamacsr/sentinelsat)`, which facilitate working with the API from e.g. a Python script or via the command line. We will give `[sentinelsat](https://github.com/ibamacsr/sentinelsat)` a try. It works on command line and provides a Python API as wel,l which makes it easy to integrate with your infrastructure. Once `sentinelsat` is installed and you have a [GeoJSON containing your Area of Interest](https://gist.github.com/ungarj/de87191dabb6084c2399), you can search and download products from the command line: ``` sentinel search --sentinel2 -s 20160201 -e 20160211 guest guest vienna\_bbox.geojson ``` This returns the following output: ``` sentinel download --md5 guest guest a3f4ac5d-6173-49dd-86a3-1251d8fb9fd0 ``` Of course it will take a while to download the 6.4 GB of data, but with these two commands you managed to both query and retrieve a Sentinel-2 product. Now, lets have a look what this product looks like using the visible bands: Interactive viewer provided by © [mapalupa.com](https://mapalupa.com/) ## Sinergise and AWS A Slovenian company called [Sinergise](http://www.sinergise.com/) courageously took the first leap and uploads the Sentinel-2 archive to an [Amazon AWS](http://sentinel-pds.s3-website.eu-central-1.amazonaws.com/). That's right, now there is not only [Landsat 8 imagery](https://aws.amazon.com/public-data-sets/landsat/) available but also Sentinel-2. The data in the [bucket](http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/) is structured by a [MGRS-like Sentinel tiling scheme instead of Sentinel data products](/2015/12/understanding-sentinel-2-satellite-data/). If we take a closer look on our example product, we see it contains data from multiple Sentinel tiles: ![Sentinel-2 Vienna image and MGRS tiles](/images/example_tiles_image.jpg) The overlap you can see at the southern image border is intended and done automatically while creating the product. If you want to download the data from Amazon, you will have to follow the data structure described [here](http://sentinel-pds.s3-website.eu-central-1.amazonaws.com/). The URL to the tile containing Vienna ([33UWP](http://sentinel-s2-l1c.s3-website.eu-central-1.amazonaws.com/#tiles/33/U/WP/2016/2/6/0/)) looks like this: This introduction should help you making the first steps to use the Sentinel-2 data offered by ESA. In the upcoming next Sentinel post, we will have a deeper look into the SAFE data structure and image formats used by the Sentinels. So stay tuned. --- --- url: /2021/01/earth-observation-data-cubes-as-a-service.md --- *Editor’s Note: The information in this blog post is part of [Stefan’s master’s thesis](http://othes.univie.ac.at/64983/), that he wrote as an intern at EOX.* Have you wondered how to extract meaningful information from loads of satellite imagery spanning several months? You are not alone! Satellite imagery is considered Big Earth Data and comes with a specific set of challenges. Earth Observation Data Cubes have set out to tackle these challenges. They allow for easy information extraction, for example in the agricultural domain. ## Checks by Monitoring In an effort to modernize the [Common Agricultural Policy (CAP)](https://ec.europa.eu/info/food-farming-fisheries/key-policies/common-agricultural-policy/cap-glance_en) the European Union is introducing a new concept called Checks by Monitoring (CbM). The idea is to automatically check all farmers’ subsidy requests using satellite imagery. To this end time stacks of satellite data are overlain by agricultural parcels and typical vegetation "signals" are modeled for each crop. Then the actual signals of all agricultural parcels are compared to the typical signal models corresponding to the farmers’ declarations. If the declaration and the actual signal agree, the parcel is flagged as compliant. Else, further investigations are triggered, and the farmer is given the chance to review their declaration. Working on putting the CbM concept into practice EOX has partnered with the Austrian paying agency Agrarmarkt Austria (AMA). Together we are exploring the various steps of the CbM workflow in a hands-on fashion. ## Earth Observation Data Cube At the beginning of the CbM workflow satellite imagery has to be pre-processed into a form that is more suitable for time series analysis. Earth Observation Data Cubes are an emerging paradigm in the remote sensing realm. Over an area of interest (AOI) images are mosaicked together and then these mosaics are stacked on top of each other. This way we receive a continuous, gridded data structure with at least three dimensions (time, x, y). In the context of CbM we are using Sentinel-1 backscatter plus derived polarisation ratios and multiple bands of Sentinel-2 plus derived vegetation, water and salinity indices. Satellite data is processed using the EOX-developed mapchete Hub cloud service. A Sentinel-2 data cube schema that includes half-monthly data from the start of the winter season up to the first checks date is detailed below. ## Tiles Processing Since the project AOI covers about 17,000 km² we tiled the input data into smaller chunks to speed-up the processing. We created a custom WMTS tile pyramid and selected six process zones corresponding to the project AOI. These zones were in turn chunked again into smaller process tiles. While process zones are used to parallelize processing on multiple machines, process tiles are distributed among the available processor cores on each machine. This setup allowed us to process 8.5 months of Sentinel-2 satellite imagery within 1:24 hours, including calculations of half-monthly averaged time slices and band indices. The resulting data cube with 17 time slices covering 17,000 km² is 48.37 GiB in size. Find the used [processing script on GitHub](https://github.com/StefanBrand/masterdatacube/blob/mapchete/datacube_purge.py). ## Conclusion The Earth observation data cube experiment presented above shows that EOX is perfectly equipped to deal with the satellite data pre-processing requirements for the upcoming CAP reform. Contact us and we will be glad to discuss your custom processing needs together. ## References * Koetz, B., Defourny, P., Bontemps, S., Bajec, K., Cara, C., de Vendictis, L., Kucera, L., Malcorps, P., Milcinski, G., Nicola, L., Rossi, L., Sciaretta, C., Slacikova, J., Tutunaru, F., Udroiu, C., & Zavagli, M. (2019, October 4). SEN4CAP - Sentinels for CAP monitoring approach. * Kopp, S., Becker, P., Doshi, A., Wright, D. J., Zhang, K., & Xu, H. (2019). Achieving the Full Vision of Earth Observation Data Cubes. Data, 4(3), 94. [https://doi.org/10.3390/data4030094](https://ec.europa.eu/jrc/sites/jrcsh/files/14-sen4cap.pdf) --- --- url: /_team/wankoe.md description: >- Geodata juggler from Tyrol, Austria exploring the realm of AI. Connected to nature, enjoys cooking, good movies, games, and unnecessary facts. --- --- --- url: /_team/eseneschal.md description: >- Geospatial engineer with expertise in remote sensing, GIS and ecology. A sports enthusiast, DIY hobbyist, and board game lover in my free time! --- --- --- url: /2017/03/eox-swarm-cryosat-science-meetings-in-banff-canada.md --- EOX is showing off a [VirES for Swarm](http://vires.services) poster at location #72 as well as giving two VirES for Swarm Introduction & Training Sessions one on Tuesday at 11:30 and the other on Thursday at 9:45 both local time and in room Aspen. Supporting material for the VirES for Swarm Introduction & Training Sessions is provided here as [PDF](/content/Vires_Presentation_Banff.pdf) or [PPTX](/content/Vires_Presentation_Banff.pptx). --- --- url: /agri.md --- ### About the EOX AgriApp #### Flexible Dashboard Service The EOX AgriApp is a flexible dashboard service for continuous remote sensing analysis of agricultural parcels or, more generally, any monitoring areas of interest. It visually presents satellite information such as vegetation and soil profiles interactively linked to image time series down to the parcel level—even allowing for comparison between parcels. #### Value-added Information Value-added derived information such as machine learning- or threshold-based markers (crop type, harvest events, mowing events, vegetation cover changes, …) are available as chart annotations and thematic map layers. A compliance rule engine evaluates specific parcel-level requirements and provides customised thematic map visualisations. #### Interactive Reports The cloud-based service is rounded off by a high-performance full-dataset search engine with dynamic filter combinations. A workspace with various interactive charts allows for detailed analysis of machine learning result and compliance rule engine outcomes. For example, the EOX AgriApp offers a confusion matrix and accuracy measures for crop type classification. #### Areas of Use Applications for the EOX AgriApp include—but are not restricted to— * Common Agricultural Policy implementation * drought and irrigation monitoring * CO2 flux visualisation and quantification * nature capital and biodiversity monitoring. In the context of the CAP Area Monitoring System the EOX AgriApp has successfully been in operative use within the Austrian and Irish Paying Agencies since the beginning of 2023, serving near real-time insights for millions of parcels. ### Learn more A blog post series introduces the AgriApp and its usage in the context of the Common Agricultural Policy, and you can also try our live demo instance. ### Customers & Partners ### Contact Do you have any questions? Let's get in touch with Stefan Brand! --- --- url: /2024/10/intergeo-2024.md --- *** Intergeo 2024 is one of the world’s leading expos for geospatial solutions, bringing together professionals, innovators, and thought leaders from across the globe. Held this year in Stuttgart, Germany, the event provided a perfect platform for industry experts to showcase the latest advancements in geospatial technologies, mapping, and data analytics. EOX was among the participants at this event. [Stefan Brand](mailto:stefan.brand@eox.at) and [Miriam Mabeya](mailto:miriam.mabeya@eox.at) had the privilege of representing the company on the expo stage, sharing insights about what we do. ## Key Highlights of the Presentation The presentation highlighted several key areas. First, it addressed the challenges of handling satellite data, emphasizing the complexity of processing such large datasets due to daily updates and cloud interference. To overcome this, the cloud-based [EOxHub](/expertise/managed-cloud-workspace) platform was introduced, offering services for data access, processing, and visualization via tools like [EO Dashboard](https://eodashboard.org/) and [AgriApp](/agri). We also talked about [EOxCloudless products](https://cloudless.eox.at/). The presentation additionally showcased successful case studies where our solutions have contributed to various projects on [Agriculture](/agri), e.g. one of the areas where we apply this technology is the Area Monitoring System which is used by government agencies to pay EU subsidies to farmers in a fair way. [Aviation](https://planefinder.net/), [EU Green Deal](https://gtif.esa.int), which is an ambition to be climate neutral by 2050 and the GTIF (Green Transition Information Factory) has tools that allow users to interactively discover the underlying opportunities and complexities of transitioning to carbon neutrality. **You can watch the [full presentation](https://www.youtube.com/watch?v=QuFjnIQJOs4) on demand.** ## General Observation at Intergeo It was striking to observe the market and geospatial community's increasing focus on UAVs/drones and 3D technologies, with numerous companies prominently exhibiting these advancements. A diverse range of mobile mapping systems, particularly more portable options, were on display. For EOX, this led to productive discussions about potential collaborations, as mobile mapping systems require global basemaps. However, Earth Observation (EO) and imagery technologies appeared to have limited representation at the event. Additionally, it was intriguing to see companies involved in ocean technology, exhibiting at this event showcasing innovations such as underwater drones and advanced underwater surveying solutions. ## Conclusion Beyond the presentation, Intergeo 2024 provided an excellent opportunity to engage with fellow professionals, potential clients, and industry thought leaders. We received a positive response to our technologies, with some expressing interest in partnerships and collaborations. The expo allowed us to network with key stakeholders and discuss how geospatial solutions can solve today’s most pressing global challenges, from climate change to greening the urban cities. We look forward to leveraging the momentum gained at the expo to further expand our impact across the various sectors. We are also planning to attend the next [Intergeo event in October 7th to 9th 2025](https://www.intergeo.de/en/welcome-to-intergeo) in Frankfurt, Germany, where we hope to network even further and gain deeper insights into emerging trends in geospatial technologies and innovations. We look forward to seeing how the industry evolves and especially new developments in areas like Earth Observation (EO) and satellite imagery, which as EOX we believe is already playing a bigger role in the industry. *** Miriam Mabeya, EOX IT Services GmbH Email: Stefan Brand, EOX IT Services GmbH Email: --- --- url: /2026/06/eoxcloudless-2025.md --- We have a new EOxCloudless release with more precise ecoregions and advanced ocean mosaicking. Check out the [preview](https://cloudless.eox.at/preview)! *** Over the past few years, our engineering focus for [EOxCloudless](https://cloudless.eox.at) has been, among others, on optimizing the BRDF corrections to eliminate equatorial striping and optimizing performance of the whole process chain. Having locked down radiometric quality and computational scale, the newest EOxCloudless dataset takes the next logical step: perfecting spatiotemporal context. We are abandoning our previous and partly arbitrary processing zones. Instead, we're letting the Earth's actual biological and physical realities dictate our algorithmic parameters, introducing precision ecoregion targeting and an entirely separate pipeline for ocean mosaicking to deliver our most accurate global map yet. ## Approximating Climate Zones A high-quality mosaic requires scenes with minimal cloud cover, taken as close to summer as possible, as a higher sun zenith creates smaller shadows. While compromises are inevitable, optimizing this time range is crucial. Fewer clouds mean fewer source products to read, which drastically improves processing performance. Additionally, deriving the mosaic from fewer scenes prevents the final map from looking patchy. To assign extraction parameters, we divide the globe into "processing regions" (not to be confused with "process zones," which manage internal data chunking). Historically, these processing regions were hand-drawn or based on political borders, leading to massive generalizations. Australia, for instance, was treated as a single region despite containing desert, tropical, and temperate climates. Climate zones indirectly control our key variables: temperature relates to the sun angle, and precipitation drives cloud cover. Because climate fundamentally determines biomes, we integrated the [RESOLVE Ecoregions and Biomes](https://ecoregions.world/) dataset to accurately fine-tune our settings. Rather than using all 846 terrestrial ecoregions independently, we combine them into larger, ecologically cohesive processing regions. By letting nature's true boundaries define our temporal search windows, we minimize visual patchiness, maximize processing speed, and deliver our most balanced mosaic yet. ## Ocean Integration Processing large water bodies alongside solid terrain requires compromises. For instance, to mitigate the blinding "mirror effect" of summer sun glint over the sea, we previously had to use summer-month time ranges because they yielded better results with land masses. We now have integrated the ocean polygons into the new biome-based processing regions. This allows us to get the best of the two worlds - specific parameters for ocean and land masses. To be able to further refine the settings, we also split up the oceans in "latitudinal" subregions. ## Building on Robust Foundations While our new biome-based regions and latitudinal ocean splits represent a major leap in our spatial logic, the robust core of the EOxCloudless processing chain remains firmly intact. We continue to use our dynamic, region-and-pixel-classified BRDF (Bidirectional Reflectance Distribution Function) correction model, which drastically reduces orbital striping over challenging terrains like the Amazon and the Sahara. We have also retained the Contrast Limited Adaptive Histogram Equalization (CLAHE) post-processing step introduced in previous releases, ensuring maintaining maximum structural sharpness and contrast in dense urban areas. Crucially, our delivery architecture remains unchanged and highly scalable. ## Usage and Applications Whether you are running complex ML pipelines using our 16-bit Exploitation Ready data via STACTA TileDirectories, or embedding our Viewing Ready single-file formats (GeoPackage and MapCache SQLite) into offline applications, the new dataset integrates seamlessly into standard [GDAL and QGIS](https://cloudless.eox.at/documentation/usage#data-usage-in-possible-applications) workflows without any disruptions. Also, EOxCloudless is being used in [agriculture monitoring](https://cloudless.eox.at/industries/agriculture), [games and simulators](https://cloudless.eox.at/industries/games-simulations), [urban planning](https://cloudless.eox.at/industries/urban-planning) and other industries. ## Explore the Data The EOxCloudless Acquisition year 2025 dataset is now available. We encourage you to head over to our [product page](https://cloudless.eox.at) to discover the improvements firsthand. Whether you are interested in commercial licensing, require off-the-shelf multispectral data, or want to discuss bespoke processing inquiries, you will find all the necessary [documentation](https://cloudless.eox.at/documentation/introduction), [data samples](https://cloudless.eox.at/documentation/examples), and [contact information](https://cloudless.eox.at/contact) directly on the new [website](https://cloudless.eox.at/)! ## Contact Information Do you have any questions? Please contact Karolína Lehotská at for sales or Joachim Ungar at for any technical matters. ## Want to Play a Game? **[EO-Guesser](https://cloudless.eox.at/eo-guesser)** is here! Are a fan of satellite imagery or the world from space? We are running a neat game and at EOX we have been playing it with great enthusiasm for some time now. If you want to try your luck or want to put your Earth Observation knowledge to test feel free to try our [EO-Guesser](https://cloudless.eox.at/eo-guesser). Being zoomed in on a random place on Earth just able to see the satellite map, can you guess where you are? How conversant are you with planet earth? Enjoy the game! ## Images Source EOxCloudless by [EOX IT Services GmbH](https://eox.at) (Contains modified Copernicus Sentinel data 2024 & 2025) *** Karolína Lehotská, EOX IT Services GmbH Email: Petr Ševčík, EOX IT Services GmbH Email: Joachim Ungar, EOX IT Services GmbH Email: --- --- url: /2019/05/eoxcloudless-for-common-agricultural-policy-cap.md --- Remote sensing goes hand in hand with extensive land monitoring, needless to say agriculture is such a case. Data analysis aiming at monitoring land and crops need high and consistent temporal resolution data supply. [CAP](https://ec.europa.eu/info/food-farming-fisheries/key-policies/common-agricultural-policy/future-cap_en#a-new-way-of-working) is an economical ecosystem to sustain and support agriculture throughout Europe. This helps to achieve better competitive conditions within and outside of agricultural industry and control the impact of agriculture on the environment. Beginning in 2020 crop monitoring with the help of remote sensing should be established in Europe as a part of modernization of the CAP ecosystem. This whole endeavor should be tailored to each Paying Agency in the member states individually. Providing specific and accurate information about state of crops, grassland, forests, etc. such remote sensing policy can provide unprecedented amount of information across whole countries in a manner of hours not days or weeks. At EOX we are cooperating with several [Paying Agencies](https://enrd.ec.europa.eu/general-info/whos-who/implementing-authorities/paying-authorities/paying-agencies_en) to introduce methods and products which will help to monitor large areas frequently and with the most precise information. *** ### Satellite Imagery The core data infrastructure should be based on remote sensing, i.e. Sentinel-1 and Sentinel-2 satellite constellations of the European Commission (EC). For the primary data dissemination and delivery the [Copernicus Data and Information Access Services](https://www.copernicus.eu/en/access-data/dias) (DIAS) exploitation is foreseen. EOX is in the DIAS [MUNDI](https://mundiwebservices.com/data) and we are trying to improve MUNDI's offer and services by evaluating, processing and developing land monitoring methods and relevant data products (see ). The data we offer ranges from RGB Natural Colors Satellite Imagery, over Vegetation Indexes (NDVI, CVI, EVI) to SAR Coherence or Backscatter maps. For technical details see the [EOxCloudless ARD Article](/2019/05/eoxcloudless-sentinel-1-and-sentinel-2-analysis-ready-data/). The description there introduces the processing methods we are using to create various data to be used for visual, machine learning (ML), land-use and land-cover analysis and more. With these data sources vegetation can be monitored over the whole vegetation period continuously, up to weekly with Sentinel-1 data and monthly with Sentinel-2. (For Sentinel-2 seasonality and the weather/cloud coverage can impact the acquisitions, from measurement every week to almost two months difference.) The gif below shows the NDVI change from March to October 2018, the greener the higher NDVI, brown and white means no vegetation. ![](/images/s2-l2a_monthly_animation.gif) NDVI change around Franzensdorf, Austria 2018 Next image shows the vegetation maximum in the area around Sporring, Denmark (north of Aarhus, DEN) represented by RGB composite of average Backscatter values from 4th of June to 17th of June 2018. ![](/images/Denmark_Fields_Aarhus_north_2018-06-04_2018-06-17-2.jpg) North of Aarhus, Denmark - Vegetation Maximum 04.06.-17.06.2018 RGB Gamma0 Average \[dB] Composite With the means and tools at hand we can offer services all over the globe for anyone, from a single field to the whole World. ![](/images/2018-Austria_480p-1.gif) Austria bi-monthly 2018 L3 Mosaics - Sentinel-2 Surface Reflectance *** ### CAP monitoring Combining the satellite data with ground truth datasets, confident monitoring structure can be established. Relying on ML and Classification such infrastructure can provide continuous stream of information able to differentiate even the slightest change over an area. Without the need of shoveling through all the data every single time. The DIAS facilities are the key for such campaigns, with the Sentinel data stored in the same cloud environment, and the possibility to use additional services available there, it has never been easier to access so much information at the same time. Additionally custom data can be uploaded to the cloud at any time, e.g. training or already trained datasets with labels for ML, weather forecasts, measurements and models, climate models, hydrological features, soil profiles, can individually shape the way to the final result. The decisions from the results can determine the distribution of the CAP resources for Paying Agencies. Each Paying Agency can decide how to implement the monitoring approach based on the tools and policy in their area. *** ### More Information For additional information visit: and contact us if you have questions. Visit us in Copenhagen, Denmark at the 55th Panta Rhei Conference from 22th to 24th of May 2019. *** Petr Ševčík, EOX IT Services GmbH Email: petr.sevcik@eox.at *Contains Modified Copernicus Sentinel-1 and Sentinel-2 Data 2018.* --- --- url: /2019/05/eoxcloudless-sentinel-1-and-sentinel-2-analysis-ready-data.md --- The term Analysis Ready Data (ARD) describes data which is preprocessed for immediate further use. For Sentinel-1 and other SAR data, calibration and corresponding terrain correction would be an example of such a preprocessing chain. For optical data this can mean for example cloud removal and atmospheric correction and for some specifications it also means bringing multiple datasets to a common grid. When using data from different sensors (e.g. optical sensors like Landsat and Sentinel-2) it could mean the data is preprocessed in such a way that data from different sensors can be compared directly (e.g. Landsat NIR band and Sentinel-2 NIR band). This facilitates further usage such as Machine Learning (ML), image classification, creating derivatives like Vegetation Indexes. The availability of ARD saves resources needed to deal with the huge amount of raw satellite imagery. We offer ARD in a form of high quality homogeneous mosaics over large areas. The mosaics can be traced back to its origin for pixel selection methods or can be statistical for which the pixel origin information is lost. Taking the EU Common Agricultural Policy (CAP) into consideration, the DIAS have received tasks and interests to improve their data collections especially the ones related to land use and agricultural applications like crop classification and crop monitoring, leading to the birth of CAP Analysis Ready Data (CARD) . We at EOX are a partner in the DIAS [MUNDI](https://mundiwebservices.com/data) and are developing more efficient ways how to disseminate the Sentinel-1 and Sentinel-2 data and information. Based on the experience of processing huge amount of satellite imagery (see [s2maps.eu](https://s2maps.eu/)) we have developed state of the art processing environment for SAR and optical imagery. Thus introducing products ready to be used as CARD. The Level-3 data reference used below is the same according to the definition from [CESBIO](http://www.cesbio.ups-tlse.fr/index_us.htm) (see: ) for both Sentinel-1 and Sentinel-2 processing. ## Sentinel-1 Level-3 Backscatter Mosaics The first example of our work was extensive Sentinel-1 processing. We have went through all data available for the year 2018 over Denmark and generated comprehensive SAR signal statistics at 10\[m] resolution. These were created to be further used to hone Machine Learning (ML) to be exploited for [agricultural Land-Use Classification](http://blogs.esa.int/philab/2019/03/14/%cf%86-lab-workshop-on-sentinel-1-sar-data/). The results are visualized in [this preview](https://eox-public.obs-website.eu-de.otc.t-systems.com/), the data itself is publicly available for free on Open Telekom Cloud (OTC) bucket: [eox-public.obs.eu-de.otc.t-systems.com](http://eox-public.obs.eu-de.otc.t-systems.com). (This data and its preview is mostly for demonstration of various tools so it is subject to change.) The gif below shows RGB composite of average values of the Normalized Flattened Scattering Radar Cross Section (Gamma0) every 14 days of 2018. ![](/images/Funen_Island_Sentinel-1_2018_EOX_small.gif) Denmark 14 days L3 Mosaics- Sentinel-1 Gamma0 The research background for this project was Machine Learning (ML) algorithm improvement for food security in the [Phi-Lab](http://blogs.esa.int/philab/2019/03/14/%cf%86-lab-workshop-on-sentinel-1-sar-data/) in ESRIN (European Space Agency). We have agreed to setup a processing environment with the result of bi-weekly Gamma0 statistics mosaic over Denmark 2018. The data of the statistic are the dual polarization (IW VV/VH) decibel values of 14-days average, minimum, maximum and standard deviation, as well as corresponding count of input data, per pixel. *** ## Sentinel-1 Level-3 Coherence Mosaics Coherence is a value which describes how similar two SAR measurements are based on their signal properties ([Definition](https://earth.esa.int/handbooks/asar/CNTR5-2.html)). Therefore it is generally used to describe changes at a location, for instance if we have high coherence, almost nothing has changed between two or more acquisitions, for instance urban areas tend to stay similar. Low coherence comes from surfaces that change frequently based on wind, precipitation, or other (surface) movement. Somewhere in between is bare soil and similar entities. Coherence is a normalized value from 0 to 1. In the image below red are values from 0.0-0.25, orange 0.25-0.5, yellow 0.5-0.75 and green 0.75-1.0. The original values are stretched to 256 colorspace for visualization with customized [geotiff.js](https://geotiffjs.github.io/). Original pixel size is 20\[m]. ![](/images/Coherence_Belgium.jpg) Belgium 7 days L3 Mosaics- Sentinel-1 Min. Coherence 25.06.-01.07.2018 This data comes from the enhanced data collection offered by [MUNDI](https://mundiwebservices.com/data/sentinel-1), currently available for [Denmark](https://mundiwebservices.com/acdc/catalog/proxy/search/Sentinel1/opensearch?q=\(footprint:%22Intersects\(POLYGON%20\(\(7.03116%2053.43760,%2015.46875%2053.43760,%2015.46875%2059.06250,%207.03116%2059.06250,%207.03116%2053.43760\)\)\)%22\)\&productType=SLC\&processingLevel=L2_), [Belgium](https://mundiwebservices.com/acdc/catalog/proxy/search/Sentinel1/opensearch?q=\(footprint:%22Intersects\(POLYGON%20\(\(1.40625%2049.21875,%207.03125%2049.21875,7.03125%2052.03125,%201.40625%2052.03125,%201.40625%2049.21875\)\)\)%22\)\&productType=SLC\&processingLevel=L2_) and Italy. Under Denmark and Belgium there are the *opensearch* links where the data can be found and accessed publicly for free (If you are interested in other Sentinel-1 Level-2 data, please [contact EOX](/contact/) or [MUNDI](https://mundiwebservices.com/contact), MUNDI Level-2 Gamma0 is available for Denmark and Italy 2018). The difference between Level-2 and Level-3 products in this case is, that the Level-2 products are disseminated in the same extend and similar product structure as the Level-1 products, with the added value via the processing. The EOX Level-3 products are reduced to single data product for defined time range and AOI. The reduction is usually performed as a statistic of the measurements. Depending on its nature the results can be selective (e.g.: Min or Max, ...) or statistical (e.g.: Average, Standard Deviation, Linear Models, ...). Additional public datasets like the one in the image above (minimum Coherence L3 Mosaics) are available as mosaics via the same interface like the Sentinel-1 Backscatter Mosaics here: [eox-public](https://eox-public.obs-website.eu-de.otc.t-systems.com/#zoom=7\&lon=11\&lat=55.5) ([OTC Bucket](http://eox-public.obs.eu-de.otc.t-systems.com)). *** ## Sentinel-2 Level-3 Mosaics Regarding the optical satellite imagery the latest Sentinel-2 Mosaics we've made were the monthly mosaics over Austria for the year 2018. The mosaics were delivered as satellite background maps at 10\[m] resolution. It has been made from the Sentinel-2 Level-2A data by a pixel selection method and each pixel is traceable to the original product. Currently available (not publicly) in EPSG:31259 and bands (red, green, blue, NIR). The data is collected from the primary month with previous or following month as fallback to reduce no-data values as much as possible (clouds, snow, etc.). The selection method for each pixel is to search for maximum [NDVI](https://www.indexdatabase.de/db/i-single.php?id=58) and [CVI](https://www.indexdatabase.de/db/i-single.php?id=391) (Chlorophyll Vegetation Index). The results are shown in the gif below. The data itself having four spectral bands allows some basic analysis like monthly NDVI, EVI, CVI, NDWI. This clean dataset is ideal to feed to trained ML environment to classify or simulate Sentinel-2 optical imagery. ![](/images/2018-Austria_480p-1.gif) Austria monthly 2018 L3 Mosaics - Sentinel-2 Surface Reflectance The Austria Mosaics were processed to serve as high resolution satellite maps for the [GeoData Office](https://geodaten.bgld.gv.at/de/kartendienste-fachdaten/oeffentliche-kartendienste.html) of Burgenland, Austria. *** ## EOxCloudless Should you be interested in processing for your Area of Interest (AOI) please visit . Under "Data Products - EOxCloudless Custom" we accommodate any individual needs regarding the Sentinel-1 or Sentinel-2 data requests. *** Petr Ševčík, EOX IT Services GmbH Email: petr.sevcik@eox.at *Contains Modified Copernicus Sentinel-1 and Sentinel-2 Data 2018.* --- --- url: /2022/03/eoxmaps-the-update-part1.md --- The update of our [EOxMaps](https://maps.eox.at/) infrastructure has been long overdue. Finally, in 2021 we have completely reworked our former EOxMaps infrastructure, which allows us to generate, alter and update EOxMaps more frequently than before. Firstly, we focused on updating the somewhat outdated map label layers. ## Data Foundation - OpenStreetMap & Natural Earth Like with most OpenData maps today we have opted to use a mix of [OpenStreetMap](https://planet.openstreetmap.org/planet/) and [Natural Earth](https://www.naturalearthdata.com/) as sources for our map labels update. ## PostgreSQL + PostGIS Database The OSM dataset is imported into a PostGIS database via an import tool: [imposm3](https://github.com/omniscale/imposm3). Here we specify the nessesary parameters to extract the attributes needed by the map layers from the OpenStreetMap `.pbf` file. Given the size of the imported data and the complexity of some spatial features, we have generalized some layers to make the vector tiles generation more performant. ### Centerlines For large geographical features like oceans, lakes, mountain ranges, etc. we use the custom centerlines of these objects for labeling. The EOX has a custom solution to generating these centerlines as described in the ["Curved labels"](https://eox.at/2015/12/curved-labels/) article. The output of these centerlines is then imported into the PostGIS database. ## Custom Tiles-API as requests handler Since numerous tools are used in this pipeline, we were forced to consolidate these tools under a single endpoint that handles input and output (I/O) from the database over the vector tiles to rendering of raster tiles in the final map layers. For this data throughput management we have utilized [FastAPI](https://fastapi.tiangolo.com/) to serve vector-tiles and rendered raster tiles as well as some metadata around the whole EOxMaps infrastructure. ### FastAPI Introduction For those unfamiliar with [FastAPI](https://fastapi.tiangolo.com/) here is a python3 ['Hello World'](https://fastapi.tiangolo.com/tutorial/first-steps/) on how to set up a simple API instantly. 1. Save a `main.py` python file with: ```python from fastapi import FastAPI app = FastAPI() @app.get("/") async def root(): return {"message": "Hello World"} ``` 2. Run the app from `bash` terminal via: ` $ uvicorn main:app --reload` 3. Open a browser at localhost: . Response should be following `json` output: `{"message": "Hello World"}`. ### Vector Tiles (Mapbox-Vector-Tiles) At EOX we are already familiar with vector tiles ([MVT](https://docs.mapbox.com/vector-tiles/specification/)) as for instance this article from last year shows: [Vector tiles with dynamically updated style in Leaflet LPvis](https://eox.at/2020/01/vector-tiles-with-dynamically-updated-style-in-leaflet-lpvis/). Having already had some idea on how we should proceed in handling the data from the PostGIS database, we decided that apart from our agriculturally driven projects we will use a similar data pipeline here as well. Vector tiles are extremely handy for anything with high I/O vector data throughput. As the files are independent from each other, this enables convenient concurrently driven utilization. There are some PostGIS native functions that create MVTs on the fly: [ST_AsMVT](https://postgis.net/docs/ST_AsMVT.html) and [ST_AsMVTGeom](https://postgis.net/docs/ST_AsMVTGeom.html). If you are comfortable with the SQL language, you can customize and add some handy SQL functions to clip and reproject the geometries and edit attributes of the vector layers. These operations can increase or decrease the performance of the MVT generation, so be careful. Finally, we serve the optimized vector tiles to the renderer via our `Tiles-API`. ## TileServer GL as renderer To make a smooth transition between our former and current maps infrastructure we still do render raster (`.png`,`.jpg`) tiles to serve them via our WMS/WMTS services. For this we have used [TileServer GL](https://github.com/maptiler/tileserver-gl) written by [Maptiler](https://www.maptiler.com/). This tool is based on the open version of [Mapbox-GL-Native](https://github.com/mapbox/mapbox-gl-native). TileServer GL allows for an easy rendering of vector tiles into raster tiles. It also comes with some handy [configuration options](https://tileserver.readthedocs.io/en/latest/config.html#) that give somewhat more control over the whole rendering procedure. It also provides vector tiles (source data), styles and rendered tiles (WMS/WMTS) [endpoints](https://tileserver.readthedocs.io/en/latest/endpoints.html). This is particularly handy if you don't want to handle the requests yourself like we do via the `Tiles-API`. ### Mapbox-GL Styles The final layer look is defined in a [Mapbox Style File](https://docs.mapbox.com/mapbox-gl-js/style-spec/), which is a json file specifying the outcome of your map. This is the most time-consuming part where fiddling with the map details comes into play. There are numerous ways on how to configure and review your styling; our primary style creation tool was [maputnik](https://maputnik.github.io/). Maputnik allows editing [Mapbox Style Files](https://docs.mapbox.com/mapbox-gl-js/style-spec/) in a nice map editor and exporting the final style so that you can use it in the TileServer GL endpoints. Example of a **mapbox-gl-js json style** config: ```json { "version": 8, "name": "osm_borders", // name of the style "metadata": { // some metadata information, this can be whatever you need "name": "osm_database", "version": 1.0, "timestamp": "2022-03-08", "info": "OSM Borders Tutorial", "sources": ["openstreetmap"], "maputnik:renderer": "mbgljs" }, "sources": { // sources where to look for vector tiles, "osm_borders": { // in this case we use the regular EPSG:3857 MVT grid "minZoom": 0, "maxZoom": 8, "tiles": ["http://{VECTOR_TILES_HOST_ADRESS}/vt/xyz/overlay_mercator/{z}/{x}/{y}.pbf"], "type": "vector" } }, "glyphs": "http://{FONTS_HOST_ADRESS}/fonts/{fontstack}/{range}.pbf", // one layer for this example should be enough "layers": [ { "id": "osm_border_linestrings_countries", "type": "line", // where to look in the sources list above "source": "osm_borders", // name of the layer in the MVT files "source-layer": "osm_border_linestrings_countries", // filter what you need based on attributes in the MVT "filter": ["all"], "layout": { "line-cap": "round", "line-join": "round", "visibility": "visible" }, "paint": { "line-color": "rgba(255, 130, 40, 1)", "line-width": {"stops": [[0, 0.5], [8, 1], [11, 2]]} } } ], "id": "12345" } ``` ## Preview and Map Layer Cache Having everything set up and running, it is nice to see your map layers in action. For the preview we have taken our own [Openlayers](https://openlayers.org/) extension - [EOxElements](https://eox-a.github.io/elements/examples/map-basic.html) package and provided it with access to the vector tiles, WMS and WMTS endpoints—all of these served by the custom `Tiles-API`. ### MapServer MapCache Serving already cached map layers increases the performance significantly. Therefore we are using [MapCache](https://mapserver.org/mapcache/), which provides a tile based caching solution and also offers various serving [endpoints](https://mapserver.org/mapcache/services.html). The caching itself happens in a command called ["mapcache_seed"](https://mapserver.org/mapcache/seed.html), where a lot of options are offered to tailor the (Map)cache into the final endpoints and services, which will utilize the cache once it has been generated. ## Overview As this is the first article about our maps technical details, we hope that it was an appropriate introduction into the EOxMaps topic on how we utilize available tools to provide up-to-date solutions for one of our many services. It might seem that a lot of tools in this scheme are somewhat doubled in the infrastructure, which is partly true. For the purpose of a flawless transition from our old EOxMaps solution we have adapted available geospatial tools and combined them into one big pipeline which is capable of serving map tiles on the fly. Here is a graph visualising the content of this article. ## What comes next? Like in my [last article](https://eox.at/2021/03/sentinel-2-cloudless-2020/), there is more stuff happening at EOX right now. After the EOxMaps labels, we have started working on even more advanced automatization of our [EOxCloudless](https://cloudless.eox.at/) data processing. As for the maps, in a bit more distant future, apart from labels we aim to update our other [EOxMaps](https://maps.eox.at/) layers (i.e. the "Terrain" and "Terrain-Light" layers) so that they get a fresh coat of paint as well. *** Petr Ševčík, EOX IT Services GmbH Email: Data ©[OpenStreetMap](https://www.openstreetmap.org/copyright) contributors and others. Contains Modified Copernicus Sentinel-2 Data 2020. Our thanks to [maptiler](https://www.maptiler.com/) for [TileServer GL](https://github.com/maptiler/tileserver-gl/blob/master/LICENSE.md) --- --- url: /2021/09/eoxserver-1-0.md --- With the release of [version 1.0](https://github.com/EOxServer/eoxserver/releases/tag/release-1.0.0) of the [EOxServer](https://eoxserver.org/) achieved it is a good time to shed some light on this workhorse software more than ten years in active development. Even before this milestone, EOxServer was and is used in quite a number of operational deployments, most notably in the [VirES](/tag/vires/) line of projects and others like the [Copernicus Space Component Data Access system (CSCDA)](https://spacedata.copernicus.eu/) (shown in the images below), [Mundi Web Services](https://mundiwebservices.com/), [EO-CAT](https://eocat.esa.int/sec/about.html), or more recently [Earth Observation Exploitation Platform Common Architecture](https://eoepca.org/). ## What is EOxServer? In simple terms: EOxServer is an EO Raster Data server that catalogs EO Products and provides access to stored data via open interfaces. The most simple case would be to have a couple of TIFF files on the local disk which shall be made accessible, but EOxServer can provide much more functionality and is very flexible in terms of storage back-ends, processing, and structure. For example, in a cloud native setup, the data is likely stored on an Object Storage. EOxServer provides both S3 and OpenStack Swift support, which means the files can be indexed from there and do not need to be stored locally. Alternatively, files can be accessed from pure HTTP as well and for some retro-enthusiasts even FTP back-ends. The benefits from that flexibility are quite obvious: EOxServer can adapt to existing storage solutions, but can also benefit from more modern architectures. In order to structure the incoming data, EOxServer provides a refined data model to both keep integrity, but also provide additional functionality on top of a pure data access. For example, an application operator may want to expose their Red-Green-Blue-Near Infrared data in a raw fashion, but also may want to provide dynamic renderings on top of that. This may be a simple true color visualization using the Red, Green, and Blue bands, but for these kinds of files often a false color composite using the Near Infrared, Red, and Green bands is also very common. Such dynamic views on the data can easily be made available using EOxServer, without the necessity to duplicate any file or pre-generate the visualizations. ## Serviced with a smile On the front side, EOxServer supports various protocols out of the box. This includes the Open Geospatial Consortium's Web Services (OGC OWS) family of protocols Web Coverage Service, Web Map Service, and Web Processing Service. The former provides access to the raw values, which is especially useful for multi-spectral or higher bit-depth data such as Digital Elevation Models (DEM). WMS provides rendered maps that can very easily be integrated in a variety of clients to provide layers of mapping applications. Here the visualizations detailed earlier come into play: each stored collection can be accessed as a layer for each rendering method available. For visualizations that produce only a single channel output, one of various color-scales can be used to colorize the output. When visualizing longer time periods resulting in more intersecting products, it may be needed to filter the resulting products based on their metadata. For this reason, the WMS interface supports the optional non-standard `cql` parameter, where filter parameters in CQL syntax can be passed. For example, this can be used to only include images that have a lower cloud coverage than a certain value: `...&cql=cloudCover < 10`. It is also possible to create custom processes and expose them via the integrated WPS. This allows to specify various kinds of parameters, synchronous or asynchronous processing and single or multiple outputs in custom formats. When dealing with large collections of EO data often a search interface is required to allow for a suitable dissemination. For this reason EOxServer provides an OpenSearch interface including its time, geo, and EO extensions to allow for fine grained filters over the indexed metadata. If even that filtering method is too crude, it is possible to pass custom filters using the same `CQL` syntax as with WMS. ## Ready-to-go Setup By providing open interfaces it is easy to integrate offered data into a user client interface application such as QGIS. But that may not even be necessary, as EOxServer ships with a client of its own: an instance of [EOxC](https://github.com/eoxc), always configured and up to date with the contents of the service. With that client, users can browse the contents of the service, inspect the data and its various configured visualizations and download specific scenes or subsets as shown in the image above and the two below. On top of the already existing functionality EOxServer is built with extensibility in mind. It is quite easy to extend its capabilities at any point in the software architecture. This is demonstrated by its usage in very specific fields, such as magnetospheric or atmospheric missions with data from the Swarm or Aeolus missions respectively. Even in planetary sciences EOxServer was successfully deployed. ## What's next? - View Server We are currently dedicating quite a lot of efforts into the [View Server](https://vs.pages.eox.at/vs/) which in its heart is relying on EOxServer and is adding cloud native configuration and deployment. We'll introduce the View Server in a future post but please [get in touch](mailto:office@eox.at) if you're already interested to use it. --- --- url: /_team/zz_guest_davise.md --- --- --- url: /2025/06/lps-vienna-2025-blogpost.md --- *** Welcome to the Living Planet Symposium 2025 in our hometown Vienna! In an effort to help meet the event's [sustainability goals](https://lps25.esa.int/sustainability-plan/), we are refraining from distributing printed materials. Our handouts are instead provided as the following [digital files and websites](https://drive.google.com/drive/folders/1_cJJgV1CSOKumdRs1vqMLaz2W1YIxeUj?usp=sharing). Find us at our **booth F13, level 0** or one of our presentations. ## EOxCloudless - Your cloudfree global satellite imagery basemap [**Get in touch about EOxCloudless via cloudless@eox.at**](mailto:cloudless@eox.at) ## EOxHub Workspaces - Where Earth observation work takes place [**Get in touch about EOxHub Workspaces**](mailto:service+tenants-hub-support-request@sd.eox.at) ## eodash - Open source tool to set up your own dashboard [**Get in touch about eodash**](mailto:office@eox.at) ## VirES for Swarm - Earth's magnetic field as observed by satellite [**Get in touch about VirES for Swarm via info@vires.services**](mailto:info@vires.services) ## VirEs for Aeolus - Atmospheric wind fields observed by satellite [**Get in touch about VirES for Aeolus via info@vires.services**](mailto:info@vires.services) ## Area Monitoring Services - The solution for managing your area-based direct payments [**Get in touch about Area Monitoring Services via ams@eox.at**](mailto:ams@eox.at) *** Karolína Lehotská, EOX IT Services GmbH Email: Stephan Meißl, EOX IT Services GmbH Email: --- --- url: /2015/09/esas-spot-1-5-via-sm-cat.md --- The Static Map Catalogue introduced by Fabian in the [last post](/2015/08/sx-cat-part-one-the-static-map-catalogue/) is now used by ESA to also [show their Spot 1-5 collections](https://tpm-ds.eo.esa.int/smcat/Spot1-5_ESA/). The other available collections are ESA's Landsat [TM](https://landsat-ds.eo.esa.int/smcat/LandsatTMCloudFreeCoverage/) and [ETM](https://landsat-ds.eo.esa.int/smcat/LandsatETMCloudFreeCoverage/) archives. Please read Fabians [excellent intro](/2015/08/sx-cat-part-one-the-static-map-catalogue/) if you're interested in more details about the software and contact us if you're interested to use it with your data. --- --- url: /2018/05/explore-landsat-8-scenes-with-cog-explorer.md --- In our [previous blog post about visualizing raster data](/2018/01/visualizing-geotiff-tiles-with-openlayers/) we demonstrated a use case for client side browser based visualization of GeoTIFFs using modern web techniques and the [geotiff.js](https://geotiffjs.github.io) library. The application uses GeoTIFFs derived from Sentinel-2 data and visualizes either single bands or calculates indices using applied parametrized color scales. We now like to present a different application of client side GeoTIFF usage, a small application we called [COG-Explorer](https://geotiffjs.github.io/cog-explorer/). This demonstrator application is built to explore the technical feasibility of browser based exploitation of high-resolution earth observation imagery in [Cloud Optimized GeoTIFF](http://www.cogeo.org/) format. The application is currently tailored to use data from the Amazon S3 Landsat 8 repository (since it provides a vast amount of datasets in the COG format), but also works with any other GeoTIFF image on the web. When a scene is selected, it is shown on the map with certain image processing applied (contrast adaption is necessary, as the Landsat 8 data is not visually representable by default). We are convinced that this approach has huge benefits in the exploitation of earth observation data in contrast to traditional server side rendering, so stay tuned for further developments! Click on the image below to try out COG-Explorer! [![](/images/cog-explorer-2.jpg)](https://geotiffjs.github.io/cog-explorer/) --- --- url: /_team/schindlerf.md description: >- Passionate about open source, geospatial software, and building things that make complex data accessible. Creator of geotiff.js and always curious about new technologies, especially AI. Enjoys hiking, climbing, running, and board games. --- --- --- url: /_team/zz_guest_schindlerf.md --- --- --- url: /2016/09/find-us-at-eo-open-science-2016.md --- Gerhard Triebnig ([@GerhardTriebnig](https://twitter.com/GerhardTriebnig)), Christian Schiller, Martin Pačes and myself ([@ungarjo](https://twitter.com/ungarjo)) will be in Rome/Frascati next week, for this year's [EO Open Science 2016](http://eoopenscience.esa.int/) conference, hosted and organized by [ESA](http://www.esa.int/). Don't hesitate to reach out to us at our table in the Exhibition Area! On Tuesday (Sept. 13th, 2016) at 4:30 p.m., Gerhard will present the *Virtual Workspaces for EO Scientists ([VirES](https://vires.services/)) - Operational for Swarm Magnetic Field Mission* in the Magellan Conference Room. ![vires](/images/vires.jpg) There will also be a [livestream](http://www.esa.int/Our_Activities/Observing_the_Earth/Watch_live_EO_Open_Science) from the conference if you couldn't make it to Frascati. --- --- url: /2017/07/foss4g-europe-2017.md --- This week the third [FOSS4G-Europe conference](https://europe.foss4g.org/2017/) took place in Paris, France. Stephan presented the EOxC client for CODE-DE: Joachim talked about generating an almost cloudless mosaic for the World using Sentinel-2 data: [presentation pdf](/content/ungarj_eox_s2cloudless.pdf) As a sponsor, EOX even had a booth! --- --- url: /2015/07/foss4g-europe-2015.md --- Last week the second FOSS4G-Europe conference took place in Como, Italy following two FOSS4G-CEE (Central and Eastern Europe) in Prague and Bucharest. It was a huge success with over 400 participants which means sold out already weeks in advance. Can you find Joachim and Stephan from EOX on the group photo? On the first day Stephan reported together with Thomas Bonfort on the current MapServer project status. Among all the great presentations I want to highlight the talk of Ian Turton on Earning Your Support Instead of Buying it which holds some really good advice. Even Rouault presented a long list of new features available in GDAL 2.0. The Pointcloud tango from Ivan Sanchez was an entertaining surprise at the closing plenary. Many thanks to everybody for a great FOSS4G-Europe. --- --- url: /terms-conditions.md --- (version as of February 5th 2019) Download pdf version [here](/content/general_terms_and_conditions.pdf) to save or print this general terms and conditions. ## Section 1 - Scope of Application a. By completing an order for any product or service of EOX IT Services GmbH ("EOX") referencing these General Terms and Conditions (the "Terms"), the Customer is accepting and agreeing to be bound by these Terms. If a person is completing a purchase order on behalf of a company or other legal entity, he/she represents to have the authority to bind that entity accordingly. b. The contractual relationship between the Customer and EOX is established upon the written acceptance of the Customer's purchase order by EOX ("Purchase Order Acceptance") and is limited to the scope of the Purchase Order Acceptance. The Purchase Order Acceptance together with these Terms and the separately agreed license terms (the "License") are collectively the Customer's agreement for the license and purchase of EOX products (the "Agreement"). c. Specifications in the Purchase Order Acceptance, the License or other existing written agreements between the Customer and EOX shall supersede any contrary provisions in these Terms. Unless explicitly stipulated otherwise the Terms apply to all products of EOX. d. Any general terms and conditions or other contractual frameworks used or communicated by the Customer, which have not been agreed to by EOX in writing, are hereby excluded for the entire contractual relationship between the Customer and EOX. ## Section 2 - Product Specification and Delivery a. Nature and scope of the product are determined by the descriptions contained in the Purchase Order Acceptance. Prior to the Purchase Order Acceptance EOX reserves the right to discontinue developing, producing, licensing or distributing any of the products and to modify, replace or add to the products at its discretion at any time. b. Our offers are not binding. Details contained in our website, catalogs, brochures and similar material as well as oral statements, including regarding prices and any delivery date and/or suggested end collection date, are only relevant if expressly confirmed in writing by EOX in the Purchase Order Acceptance. ## Section 3 - Payment and Delivery Terms a. The Customer shall pay EOX for the products at the fee set forth in the Purchase Order Acceptance. Payment is possible via credit card or invoice. The Customer shall pay the fee in the currency specified by the website's payment functionality. No payment shall be deemed to have been made until EOX has received payment in cleared funds. b. EOX shall deliver the product to the Customer after receiving full payment in cleared funds. EOX shall not be obliged to deliver any part or element of the product before receiving such payment. In case it has been expressly agreed with the Customer that EOX shall deliver the product prior to receiving full payment, all rights to the delivered product shall remain exclusively with EOX until full payment is received and the Customer is not entitled to utilize the product in any way without the prior written authorization by EOX. c. In the case of payment via invoice, the Customer must make all payments by the date set forth in the invoice but no later than within thirty (30) days following the date of invoice. d. If the Customer does not make payment on or before the date on which it is due, yearly interest shall be payable on the overdue amount at the rate of 8% above the base rate of the Austrian National Bank, but no less than 12% per year. Interest shall be payable at this rate both before and after any judgment is made against the Customer until the date on which payment in cleared funds is received in full, including all accrued interest. The Customer shall indemnify EOX for all costs and expenses reasonably accrued in the collection of the delayed payment. e. In case of late payment or non-payment, EOX reserves the right to (i) suspend current orders until full payment of the invoice, (ii) refuse future orders or (iii) terminate the existing Agreement with the Customer, after granting a reasonable notice period. In addition, EOX reserves the right to assert any damages relating to the delay of payment against the Customer and claim all accrued costs, including loss of profits. f. The Customer waives the right to withhold payment based on any claims of incomplete, deficient or otherwise non-conforming delivery by EOX. The Customer shall make all payments due without any deduction by way of set-off, counterclaim, discount or otherwise unless the Customer has a valid confirmatory court order from an Austrian court or EOX has accepted such set-off, counterclaim or discount in writing. ## Section 4 -Cancellation Terms a. After a Purchase Order Acceptance the order may only be canceled by the Customer under the condition of paying a cancellation fee of 100% of the agreed fee. All cancellation fees will be invoiced by EOX following cancellation. Customer will pay all cancellation fees as set forth in Article 3 above. ## Section 5 - Taxes a. Unless specified otherwise, indicated fees do not include any taxes, fees, excises or tariffs ("Taxes"). Customer is responsible for, and if applicable will reimburse EOX within thirty (30) days of request, all Taxes and any related penalties, except for taxes imposed on EOX's net income. If Customer is required to withhold Taxes from payments, the amount due and payable to EOX must still equal the fee that would otherwise be payable had the Taxes not been withheld, and Customer must provide EOX with a receipt or other acceptable documentation from the applicable taxing authority evidencing the payment of the Taxes. ## Section 6 - Resale and Lawful Use a. Unless expressly agreed otherwise, the Customer may not *directly* redistribute the product to third parties - neither for sale nor for free. b. The Customer irrevocably and unconditionally warrants and undertakes that it will only use the products for lawful purposes. c. The Customer shall indemnify EOX and hold EOX harmless for any liability whatsoever that EOX suffers by reason of breach by the Customer of the warranty and undertaking in clause [6(b)](#lawful). ## Section 7 - Intellectual Property and Confidentiality a. The Customer acknowledges and agrees that the copyrights and similar rights, patents, trade secrets, and all other intellectual property rights of whatever nature in, or related to, the products (hereinafter "EOX's Proprietary Rights") are and shall remain the property of EOX, and nothing in the Agreement shall be construed as transferring any aspects of EOX's ownership, title or rights to the Customer or any third party. b. The Customer or any third party shall not have any right, title, or interest in EOX's Proprietary Rights other than the Customer's right to use the same in accordance with the terms of the Agreement. c. The Customer acknowledges and agrees that the product contains confidential and proprietary information of EOX. The Customer agrees to keep the product strictly confidential and not to disclose any information related to the product to any third party, unless such disclosure is absolutely necessary for the use of the product as expressly permitted by the terms of the Agreement. ## Section 8 - Data Protection a. To the extent Customer's use of the product or the execution of the Agreement will involve the processing of personal data submitted by EOX to the Customer, Customer shall comply at all times with applicable data protection laws, regulations and other legal requirements (in particular the EU General Data Protection Regulation). Customer will ensure that it has in place adequate privacy policies and employs adequate technical and organizational security measures to protect personal data against breaches. Customer shall indemnify and hold harmless EOX for any claims or damages arising out of, or in connection with, violations of data protection laws by the Customer. ## Section 9 - Termination a. EOX may terminate the Agreement with immediate effect upon giving written notice if the Customer: 1. commits a material breach of his contractual obligations which is incapable of remedy; or 2. commits a material breach of his contractual obligations which is capable of remedy but the Customer fails to remedy that breach within 30 calendar days of being notified of the breach. b. For the avoidance of doubt, failure or delay of payment of the agreed fee, breaches of applicable license terms and violations of intellectual property rights constitute material breaches of contractual obligations. c. Upon expiration or termination of the Agreement: 1. the Customer will return the product to EOX and destroy any other copies of any portion of the product in its possession, and 2. the Customer will have no further right to possess or use the product. ## Section 10 - Disclaimer of Warranties and Limitations of Liability 1. **Unless otherwise separately undertaken by EOX, to the extent possible, EOX offers the product as-is and as-available, and makes no representations or warranties of any kind concerning the product, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. To the extent disclaimers of warranties are not allowed by applicable law in full or in part, this disclaimer may not apply to the Customer.** 2. **To the extent possible, but with the exception of intentional damages, in no event will EOX be liable to the Customer on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of the contractual relationship with the Customer or use of the product, even if EOX has been advised of the possibility of such losses, costs, expenses, or damages. To the extent a limitation of liability is not allowed by applicable law in full or in part, this limitation may not apply to the Customer.** 3. EOX and the Customer agree that the disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability. 4. The Customer shall release EOX from any liability in connection with Customer's use of the product and indemnify and hold harmless EOX from and against any claims. Any costs incurred by EOX for necessary legal action or defense in this regard shall be reimbursed by the Customer. 5. The Customer agrees to inform EOX immediately about any actual or imminent infringement of or damage to the product. If EOX shall decide to resort to legal action to avert such infringement or damage, the Customer agrees to make every effort to assist EOX upon request. Without EOX's prior explicit written consent, the Customer shall not initiate any legal action in this regard. 6. **In no event will the total liability of EOX, its affiliates or its suppliers arising out of or in connection with the products exceed the fees paid by the Customer for the products giving rise to the claim.** ## Section 11 - Final Provisions 1. The Agreement may be amended or supplemented only by a writing that refers to the Agreement (or parts thereof) and that is signed by both parties. 2. To the extent possible, if any provision of the Agreement is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from the Agreement without affecting the enforceability of the remaining terms and conditions and replaced by a provision which best corresponds to the intention and economic purpose of the invalid provision. 3. No term or condition of the Agreement will be waived and no failure to comply consented to unless expressly agreed to by EOX. 4. The failure or delay by a party to require performance of any provision of the Agreement does not constitute a waiver. All waivers must be in writing and signed by the party granting the waiver. The waiver by a party of any of its rights or remedies in a particular instance will not be construed as a waiver of the same or a different right or remedy in a subsequent instance. 5. Neither the Agreement nor any rights or obligations thereunder shall be transferred, assigned, delegated, pledged or otherwise encumbered by the Customer without the prior consent of EOX. 6. For the avoidance of any doubt, the provisions of these Terms survive the termination or expiry of the Agreement and they shall also extend to future transactions between the contracting parties even if not referred to again in case of future conclusion of contracts. 7. The Agreement shall be governed by and construed in accordance with the laws of Austria, excluding its international private law provisions governing conflicts and the UN Convention on Contracts for the International Sale of Goods. All disputes in connection with any matters arising out of the Agreement shall be finally decided by the competent courts of the Inner City of Vienna, Austria. EOX remains free to sue the Licensee at any other competent court worldwide. --- --- url: /_team/triebnigg.md description: >- Four decades of dedication to IT and Earth observation service concepts and implementations, over a rainbow of affiliations in academia, public applied research, the international institutions ESA and EC; now promoter of finest company in geospatial industry. --- --- --- url: /_team/zz_guest_moldenhauerh.md --- --- --- url: /_team/herglotzh.md description: >- IT project management and Green Finance agenda @eox. Background in business administration and software development. Into music and cooking. --- --- --- url: /2022/02/how-eo-expertise-powers-agri-monitoring-decisions-part-1.md --- *This blogpost is the first part of a multi-post series.* ## Introduction The European common agriculture and rural development policy is ambitious to implement a performance based and compliance-driven delivery model of area based payments. Strong emphasis is put on the use of earth observation (EO) and of recently available new technologies, such as big data-GIS, IoT solutions, cloud services or artificial intelligence. Such new EO-data analysis technological approaches had increased the meaning and use of open value chains, data sharing, and multi annual monitoring. The fundamental paradigm shifts of checks by monitoring - meaning the control on agricultural parcel level the area based payments - and of Area Monitoring System (AMS) has been recently codified in EU regulations. AMS is defined as a “regular and systematic observation, tracking and assessment of agricultural activities and practices” by satellite data, primarily Copernicus Sentinels. Thus, it is now clear that Paying Agencies and the agricultural administration is facing the challenge not only as a technological step forward, but this paradigm shift will enforce redesigning some fundamental elements of the existing robust Integrated Administration and Control Systems (IACS), while continuously functioning should still be ensured. The EO-WIDGET project aimed at developing a robust technological chain linking EO data processed in cloud environments to a web interface that delivers the data and the result towards the end-users. Agri service platform integrates mini-applications providing visualization access to monitoring products and related quality assessments, while at the same time supporting expert judgment, and user activities in GIS environment. One of the key point is that the end-user platform presents real-time processed EO signals, while the back end processing can be distributed among several cloud locations and processing chains. The Agri service platform is designed with full interoperability in mind, allowing * to consume data products upstream from service providers of choice and * to serve downstream APIs consumed by WebGUI applications like the AgriApp or by data-exchange logic to integrate with the PAs environment. Monitoring products from multiple sources can be sustained in the platform and analysed there for conformance and quality. A monitoring product is the combination of EO-signals, vegetation status indexes, spatial indicators and markers derived for predefined group of declared parcels. The applied decision parameters are combined along the rules defined by the Paying Agencies. The app is also willing to serve to “look behind the algorithm”, while it can become the user interface of fine tuning or setting the acceptance limits of an automatized image processing. ## Parcel Explorer ### Main Interface On the initial application visit, the user is greeted with a dashboard-like interface consisting of multiple interactive elements: Welcome to the Parcel Explorer! The Parcel Explorer is the main tool for visually inspecting compliance (or non-compliance) of various parameters and markers in the scope of CbM. Current supported use cases are **Cultivated Crop Type** (`Crop Type Conformance` and `Declared Groups`), **Grassland Mowing** (`Mowing Events` and `Mowing Events Conformance`), **Harvest** (`Harvest Detection Conformance`) and **Winter Catch Crops** (`Winter Catch Crops Conformance`). Each of these use cases can be defined and configured individually in terms of dashboard layout - map and chart elements, color schemes, visualized data, classification/categorisation of the data (etc) -, to fulfil the needs of the application, data availability and user screen resolution. The basic unit of evaluation is the agricultural parcel and the incomming vector and raster databases are processed and published on-the-fly via cloud environments. For further technological details see Part 2. ### Parcel Map The heart of the Parcel Explorer is the map element, which allows the user to browse and visually inspect agricultural parcel data, glimpse into parcel metadata (mouseover on a single parcel), switch between data product visualizations (e.g. `Crop Type Conformance` and `Declared Groups`), show/hide map layers, and draw reference areas (enabling a comparison of statistical parameters of a selected parcel and the average value of the same crop in its user defined surroundings and further similar functionalities). An overview map helps to analyse the bigger picture. ### Parcel Chart The chart element offers insights into the changing of EO signals or markers over time. Individual datasets (NDVI, coherence and more) can be shown / hidden, and the reference signals can also be visualized. Time limitations and compliancy/non-compliancy windows of markers can also be visualized on the chart. Selecting a data point in the chart also sets the date for the whole Parcel Explorer interface, synchronizing the date picker and image grid (described below) with the currently selected date. ### Parcel Image + Image Grid The image grid and single image view are tools that enable a direct look at the satellite data used for product generation. The user can display an overview of imagery for a selected parcel with weekly or daily granularity, and can browse through the selected parcel on individual images one by one. Visualization of the images and processed EO-signals might vary and can be switched by the user: pre-defined color composites; continuous and classified scaling of spectral indexes or cluster maps or also cloud cover information. ## Quality Assessment Tool Besides the parcel explorer, the application also includes some data-science-driven app views. These provide an overview over all algorithm results that are applicable for each use-case. Where prepared by the paying agency, also compliance rules for the whole territory are evaluated. In the context of crop type classification a confusion matrix, an omission/commission chart and an f1-score chart is displayed. All of these are configurable inside the app. Settings like the exclusion of small parcels and training parcels, and if the results are weighted by parcel area, are defined by the user. Also available is an interactive report for the grassland mowing use-case. Two charts are available, each in two different modes. The chart types are an overview bar chart and a mowing progress chart by crop type. Inside the app the user can switch between a view for the number of mowing events or a view for mowing event compliance values. The harvest use-case also features two charts. One of them is an overview bar chart similar to the one in the grassland mowing report. The other one is a chronological sequence chart that displays the harvest events by crop type and calendar week. The harvest events can be filtered by confidence values (e.g. all harvest events with a confidence greater than 85%). ## Expert Judgement & ToDo Workflow In the Reporting View, the user can search and filter the entirety of the parcel database and find parcels of interest for the current task. It is then possible to export group of parcels to analyse further in other parts of the application. For example, it is possible to filter parcels also by any quality description value, practically to list parcels contributes to an element of the confusion matrix. In the ToDo element, the user can click through the previously selected parcels one by one to do photo interpretation (CAPI) in relation to markers and compliance decisison. Each parcel can have several status assigned storing the metainformation of the expert validation. ## Internationalization As the aim is to make the AgriApp accessible to a wide professional audience from several member states, a continuous translation has been integrated into the workflow. If interested, paying agency employees can even review and improve translations on their own. Translation efforts even go so far as to provide database columns in local language. Since the AgriApp is data source agnostic, it does not know about the database columns upfront. Via a translations endpoint it is guaranteed that even those database columns are understood by everyone. ## Agri Blogpost Series This blogpost is part of a multi-post series. In [the next blogpost](/2022/05/how-eo-expertise-powers-agri-monitoring-decisions-part-2/), the services and APIs powering the AgriApp will be covered, plus a description on how to get on board and reap benefits for your workflow! --- --- url: /2022/05/how-eo-expertise-powers-agri-monitoring-decisions-part-2.md --- *This blog post is the second part of a multi-post series.* ## Introduction The [first part of this blog post series](/2022/02/how-eo-expertise-powers-agri-monitoring-decisions-part-1/) introduced the Area Monitoring System (AMS) and the EO-WIDGET project. Furthermore, it covered the user interface components developed in the EO-WIDGET context. This second part will go into detail about the API (Application Programming Interface) services powering the user interface (UI). While [vector tiles are at the center](/2021/11/foss4g-2021-vector-tiles-that-would-be-great/) of data exchange between frontend and backend services of the AgriApp, several RESTful APIs, WMS endpoints and other interfaces are provided, which a paying ageny can plug into their own system. ## Open Value Chain The EO-WIDGET project envisions an open value chain. This means that the system architecture can interface with any set of data and/or algorithm providers. The system is also agnostic to the API consumer: While the AgriApp consumes the API via a customizable user interface (presented in [part 1 of this blog post series](/2022/02/how-eo-expertise-powers-agri-monitoring-decisions-part-1/)), the **modular API components** can also be readily integrated into a paying agency's in-house tools. In more technical terms, all satellite products of Level 2 (atmospherically corrected imagery) through Level 4 (derived variables, i.e. markers) are offered, complimented by compliance results following the Checks by Monitoring (CbM) / Area Monitoring System (AMS) decision processes. ### Satellite Imagery Catalog & WMS The AgriApp is backed by powerful satellite imagery cataloging and viewing technology. This enables the operator to visualize the phenological development of a parcel over time using multiple sensors & processed products (i.e. Sentinel-1, Sentinel-2, Planet Fusion, different vegetation indexes, etc.). Visualizations are also available for direct integration into internal paying agency tools. Within the AgriApp, a calender widget allows interactive selection of single image chips and an image grid provides an overview of crop development across multiple months. The latter takes advantage of cloud cover information and surfaces the best image for a week accordingly. The structure and the content of the image chips are optimised to support validation of different thematic products. Additionally, satellite images can be viewed as true and false color composites for easier insights by Computer Aided Photo Interpretation (CAPI) experts. Different vegetation indexes – such as NDVI – can also be presented. ### Signals In the context of CbM *signals* are time series of spatially aggregated satellite data, making them a Level-3 satellite product. Examples are vegetation index (NDVI) or SAR backscatter curves. Such signals are both used to visually inspect the development of vegetation over an agricultural parcel and also serve as input to machine learning algorithms. The most-common sources for signals are the Sentinel-1 and Sentinel-2 imagery. For parcels too small to be monitored with the Sentinels' spatial resolution, however, signals are derived from Planet Fusion or similar data products. The signals for all agricultural parcels can be interactively explored in the chart widget of the AgriApp. Additionally, a web endpoint of JSON format is exposed to paying agencies, which can be used to integrate signals into internal paying agency tools. ### Markers Derived from a signal, markers indicate a kind of farming practice or vegetation change on a parcel at a given time during the agricultural season. For example, machine learning algorithms are able to detect harvest or mowing events by searching for signal drops in the vegetation index time series. The EO-WIDGET architecture is not constrained to any algorithm provider—several kinds of algorithms have already been integrated successfully into our agri workspace. ### Compliance Results Our agri workspace is also the place where compliance rules as provided by the paying agency are evaluated. The paying agency can define requirements a parcel has to meet in compliance with a specific payment scheme. For example, one crop type must be mowed at least once and at the most twice. If a specific parcel's markers show compliance with this rule, the parcel is annotated with a "compliant" flag. If not, "non-compliant" is annotated. In some cases the algorithm cannot take a confident decision based on the input data presented to it. Such parcels are flagged as "inconclusive" and delegated for decision by a human operator. ### Business Decisions A batch of "inconclusive" parcels can be assigned to an operator. This batch will be added to an interactive to-do list. For each parcel in the list the operator can make a decision based on the signals and markers and auxiliary information such as satellite imagery, on-the-spot-checks commentary, etc. These human decisions are also stored in the agri database and are prepared to be imported into the paying agency's system via defined interfaces, together with any other information gathered in the AMS process. ## Conclusion EOX strives to build an open value chain for the Area Monitoring System. Therefore our agri workspace can interface with a diverse set of input data and algorithm providers. All processing results such as signals and markers, as well as compliance values and business decisions are readily available to the paying agency in the form of on-demand access and batch data transfers. ## Agri Blogpost Series This blogpost is part of a multi-post series. Stay tuned for [part 3](/2022/07/how-eo-expertise-powers-agri-monitoring-decisions-part-3/) of this blog post series, where you will learn about the data pipelines that interface with the paying agency and algorithm providers. You will also find out how to get on board and benefit from the open-value-chain AMS. --- --- url: /2022/07/how-eo-expertise-powers-agri-monitoring-decisions-part-3.md --- *This blog post is the third part of a multi-post series.* ## Introduction The [first](/2022/02/how-eo-expertise-powers-agri-monitoring-decisions-part-1) and [second](/2022/05/how-eo-expertise-powers-agri-monitoring-decisions-part-2) parts of this blog post series introduced the Area Monitoring System (AMS) and the EO-WIDGET project. Furthermore, it covered user interface components, APIs (Application Programming Interface) and backend services developed in the EO-WIDGET context. This third part describes the data pipelines integrating with paying agency systems, satellite data archives and monitoring products providers. It also gives details on how to get on board and benefit from our free demo service offer. ## Data Pipelines Our agri workspace and user interface are no islands. They integrate in various ways with inbound and outbound data pipelines. So-called "glue code" is set up to make the workspace inter-operable with any external data interface. ### Paying Agency The fundamental data source of the AMS are the Geo-Spatial Aid Applications (GSAA), in which farmers apply for area-based direct payments by declaring their parcels. These GSAA are provided by a paying agency to their individual secured data landing zone and imported into the agri workspace on a fixed schedule (e.g. monthly). The same may apply to other datasets like in-situ data or geo-tagged photos. While this pipeline code to transfer the data from the landing zone to the agri workspace is specific per paying agency, it leverages the agri workspace API and reusable code blocks for fast onboarding and safe code evolution. The data pipeline itself is integrated into our operations stack for monitoring, alerting and problem mitigation. When the monitoring results are available in the agri workspace, another data pipeline is invoked for delivery to the paying agency in the desired data formats. This pipeline code is specific per paying agency and again exclusively leverages the agri workspace API to include any attributes that are useful to the paying agency, as well as manual annotations paying agency staff has made during the expert judgment routines (see [first blog post](https://eox.at/2022/02/how-eo-expertise-powers-agri-monitoring-decisions-part-1/#expert-judgement-todo-workflow)). Level of integration into the Integrated Administration and Control System (IACS) might vary, determining the data exchange frequency, the complexity of marker-based traffic light decisions per lane (group of parcels in the same EO-decision flow) and the share of information stored/archived in the EO-WIDGET Agri Workspace. ### Satellite Data Viewing freely available products (Sentinel 1 & 2) mark the most basic case of our satellite data integrations -- they are simply registered by our View Server and served via WMS. For commercial satellite products, like Planet Fusion, a gate-keeper functionality is provided. This means that only those signal values and spatial extents are used and shown to the end user that have been activated by the paying agency. The advantage is that the paying agency does not have to buy entire satellite image scenes, but only pays for the area corresponding to "activated" parcels (for example all small parcels or a group of inclonclusive parcels). Phrasing it differently, Application Ready Data is distributed with a use-case-specific business model. In general, monitoring algorithms need time series of numerical spectral values as input data. We continuously pre-process all available raw satellite imagery to speed up the training and evaluation of the monitoring algorithms and the response times in the user interface's chart widget. Several pre-processing methods can be applied, the most common being: * cloud masking, * harmonization along the time axis, * generation of different spectral indexes. Often the time series are aggregated over a parcel (object-based algorithms), but we can also make available all data points (pixel-based algorithms). Different pixel-based sampling methods can be applied before training of machine learning algorithms. ### Monitoring Algorithms & Products Providers Since we envision an open value chain (see [second blog post](https://eox.at/2022/05/how-eo-expertise-powers-agri-monitoring-decisions-part-2/#open-value-chain)), our platform is open to any kind of monitoring algorithm and can exchange data with various product providers. If, for example, a paying agency or some external provider has developed a very well performing algorithm on a small area of interest (AOI), we can scale it up country-wide and deliver the results for each parcel declaration. The data pipeline concept introduced above is also leveraged here, ensuring that data may only enter or leave the agri workspace via a well-defined interface as provided through the agri workspace API. Other scenarios involve potent image processing partners, who want to benefit from our interactive [user interface](https://eox.at/2022/02/how-eo-expertise-powers-agri-monitoring-decisions-part-1/#parcel-explorer) and [validation dashboards](https://eox.at/2022/02/how-eo-expertise-powers-agri-monitoring-decisions-part-1/#quality-assessment-tool). We are excited about industry collaborations, which bring maximum value to our paying agency customers. ## Free Demo Service A free demo setup of our AMS solution is offered to interested paying agencies. All you need to provide are GSAA data (geometries & crop types) and you get your own cloud workspace and an instance of the AgriApp web interface configured with your data. Additionally, affordable demo licenses for a defined AOI are available for processing of signals, monitoring products and compliance rules. All of these will be viewable in the AgriApp web interface and are available for internal inspection via APIs. A live example of our services can be found at . Contact [Gerhard.Triebnig@eox.at](mailto:Gerhard.Triebnig@eox.at?subject=Requesting%20a%20free%20Agri%20Workspace%20Demo) for further information! ## Conclusion Our agri workspace offers flexible data pipelines towards the three main stakeholders of the AMS: A paying agency, satellite data providers and monitoring algorithm/product providers. There is a free demo service available for you to get on board right away and to try out the agri workspace with your own data. --- --- url: /_team/zz_guest_raubergj.md --- --- --- url: /_team/ungarj.md description: >- Map maker, data shoveler and Open Source GIS enthusiast from Salzburg, Austria. Worked as a researcher and web developer before EOX. Also enjoys cooking, football and good rock music. --- --- --- url: /_team/prennerj.md description: >- Working as Software Engineer for over 10 years for various companies and operating different stacks from bare-metal over Cloud environments to Kubernetes. Maker of the famous Unreal Engine geodata extension "Landscaping". --- --- --- url: /_team/grubek.md description: >- Pianist, Free Software Advocate. A proudly lazy administrator who prefers automation and building better services over fixing broken systems. --- --- --- url: /_team/lehotskak.md description: >- A geographer loving maps since kindergarten. Likes languages, knitting, swimming, nordic walking, board games and amusement parks. --- --- --- url: /_team/dolezall.md description: >- Open Source GIS enthusiast. Feels great when a hidden bug gets tracked and fixed. Loves escaping cities on a bike, fantasy books and board/video games. PyLadies Vienna co-organizer. --- --- --- url: /_team/zz_guest_kadlecl.md --- --- --- url: /expertise/managed-cloud-workspace.md --- ##### EOxHub [EOxHub](https://hub.eox.at/) is a unified cloud workspace designed where Earth Observation (EO) work takes place. Built for both academic and industrial research teams, it provides the necessary infrastructure to bridge the gap between massive raw satellite data archives and the exploitation of these sources for analysis and decision-making. The platform provides managed JupyterLab instances featuring specific tooling and customizable environments. These setups are ideal for accelerating science projects, running community workshops, and supporting various commercial and non-profit application domains. [Visit EOxHub](https://hub.eox.at/){.button} If you are interested to offer an EO exploitation platform to your community please download the product brochure:\ [Download EOxHub brochure ⤓](/content/EOX_Brochure_-_EOxHub_web.pdf) Check out general purpose subscriptions for managed cloud workspaces:\ [Register for a free trial ↗](https://workspace.freetrial.hub-otc-sc.eox.at/) ###### Supported Workflows * **Algorithm Development:** Build and test EO algorithms in a scalable, cloud-native environment. * **Result Generation:** Process high-volume workloads efficiently. * **Algorithm as a Service:** Deploy and share your algorithms with broader communities. * **Insight Publishing:** Visualize and communicate your findings seamlessly. Whether you are hosting university courses, organizing community workshops during conferences, or driving commercial and non-profit science projects, EOxHub provides the established environment you need. It is already the platform of choice for over 15 European Space Agency (ESA) projects, including EarthCODE, VirES, and PolarTEP. [Explore EOxHub Documentation](https://documentation.hub.eox.at/){.button} ###### Who is Using EOxHub? There are various users that already make use of the EOxHub technology among which are: * **ESA-supported projects**: * **[Rapid Action for Citizens with Earth Observation (RACE)](https://race.esa.int)** * **[EO Dashboard](https://eodashboard.org)** * **[ESA Climate Change Initiative](https://climate.esa.int/en/)**: The ESA Climate Office uses satellite observations to develop global, long-term data records that evidence our changing climate and inform international action. * **[Euro Data Cube](https://eurodatacube.com/)** * **Green Transition Information Factories (GTIF)**: * **[GTIF Austria](https://gtif-austria.info/)** * **[Baltic GTIF](https://gtif-baltic.info/)** * **[GTIF UK, Ireland, France](https://gtif-ukif.github.io/gtif-ukif-client/)** * **[Cerulean Information Factory](https://cif.eox.at/)** * **VirES VREs**: The Virtual Workspace Environments offer a "batteries included" development environment with direct and powerful access to Earth Explorer missions (**[Swarm](https://vre.vires.services)** and **[Aeolus](https://vre.aeolus.services)**). * **[EOxCloudless](https://cloudless.eox.at/)**: Provides cloud-free satellite data products thanks to large scale processing on the EOxHub platform. * **[NCS](https://ncs.eox.at/)**: The Natural Capital Solutions platform bridges satellite-driven ecological insights and agricultural practices with financial markets to quantify and report ecosystem services and land use impacts. * **[Polar TEP](https://polartep.polarview.org/)**: The Polar Thematic Exploitation Platform provides a complete working environment where users can access algorithms and data remotely.Destination Earth Sea Ice Decision Enhancement service providing comprehensive sea ice and related information for policy and operational decision makers in the Baltic Sea, European Arctic Ocean, and other polar regions. --- --- url: /2022/06/mhub2022.md --- In our [previous post](/2022/05/sentinel-2-cloudless-2021/) we briefly explained how we improved our processing pipeline for the new [Sentinel-2 cloudless 2021](https://s2maps.eu/) release. In this post we will go deeper into the details of what changed and where we are technologically. Parts of these developments are results from the LOOSE project funded by the [European Space Agency](https://www.esa.int/). ## Mapchete Our core processing tool is the in-house developed open source package [mapchete](https://github.com/ungarj/mapchete). It allows applying a user-defined piece of Python code to geospatial data. This data can either be in the form of rasters or feature datasets. Mapchete uses the approach of tiled slippy maps when processing by splitting up the geographical area into zoom levels and tiles. This allows for processing large, up to global, areas because these tiles or data chunks can be dimensioned in a way that a single process can handle them in memory. Because a process area is nothing else than a (potentially large) set of tiles, each of which can be processed by one worker, this processing job can easily be paralellized over a number of workers. These workers can either run as separate threads or processes on a single machine or on a cluster, but more on that later. Mapchete is designed to be extensible via plugins. Per default it is able to read and write the most common raster and vector formats such as GeoTIFF, COG, GeoJSON, GeoPackage, etc. but also more complex inputs or outputs can be implemented such as a satellite data input driver for a Sentinel-2 archive. For our [EOxCloudless](https://cloudless.eox.at) product line we developed such an input driver for Sentinel-1 and Sentinel-2 data. Mapchete Hub contains the core package for all processing capabilities but enhances it by adding a REST interface, a database to hold job metadata and allows for distributed, asynchronous job execution. Every processing job too large for a single Laptop can be sent to mapchete Hub where it will be processed and tracked. ## Previous architecture Processing a global satellite mosaic with 10 meters ground resolution would take too long to run on a single machine, but thanks to the cloud compute resources are easily accessible. Our initial solution for the issue of distributing the work load on multiple workers was to split up the entire world into squares called "process zones" and assign each worker one zone. Mapchete would then take over, split up the process zone into tiles and process the tiles in parallel on the worker. The distribution part didn't require many additions to the existing software. Mapchete was the core package taking the process configuration and just running as if it would run on a local machine. Around that [Celery](https://docs.celeryq.dev/en/stable/) and [RabbitMQ](https://www.rabbitmq.com/) were used to queue and distribute the jobs. For job submission a REST API was built using [Flask](https://flask.palletsprojects.com/en/2.1.x/) and as permanent metadata storage a [MongoDB](https://www.mongodb.com/) instance was deployed. The processing campaigns for EOxCloudless were conducted yearly since 2017 and used different Sentinel-2 archives in different stages. In all cases data archives from AWS (Amazon Web Services) were used. In the first years the [JPEG2000 archive](https://registry.opendata.aws/sentinel-2/) managed by [Sinergise](https://www.sinergise.com/) was used. Because the JPEG2000 format is not optimized for being read directly from an HTTP or S3 endpoint, a caching step was introduced downloading all Sentinel-2 JPEG2000 files onto the worker and converting them into GeoTIFFs. In the recent runs the [COG archive](https://registry.opendata.aws/sentinel-2-l2a-cogs/) managed by [Element 84](https://www.element84.com/) could be used which omitted the necessity of caching and converting the image data. This setup worked reasonably well since the first EOxCloudless campaign conducted in 2016/2017 but from an operators view it was at times tedious to handle and in the end lacked some key features. First and foremost we didn't have a proper deployment approach. The server, database and broker containers were all run on on-demand instances on AWS but we had to manually initialize the worker instances and terminate them once the jobs were finished. Apart from that we faced eventually unresolvable issues when e.g. cancelling/deleting a queued job or in case a worker became unresponsive. Furthermore we wanted to move all of the deployment responsibilities over to [EOxHub](https://hub.eox.at/). ## New architecture Two main points of improvement were identified: 1. Job tasks should be distributed on workers, not full jobs. 2. Workers should automatically start before and terminate after jobs. ### All the power to dask [`dask`](https://www.dask.org/) is an amazing project which got very popular in the data science and geospatial fields recently. It is mainly famous for extending Numpy, pandas and scikit-learn to run on a cluster but dask also replicates the python core package `concurrent.futures` interface to schedule arbitrary tasks. Dask provided us with some attractive additional features: * A dask cluster does not require a messaging component like RabbitMQ. This is being handled by the dask scheduler. * The dask scheduler tries to optimize which workers execute which task depending on the task graph. * A task graph can be defined, adding dependencies between tasks. * [`dask-kubernetes`](https://kubernetes.dask.org/en/latest/) is a package built to integrate dask cluster management within Kubernetes. * The dask API is simlar to `concurrent.futures`, thus could be integrated into the [`mapchete`](https://github.com/ungarj/mapchete) core package without much effort. * Once dask is integrated, the same client (`mapchete`) code can handle both local and distributed processing. Extending the internal calls to `concurrent.futures` with a [`dask`](https://www.dask.org/) object also allowed for distributing the job tasks over the cluster rather than sending jobs to workers individually like we did when using Celery. For the operator triggering the processing this makes it much more convenient because before we had to manually splilt up large regions (e.g. continents) into smaller jobs ("process zones"). Now, the operator can just submit one large job which then gets split up and distributed over a larger number of workers. Previously all of the required steps when creating a cloudless mosaic had to be processed one after the other leading to unused workers at the end of each step. Furthermore it required caching task results so that a follow-up task could pick up from there and process the next step. We did this using an object storage which produced costs (requests and storage). [`dask`](https://www.dask.org/) enables us to build task graphs where task results can be directly shared with the follow-up task. The tasks of the processing steps can now start any time when the dependent tasks finished successfully; thus the workers can always be busy and less resouces are wasted. ### EOxHub and Kubernetes The dask ecosystem contains packages called [`dask-kubernetes`](https://kubernetes.dask.org/en/latest/) and [`dask-gateway`](https://gateway.dask.org/) which allow integration into a Kubernetes cluster. This means custom dask clusters can be requested individually for each job and Kubernetes handles all the cloud resources required. Apart from autoscaling it was important to finally make the deployment more solid. Automated testing, tagged releases and building docker images via CI/CD had been already implemented in previous years so handing over the deployment to [EOxHub](https://hub.eox.at/) was straightforward. Deploying on EOxHub also means we can fall back to a prior version if needed by just editing the configuration. Everything else is handled by Kubernetes. ## Summary Our mapchete Hub system has grown organically and has evolved quite a bit since our first Sentinel-2 processing campaigns in 2016 and 2017. We came a long way from manually handling AWS spot instances via installation scripts and without docker images (!) to a fully kubernetes-managed, autoscaling cluster. There are still challenges ahead, especially towards optimizing resource usage (i.e. reducing costs) and error handling (i.e. reducing detective work). The way the system is designed allows us to easily plug in other processes and data backends. Apart from creating cloudless mosaics we are also processing Sentinel-1 data as well as time series preparation tasks for creating [agricultural signals](/2022/05/how-eo-expertise-powers-agri-monitoring-decisions-part-2/#signals) on a large scale. We aim to achieve a transparent computing environment resilient to errors and efficient workload distribution for us at EOX and also for you as our customer. --- --- url: /_team/pacesm.md description: >- Passionate about mathematical modelling, numerical methods, analytics, high-performance optimization and programming in imaging, visualization and geospatial software endeavours. --- --- --- url: /_team/zz_guest_mabeyam.md --- --- --- url: /_team/ziselsbergerm.md description: >- Likes to automate things, and spend her free time outdoors, in a coffee/tea shop or at the local tech meetup. --- --- --- url: /_team/zz_guest_riedem.md --- --- --- url: /_team/abdallam.md description: >- Geomatics engineer, passionate about geospatial and open source software, loves to learn about different cultures, cooking and watching soccer. --- --- --- url: /2016/05/nautical-eoxploration.md --- The EOX team went on an inspiring sailing trip to southern Croatia. Enjoy some impressions we don't want to hide from our dear readers. The next post will be more technical again, promised. Thanks to everybody for the great days and special thanks to Viktor our skipper. --- --- url: /2016/12/new-vires-service-version-out-for-xmas.md --- We released **[VirES for Swarm 1.2](https://vires.services)** on 21 December 2016, bringing to users improved service utilization comfort, additional visualization elements, and better control over data downloading. This evolution of VirES is  responding to new requirement inputs from leading experts in geomagnetic field research to ensure best possible user experience and accurate scientific validation of the service. **The version 1.2 update** brings additional date navigation functions and now supports more precise time filtering for data downloading. An even richer set of color scales and additional geographical overlays (geomagnetic and geographic graticule, geographic borders) can now be included in VirES' attractive data visualizations and publication-ready graphics. Also new is the "personalized view" function through which the service keeps status and preferred configurations from the previous user sessions. Technical details of version 1.2 are described in the [service changelog](https://vires.services/accounts/changelog/). The virtual research service “[VirES for Swarm](http://vires.services)” adds **discovery and visual analytics** capabilities to the European Space Agency’s online data access services established for the [Swarm geomagnetic satellite mission constellation](https://earth.esa.int/web/guest/missions/esa-operational-eo-missions/swarm). VirES provides a highly interactive data manipulation and retrieval web interface for the official Swarm product archive and for a number of ancillary data sets. It includes multi-dimensional geographical visualization, interactive plotting, and on-demand processing tools for studying Earth magnetic models and their time variations for comparing them to the Swarm satellite measurements at given global context of space weather. Subsets of Swarm data selected by versatile filtering methods can be downloaded in different encoding formats. The data downloaded can be combined to fit various use cases. VirES is also an environment for preparation of publication-ready graphics and charts through an intuitive and powerful yet customizable interface. An embedded online tutorial introduces the features of the VirES service to first-coming users. A [video tutorial](https://www.youtube.com/watch?v=7IuChwuye8c) is also available: --- --- url: /_team/baudoinn.md description: >- A black-belt in Ansible with a katana-sharp tongue. Forged in a Japanese dōjō, his soul channels disciplined focus into dark automations. When not summoning servers, he is warding off mischievous spells from the shadows realm. --- --- --- url: /_team/jankovicn.md description: >- Geospatial engineer and developer with a background in GIS. Likes science, tinkering and figuring out how things work, open-source software, gaming, traveling --- --- --- url: /2024/02/open-science-catalog.md --- Just like the European Space Agency (ESA), we advocate for and actively support **Open Science**, as we believe in the significance of collaborative efforts in advancing scientific knowledge and addressing global challenges. We acknowledge the transformative power of **Open Science** in driving interdisciplinary collaboration, facilitating reproducibility, and ultimately contributing to a more sustainable and resilient future for our world. We are proudly supporting ESA in their Open Science Catalog (OSC) initiative through frontend development. The OSC is one part of a larger initiative by ESA called EarthCODE, which aims to provide an Earth Science Collaborative Open Development Environment. The OSC aims to advance science initiatives and projects by providing a platform to store catalog records for science themes, variables, projects, products, and contributing (EO) missions. This system allows users an easy way to browse, search, and access metadata information associated with these records. This blog post provides some technical details about the OSC. The frontend application of the Open Science Catalog can be accessed from its [public URL](https://opensciencedata.esa.int/). This application will communicate with both the resource catalog, powered by PyCSW, allowing for dynamic searches, as well as a [static catalog hosted on GitHub pages](https://eoepca.github.io/open-science-catalog-metadata/). The frontend is built using the popular Vue framework and uses various elements of the [EOxElements library](https://github.com/EOX-A/EOxElements). Additionally, it incorporates the [STAC browser](https://github.com/radiantearth/stac-browser), a fully fledged browser application to browse the contents of a STAC catalog or API. The contents of the catalog is actually managed by a public GitHub repository, allowing for a full history of changes to the contents of records. Additionally changes to the repository are done using the mechanism of a Pull Request. Users can suggest changes, which can be revised by the data administrators of the Open Science Catalog, and run through rigorous checks using GitHub actions, which perform checks on the validity and the integrity of the changes. When a Pull Request is accepted, the changes are incorporated in the main branch of the catalog, triggering a new release of the static catalog, which is subsequently merged into the resource catalog. The Open Science Catalogs data model is expressed in Spatio-Temporal Asset Catalog objects. This allows third party software to also access and digest the contents of the catalog. This is why the Open Science Catalog is also listed in the popular [STAC Index](https://stacindex.org/catalogs/osc). As STAC is usually used for upstream earth observation data and not for derived science products, an [Open Science Catalog specific extension](https://github.com/stac-extensions/osc) was created to add the specific metadata required to describe the science data records. Additional community extensions are used for various other metadata, such as contacts, or the scientific extension. The Open Science Catalog is a versatile and robust tool, built on open standards and open source software in order to facilitate **Open Science**. Stay tuned for further updates about our **Open Science** activities, like EarthCODE. --- --- url: /_team/sevcikp.md description: >- Cartographer from the Czech Republic with experience of governing GIS data from remote sensing sources. Likes climbing, hiking, music and gaming. --- --- --- url: /privacy-notice.md --- (version as of November 17th 2025) Download pdf version [here](/content/privacy_notice.pdf) to save or print this privacy notice. ## Section 1 - Introduction a. This Privacy Notice (this "Notice") explains which personal data are collected when you visit one of our websites under the `eox.at` domain like , https://cloudless.eox.at]\(https://cloudless.eox.at), or [https://eox.at](/) itself (our "Website" or "Websites") and how this data is processed by EOX IT Services GmbH as the data controller ("EOX" or "we"). b. This Notice is addressed to any visitor of our Websites ("data subject" or "you"). c. We process your personal data in accordance with the EU General Data Protection Regulation (GDPR) and applicable national data protection laws. Unless otherwise defined in this Notice, the terms used herein shall have the same meaning as defined in the GDPR. ## Section 2 - What Personal Data We Collect and How We Use It a. When you use our Websites we may process those personal data which you voluntarily provide to us (e.g. by means of a sign-up or registration form, our product order forms, or the Chat Bot). b. However, you can also visit our Websites without actively providing us with information about you. In this case we collect certain data that your browser transmits to our Website server (i.e. log files) as well as data that we collect via the use of cookies and similar technologies. c. The following explanations shall serve to inform you about the different ways we may collect personal data about you on our Websites and for what lawful purposes we may use them. d. **Personal data you actively disclose to us** 1. When you actively communicate with us via our Websites, we process those personal data that you voluntarily provide to us. In particular, this relates to our following services: A. **EOxHub Workspaces**: When you choose to use one of our online services via our Websites we ask you to provide certain mandatory information about you to sign-up or register for the online service (e.g. name, username, password, and email). Unless communicated otherwise to you, we process these data exclusively for the purpose of providing the online service (Legal basis: Art 6(1) lit b GDPR - *performance of contract*). Please note that integrated external tools may also have connections to other external service providers and could forward data to them. This is beyond our control and cannot be monitored or prevented by us (e.g., JupyterLab uses a link to Google Fonts). B. **B2B Web Shop**: When you choose to purchase one of our products (e.g. online service subscription or data product) via our Websites we ask you to provide certain mandatory information about you and/or your company to complete your order (e.g. name, company, VAT number, address, email, order details and payment information such as credit card or invoice details). Unless communicated otherwise to you, we process these data exclusively for the purpose of processing your order, facilitating your payment and delivering our product (Legal basis: Art 6(1) lit b GDPR - *performance of contract*). C. **Email contacts and Chat Bot**: You may decide to get in touch with us via one of the email contacts or the chatbot function ("Chat Bot") provided on our Websites. A chatbot is an artificial intelligence (AI) program that simulates interactive human conversation by using key pre-calculated user phrases and text-based signals. When you choose to contact us via our email contacts or the Chat Bot the personal data you provide to us (e.g. name, email address, and your correspondence data) will be processed for the purpose of creating a contact file, answering your questions, fulfilling your requests or otherwise communicating with you (Legal basis: Art 6(1) lit b and f GDPR - *performance of (pre-)contractual duties and legitimate interest in adequate customer relationship management*). The Chat Bot is hosted by HubSpot, Inc., acting as a data processor on our behalf (see [Section 3](#section-3-to-whom-we-may-disclose-your-personal-data) below). D. **Marketing Emails**: If you subscribe to our marketing Email list via the Websites or otherwise provide us with your business contact details, we may send you publications, event invitations and/or news which may be of interest to you (Legal basis: Art 6(1) lit b and f GDPR - *performance of (pre-)contractual duties and legitimate interest in adequate customer relationship management*). 2. Some of the data we request in connection with our above services may be marked as mandatory fields. You are not required to provide these data. However, without providing this information we may not be able to process your request or provide our services. e. **Log files** 1. You may also visit our Websites without actively providing us with information about you. In this case we collect certain data that your browser transmits to our Website server (i.e. log files). 2. Our log files contain the following information: (i) date and time of retrieval of our Website, (ii) type, version and settings of your web-browser, (iii) your operating system and internet service provider, (iv) requested pages and files, (v) website used prior to visiting our Website as well as (vi) your IP-address. The IP address is a specific number assigned to your computer which enables your device to communicate in a network using the Internet Protocol (IP). IP addresses may qualify as personal data as they technically allow the identification of the user in certain circumstances. 3. The processing of these log files is necessary for us to maintain the functionality, stability and security of our Website. We may also process them for the purpose of forensic investigations in the case of a security incident or in order to generate user traffic statistics. Legal basis: Art 6(1) lit f GDPR - *legitimate interest in developing and maintaining the functionality, stability and security of our Website*. f. **Cookies** 1. In addition, this Websites use cookies. These are small text files that may be placed on your device while browsing our Websites which store certain information about you. Cookies cannot access, read, or modify other data stored on your device. When we refer to "cookies" we include other technologies with similar purposes, such as pixel tags. 2. We use two types of cookies on our Websites: A. **Necessary cookies**: Without necessary cookies the proper functioning of our Websites would not be possible or only to a limited extent. Necessary cookies are used to save your authentication state, information during the checkout process in our web shop and to properly process your provided payment information. In addition, necessary cookies are used to store your optional cookie consent (see below), session preferences and settings (e.g. for the Chat Bot). The use of necessary cookies on our Website is possible without your consent. However, you can deactivate cookies at any time by modifying your browser settings. Legal basis: Art 6(1) lit f GDPR (*legitimate interest*) B. **Optional cookies**: These types of cookies may be used to improve our Website, optimize your user experience or analyze user behavior. Optional cookies are used by our web analytics software Matomo (see [Section 2(g)](#matomo) below). Optional cookies are also placed by our data processor [HubSpot, Inc.](https://legal.hubspot.com/privacy-policy)[1](#hubspot) ("third party cookie") which tracks visitors using browser cookies. Every time you land on our Websites, HubSpot will check for an existing tracking cookie. If one does not exist, a cookie will be associated with you and will log every page visited moving forward. These third party cookies are used for the purpose of analyzing visitors' behavior as well as optimizing visitors' experience and marketing campaigns. Optional cookies may also be placed by external advertising companies or social media ("third party cookies"). Optional cookies will only be used upon your consent which you may provide by clicking "OK" on our Website's cookie banner. This consent can be withdrawn at any time with effect for the future. Legal basis: Art 6(1) a GDPR (*consent*) g. Web Analytics - Matomo 1. In case you have consented to the use of cookies by us, our Websites use the open source software Matomo, a web analytics software developed by InnoCraft Ltd., 150 Willis St, 6011 Wellington, New Zealand. Matomo uses cookies to help the Websites analyze how users use the site. 2. The information generated by the cookie about your use of our Websites (including your IP address and the URLs of the accessed pages) will be stored by EOX and will not be disclosed to third parties. Your IP addresses is made anonymous by truncating it. 3. We will use this information for the purpose of evaluating your use of our Websites, compiling reports on Websites activity and providing other services relating to Websites activity and internet usage. We will not associate your IP address with any other data held by us. 4. Our processing using the Matomo software is based on your consent which you may provide by clicking "OK" on our Websites' cookie banners (Art 6(1) lit a GDPR). This consent can be withdrawn at any time with effect for the future. You may also refuse the use of cookies by selecting the appropriate settings on your browser (see [Section 2(h)](#cookiescontrol) below). However, please note that if you do this, you may not be able to use the full functionality of our Websites. h. How to control and manage the use of cookies 1. By clicking on the "OK"-button in the Websites' cookie banners you agree to the use of optional cookies on our Websites. 2. By clicking on the "Not OK"-button in the Websites' cookie banners you refuse the use of optional cookies on our Websites. 3. Your consent or refusal can be withdrawn (for all or individual cookies) at any time with effect for the future by directing your request to or (for all cookies) by clicking the "Open Cookie Banner" button on our Websites and selecting "OK" (consent) or "Not OK" (refusal). 4. You may also refuse the use of cookies by selecting the appropriate settings on your browser or by deleting the cookies from the device and browser. Most browsers accept cookies automatically, but you can alter the settings of your browser to erase cookies or prevent automatic acceptance if you prefer. Generally you have the option to see what cookies have been placed and delete them individually, block third party cookies or cookies from particular sites, accept all cookies, to be notified when a cookie is issued or reject all cookies. Visit the 'options' or 'preferences' menu on your browser to change settings, and check the following links for more browser-specific information: Cookie settings in Microsoft Edge: Cookie settings in Firefox: Cookie settings in Chrome: Cookie settings in Safari: 5. You should be aware that any preferences will be lost, if you delete all cookies and many websites will not work properly or you will lose some functionality. We do not recommend turning cookies off when using our Website for these reasons. ## Section 3 - To Whom We May Disclose Your Personal Data a. For the above mentioned purposes we may share your personal data with the following recipients: 1. IT service providers who provide hosting, maintenance, and security services as well as certain functionalities for our Websites and/or internal business activities 2. dedicated servers and cloud infrastructure providers 3. payment service providers 4. accounting service providers 5. log collection and monitoring service providers 6. Where disclosure is required (i) by law or regulation or (ii) to establish, exercise, or defend legal claims, we may also disclose personal data to a competent authority, such as supervisory, regulatory or criminal authorities, courts of law, or other third parties who advise us in this context (e.g. lawyers or forensics experts). b. Some of these recipients may be located in countries outside the EU/EEA for which an adequate level of data protection has not yet been established by the EU Commission. In particular, this may include our IT service providers (e.g. [HubSpot, Inc.](https://legal.hubspot.com/privacy-policy)[1](#hubspot), [Stripe Payments Europe, Ltd.](https://stripe.com/privacy-center/legal)[2](#stripe), and [Slack Technologies, Inc.](https://slack.com/privacy-policy)[3](#slack)), who may transfer personal data to third countries, including the United States of America, in the context of providing data processing services to us. It should be noted that the level of data protection in such countries may not be the same as within the EU/EEA. Also, subject to local laws and regulations data may be accessible to local authorities or courts. c. However, where personal data is transferred to such third countries we ensure that your rights are protected in accordance with the GDPR. This includes the selection of recipients who are certified under recognized protection mechanisms pursuant to Art 45 GDPR (such as HubSpot, Stripe, and Slack who are certified under the EU-U.S. Privacy Shield Framework) and/or the conclusion of the EU Commission's standard contractual clauses for the transfer of personal data (Art 46(2) lit c GDPR). Further details on the implemented safeguards as well as copies of the respective agreements are available on request at . ## Section 4 - How Long We Keep Your Personal Data a. Log files (see [Section 2(e)](#logfiles) above) are generally kept for a period of less than a year. Beyond this time period log files will only be stored for the purpose of investigating irregularities or security incidents in our system. b. Cookies (see [Section 2(f)](#cookies) above) are usually valid for a short term (a day, a week or a month), though in some cases they may remain valid for up to 2 years. c. Data which you voluntarily provide to us is generally retained for as long as this is necessary for the fulfillment of the purpose for which they were obtained. Thus, in any case we process your personal data for the duration of our contractual or service relationship with you (see [Section 2(d)](#activelydisclosed) above). Beyond this time period we keep your personal data to comply with statutory retention obligations (e.g. to fulfill the seven year retention obligation under applicable tax and company law). Where necessary we may also keep your data for as long as potential legal claims against us are not yet time-barred; for certain claims the statutory limitation period may be up to 30 years. d. As soon as there are no legitimate grounds for the further storage of personal data available, they will either be deleted or made anonymous. ## Section 5 - Your Rights as a Data Subject a. As a data subject you have *inter alia* the following rights under the statutory conditions: 1. to check whether and what kind of personal data we hold about you and to request copies of such data (*right of access*) 2. to request correction, supplementation, or deletion of your personal data that is inaccurate or processed in non-compliance with applicable requirements (*right to rectification and erasure*) 3. to request us to restrict the processing of your personal data (*right to restriction*) 4. in certain circumstances, to object for legitimate reasons to the processing of your personal data or to revoke consent previously granted for the processing (*right to object or withdraw consent*) 5. to receive the personal data you provided to us in a structured, commonly used, and machine-readable format and to transmit those data to another controller (*right to data portability*) b. We do not process your personal data for the purpose of taking decisions based solely on automated processing, including profiling, which produce legal effects concerning you (Art 22 GDPR). c. To exercise any of the above rights kindly send an email to . In addition, you have the right to lodge a complaint with a supervisory authority, if you believe your data protection rights have been violated. For Austria the competent authority is the Data Protection Authority (*Datenschutzbehörde*). ## Section 6 - Updates to this Notice a. We may update this Notice to reflect legal, technical, or business changes. When we update this Notice, we will take reasonable steps to inform you about the changes made. You will find the date of the "last update" at the beginning of this Notice. ## Section 7 - Disclaimer a. The Websites contain links to third-party websites. We have no control over the content or privacy practices of these other websites. Please read the respective data protection provisions of other websites that you visit. ## Section 8 - Our Contact Details a. Should you have any requests or questions in relation to the processing of your personal data by us, kindly address them to b. Our office address is: Thurngasse 8/4, 1090 Wien, Austria *** [\[1\] https://legal.hubspot.com/privacy-policy](https://legal.hubspot.com/privacy-policy)\ [\[2\] https://stripe.com/privacy-center/legal](https://stripe.com/privacy-center/legal)\ [\[3\] https://slack.com/privacy-policy](https://slack.com/privacy-policy) --- --- url: /2020/06/race-dashboard.md --- The [Rapid Action coronavirus Earth observation dashboard](https://race.esa.int), otherwise known as RACE, presents the results of the joint cooperation between [ESA](https://www.esa.int/) and the [European Commission](https://ec.europa.eu/info/index_en) on Covid 19 and EO. The RACE platform demonstrates how the use of Earth observation data can help shed new light on societal and economic changes currently taking place owing to the coronavirus pandemic. EOX is proudly supporting this initiative as part of [Euro Data Cube](https://eurodatacube.com) and had the privilege to be tasked with the development of [eodash](https://github.com/eurodatacube/eodash), the dashboard front-end. We had the privilege to work on this dashboard in close  contact with ESA and managed to kick-start the application from 0 to public release within just one month. Together with Euro Data Cube services it is powering the [RACE dashboard](https://race.esa.int) available at shown in some screenshots below. ![](/images/race-screenshot.jpg) RACE dashboard ![](/images/race-screenshot-current-indicators-list.jpg) Current indicators list ![](/images/race-screenshot-air-quality-indicator.jpg) Air Quality indicator ![](/images/race-screenshot-water-quality-indicator.jpg) Water quality indicator The eodash software powering the RACE dashboard is [openly available unter MIT license](https://github.com/eurodatacube/eodash). Feel free to report bugs or request features directly in the repository. --- --- url: /_team/pasparugar.md description: Finding bugs by moonlight, fixing them by daylight --- --- --- url: /2020/09/raising-the-bar-for-earth-observation-data-processing.md --- “Enable researchers to focus more on the science and less on the plumbing” is the core message of [this great article](https://eos.org/features/a-geodata-fabric-for-the-21st-century) from Jeff de La Beaujardière, where he summarizes the upcoming challenges and opportunities with Earth Observation (EO) data as well as describes the path to a generic EO data processing solution - by introducing the concept of a Geodata Fabric. We at EOX are following these ideas and concepts since a while, for our own large-scale processing needs with [our cloudless mosaic products](/2020/03/sentinel-2-cloudless-2019), and in our project work like for [Euro Data Cube](https://eurodatacube.com) (EDC), an initiative supported by the European Space Agency (ESA) to create a concrete representation of such Geodata Fabric - we called it EO Information Factory - to foster the long term vision of building up a Digital Twin Earth. The [ESA EO Φ-Week](https://phiweek.esa.int) virtual event provides an opportunity to acknowledge the latest achievements going in that direction, with EDC being presented and showcased in [two side-events](https://eurodatacube.com/news/phiweek-2020). If you are following the trends in EO data processing you are probably familiar with the great blog post series by Chris Holmes on [Cloud Native Geospatial](https://medium.com/planet-stories/cloud-native-geospatial-part-1-basic-assumptions-and-workflows-aa67b6156b53) and [Analysis Ready Data](https://medium.com/planet-stories/analysis-ready-data-defined-5694f6f48815) - setting up the stage why EO data should be cloud friendly and analysis ready. Cloud friendly data enables analysis straight on the source data files by relying on structured metadata for partial data reads as requested by the spatial and temporal extents of interest - without requiring the download of the whole file. Several such cloud-optimized formats have gained popularity, with [Cloud Optimized GeoTIFF](https://www.cogeo.org/) (COG) and [Zarr](https://zarr-specs.readthedocs.io) being the most prominent ones, both on the way to become an agreed standard. Relying on them for persistent storage allows lazy loading of data, materializing only these chunks of data which are really needed for processing. Analysis ready datasets (ARD) are defined to have all necessary transformations and aggregations applied and the required metadata included, making them directly consumable for domain-specific processing - from temporal analysis to statistical forecasts to AI and Machine Learning (ML) use cases. If ARDs are stored in a cloud friendly format, the only missing piece is an elastic and scalable compute layer - a workspace environment with all the infrastructure resources to run processes in a cost-effective manner, enabling reusable and reproducible workflow execution, both interactively and in a headless fashion (for on demand, systematic and scheduled invocation). ![](/images/cloudworkspace-process.jpg) We at EOX operate such cloud workspaces, providing different flavors of computational resources and storage options in various clouds, to run customer workloads close to the particularly needed EO data archives, like [Amazon Public Data Sets](https://aws.amazon.com/blogs/aws/tag/public-data-sets/) (PDS) or the [Mundi Dias data offerings](https://mundiwebservices.com/data) . Please [reach out to us](/contact/) for details. Note: our AWS EU-Central-1 cloud workspace offering, the [EDC EOxHub Workspace](https://eurodatacube.com/marketplace/infra/edc_eoxhub_workspace), is accessible in self-service mode! But also in the case the needed data is not in cloud optimized format yet or still at a lower processing level (i.e. not analysis ready), you are not on your own with the data preparation steps. One option is to use managed service offerings like [EDC Sentinel Hub Batch processing](https://eurodatacube.com/marketplace/services/edc_sentinel_hub) or the [EDC xcube Generator](https://eurodatacube.com/marketplace/services/edc_xcube_generator), both taking care to connect to a data archive, transforming the loaded data based on a supplied processing script and staging the result data into object storage - in a cloud friendly and (now hopefully) analysis ready way, to be used for subsequent domain-specific processing as laid out before. Another possibility is to leverage cloud workspaces like the above mentioned [EDC EOxHub Workspace](https://eurodatacube.com/marketplace/infra/edc_eoxhub_workspace) also for pre-processing. There are [tons of great EO libraries](https://github.com/acgeospatial/awesome-earthobservation-code) for accessing and transforming EO data, many of them freely and publicly available, with a curated selection of them bundled within the individual cloud workspace offerings and ready for use. We will host a science user testimonial at ESA EO Φ-Week demonstrating such data preparation step based on a concrete example. So if you are not only interested in how Henrik Fisser could [detect trucks on roads across Europe using free 10m imagery](https://eurodatacube.com/marketplace/notebooks/contributions/Detect_Trucks_Sentinel2.ipynb), but also on how he used the [xcube-sh library](https://eurodatacube.com/marketplace/notebooks/getting-started/EDC_Sentinel_Hub-Data-access.ipynb) (a python package bundled within EDC) to shuffle 38 TB of imagery data with tens of millions API calls from the [EDC Sentinel Hub API](https://eurodatacube.com/marketplace/services/edc_sentinel_hub) to his cloud workspace before he applied the truck detection algorithm, you should definitely checkout this [](https://eurodatacube.com/news/phiweek-2020/)[Φ-Week side-event](https://eurodatacube.com/news/phiweek-2020/). --- --- url: /2017/09/s2maps-eu-stumbling-into-half-a-million-visitors.md --- How it happened and what we learned. On Friday, August 18, thanks to Joachim's great work, we announced our worldwide [Sentinel-2 cloudless map](https://s2maps.eu) and were quite happy about a couple of hundreds of visitors over the next days. We maintained a steady stream of about 300-1000 visitors a day mainly because of tweets from the European Space Agency (ESA). On Monday, August 28 ESA decided to announce Sentinel-2 cloudless on their EO-Portal and send the link to some 60.000 subscribers of their EO newsletter and share it on Facebook to some 800.000 followers. We were celebrating and prepared our servers for a couple of thousands visits which we expected based on experience from the release of our Europe version back in March. Personally, I was prepared to monitor our servers for a couple of days and then take off for some long awaited family vacations over a long weekend. Then it went crazy. On Wednesday, the day before my vacations were supposed to start, the German IT news portal heise online posted a [blog post about our map](https://www.heise.de/newsticker/meldung/ESA-Satelliten-liefern-Daten-fuer-Satellitenkarte-der-Welt-ohne-Wolken-3817252.html). I was jokingly asking my colleagues, will this be a *hug of death* for our service? It really boosted visitor numbers as the graph below shows but I couldn't have been more wrong about traffic peaks. On Friday, I woke up and checked our servers and saw some crazy numbers which I didn't believe at first. At second look I found the [Daily Mail in the referrers](http://www.dailymail.co.uk/sciencetech/article-4841958/80-TRILLION-pixel-interactive-cloudless-Earth-image.html), holy moly. The number of visitors peaked at 35.000 on Friday. My family wouldn't allow me to spend more than 2 hours trying to increase server capacity for our tile service which of course was not nearly enough as we hadn't planned for anything near these numbers. Nevertheless, Cloudflare, which we were already using for the website, proved to be really great and Hetzner proved to be reliable with network bandwidth, at least to the extent our server was able to fill it. HTTP response times recovered a little on Saturday and I thought, well this wasn't too bad after all. Again, I couldn't have been more wrong as Sunday morning showed. Numbers were again an order of magnitude higher than on Friday. What had happened this time? I Fucking Love Science had written [a piece about us](http://www.iflscience.com/technology/check-out-this-stunning-80trillionpixel-cloudless-view-of-earth/) and shared it around midnight to their 2 million followers on Facebook. Perfect timing for me being on vacations and, well, sleeping. Sorry for the performance that night. The post was trending and got over 38.000 shares and resulted in more than 140.000 visits on Sunday alone with a peak hour at midnight of 25.000 visits. We quickly added a banner to apologize about not having planned for such a traffic, many thanks Silvester. Again I got 2 hours from my family in which I managed to upgrade and improve our server to serve 65GB of tiles per hour during Sunday afternoon. Still not enough, but way better than before. Back home on Sunday evening, I bought and configured the new domain s2maps-tiles.eu to serve tiles via Cloudflare. Luckily s2maps.eu had always been behind Cloudflare. On Monday I added load balancing to the Cloudflare configuration using Amazon EC2 instances. These have a very basic configuration where MapCache is serving map tiles from SQLite tile caches loaded to Amazon EFS. On Tuesday alone more than 120.000.000 tile requests have been made to the new domain. I was expecting Cloudflare to save maybe around 50% of bandwidth but statistics show it served 1.63TB cached and only 179.16GB un-cached on Tuesday alone which corresponds to a huge saving of 85% of bandwidth. Since enabling Cloudflare 360.000.000 tiles have been requested and served successfully in the sense that the HTTP response time has never gone beyond five seconds again. The graph below shows the number of unique visitors of s2maps.eu since our announcement. Can you still spot the traffic spike when ESA announced it on their EO-Portal on Monday, August 28th? ![Unique Visitors](/images/cloudflare_visitors.jpg) Let me drop some further statistics here: We see 90 referrers with more than 100 visits and 440 in total. The average visit duration is 1:52 minutes with 3.6 actions average and a bounce rate of 16%. Our most interested visitor made a stunning 155 actions on the page. Direct entries are at 34% but that's mostly due to websites including rel="noopener noreferrer" in their links to s2maps.eu. Interest was worldwide, with at least 10 visits from 161 countries. On a different note, we added a link to a [3D widget](https://mapalupa.com/v1/w/363) using our [mapalupa service](https://mapalupa.com) which got over 100.000 visits as well. Fairly big news pages like [galileo.tv](https://at.galileo.tv/weltweit/zoom-dich-hier-durch-eine-80-billionen-pixel-karte-unserer-erde) integrated it directly. Quite a validation of our idea, isn't it Daniel and Silvester? What did we learn stumbling along the way? * Earth Observation isn't that boring after all and does get attention from the general public. * Preparations for increased traffic should start at the first indication. In our case that would have been heise online which would have saved us some rather poor performance impressions on my vacation weekend. * Cloudflare really takes away a lot of traffic, 97% of bandwidth for s2maps.eu, even for an interactive map, 85% of bandwidth for s2maps-tiles.eu. * Open Source software like MapCache, Apache, etc. is great. * (Media) people don't give a damn about attribution requirements. * Certainly we'll make sure to have spare server capacities for future announcements. Many thanks for your interest and please check back if you encountered a poor performance, it's better now, promised. --- --- url: /2020/06/saharan-dust-event-from-space.md --- ## EO Data visualization on the browser Here at EOX we have already talked about how we manage and visualize data in the browser, here some references to previous blog posts if you want to refresh your memory: * [Explore GeoTIFFs like Landsat 8 scenes with COG-Explorer in the browser](/2018/05/explore-landsat-8-scenes-with-cog-explorer/) * [Visualizing Raster Data](/2016/02/visualizing-raster-data/) To help us with directly visualizing EO data in the browser we have developed a modular set of tools with the intent of being able to use them interchangeably in multiple projects allowing us to further develop and enhance them based on each of the projects' needs. We are very happy to see how well we managed to make use of and integrate our tools in the TOP project as part of the DAVE visualization client, here we see a recent example of visualization of simulation (CAMS) and satellite (Sentinel5P) data over imposed and directly rendered on a 3D globe showing the Sahara dust event from the 24th of June! On this occasion, we would like to again acknowledge the awesome cooperation with Sistema and ZAMG, our partners in the TOP initiative! --- --- url: /2015/07/sentinel-2.md --- Just two weeks ago the European  satellite Sentinel-2 has been [launched into orbit](http://www.esa.int/spaceinvideos/Videos/2015/06/Sentinel-2A_liftoff). Sentinel-2 is going to observe the Earth's surface with an optical camera. In fact the [first images have already been acquired and sent down to Earth just four days after launch](http://www.esa.int/Our_Activities/Observing_the_Earth/Copernicus/Sentinel-2/Sentinel-2_delivers_first_images). These images give an impression of the great capabilities  of Sentinel-2 like it's wide swath and short revisit time. This first image shown below was acquired on June 27th and covers the Po Valley, framed by the Alps in the north and the coastal mountains of France and Italy in the south. Arianespace has released a [poster](http://www.arianespace.com/news-image_library/posters.asp) from the launch. Our office is already decorated with it ;) The data coming from the Sentinel-2 satellite will hopefully keeps us busy the next years or even decades. Expect more posts to come in the future :) --- --- url: /2017/03/sentinel-2-cloudless.md --- Eternal summer where the sun shines every day the whole year does not exist on Earth. At least [not everywhere](https://en.wikipedia.org/wiki/Atacama_Desert). And that is a good thing too since plants need the occasional rainfall to prosper and we need plants for food and oxygen. When dealing with satellite imagery though, clouds cloak the Earth's surface and the shadows they throw obscure features on the ground. From certain areas, it even is [close to impossible](https://medium.com/planet-stories/one-of-the-worlds-largest-reservoirs-lies-hidden-in-the-mountainous-center-of-borneo-1300215f26bf) to get [a cloud-free view](http://www.esa.int/spaceinimages/Images/2013/01/A_rare_cloud-free_view_of_Ireland_Great_Britain_and_northern_France). In the end, almost [70% of the Earth's surface is covered by clouds every day](http://earthobservatory.nasa.gov/IOTD/view.php?id=85843). But somehow, when looking at the satellite layer of one of the big mapping companies like Google, Bing or Mapbox, there are no clouds and on the whole Earth there seems to be summer. How does this work? We answer this question in the following and explain which process we applied in the generation of our Sentinel-2 cloudless map. The key here is to get multiple images and merge them in a way that the best pixels (i.e. no cloud or cloud shadow) are selected for the target mosaic. One approach is, to use cloud masks for each image if available and replace the masked patches with cloud-free areas from other images. If the images are manually pre-selected it can lead to [pretty good results](http://www.esa.int/spaceinimages/Images/2016/05/African_mosaic) however it is very likely that the image and cloud mask borders can be seen within the mosaic because light or vegetation conditions vary between images depending on the date they were acquired from. Furthermore, manual pre-selection is tedious and does not scale well for larger areas. A more brutal approach is to apply the pixel sorting method used by [Mapbox](https://www.wired.com/2013/05/a-cloudless-atlas/) and [Google](https://www.theatlantic.com/technology/archive/2016/06/google-maps-gets-a-satellite-makeover-mosaic-700-trillion/488939/). Here, a time stack of pixels is sorted by color brightness and one or multiple pixels not showing very bright (cloud) or very dark (shadow) extremes are selected, merged and used for the target mosaic. The latter approach has two main advantages: 1. If the numbers of images within the timestack is large enough, the borders between images get smoothed out which makes the mosaic appear seamless. 2. It is fully automatable. Because we want to map the world and the results shown by Mapbox and Google looked very promising, we decided to go with the latter approach. Visit to browse through the map. To provide a preview, this is how the result looks on selected locations: ## Data In previous blog posts we already covered the [Sentinel-2 data structure](/2015/12/understanding-sentinel-2-satellite-data/) and [how to get the data](/2016/02/downloading-sentinel-2-data/). To create this mosaic we decided to get the data from [Sinergise hosted by AWS](http://sentinel-pds.s3-website.eu-central-1.amazonaws.com/) because the data access is more reliable and, more importantly, data is structured by the Sentinel tiling scheme (which is based on [MGRS](https://en.wikipedia.org/wiki/Military_Grid_Reference_System)). Unfortunately, the only product level currently available is Level-1C, which means the images are orthorectified but the image values are "top of the atmosphere" (TOA) which means that atmospheric correction has not been applied. There is the possibility to calculate the atmospheric correction by using the semi-open-source tool [SEN2COR](https://github.com/umwilm/SEN2COR) but processing times per scene were too long and would have slowed down our processing chain significantly. The main challenge here for us was to find a way do do it quicker with acceptable results. There is a great article from [Planet](https://www.planet.com/pulse/color-correction/) describing color correction using image software and simply stretching separately the different bands to approach realistic colors. So, for this version we are using a static stretching for all of the data. ## The Tools Processing such an amount of data requires the right set of tools. We mainly use Python as there are great packages available to handle spatial data ([rasterio](https://mapbox.github.io/rasterio/)/[GDAL](http://gdal.org/)) and arrays ([NumPy](http://www.numpy.org/)). The core pixel selection algorithm, which is sort of a middle[-](http://www.piedpiper.com/)out approach as described above, was implemented in [Cython](https://cython.readthedocs.io/en/latest/) to increase performance. Based on the experiences made during the creation of the European coverage, improvements will be made for future processing. ### Mapchete The framework gluing it all together is developed in-house, open source and called [Mapchete](https://github.com/ungarj/mapchete). It applies a geoprocess written in Python to a large dataset by chunking the input data into smaller tiles. Look forward to a deeper introduction in a later blog post. ### Sinergise Catalog Having access to the S3 bucket holding the Sentinel-2 archive alone is not sufficient for these processing jobs. At some point, a preselection has to be made which datasets are being used based on their geographical location and timestamp. This is where the catalog service from our friends at Sinergise come into play. ### Amazon Web Services We used a bunch of EC2 instances to scale up the process and [Celery](http://www.celeryproject.org/), a Python-based task queuing framework. This allowed us to spin up a large number of virtual machines working through the CPU intensive processing steps. ## Using The Layer The Sentinel-2 cloudless layer is provided under a [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/). This means everyone is allowed and encouraged to use the layer as he/she sees fit while providing attribution that reads *"[Sentinel-2 cloudless](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at) (Contains modified Copernicus Sentinel data 2016)"* and include the links as shown here where possible. ### WMTS Endpoint Among other layers provided by [EOX Maps](http://maps.eox.at/), the Sentinel-2 layer is available via WMTS and can be used in QGIS, OpenLayers or other OGC-compliant viewers: [WMTS GetCapabilities](https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml) ## Contact Questions? Suggestions? Do you want to have your algorithm to be processed on the Sentinel-2 archive? Feel free to contact us via [twitter](https://twitter.com/eox_a) or [e-mail](mailto:office@eox.at)! ## Sources All images contain modified Copernicus Sentinel data 2016. --- --- url: /2019/02/sentinel-2-cloudless-2018.md --- Good news everyone, there is an updated version of the Sentinel-2 cloudless map available! After being first in having processed the whole Sentinel-2 archive to create a cloudless mosaic for 2016 it was time to use our experience and improve the processing to create an updated, new version. With the help of our new colleague [Petr](/team/) from [Š.cartography](https://sevcikcartography.wixsite.com/cartography) we were able to develop various mosaicking methods which resulted in a new product: [EOxCloudless](https://cloudless.eox.at). In direct comparison to the old layer from 2016 you will see a lot of visible improvements in the new version from 2018. There are a lot less clouds (we are approaching real cloud-free!), the colors are much more balanced and the overall integration with other map elements such as the ocean bathymetry was completely reworked. ## Less Clouds As much as we learned to love the cloudy stripe over Ireland in 2016, we were glad that we managed to have a cloud-free view of Ireland now. Also, there were many other problematic areas which now have much improved in the new version. There are a couple of reasons the new version dealt better with cloudy areas. First, we developed and tested other methods of detecting and selecting cloud-free pixels from time stacks of data. This helped us to get good pixels with a higher probability than with the old method. Second, since 2017 we now have data from the second of the two Sentinel-2 satellites at our disposal. This means we now have roughly twice as much observations as we had before but it also meant we had to process twice as much data. Therefore we also had to improve on pre-selecting the observations in a more intelligent way. This especially paid of in regions which had poor coverage in 2016 such as the Americas and Asia. We also used the multi-spectral properties of the Sentinel-2 sensors to better detect clouds and appended the near-infrared band to our [output mosaics](https://cloudless.eox.at). ## Post-processing After extracting our cloudless pixels we also completely reworked the color balancing before we rendered the map. There is no atmospherically corrected data available on a global scale yet so we had to find a way how to create a satellite map which appears to be atmospherically corrected and which is visually appealing. For this we made use of the great open-source tool [rio-color](https://github.com/mapbox/rio-color) which goes way beyond the linear color stretching we applied for the old version. The availability of the near-infrared band also enabled us to distinct between different land cover classes and apply fine-tuned color correction patterns for each patch. This now results in brighter colors in higher latitudes and much more balanced colors for deserts and forests while maintaining a good brightness in build-up areas such as cities. In the end we also applied some selective smoothing and sharpening to accentuate water and land areas. ## Ocean Mask Instead of having a discrete mask like in the old version to distinguish Land from Ocean, we now use a fuzzy mask to ensure a smooth transition. The buffer width is also dependent on the zoom level so as you zoom in, more and more from the Ocean is revealed. We also now not only include land masses but also reefs which were not present in the old version. This should also make Oceanographers happy! ## License Change The 2016 version is provided for free under the [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0) and the 2018 one under [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NS-SA 4.0). The required attribution is "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu/) by [EOX IT Services GmbH](https://eox.at/)(Contains modified Copernicus Sentinel data 2016 & 2017)*" for 2016 and "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu/) by [EOX IT Services GmbH](https://eox.at/)(Contains modified Copernicus Sentinel data 2017 & 2018)*" for 2018 respectively. It shall include the links as given here in on-line usage. For commercial usage we [offer the data](https://cloudless.eox.at) under the [EOX Commercial Attribution-RestrictedUse 1.0 License](https://cloudless.eox.at/license-deed) which is based on CC BY. The main change over CC BY is that EOX reserves the right to provide the data with this license. You're entirely free to use it in your services and products, you just can't transfer this license to your users. Usage examples include a WMTS you provide under your name or a product you sell that bundles the data for offline usage. ## EOxCloudless Are you interested in getting the rendered map, the mosaic used, or order a custom mosaic? Come over to [EOxCloudless](https://cloudless.eox.at) and let us help you out! --- --- url: /2020/03/sentinel-2-cloudless-2019.md --- The annual update of our [EOxCloudless](https://cloudless.eox.at/) global satellite layer known as [Sentinel-2 cloudless](https://s2maps.eu) is here! ## Even less clouds than before Due to the usage of atmospherically corrected data we can now present a much clearer and less hazier image of the Earth than ever before. However with just one year worth of data at hand we faced a difficult challenge again. Given that many regions are covered by clouds most time of the year. Just take a look at the map below which shows the median from a whole year worth of data from the Suomi-NPP satellite for the year 2018. [![](/images/VIIRS_SNPP_CorrectedReflectance_TrueColor_4326_median.vrt_2000px.jpg)](https://hannes.enjoys.it/blog/2019/01/average-earth-from-space-2018/) [VIIRS SNPP CorrectedReflectance TrueColor median 2018](https://hannes.enjoys.it/carto/VIIRS_SNPP_CorrectedReflectance_TrueColor_median/) Some regions have white albedo all year long, apart from glaciers in the arctic, antarctic, and higher mountain ranges, the northern Andes, west coast of central Africa, and of course Indonesia are especially troublesome as the cloud cover around the acquisition time in some regions seems to be permanent. To finalize the comparison this is the global Sentinel-2 cloudless 2019: ![](/images/global_no_coords_2019-scaled.jpg) Sentinel-2 Cloudless 2019 ## Bottom of the atmosphere reflectance One of the major changes is utilizing the Bottom of the Atmosphere (BOA) reflectance. This makes the values returned from the mosaicing more accurate in terms of dealing with the haze and other atmospheric effects. This upgrade is visible over the whole globe but the major impact can be observed over the tropics and subtropics. ![](/images/amazonas_2018_2019-scaled.jpg) Major haze influence in the blue channel in 2018 (bottom) vs. clearer look in 2019 (top) ## Optimizing the climate zones Finding the ideal time period from which to generate the satellite map is something quite variable and it is almost impossible to define a single time window for whole continents or even individual climate zones. As we can see in the image below the vegetated and not vegetated areas affect the reflectance directly, so that we are still tuning the optimal definitions for various regions and corresponding time intervals. As a rule of thumb, we try to keep the time window borders in the ocean or just along the coastline, but sometimes we do define the border going straight through the landmass. In most cases it lies somewhere in the subtropical zones. ![](/images/global_NDVI_no_coords_2019-scaled.jpg) EOxCloudless 2019 NDVI render With the changing sun zenith angle over the year some regions might appear brighter or darker, some objects will cast longer shadows with lower sun zenith. This provides better contrast especially noticeable with taller objects such as various landmarks and buildings, for instance the Giza Pyramids in Egypt. ![](/images/pyramids_2018_2019-scaled.jpg) Giza Pyramids, Egypt in 2018 (left) and 2019 (right) On the other hand sun and water present another dilemma on its own. In the summer sea surface often acts as a giant mirror. This causes large scale reflections of what seems to be white surface over the sea. A natural solution is to use measurements with lower reflectance. For processing the 2018 version of the mosaic we used the (north) summer months to process the Caribbean. For the year 2019 we used the (north) winter months for processing most of the northern subtropical regions. These reach up to the Mexico border with the U.S.A., whole Africa, through the Middle East, Hindu Kush, just north of the Himalayas, and through southern China. ![](/images/bahamas_2018_2019-scaled.jpg) Great Exuma and Long Island, The Bahamas in 2018 (left) and 2019 (right) ## What's next? We all like nice images, that's why we do what we do, so the answer to the question is naturally: "More maps!". Our roadmap is to release yearly global updates for the foreseeable future to provide our customers a steady update and the possibility to track changes across the globe. Additionally we will keep improving the Sentinel-2 cloudless to offer an unique sight at our beautiful planet, so go ahead and take a look yourself: [s2maps.eu](https://s2maps.eu/). If you are interested in using our maps commercially: [cloudless.eox.at](https://cloudless.eox.at/) is the site you are looking for. *** ## License The 2016 version is provided for free under the [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0), the 2018 and 2019 under [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NS-SA 4.0). The required attribution is "*[Sentinel-2 cloudless -](https://s2maps.eu/)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2016 & 2017)*" for 2016, "*[Sentinel-2 cloudless -](https://s2maps.eu/)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2017 & 2018)*" for 2018, and "*[Sentinel-2 cloudless -](https://s2maps.eu/)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2019)*" for 2019 respectively. It shall include the links as given here in on-line usage. For commercial usage we [offer the data](https://cloudless.eox.at) under the [EOX Commercial Attribution-RestrictedUse 1.1 License](https://cloudless.eox.at/license-deed) which is based on CC BY. The main change over CC BY is that EOX reserves the right to provide the data with this license. You're entirely free to use it in your services and products, you just can't transfer this license to your users. Usage examples include a WMTS you provide under your name or a product you sell that bundles the data for offline usage. The [EOX WMTS](https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml) and [EOX WMS](https://tiles.maps.eox.at/wms?service=wms\&request=getcapabilities) are free and open to use for non-commercial applications given that when publishing the corresponding [attribution](https://s2maps.eu/#license) is provided in the publication. *** Petr Ševčík, EOX IT Services GmbH Email: petr.sevcik@eox.at *Contains Modified Copernicus Sentinel-2 Data 2018 and 2019.* *The median cloud coverage image credit goes to: * --- --- url: /2021/03/sentinel-2-cloudless-2020.md --- Another year went by and our most recent update of the [EOxCloudless](https://cloudless.eox.at/) satellite map [Sentinel-2 cloudless](https://s2maps.eu) has been released! ## Layer comparison from 2016 to 2020 Apart from major technical improvements of the data foundation the website [Sentinel-2 cloudless](https://s2maps.eu/) has got a new coat as well. You can now browse all years of our global satellite layers by clicking the clock symbol top left. Can help us to find more interesting areas that changed over the past years in our [Sentinel-2 cloudless](https:s2maps.eu)? Feel free to share with us any beautiful examples on [Twitter](https://twitter.com/eox_a). ## Less striping effects in the map The biggest change is the implementation of a Bidirectional Reflectance Distribution Function (BRDF) to correct the differences in brightness across the Sentinel-2 acquisitions in west-east direction. This improves the overall smoothness of our yearly map. The most challenging regions are around the equator, where the whole year of data has been used as input data. Let’s revisit the same region like the [blog from last year](https://eox.at/2020/03/sentinel-2-cloudless-2019/) to see the improvements. ## Even smoother transitions between climate zones The BRDF correction alone is not enough to compensate for the differences between the seasons, therefore we tried to define time intervals that are not too large to keep the mosaic as homogeneous as possible. As you can see in the image above the borders of the climate zones over land are following state or province borders of various countries. Given that most of the borders between countries follow distinct geomorphological features thus making it easier to split the regions naturally. We went even further and made these borders fuzzy to make these transitions even less apparent. In the example below the climate zones border at the Himalayas has clearly improved since the 2019 mosaic. ## Following next... Apart from the next satellite map for the year 2021 which will be out around this time next year, the biggest news regarding the [EOX map products](https://maps.eox.at/) is, that we are currently working on an update for our custom [OpenStreetMap](https://www.openstreetmap.org/) rendering, labels, and more. Finally, if you are looking for a nice screensaver to watch while on a coffee break click the *"Tour"* button at the bottom left of and enjoy the ride. Also please note that our maps are free to use non commercially with proper attribution, if you aim to use our maps in your business please head to or contact us directly via [cloudless@eox.at](cloudless@eox.at). *** ## License The 2016 version is provided for free under the [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0), the 2018 - 2020 under [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NS-SA 4.0). The required attribution is "*[Sentinel-2 cloudless -](https://s2maps.eu/)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2016 & 2017)*" for 2016, "*[Sentinel-2 cloudless -](https://s2maps.eu/)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2017 & 2018)*" for 2018, and "*[Sentinel-2 cloudless -](https://s2maps.eu/)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2019)*" for 2019 and "*[Sentinel-2 cloudless -](https://s2maps.eu/)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2020)*" for 2020 respectively. It shall include the links as given here in on-line usage. For commercial usage we [offer the data](https://cloudless.eox.at) under the [EOX Commercial Attribution-RestrictedUse 1.1 License](https://cloudless.eox.at/license-deed) which is based on CC BY. The main change over CC BY is that EOX reserves the right to provide the data with this license. You're entirely free to use it in your services and products, you just can't transfer this license to your users. Usage examples include a WMTS you provide under your name or a product you sell that bundles the data for offline usage. The [EOX WMTS](https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml) and [EOX WMS](https://tiles.maps.eox.at/wms?service=wms\&request=getcapabilities) are free and open to use for non-commercial applications given that when publishing the corresponding [attribution](https://s2maps.eu/#license) is provided in the publication. *** Petr Ševčík, EOX IT Services GmbH Email: Contains Modified Copernicus Sentinel-2 Data 2016, 2017, 2019, and 2020. --- --- url: /2022/05/sentinel-2-cloudless-2021.md --- This year we challenged ourselves by overhauling our processing infrastructure in order to give us more room for improvements in the future. This caused a delay of our usual release interval, but we think it was well worth the effort! With a bit of sweat and blood we can finally announce the latest release of our annual [EOxCloudless](https://cloudless.eox.at) satellite map [Sentinel-2 cloudless](https://s2maps.eu). ## Layer comparison from 2016 to 2021 Apart from major technical improvements of the data foundation the website [Sentinel-2 cloudless](https://s2maps.eu) has got a new coat as well. You can now browse all years of our global satellite layers by clicking the clock symbol top left. The latest visual change of the map is darker color rendering over the Sahara, Northern Africa, Near East, and Arabian Peninsula. Can you help us to find more interesting areas that changed over the past years in our [Sentinel-2 cloudless](https://s2maps.eu)? Feel free to share with us any beautiful examples on [Twitter](https://twitter.com/eox_a). ## Processing development This technology chapter tries to illustrate how we have re-implemented our whole processing deployment infrastructure. We are using an in-house open source solution for geoprocessing ([mapchete](https://github.com/ungarj/mapchete)) since the first iteration in 2016. However, in scaling it up to be able to process the global dataset on a large computing cluster always required some additional technologies. Up until this campaign [Celery](https://docs.celeryq.dev/en/stable/) and [RabbitMQ](https://www.rabbitmq.com/) were used to distribute jobs over a manually managed set of compute instances. Lacking some desired features we have reevaluated the deployment procedure to run on a [Kubernetes](https://kubernetes.io/de/docs/concepts/overview/what-is-kubernetes/) cluster on AWS. This allowed us to make use of autoscaling and took the burden of managing instances from the operator's shoulders. To handle the distributed computing part Celery was replaced by [dask](https://github.com/dask/dask) and [dask-distributed](https://github.com/dask/distributed) mainly because of the simpler API as well as the large momentum dask gained in recent years. Also, it supports more complex computation graphs but this topic is worth a separate blog post. Dask also allowed us to make use of other packages in the ecosystem such as [dask-gateway](https://github.com/dask/dask-gateway) and [dask-kubernetes](https://github.com/dask/dask-kubernetes) which can handle requesting, provisioning, and down-scaling a compute cluster on demand. To round it up the exposed REST API was built using [FastAPI](https://fastapi.tiangolo.com/) which we use in other projects in EOX as well. We now have a full processing ecosystem in our hands which can be triggered by just interacting with the exposed REST interface. To facilitate usage we developed a command line utility (`mhub`) which allows job submission and cancellation as well as some additional productivity tools such as showing the progress of a running job and querying jobs by their metadata. The internal metadata representation is based on GeoJSON so all jobs can be dumped and visualized in a GIS if need be. Once a job is submitted, mapchete determines the processing chunks and requests a dask cluster with certain specifications to dask-gateway. The gateway then passes on the request to Kubernetes and if there are resources available (e.g. AWS spot instances), Kubernetes will start nodes and the desired dask scheduler and worker pods which we refer to as dask cluster. As soon as the cluster is up and running, mapchete will submit its tasks and observe the job progress while keeping the job metadata database up to date. These developments now bring us one step further to achieving full automation where a full process can be triggered by an event (e.g. a custom mosaic request) rather than an operator triggering and observing the process. Also this flexibility allows us to use the platform in [other projects](/2022/02/how-eo-expertise-powers-agri-monitoring-decisions-part-1/). ## EO-Guesser Have you heard of GeoGuesser and are a fan of satellite imagery or the world from space? We have developed a neat new game and at EOX we are playing it with great enthusiasm since weeks. If you want to try your luck or want to put your Earth Observation knowledge to test you have to try our [EO-Guesser](https://s2maps.eu/eo-guesser). Being zoomed in on a random place on Earth just able to see the satellite map, can you guess where you are? How well do you know our planet? ## Beyond True Color Our data can be used to generate incredible map and visualisation products e. g. the image in the top banner of this blogpost. Most of our customers are interested in the True Color imagery, but recently we had a positive feedback on alternatives such as False Color Infrared (Sentinel-2 band combination: B08 - NIR, B04 - red, B03 - green) as False Color RGB or NDVI RGB render. So if you would like to see more of these or would like to see those in our default product offer, please let us know. ## Following next... The next blogpost for under the EOxCloudless and EOxMaps topic will be introducing the latest version of *mapchete-hub* the software that handles the processing of our products. Other topics and updates will be arriving as well so stay tuned. For more information download our [EOxCludless brochure](/content/EOX_Brochure_-_EOxCloudless_web.pdf). Finally, if you are looking for a nice screensaver to watch while on a coffee break click the *"Tour"* button at the bottom left of and enjoy the ride. Also please note that our maps are free to use non commercially with proper attribution. If you aim to use our maps in your business please head to or contact us directly via . *** ## License The 2016 version is provided for free under the [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0), the 2018 - 2021 versions under [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NS-SA 4.0). The required attribution is "*[Sentinel-2 cloudless -](https://s2maps.eu)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2016 & 2017)*" for 2016, "*[Sentinel-2 cloudless -](https://s2maps.eu)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2017 & 2018)*" for 2018, and "*[Sentinel-2 cloudless -](https://s2maps.eu)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2019)*" for 2019, "*[Sentinel-2 cloudless -](https://s2maps.eu)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2020)*" for 2020 and "*[Sentinel-2 cloudless -](https://s2maps.eu)  by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2021)*" for 2021 respectively. It shall include the links as given here in on-line usage. For commercial usage we [offer the data](https://cloudless.eox.at) under the [EOX Commercial Attribution-RestrictedUse 1.1 License](https://cloudless.eox.at/license-deed) which is based on CC BY. The main change over CC BY is that EOX reserves the right to provide the data with this license. You're entirely free to use it in your services and products, you just can't transfer this license to your users. Usage examples include a WMTS you provide under your name or a product you sell that bundles the data for offline usage. The [EOX WMTS](https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml) and [EOX WMS](https://tiles.maps.eox.at/wms?service=wms\&request=getcapabilities) are free and open to use for non-commercial applications given that when publishing the corresponding [attribution](https://s2maps.eu/#license) is provided in the publication. *** Petr Ševčík, EOX IT Services GmbH Email: Contains Modified Copernicus Sentinel-2 Data 2016, 2017, 2019, 2020, and 2021. --- --- url: /2023/10/sentinel-2-cloudless-2022.md --- *** Introducing the latest marvel in Earth observation: Sentinel-2 Cloudless, the pinnacle of usability clarity in satellite imagery. This newest version takes your visual exploration to unprecedented heights, delivering pristine, cloud-free views of our planet with breathtaking detail and accuracy. Every pixel tells a story as Sentinel-2 Cloudless unveils the Earth's beauty in vibrant colors, making it an indispensable tool for scientists, environmentalists, urban planners, and explorers alike. Immerse yourself in landscapes and cityscapes like never before, with crystal-clear rivers, lush forests, and sprawling metropolises, all devoid of those pesky clouds that once obscured your view. Whether you're monitoring environmental changes, studying land use, or simply marveling at the Earth's wonders, Sentinel-2 Cloudless is your passport to a world of visual splendor. Don't miss your chance to witness our planet in its most unobstructed glory—upgrade to Sentinel-2 Cloudless today! We now finally announce this year's latest release, Sentinel-2 cloudless 2022 which has a completely new look. We do this annually to give our customers access to the latest datasets and updates. If you are Interested in cloudless satellite imagery, check the links below on our 2022 release. We preprocess raw satellite imagery to cloudless and seamless satellite data coverage so it has less clouds, less stripes and a sharper look with more balanced colors. Our improved post-processing yields much better results in the various environments. See the latest realease by clicking on these links [EOxCloudless](https://cloudless.eox.at) satellite map [Sentinel-2 cloudless](https://s2maps.eu). ## Processing updates ### Sentinel-2 Processing Baseline Processing baselines in the context of Sentinel-2 refer to the different versions or releases of the processing software and algorithms used to generate and distribute Sentinel-2 data products. These processing baselines are important because they can affect the quality, accuracy, and consistency of the data products delivered to users. As new processing baselines are developed and released by [ESA](https://sentinel.esa.int/web/sentinel/home), they may include improvements in calibration, atmospheric correction, geometric accuracy, and other aspects of data processing. Here are some key points about Sentinel-2 processing baselines: * Data Quality Improvement: New processing baselines often include enhancements to improve the overall quality of Sentinel-2 data. This can involve refining radiometric and geometric calibration, reducing noise, and correcting for various artifacts. * Algorithm Updates: Processing baselines may incorporate updates to the algorithms used for atmospheric correction, cloud detection, and other data processing steps. These updates can lead to more accurate and reliable data products. * Consistency Across Time: One of the goals of maintaining processing baselines is to ensure the consistency of data products over time. This is important for long-term environmental monitoring and trend analysis. * Product Format Changes: New processing baselines may introduce changes to the format and structure of Sentinel-2 data products. Users need to be aware of these changes to ensure compatibility with their analysis workflows. * Metadata and Documentation: Along with each processing baseline release, updated metadata and documentation are usually provided to help users understand the changes and improvements made in that version. * Archiving Previous Versions: While new processing baselines are introduced, previous versions of data products are often archived and remain accessible to users who need to maintain consistency in their analyses or have specific requirements. In the beginning of 2022 a new processing base line version (04.00) was released which required adaption of our code to product Mosaics. The detector footprint format required for the BRDF correction changed and had to be adapted as well as the classification (including clouds) and quality masks. Furthermore due to the updated L2A processor the original reflectance values can now contain negative values, thus an offset to the original data was made in order to fit all values to the desired data range (uint16). ## EO-Guesser Have you heard of GeoGuesser and are a fan of satellite imagery or the world from space? We have developed a neat new game and at EOX we are playing it with great enthusiasm since weeks. If you want to try your luck or want to put your Earth Observation knowledge to test you have to try our [EO-Guesser](https://s2maps.eu/eo-guesser). Being zoomed in on a random place on Earth just able to see the satellite map, can you guess where you are? How conversant are you with planet earth? ## Following next... We will take part in this years [Big Data from Space](https://www.bigdatafromspace2023.org/) Conference in Vienna from November 6 to 9. If you want to connect personally, look out for EOX staff. For more information download our [EOxCludless brochure](/content/EOX_Brochure_-_EOxCloudless_web.pdf). Finally, if you are looking for a nice screensaver to watch while on a coffee break click the *"Tour"* button at the bottom left of and enjoy the ride. Also please note that our maps are free to use non commercially with proper attribution. If you aim to use our maps in your business please head to or contact us directly via . *** ## License The 2016 version is provided for free under the [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0), the 2018 - 2022 versions under [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NS-SA 4.0). The required attribution is "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2016 & 2017)*" for 2016, "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2017 & 2018)*" for 2018, and "*\[Sentinel-2 cloudless - https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2019)*" for 2019, "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2020)*" for 2020, "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2021)* for 2021" and "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2022)*" for 2022 respectively. It shall include the links as given here in on-line usage. For commercial usage we [offer the data](https://cloudless.eox.at) under the [EOX Commercial Attribution-RestrictedUse 1.1 License](https://cloudless.eox.at/license-deed) which is based on CC BY. The main change over CC BY is that EOX reserves the right to provide the data with this license. You're entirely free to use it in your services and products, you just can't transfer this license to your users. Usage examples include a WMTS you provide under your name or a product you sell that bundles the data for offline usage. The [EOX WMTS](https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml) and [EOX WMS](https://tiles.maps.eox.at/wms?service=wms\&request=getcapabilities) are free and open to use for non-commercial applications given that when publishing the corresponding [attribution](https://s2maps.eu/#license) is provided in the publication. *** Miriam Mabeya, EOX IT Services GmbH Email: Petr Ševčík, EOX IT Services GmbH Email: Joachim Ungar, EOX IT Services GmbH Email: Contains Modified Copernicus Sentinel-2 Data 2016, 2017, 2019, 2020, 2021 and 2022. --- --- url: /2024/08/sentinel-2-cloudless-2023.md --- *** We are thrilled to announce the latest release of EOxCloudless 2023 imagery. Whether you are an existing customer or new to EOxCloudless, this latest update offers clarity with beautiful sceneries and is designed to meet the needs of our users in a convenient way. With the everchanging and more challenging accessibility of satellite data, like the recent large digital infrastructure project [Copernicus Dataspace Ecosystem](https://dataspace.copernicus.eu/about), which provides centralized access to [Copernicus](https://www.copernicus.eu/en) datasets, we at EOX aggregate the huge volume of data into a value added product in a form of our satellite maps to make it easier for our customers to get hands on satellite imagery. For several years now, we offer an unchanged way of reaching our data for non-commercial use via [maps.eox.at](https://maps.eox.at) or for commercial use via the [cloudless.eox.at](https://cloudless.eox.at). This includes off-the shelf datasets and WMTS endpoint as well as offering custom solutions for various applications, being it simply more colorful satellite basemaps or use case tailored products for agriculture, environmental monitoring, aviation simulation, security, etc. To take a look at the latest annual mosaic of our planet Earth derived from the Sentinel-2 data head to [s2maps.eu](https://s2maps.eu)! ## Why Update or Purchase EOxCloudless 2023? Our most recent datasets show our dedication to providing quality satellite images. Here are some key reasons why you should consider updating your existing satellite map or buying EOxCloudless for the first time: 1. **Enhanced Color Correction** One of the standout features is the improved color correction. We have refined our processing algorithms to produce more natural and vibrant colors, making the imagery more visually appealing for scientific and practical applications. 2. **Clarity and Detail** The quality of this release is consistent with improvements, which can be seen at first glance. Whether your focus lies in any scientific or industrial geospatial, remote sensing or any other field that relies on satellite data, this will significantly improve your analysis and decision-making processes. 3. **Comprehensive Global Coverage** The dataset provides complete global coverage, ensuring you have access to dependable imagery no matter where your area of interest lies. From remote wilderness areas to bustling cityscapes we have the solution for you to provide extended and sustainable monitoring across the globe. 4. **Reduced Cloud Coverage** As the name suggests, EOxCloudless is renowned for its minimal cloud coverage. Our latest release continues this tradition, utilizing advanced cloud removal techniques to provide clear, unobstructed views of the Earth’s surface. This is particularly beneficial for users in regions frequently affected by cloud cover. ## Enhanced Color Correction - What is Different? Do some parts of our map from [s2maps.eu](https://s2maps.eu) look different than the 2022 version? Let's take a quick look behind the curtain to see what has been done differently this time. The core change was introducing so called `contrast limited adaptive histogram equalization (CLAHE)` to our post-processing color correction of the mosaic data. These histogram equalization methods had been around in image processing for a some time based on following papers: [Contrast-Limited Adaptive Histogram Equalization: Speed and Effectiveness; Pizer, S. M., Johnston, R. E., Ericksen, J. P., Yankaskas, B. C., & Muller, K. E. (n.d.); (1990)](https://sci-hub.se/10.1109/vbc.1990.109340) and [Effectiveness of Contrast Limited Adaptive Histogram Equalization Technique on Multispectral Satellite Imagery; Vidhya, G. R., & Ramesh, H.; (2017)](https://sci-hub.se/10.1145/3177404.3177409). The implementation of these methods can be found in modern image handling software packages like [OpenCV: cv2.createCLAHE()](https://docs.opencv.org/4.x/d5/daf/tutorial_py_histogram_equalization.html) and [scikit-image: exposure.equalize_adapthist()](https://scikit-image.org/docs/stable/auto_examples/color_exposure/plot_equalize.html). We have taken the `OpenCV` solution for the 2023 color correction and combined with the other post-processing steps we did in the past, with this approach significant change in sharpness especially in urban areas was achieved, making the viewing experience more "crispy". This rather small change to our data refining chain, has made more impact than we initially expected and it will be included in our future data generation where applicable. With the BRDF and CLAHE additions over the years, our satellite maps are gradually achieving more satisfying results. These tools and improvements allow us to overcome major obstacles old and new, when dealing with big data from space. ## How to Update or Purchase EOxCloudless 2023? If you are considering satellite imagery, then getting started with EOxCloudless is simple. Our offer is suited for different needs and budgets, ensuring you can find the right solution for your specific requirements and even be flexible with our custom solutions. You can purchase our products directly via [cloudless.eox.at](https://cloudless.eox.at/) by clicking **buy now** button. Additionally, we provide detailed documentation and samples to help you get started for free and to make sure our delivery formats are suitable for your use case. We are here to help you make the most out of your application with our data! Feel free to reach out to us if you need any further assistance or have specific requirements via: . ## Looking Ahead of the Curve The work on the 2024 release has already begun and will have even more shiny improvements as well as updated processing chain and data storage infrastructure to provide even better EOxCloudless experience for our customers. Reaching forward and outwards, we would hereby like to invite you to join us at the space events happening around the globe: * [Intergeo](https://www.intergeo.de/en/welcome-to-intergeo) 24 - 26th of September 2024 in Stuttgart, Germany * [Living Planet Symposium 2025](https://lps25.esa.int/) 23 - 27th of June 2025 in our beautiful hometown Vienna, Austria Look for the EOX staff at these event and don't miss out on this opportunity to chat with us and inquire about what EOX can offer you. ## Want to Play a Game? **EO-Guesser** is here! Are a fan of satellite imagery or the world from space? We are running a neat game and at EOX we have been playing it with great enthusiasm for some time now. If you want to try your luck or want to put your Earth Observation knowledge to test feel free to try our [EO-Guesser](https://s2maps.eu/eo-guesser). Being zoomed in on a random place on Earth just able to see the satellite map, can you guess where you are? How conversant are you with planet earth? Enjoy the game! Finally, if you are looking for a nice screensaver to watch while on a coffee break click the "Tour" button at the bottom left of [https://s2maps.eu](https://s2maps.eu/) and enjoy the ride. *** ## License The 2016 version is provided for free under the [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0), the 2018 - 2023 versions under [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NS-SA 4.0). The required attribution is * 2016: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2016 & 2017)*" * 2018: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2017 & 2018)*" * 2019: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2019)*" * 2020: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2020)*" * 2021: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2021)*" * 2022: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2022)*" * 2023: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2023)*" It shall include the links as given here in on-line usage. For commercial usage we [offer the data](https://cloudless.eox.at) under the [EOX Commercial Attribution-RestrictedUse 1.1 License](https://cloudless.eox.at/license-deed) which is based on CC BY. The main change over CC BY is that EOX reserves the right to provide the data with this license. You're entirely free to use it in your services and products, you just can't transfer this license to your users. Usage examples include a WMTS you provide under your name or a product you sell that bundles the data for offline usage. The [EOX WMTS](https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml) and [EOX WMS](https://tiles.maps.eox.at/wms?service=wms\&request=getcapabilities) are free and open to use for non-commercial applications given that when publishing the corresponding [attribution](https://s2maps.eu/#license) is provided in the publication. *** Miriam Mabeya, EOX IT Services GmbH Email: Petr Ševčík, EOX IT Services GmbH Email: Joachim Ungar, EOX IT Services GmbH Email: Contains Modified Copernicus Sentinel-2 Data 2016-2023. --- --- url: /2025/03/sentinel-2-cloudless-2024.md --- *** We are excited to announce the release of EOxCloudless 2024, our latest update to the Sentinel-2 cloudless imagery collection. This year's dataset brings enhanced clarity, improved consistency, and a more seamless visual experience for all your mapping and geospatial needs. Accessing Sentinel-2 data under the Copernicus program has become increasingly challenging for large-scale processes, especially due to its cloud-covered nature and the extensive effort required for cloud removal. EOxCloudless addresses these challenges by providing cloud-free data and streamlining large-scale processing, making the maps easier to use for visualization, academic and scientific research, flight simulator background or as visually pleasing basemaps in your applications. To explore the latest annual mosaic, please visit [s2maps.eu.](https://s2maps.eu) ## A Thorough Look at What's Different in the EOxCloudless 2024 The main improvement was to rewrite and fine-tune the BRDF (Bidirectional Reflectance Distribution Function) correction. A BRDF correction adjusts satellite imagery for variations in surface reflectance caused by differences in viewing and illumination angles. This ensures more consistent and comparable reflectance values across images taken at different times, angles, or sensor positions. Without this correction, the Sentinel-2 data stripes are visible when zooming out the map. Previously, the [HLS Normalization](https://doi.org/10.3390/rs11060632) was used for the whole globe. While in general this improved the visual quality substantially (as can be seen in the difference between [2019](https://eox.at/2020/03/sentinel-2-cloudless-2019/) and [2020](https://eox.at/2021/03/sentinel-2-cloudless-2020/) releases) but still left some visible stripes over tropical regions as well as larger deserts. For our 2024 dataset an approach was adopted based on region and pixel classification increases or decreases the weight of the BRDF correction. This is now prominently visible over the Amazonas as well as the Saraha desert. A further improvement made this year was to review and update the combination of masks used to weed out undesired pixels. Previously most of the regions still relied on the Level-1C masks due to the fact that initially, Level-2A masks did not show satisfying results. Meanwhile, ESA improved their [Sentinel-2 processing pipeline](https://sentiwiki.copernicus.eu/web/document-library#Library-S2-Documents) and EOxCloudless can now benefit from these improvements. Let’s zoom in to a few scenes to explore what has changed since the EOxCloudless 2023. ## Why Buy the EOxCloudless 2024? With the most recent version, you’ll get a cleaner, more accurate, and visually improved satellite imagery dataset, making it more reliable for mapping, analysis, and visualization tasks. Below are some of the compelling reasons why you should update your dataset: 1. **Enhanced Visual Clarity** Significantly reduced striping artifacts, especially in regions like deserts. Maps are looking smoother, cleaner and more professional. There are no more distracting patterns or inconsistencies across different terrains therefore making this a more reliable data. 2. **Maintained Color Accuracy** Colors in our imagery always closely resembles true-to-life landscapes. We have mantained this in this release to ensure that greens, browns, blues, and other natural tones appear as they should providing a more realistic representation of vegetation, water bodies, and urban areas. 3. **Stay Up to Date** Keeping your geospatial data current is crucial, especially in industries like Agriculture or Environmental Monitoring. By upgrading your dataset you ensure that your analysis and decision making are based on the lastest EOxCloudless 2024. ## How to Buy the EOxCloudless 2024? If you are considering satellite imagery, then getting started with EOxCloudless is quite simple. We offer flexible solutions tailored to different needs ensuring you find the right fit for your specific requirements. Whether you need ready-to-use data or a custom solution, we've got you covered: 1. You can purchase our products directly via [cloudless.eox.at](https://cloudless.eox.at/) by clicking **BUY NOW** button. 2. Explore our free samples and documention to ensure our delivery formats match your needs. 3. Need assistance or a custom solution? Reach out to us anytime via: . Get started today and experience the power of cloud-free imagery! ## Looking into the Future The development of the 2025 release is already underway, promising a host of significant enhancements, including an optimized processing chain and upgraded data storage infrastructure. These improvements aim to deliver an even more refined and seamless EOxCloudless experience for our customers. As we look ahead and broaden our horizons, we warmly invite you to join us at upcoming space industry events around the world: * [Living Planet Symposium 2025](https://lps25.esa.int/) 23 - 27th of June 2025 in our beautiful hometown Vienna, EOX as a member of [Austrospace](https://www.austrospace.at/) is helping to organize this great event and we would be thrilled to invite you to Austria for this highlight of the year 2025. We will be engaging with the audience through talks and presentations and we will also be exhibiting. Come, let's collaborate and exchange ideas with us! * [Big Data from Space 2025 (BiDS'25)](https://www.bigdatafromspace2025.org/) 29 September - 3 October 2025 in Riga, Latvia. Look for the EOX staff at these events and don't miss out on this opportunity to chat with us and inquire about what EOX can offer you. Want to schedule a meeting with us? Let us know at in advance. ## Have you tried EO-Guesser? **EO-Guesser** is a game for all who want to challenge their knowledge of Earth. Do you love staring at satellite images? Then [try EO-Guesser.](https://s2maps.eu/eo-guesser) Every day you’ll be placed on a different place on Earth and your goal is to guess the place where you are. Have fun and don’t forget to share your results on social media! Finally, if you are looking for a nice screensaver to watch while on a coffee break, click "Explore" and then the "Tour" button at the bottom left of [https://s2maps.eu](https://s2maps.eu/) and enjoy the ride. *** ## License The 2016 version is provided for free under the [Creative Commons Attribution 4.0 International Public License](https://creativecommons.org/licenses/by/4.0/) (CC BY 4.0), the 2018 - 2024 versions under [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/) (CC BY-NS-SA 4.0). The required attribution is * 2016: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2016 & 2017)*" * 2018: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2017 & 2018)*" * 2019: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2019)*" * 2020: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2020)*" * 2021: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2021)*" * 2022: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2022)*" * 2023: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2023)*" * 2024: "*[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at/) (Contains modified Copernicus Sentinel data 2024)*" It shall include the links as given here in on-line usage. For commercial usage we [offer the data](https://cloudless.eox.at) under the [EOX Commercial Attribution-RestrictedUse 1.1 License](https://cloudless.eox.at/license-deed) which is based on CC BY. The main change over CC BY is that EOX reserves the right to provide the data with this license. You're entirely free to use it in your services and products, you just can't transfer this license to your users. Usage examples include a WMTS you provide under your name or a product you sell that bundles the data for offline usage. The [EOX WMTS](https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml) and [EOX WMS](https://tiles.maps.eox.at/wms?service=wms\&request=getcapabilities) are free and open to use for non-commercial applications given that when publishing the corresponding [attribution](https://s2maps.eu/#license) is provided in the publication. *** Miriam Mabeya, EOX IT Services GmbH Email: Petr Ševčík, EOX IT Services GmbH Email: Joachim Ungar, EOX IT Services GmbH Email: Karolína Lehotská, EOX IT Services GmbH Email: Contains Modified Copernicus Sentinel-2 Data 2016-2024. ## Contact Information Please contact Miriam Mabeya for sales or Petr Sevcik for any technical matters. --- --- url: /2017/03/sentinel-2-cloudless-original-tiles-available.md --- The original GeoTIFF files [used to create](/2017/03/sentinel-2-cloudless/) the [Sentinel-2 cloudless](https://s2maps.eu/) layer are now available on an AWS S3 bucket. The bucket uses the "requester pays" setting which means you need to access it using your AWS credentials. Bucket ID: `eox-s2maps` Region: `eu-central-1` ## Structure Tiles are stored as GeoTIFFs containing three 8 bits bands (red, green, blue) each in a non-projected geographic coordinate system (WGS84, EPSG 4326). Compression used is LZW (lossless), tile shape is 512x512 pixels. Tiles are structured in a tile pyramid scheme based on the [WMTS-simple profile](http://docs.opengeospatial.org/is/13-082r2/13-082r2.html#30) defined by the OGC. To optimize tile numbers, they are combined into 2x2 bigger tiles (metatiles), hence the usual WMTS tile indices (zoom, column, row) don't apply exactly. To identify the correct tiles per zoom level, one must use the columns and row identifier from the zoom level above (z-1). This means, URLs to the tiles are following the schema: `s3://eox-s2maps/tiles/{zoom}/{column}/{row}.tif` Zoom levels available are 7 to 13, where 13 best resembles the original resolution (10 m). ## License Sentinel-2 cloudless by EOX IT Services GmbH is licensed under a [Creative Commons Attribution-ShareAlike 4.0 International License](http://creativecommons.org/licenses/by-sa/4.0/). The attribution shall read "[Sentinel-2 cloudless](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at) (Contains modified Copernicus Sentinel data 2016)" and include the links as shown here where possible. --- --- url: /2017/08/sentinel-2-global-cloudless-mosaic.md --- Good news everyone: The first global, (almost) cloudfree mosaic of the world is released! Have a look at ****. In March during the Sentinel-2B launch we released a cloudless mosaic covering Europe. The initial release helped us to identify weak spots and to overall assess our distributed processing approach using 20 VMs. After this first processing campaign we optimized and streamlined the process and underlying software. These optimizations enabled us to speed up the processing time by around 50%. This layer now is integrated into [EOX::Maps](https://maps.eox.at) and can be used like the Terrain Layer or OSM. ## Method The target was to create the first global cloudfree mosaic using the newly available Sentinel-2 data. The use case is to have a pure visual product to be used for mapping applications as a background layer. The data shall be from one year to find out whether it is feasible to provide an annual mosaic. To get rid of the clouds, a combination of the provided cloud masks and a pixel selection algorithm which attempts to select a pixel without clouds or cloud shadows out of a time series is used. Because the selection happens pixel by pixel, it at least reduces significantly the spatial borders appearing between each Sentinel-2 scene. To have a visually appealing mosaic, it is desired to show the Earth during Summer, when vegetation reaches its annual peak. This of course means having different time spans on the Northern and on the Southern hemisphere. Furthermore, the tropical regions require data from one whole year as they show a high degree of cloud coverage due to the [innertropical convergence zone](https://en.wikipedia.org/wiki/Intertropical_Convergence_Zone), a zone where the trade winds from North and South meet and cause a relatively stable band of clouds around the globe. Therefore, the time spans used differ over the globe. On the Northern hemisphere data from *May 2016 until September 2016* is used, whereas on the Southern hemisphere data from *November 2016 until March 2017* is used. The tropical regions contain data from *May 2016 until April 2017*. For a more technical summary of the processing method, please have a look at the [presentation slides](https://git.osgeo.org/gogs/foss4g-europe/foss4g-europe-2017-paris/src/master/presentations/2017-07-19/general_track/foss4g-europe-2017-A_cloudfree_Europe_with_sentinel2-JUngar-SMeissl.pdf) from the last FOSS4G-Europe in Paris. ## Data We already covered the topics of Sentinel-2 [data structure](/2015/12/understanding-sentinel-2-satellite-data/) and [how to get the data](/2016/02/downloading-sentinel-2-data/) in previous posts. What will change in future though is a better temporal coverage due to the launch of Sentinel-2B in March 17. A better temporal coverage means a higher chance to extract a cloudfree pixel from the time stack which results in a better looking mosaic. Current plans over Asia and the Americas are not providing sufficient temporal coverage for a cloudless quality as can be obtained by our method over Europe and Africa. According to ESA's [acquisition plans](https://sentinel.esa.int/web/sentinel/missions/sentinel-2/acquisition-plans), the revisiting time of one of the two Sentinel-2 satellites will be up to every five days at the equator (even more up to almost daily towards higher latitudes) but at least every ten days with a preference on Europe, Africa and Greenland. The new global cloudfree tiles are also available via an AWS S3 bucket as described [in a previous blog post](/2017/03/sentinel-2-cloudless-original-tiles-available/). ## License Sentinel-2 cloudless by EOX IT Services GmbH is now licensed under a [Creative Commons Attribution 4.0 International License](http://creativecommons.org/licenses/by/4.0/). Note that we dropped the Share-Alike clause although we love to hear from you how you use the data. Feel free to use the provided service endpoints ([WMTS](https://tiles.maps.eox.at/wmts/1.0.0/WMTSCapabilities.xml) or [WMS](https://tiles.maps.eox.at/wms?service=wms\&request=getcapabilities)) directly in your application. You may also download the data either as rendered tiles from these services or as source GeoTIFFs from the AWS S3 bucket eox-s2maps in the eu-central-1 region. Please [get in touch](https://s2maps.eu/#contact) if you need anything else. Whenever you use Sentinel-2 cloudless an attribution has to be provided. It shall read "[Sentinel-2 cloudless - https://s2maps.eu](https://s2maps.eu) by [EOX IT Services GmbH](https://eox.at) (Contains modified Copernicus Sentinel data 2016 & 2017)" including the links as given here where possible. --- --- url: /2018/05/sentinel-2-product-density-maps.md --- Big things have small beginnings, working with Earth observation data is no exception. Providing, distributing, visualizing, interpreting, and much more are only a few activities connected to the topic. It takes numerous steps to create a comprehensible output out of a pile of data. Some of the tasks might even take thousands of hours to complete and sometimes even after such a tremendous effort, projects still might be in their initial phase. So how to deal with the [big data](https://en.wikipedia.org/wiki/Big_data) before you actually start shoveling it one terabyte at a time? For instance, the idea could be to offer a cloudless mosaic of the entire Earth's landmass like our [Sentinel-2 cloudless](https://s2maps.eu/), but the resources are limited and decisions should be based on knowledge to see if the project is at least feasible. Let's look at the example, the  [Sentinel-2 cloudless](https://s2maps.eu/) result is determined by a lot of factors, to simplify this example we will pretend that the processing was done instantly and therefore we are looking solely at the input and the output. The most crucial factors are the source data and its time window. If we look at the map below we can see some cloud artifacts mostly present around the equator, these are caused by the intense evaporation and rainfall in the tropical regions as well as the connection to the [innertropical convergence zone](https://en.wikipedia.org/wiki/Intertropical_Convergence_Zone) (ITZ). Other cloudy scenes are located vastly in East Asia, Tibet Plateau, parts or Siberia, and certain swaths in North America. To a certain degree weather at the moment of the individual acquisitions is the reason for these cloudy scenes outside of the ITZ. Nonetheless, if we take a closer look at the minimaps, we can distinguish the lesser acquisition density for products with less cloud coverage, if we take the other maps in this article into consideration we can say that, even if there are enough cloud-free pixels in other scenes, there is a higher chance that the scenes can be more severely affected by water vapor or light [cirrus clouds](https://en.wikipedia.org/wiki/Cirrus_cloud), which might have been considered as cloud-free in the processing. This practice is not a set law, but we can use this reference as a rule of thumb. *** ### Summary of the Sentinel-2 development until early 2018 A lot has changed since the launch of the [Copernicus](http://www.copernicus.eu/) satellite Sentinel-2A on the 23th of June 2015. Like with most of [Copernicus](http://www.copernicus.eu/) satellite missions, Sentinel-2 is a satellite constellation which consist of a pair of operational satellites. The twin Sentinel-2B joined its sibling on the 7th of March 2017 enhancing the Earth observation capabilities of the mission. The mission is operational since the 3rd of December 2015 and fully operational since 6th of June 2017. Currently the mission is finishing its [ramp up phase](https://sentinel.esa.int/web/sentinel/missions/sentinel-2/operations-ramp-up-phase) to meet the mission requirements. The increase of datavolume can be clearly seen between the years 2016 and 2017. Yet this is not the reason behind increasing the product numbers so drastically, the naming and the distribution structure of the Sentinel-2 products has been [changed](https://earth.esa.int/web/sentinel/user-guides/sentinel-2-msi/naming-convention) since the 6th of December 2016. This leads to publishing single 100x100km "granules" instead of "granule packages". This has lead to a tremendous increase of product entries of the Sentinel-2 data. Up to the middle of March 2018 the Sentinel-2 [Level-2A](https://earth.esa.int/web/sentinel/user-guides/sentinel-2-msi/product-types/level-2a) products are to be considered as preliminary data. Since then the Level-2A became an operational product, beginning with coverage of the Euro-Mediterranean region, with a gradual ramp-up to systematic worldwide coverage planned for the summer of 2018. The Level-2A operational processor generates the output, from algorithms of scene classification and atmospheric correction, Level-2A (Bottom of the Atmosphere - BOA reflectance) products from [Level-1C](https://earth.esa.int/web/sentinel/user-guides/sentinel-2-msi/product-types/level-1c) products. This concludes the description of the properties of the following maps. I hope it has provided some insight of how inhomogeneous the Sentinel-2 data is, based on both the physical properties and the datastructure itself. *** ### Here are some maps, that might help you when dealing with Sentinel-2 satellite data. The maps are sorted out by processing level, cloud coverage for the summer and winter based on the seasons in the northern hemisphere, the coverage is global, therefore applicable for the southern hemisphere analogically. *** #### Sentinel-2 Level-1C density maps for the year 2017 *** *** #### Sentinel-2 Level-2A density maps for the year 2017 *** #### Sentinel-2 Level-1C density maps for the year 2016 *** Thank you for stopping by and if you have any questions, remarks or just want to discuss something write me an email, on the 2nd of May 2018 in Vienna, Austria Petr Ševčík, EOX IT Services GmbH Email: petr.sevcik@eox.at *Contains modified Copernicus Sentinel information 2015 to 2018.* *Maps made in QGIS 2.18.* --- --- url: /service-terms-and-conditions.md --- (version as of April 1st 2022) Download pdf version [here](/content/service_terms_and_conditions.pdf) to save or print this service terms and conditions. ## Section 1 - These EOX Service Terms EOX IT Services GmbH ("EOX") develops, operates, and provides various online services ("EOX Services"), for example hosted JupyterLab instances an online service to provide an interactive computational environment to help exploit Earth Observation data. The term "Service Provider" used below refers to EOX. Access to, and use of, any EOX Service constitutes acceptance of the following EOX Service Terms. ## Section 2 - Use of an EOX Service To use an EOX Service the user is required to sign-up to receive an account either through direct registration or by using a social media provider. The user is required to provide his/her real name and warrants that all data provided are accurate and up-to-date. A user may at any point request cancellation of this account by sending an email to . When using an EOX Service the user shall be obliged to observe the following: a. The user does not have any right or entitlement to be registered for the EOX Service. The Service Provider may refuse registration of the user without giving reasons. b. The user shall comply with all applicable laws when using the EOX Service and not misuse or in any way interfere with the EOX Service, nor shall the user attempt to circumvent or bypass any access restriction implemented on the EOX Service. c. The user shall utilize any data provided on the EOX Service exclusively in accordance with the provisions of the respective data terms and license. All other rights in relation to the EOX Service or any content displayed on the EOX Service, which are not explicitly granted to the user under the respective data terms and license or hereunder, are reserved by the Service Provider or their suppliers. d. The user shall not delete any author attributions, legal notices, proprietary designations or labels from any file that is downloaded from the EOX Service nor falsify the origin or source of software or other material contained in a file that is downloaded. e. The user is solely responsible for keeping the user account and login-data secure and accepts to be fully liable for all activities occurring under that account. The user will promptly notify the Service Provider, if the user becomes aware of any unauthorized use of, or access to, the EOX Service through the user's account, including any unauthorized use of the user's password or account. f. Any use of the EOX Service under a free or promotional plan shall be strictly limited to non-commercial purposes. In particular, the free EOX Service shall not be used to create and/or provide commercial products or services. In the case of an intended commercial use of the free EOX Service special permission by the Service Provider is required (please contact: ). g. The user shall be solely responsible for the content of, and for any harm resulting from, any data posted, uploaded, linked to or otherwise made available via the EOX Service ("User-Generated Content"). In particular, the user warrants that such user data does not contain harmful components, violate any applicable laws or infringe third-party rights. The Service Provider disclaims all responsibility and liability for User-Generated Content made available on the EOX Service. h. The Service Provider does not pre-screen User-Generated Content but retains the right to access and review such content at its sole discretion (e.g. for maintenance purposes or in the case of irregularities). The Service Provider reserves the right - though without accepting any obligation - to refuse or remove any User-Generated Content that, in the Service Provider’s sole discretion, violates these Terms or otherwise threatens the security, integrity or availability of the EOX Service. A violation of the foregoing, abuse (in particular through unlawful use), excessively frequent requests of, or uploads to, the EOX Service, or any other use of the EOX Service that in the Service Provider's judgment threatens the security, integrity, or availability of the EOX Service may result in the temporary or permanent suspension of the user account, in individual cases, it may also entitle the Service Provider to claim for damages. The Service Provider, in its sole discretion, will determine unlawful, abusive, excessive, or otherwise inappropriate usage. ## Section 3 - License for User-Generated Content on the EOX Service The user retains ownership of his/her User-Generated Content. The user grants the Service Provider and its legal successors the right to store, parse, and display User-Generated Content and make incidental copies as necessary to provide the EOX Service including storing the content in own data bases, creating backups, parsing it into a search index, or otherwise analyzing it on own servers and executing it. The license does not grant the Service Provider the right to sell User-Generated Content or otherwise distribute it, unsolicitedly display it, or share it with other users, or use it outside of the provision of the EOX Service. The user will not receive any remuneration for any of the rights granted hereunder and the license will end when the user removes the content from the Service Provider’s servers. The user retains all moral rights to the User-Generated Content uploaded, published, or submitted to any part of the EOX Service, including the rights of integrity and attribution. However, the user waives these rights and agrees not to assert them against the Service Provider, to enable the Service Provider to reasonably exercise the rights granted hereunder, but not otherwise. To the extent this agreement is not enforceable by applicable law, the user grants the Service Provider the rights needed to use the User-Generated Content without attribution and to make reasonable adaptations of User-Generated Content as necessary to provide the EOX Service. By making available User-Generated Content on the EOX Service the user grants each user of the EOX Service a nonexclusive, worldwide license to - exclusively for non-commercial research purposes - use, display, and perform the content through the EOX Service and to reproduce the content solely on the EOX Service as permitted through EOX Service's functionality. If the user is uploading content that the user did not create or owns, the user is responsible for ensuring that the content is licensed under terms that grant these permissions to other users of the EOX Service. ## Section 4 - User Feedback To the extent the user provides any comments, ideas, changes, or other feedback to the Service Provider regarding the EOX Service, the user grants the Service Provider a perpetual, irrevocable, worldwide, non-exclusive, transferable, sublicensable, royalty-free, fully paid-up right to use and commercially exploit such information in any manner the Service Provider deems fit. ## Section 5 - Disclaimer of Liability, Warranties, and Indemnification Please note that users browse and use the EOX Service at their own risk. Neither the Service Provider nor any other party involved in creating, producing or delivering the EOX Service shall be liable for any direct, incidental, consequential, indirect, or punitive damages arising out of Users' access to, or use of, the EOX Service. To the extent required by mandatory law, this limitation of liability shall not apply in case of intent or blatant gross negligence of the Service Provider or in the event of personal injury. Though the Service Provider uses the best technical standards to control its EOX Service, material on this website may inadvertently include technical inaccuracies or other errors. The Service Provider makes no warranties, whether express, implied, statutory, or otherwise, as to the uninterrupted or error-free availability of the EOX Service, fitness of the EOX Service for any particular requirements or purposes of the User, merchantability, non-infringement of third party rights or the absence of viruses or other harmful components. Users are obliged to notify the Service Provider of any harmful components, errors, or inaccuracies they may find. The User shall hold the Service Provider harmless from and assist in the defense against any and all claims and demands raised by third parties, including associated expenses (particularly costs of appropriate legal representation and/or experts' fees), insofar as the claims/demands can be attributed to the User's access to, or use of, the EOX Service. The Service Provider reserves the right to update, modify, or reconfigure the EOX Service or change material on the EOX Service at any time without prior notice. ## Section 6 - Access and Availability of EOX Service and Links EOX reserves the right to leverage dynamically provisioned cloud resources to offer its services. This start-up lead time (e.g. spinning up a GPU node for JupyterLab, preparing a target environment for job execution) doesn't count towards availability measurement. If not covered by a custom agreement all commercial EOX services will be available 99% of the time every month and this Service Level Agreement is the sole and exclusive remedy for unavailability of our services. If EOX does not meet this service level guarantee Service Credits (as described below) will be granted. EOX services are considered unavailable when functionality covered by a commercial agreement is inaccessible from any location. A period of unavailability is excluded from the service level guarantee if the unavailability * is due to scheduled maintenance, * is caused by factors outside of EOX reasonable control (e.g. force majeure event, internet access problems, etc.), * is a result of missing data if provided through a third party, * is based on a suspended account due to violations of service agreements, the exceeding of consumption quotas or outstanding payment obligations. Service Credits are calculated as a percentage of the aliquote monthly service fee against the overall monthly availability as follows: | Availability per month | Service Credits | |-|-| | > 99% | 0% | | 95% - 99% | 10% | | < 95% | 20% | To receive Service Credits EOX must be contacted via email within 30 days following the end of the unavailability, stating affected functionality as well as timerange of unavailability. If confirmed and the uptime percentage in a month is below 99% the Service Credits will be added to the end of the term for the service, and cannot be exchanged for or converted to monetary compensation. On top of these general agreements EOX provides Service Level Objectives for specific individual services, e.g. headless processing jobs with an overall execution duration <24h are guaranteed to run non-interrupted 99% of the time, while for processing jobs executed interactively through hosted JupyterLab instances no promises on interruptions are given. The EOX Service may contain links and references to third-party websites. These are provided for the convenience and interest of the user and on the part of the Service Provider this implies neither responsibility for, nor approval of, information contained in these websites. The Service Provider makes no warranty, either express or implied, as to the accuracy, availability or content of information, text or graphics on third-party websites. The Service Provider has not tested any software located on other sites and does not make any representation as to the quality, safety, reliability, or suitability of such software. ## Section 7 - Severability, Jurisdiction, and Applicable Law If any part of these EOX Service Terms is held invalid or unenforceable, that portion of the EOX Service Terms will be construed to reflect the parties’ original intent. The remaining portions will remain in full force and effect. Any failure on the part of the Service Provider to enforce any provision of these EOX Service Terms will not be considered a waiver of the right to enforce such provision. The rights of the Service Provider under these EOX Service Terms will survive any termination of the contractual relationship or deletion of the user account. All disputes between the user and the Service Provider including the question of the valid conclusion of these EOX Service Terms as well as its preliminary and subsequent contractual effects, shall be exclusively settled by the court having jurisdiction over the subject matter in Vienna, Austria. The Service Provider however may sue at any other competent court. Except to the extent applicable law provides otherwise, these EOX Service Terms and any access to or use of the Service shall be governed by, construed and enforced in accordance with the substantive laws of Austria excluding application of its conflict of laws provisions as well of the Convention on International Sales of Goods (CISG). --- --- url: /_team/paris.md description: >- One brain hemisphere for music and computing, one for travel, international culture and organization, both for meeting challenges @eox --- --- --- url: /solutions.md --- #### Products & Services Get the geospatial data, process them and visualize them using our solutions. #### Open Source Tools We actively develop and contribute to Open Source Software and frameworks for the geospatial community. See our GitHub organization for more details. --- --- url: /_team/madhavans.md description: >- Assisting startups over 7+ years in software engineering and development. A cricket fanatic, political observer, art admirer, financial market explorer, regional Indian movie buff, loves riding motorcycles & an amateur designer. --- --- --- url: /_team/achtsnits.md description: >- pragmatist with > 15 years enterprise software experience seeking to constantly challenge the not-invented-here-syndrome --- --- --- url: /_team/brands.md description: >- Loved by Christ, blessed by the Father. Never satisfied with superficial answers. Youth ministry leader. Reading, cooking, (train) traveling, board games, jigsaw puzzles, pilgrimages. --- --- --- url: /_team/meissls.md description: >- Free and open source software, open data, maps, space, entrepreneurish, at @eox_a, skiing, climbing, occasionally Sherlock. --- --- --- url: /2015/08/sx-cat-part-one-the-static-map-catalogue.md --- I'm excited to introduce you to our newest software development: the Simple Online and the Static Map Catalogue (short: SO-Cat and SM-Cat; even shorter: Sx-Cat). In this post I will focus on the Static Map Catalogue, at a later time I will also show the Simple Online Catalogue. The idea behind the Static Maps is to generate a simple map view of an existing archive, which can then be shipped and deployed independently, requiring nothing more than a simple web-server. This is how it looks (© ESA): [![](/images/smcat-intro.jpg)](https://landsat-ds.eo.esa.int/smcat/LandsatTMCloudFreeCoverage/) As you can see, the static maps consist of a simple web-view that can be navigated by clicking into the rectangular areas of the images that contain data. Areas that do not contain images are not selectable. The map can be navigated for three layers, at the end of which there is a result list of datasets in that area. This is slightly different for WRS and non-WRS collections, but more on that in a minute. The actual "map-widget" of the generated maps might look a bit anachronistic and inflexible at first, but that is done on purpose: no additional web services are required, making the static maps *robust* and *fast* but still allow a quick and easy navigation on the map. If you don't believe me, [try for yourself](https://landsat-ds.eo.esa.int/smcat/LandsatTMCloudFreeCoverage/)! The static maps come in two flavors: the ones with an underlying [World Reference System (WRS)](http://landsat.usgs.gov/worldwide_reference_system_WRS.php) and the ones without. The difference between those two is actually only visible on the deepest map level: the WRS type maps allow to select single WRS points (or associated footprints of those points) whereas non-WRS maps stick to the good old rectangles for the last level. When we say WRS, we are not specific to the Landsat WRS systems: Sx-Cat supports all similar systems such as track/frame, or K-J system of the SPOT missions. The last "level" of the static map is actually a list of scenes for that WRS coordinate or ultimate rectangle (in non-WRS maps). This list displays a configurable subset of the scenes metadata and also provides URLs for the full metadata, browse images and the complete product. For convenience, each page of a static map provides a list of download links for *all* scenes contained in the current view. This can be used with browser plugins like [DownThemAll!](https://addons.mozilla.org/en-US/firefox/addon/downthemall/) to easily download all products of interest. The look and feel of the generated static maps depend on the configuration, and there is a lot of possible configuration options: the grid size, the actually rendered features (any combination of center point, footprint and browse), style and color of center point and footprint, etc. Each of those values can be set per level, allowing a really fine-tuned configuration for the desired maps. Since static maps can get really crowded, we implemented two distinct methods of selecting WRS points on the last map level: by footprint and by scene centre. The generation of the static maps is pretty fast: for example the Landsat ETM Cloudfree map was generated in roughly 10 minutes (not counting the download times) including 638 records on 473 distinct WRS locations. This is due to a number of facts: the underlying search engine and the efficient caching of browse images which are downloaded and reprojected just once. When no browses shall be rendered (just footprints or scene centers), the rendering is even faster by magnitudes. Regarding the search engine, I will go into detail when I demonstrate the SO-Cat in a later post. As an additional bonus, all the scenes that contribute to a map are also sorted according to their cloud coverage value, when available. This results in mosaicked maps that are not only practical but also nice to look at (also on mobile devices!): Map enthusiasts might have noticed the terrain background map, which is taken from our [EOX :: Maps](https://maps.eox.at/) project. Both Static Map and Simple Online Catalogue developments are part of the Payload Data Ground Segment (PDGS) Evolution programme of the European Space Agency for Earth Observation satellite missions, that aims to improve its overall PDGS capabilities and to allow the storage and dissemination of legacy missions as well as new and upcoming missions such as Sentinel-2. The role of SO-Cat and SM-Cat is to provide a lightweight catalogue service for visual exploration and metadata dissemination. There are [already](https://tpm-ds.eo.esa.int/smcat/Spot1-5_ESA/) [several](https://landsat-ds.eo.esa.int/smcat/LandsatTMCloudFreeCoverage/) [instances](https://landsat-ds.eo.esa.int/smcat/LandsatETMCloudFreeCoverage/) of SM-Cat in operational use. I hope that I could give you a good overview of the Static Map functionality of our new software! Stay tuned for part 2: the Simple Online Catalogue! --- --- url: /tag/3d.md --- --- --- url: /tag/animation.md --- --- --- url: /tag/cesium.md --- --- --- url: /tag/earth-observation.md --- --- --- url: /tag/globe.md --- --- --- url: /tag/mapalupa.md --- --- --- url: /tag/nasa.md --- --- --- url: /tag/ndvi.md --- --- --- url: /tag/vegetation.md --- --- --- url: /tag/video.md --- --- --- url: /tag/esa.md --- --- --- url: /tag/plotty.md --- --- --- url: /tag/graphly.md --- --- --- url: /tag/geotiff-js.md --- --- --- url: /tag/atmosphere.md --- --- --- url: /tag/eoxmaps.md --- --- --- url: /tag/eoxcloudless.md --- --- --- url: /tag/maps.md --- --- --- url: /tag/sentinel-2.md --- --- --- url: /tag/guest.md --- --- --- url: /tag/catalog.md --- --- --- url: /tag/eo-data.md --- --- --- url: /tag/landsat-8.md --- --- --- url: /tag/open-data.md --- --- --- url: /tag/satellite-data.md --- --- --- url: /tag/data-access.md --- --- --- url: /tag/swarm.md --- --- --- url: /tag/vires.md --- --- --- url: /tag/eoxhub.md --- --- --- url: /tag/eurodatacube.md --- --- --- url: /tag/processing.md --- --- --- url: /tag/client.md --- --- --- url: /tag/code-de.md --- --- --- url: /tag/dlr.md --- --- --- url: /tag/opensearch.md --- --- --- url: /tag/covid-19.md --- --- --- url: /tag/course.md --- --- --- url: /tag/open science.md --- --- --- url: /tag/labels.md --- --- --- url: /tag/natural-earth.md --- --- --- url: /tag/openstreetmap.md --- --- --- url: /tag/voronoi-diagram.md --- --- --- url: /tag/ml.md --- --- --- url: /tag/data.md --- --- --- url: /tag/data-cubes.md --- --- --- url: /tag/cloud-processing.md --- --- --- url: /tag/s2cloudless.md --- --- --- url: /tag/agri.md --- --- --- url: /tag/cap.md --- --- --- url: /tag/mundi.md --- --- --- url: /tag/sentinel-1.md --- --- --- url: /tag/ai.md --- --- --- url: /tag/ard.md --- --- --- url: /tag/card.md --- --- --- url: /tag/satellite.md --- --- --- url: /tag/vector-tiles.md --- --- --- url: /tag/eoxserver.md --- --- --- url: /tag/visualization.md --- --- --- url: /tag/landsat.md --- --- --- url: /tag/smcat.md --- --- --- url: /tag/spot.md --- --- --- url: /tag/cog.md --- --- --- url: /tag/geotiff.md --- --- --- url: /tag/tiff.md --- --- --- url: /tag/django.md --- --- --- url: /tag/lpvis.md --- --- --- url: /tag/openlayers.md --- --- --- url: /tag/vue-js.md --- --- --- url: /tag/cloudless-mosaic.md --- --- --- url: /tag/eoxc.md --- --- --- url: /tag/foss4g.md --- --- --- url: /tag/agriculture.md --- --- --- url: /tag/eodashboard.md --- --- --- url: /tag/agriapp.md --- --- --- url: /tag/mapchete.md --- --- --- url: /tag/eoxploration.md --- --- --- url: /tag/sailing.md --- --- --- url: /tag/geomagnetic-field.md --- --- --- url: /tag/earthcode.md --- --- --- url: /tag/osc.md --- --- --- url: /tag/data visualization.md --- --- --- url: /tag/dashboard.md --- --- --- url: /tag/cams.md --- --- --- url: /tag/sentinel-5p.md --- --- --- url: /tag/pdgs.md --- --- --- url: /tag/sxcat.md --- --- --- url: /tag/frameworks.md --- --- --- url: /tag/javascript.md --- --- --- url: /tag/data-usage.md --- --- --- url: /tag/top-5.md --- --- --- url: /tag/copernicus.md --- --- --- url: /tag/eox.md --- --- --- url: /tag/jupyter.md --- --- --- url: /tag/models.md --- --- --- url: /tag/python.md --- --- --- url: /tag/leaflet.md --- --- --- url: /tag/co2.md --- --- --- url: /tag/aeolus.md --- --- --- url: /tag/earth-explorer.md --- --- --- url: /tag/workshop.md --- --- --- url: /tag/vre.md --- --- --- url: /tag/d3.md --- --- --- url: /tag/ogc.md --- --- --- url: /tag/webgl.md --- --- --- url: /tag/documentary.md --- --- --- url: /tag/habitat.md --- --- --- url: /tag/movie.md --- --- --- url: /tag/nature.md --- --- --- url: /tag/satellite imagery.md --- --- --- url: /tag.md --- --- --- url: /2021/03/amida-data-triangle.md --- We are about to finalize an ESA project called AMiDA, and thus I think it is a good moment to recapitulate and look back at what things we have achieved and explored technology-wise during this project. Just a very quick introduction, the project was lead by [SISTEMA](http://www.sistema.at/) and participating in the activity was [ZAMG](https://www.zamg.ac.at/), [cloudflight](https://www.cloudflight.io/), as well as us (EOX). The scope of the project was to provide entities active in the atmospheric domain in Austria a platform to handle day-to-day work in an efficient way. Five use cases were conducted where, apart from cloudflight and ZAMG, [LuftBlick OG](http://luftblick.at/) and the Solar Radiation and Air Quality group ([Institute for Biomedical Physics](https://www.i-med.ac.at/dpmp/bmp/) - Medical University of Innsbruck) participated. Our part in this project was the web browser data visualization module (called DAVE). There are many very interesting aspects to this project, related for example to processing. However, I will be focusing on technologies we used and applied. ## Data visualization on a web client When developing the visualization component running in a web browser a great deal of consideration has to be taken to decide where the actual rendering should take place. It is not easy to find the best balance: You can do your rendering (image/raster creation) on the server or (thanks to modern browsers) directly on the web browser. Each approach has its pros and cons. In AMiDA we focused on the "data triangle", i.e. data coming from ground stations, satellite data and model data. The processing aspect previously mentioned had the main task of facilitating comparison and correlation of these datasets (but more on that maybe for another time). As for the web client visualization we decided it would be a good approach to do the rendering completely on the client (web browser). This allowed us to use multiple tools in the client to access and manipulate the data from these three different sources. Having the actual values on the client allows quick manipulation of the data, its visualization as well as comparison of actual values through picking, for any location loaded. This provides very quick interactivity once the data is loaded, but also means longer loading times and high load on the resources of the user's machine. ## Tools used To load raster data to the client we used a [Web Coverage Service (WCS)](https://www.ogc.org/standards/wcs) interface. This allowed to retrieve GeoTIFFs for the relevant areas and time selected by the user. We can then read the data with help of [geotiff.js](https://github.com/constantinius/geotiff.js) and render it using [plotty.js](https://github.com/santilland/plotty). We already presented these tools a while back in the [Visualizing Raster Data](https://eox.at/2016/02/visualizing-raster-data/) blog post. It is great that we were able to incorporate these two tools in new projects allowing us to keep evolving and improving them through time. Additional to both these tools we also integrated [graphly](https://github.com/EOX-A/graphly) to allow quick and efficient plotting of data points pierced through the time stacks of data as well as ground station measurements. Although there is a really large selection of good plotting tools, it was quite difficult to find one that really fit our needs. In other services such as [VirES for Swarm](https://vires.services/) or [VirES for Aeolus](http://aeolus.services/) we needed very good performance to render multiple thousands of points, as well as interactivity, and configurability. Some solutions fit in some aspects, but changing the aspects that do not fit imposed quite a big challenge, which would mean possibly changing core concepts of the selected tool. Thus graphly was born, initially as a high performing plotting tool for scatter plots where the rendering is done in WebGL. With the help of logic built into the pixel shader it has evolved into a more capable configurable plotting tool. This allows for—among other things—multiple stacked synchronized plots which are highly customizable. In the animation above you can see a comparison between Sentinel 5P NO2 total column data and NO2 total column measurements from a ground-based Pandora station. The DAVE platform implements also a modifier expression field which allows application of basic arithmetic functions on the retrieved data. In the example shown this is important because the sources use different units of measurement. Having the data on the client allows quick manipulation through the function to convert the unit of measurement into a comparable unit. ## Lessons learned We learned from user feedback that the functionality provided by the DAVE component is very much appreciated and overall described as an ideal tool to explore and browse the data to find interesting atmospheric events. Through the possibility of animations of the models and picking through time also the temporal evolution can be explored very intuitively. The downside is that some users experienced the loading time for larger time periods to be too long, as well as some crashes due to loading large amounts of data. It is very difficult to implement safeguards to limit the access to data to fit the available resources on the user's machine. This is because from within the web browser you can't really find out, for example, available RAM in the system. Reviewing the different approaches users take to look at the data we think a possible middle ground solution is to better manage different "levels of detail" for the data. It would be possible to dynamically change the detail requested for the data depending on how much data is requested, i.e. how large the area of interest is as well as the time period of interest. Still this would need to be communicated to the user in an easy-to-understand way. All in all we are very happy with the results and hope we will be able to further integrate our experiences and evolve our technologies in future endeavors. --- --- url: /2020/06/the-joy-of-using-leaflet-openlayers-with-vue-js.md --- ## Introduction Webapps are becoming more and more complex, and the user interface requirements towards reactivity and snappiness are becoming more and more state of the art. In such an environment, it is helpful or even necessary to find helpers for all kind of things, to be able to prototype more quickly, to make code more modular and scalable, and better maintainable. One of these helpers are the modern JavaScript frameworks. ## Why use modern JavaScript frameworks? Modern JavaScript frameworks like Angular, React or Vue.js are hot nowadays! You have to look hard to find front-end dev job descriptions that don’t mention them, and there are cult-like fandoms that argue about which JavaScript framework religion one does belong to. But on the same time, if you’re a beginner doing your first steps, it might seem like a complex and time-consuming task to understand what this bunch of dependencies, code structures and strange syntax really does - and why you even need it! What comes to mind when you think about a JavaScript framework? Well, it helps with structure and reduces development time, helps maintaining your code by splitting it up in modular components and providing rendering templates, speeds up your workflow by providing powerful third party libraries and is generally the go-to method to write single page applications. These facts are all nice and handy, but they are missing the crucial benefit a modern JavaScript framework provides: **Keeping the User Interface in Sync with the State!** In the current day and age, users expect more and more fluid, easy to use and feature-packed user interfaces. You have elements of the page reacting to changes that are triggered by user input all the way at the other end of the page, buttons changing text while loading, hint texts providing feedback to users, automatically self-updating items and net sums in checkout flows, lists dynamically filling after API requests, drag-and-drop-and-move-and-delete-and-change… If you have some experience with such workflows, this may sound like a nightmare to you - keeping track of the first one or two UI changes is pretty easy, but afterwards, it becomes messy. It will inevitably happen that the user, after clicking multiple things, eventually ends up with an interface that displays something completely different than the code running in the browser itself “thinks” it is displaying. Let’s illustrate this with an example: Let’s say we have an input field where the user can add a to-do item, which then gets added to a to-do list. Here is approximately how the (simplified) adding function would look like in HTML + Vanilla JavaScript: ```md
    ``` This is far from complete, we didn’t even get started with event listeners and preparing classes to apply styling to these elements! The problem with this approach is not only that it is painful to write and read, but it is also extremely fragile. Especially when you add interaction with a server, things get out of hand real fast, and with any minimal mistake, the UI will be out of sync from the actual data! This means missing information, showing wrong information, or completely messed up with elements not responding to the user or even worse, triggering wrong actions (e.g. clicking on a delete button deletes the wrong item). But what about jQuery, you might ask? jQuery is still used a lot nowadays for DOM manipulation, but it really excels at one thing: querying the DOM and addressing its objects. But since it is so powerful and quick to use, it doesn’t solve the actual mentioned problem, it just helps you achieve a big mess faster than with Vanilla JavaScript… Modern frameworks to the rescue! Let’s look at how the same code example would look like in Vue.js: ```md ``` Nice! What has happened here? We have created the to-do list structure with a HTML template that uses dynamic parts like {{ todo.title }}, and where we loop through all the to-dos in the todo array and fill the dynamic fields accordingly. But there is something missing, you might ask! After adding a new to-do, how do I update the UI? Well, my friend, you don’t have to. Vue.js is here for you. Just push to the array, and the UI automagically updates! Other way round works too: just check the checkbox, the todo item in the array will update as well. It’s almost too good to be true! By the way, this powerful simplicity is one of the reasons we here at EOX are leaning towards Vue.js instead of other JavaScript frameworks. Although Angular (by Google, the oldest of the bunch) and React (by Facebook, the most used) still have far more users than Vue.js, the extremely motivated open source community behind Vue, the small bundle size, the ease of use and the clean syntax gives it an edge over the competitors. The use of Vue by respectable companies such as Alibaba and Gitlab and their positive experience gives us even more reasons to do so (see Gitlabs posts on [why they choose Vue.js](https://about.gitlab.com/blog/2016/10/20/why-we-chose-vue/) over the other frameworks, and [their evaluation one year later](https://about.gitlab.com/blog/2017/11/09/gitlab-vue-one-year-later/)). Plus, we can look forward to Vue 3, which brings even better performance and even more features! ## Map libraries and Vue.js So, how does these concepts translate to the EO world and the usage of interactive maps? For the following examples I am using OpenLayers, since it brings powerful features, especially for raster data, but the same concepts apply to Leaflet. You can use both mapping libraries in a “vanilla” way, but both have (to various degrees complete) wrappers that use the power of Vue.js from the start. You can check out [VueLayers](https://vuelayers.github.io/#/) (used in the example below), [vue-openlayers](https://www.npmjs.com/package/vue-openlayers) or [wegue](https://github.com/meggsimum/wegue) for OpenLayers wrappers, or [Vue Leaflet](https://vue2-leaflet.netlify.app/) and others for Leaflet wrappers. Let’s see how a basic map setup looks like, including an OpenStreetMap (OSM) tile layer: ```md ``` Pretty sweet! What’s cool about this: same as the to-do example above, the UI (in this case the map) stays in sync with the state thanks to Vue.js. So if you were to programmatically change the zoom value in the data function, the map would instantly react. The same goes the other way round, when you zoom inside the map, the zoom value in the data function will update with the new zoom value accordingly. It’s really cool to “always know what’s going on” in the UI! Let’s see how looping through layers can look like within a template. For this example, we want to create a tile layer for each configuration in the “baseLayer” array, and for the overlay layers (in the “overlayLayers” array), we want to configure the sources to display our EOX layers (Sentinel-2 cloudless 2019, Terrain Light, Overlay Bright), we want the visibility to be handled programmatically, and we are mixing WMTS sources and XYZ sources, so the layer should be rendered accordingly. This would already be a quite complex task with vanilla OpenLayers: ```md ``` Awesome! Let’s have a look how rendering of vectors from GeoJSON features and interactions would look like. As we have stated, with Vanilla JavaScript it’s always a pain to track user interaction and keep it in sync with the state. Here, it’s as simple as in the to-do example above, by syncing the UI with an array in the data function: ```md [...] data: () => ({ zoom: 5, center: [13, 43], rotation: 0, selectedFeatures: [], }); ``` Sweet! We have created a feature vector layer that syncs the features from our GeoJSON loaded by the computed property “getFeatures” (as soon as a feature changes, it is synced to the map), and we added some simple styling of the features. Then, we have an interaction layer that registers user interactions with the map features and syncs the “select” interaction with the selectedFeatures array, meaning, every time the user selects a feature on the map it is instantly pushed to the array, and if the user deselects it, it is removed. Of course it also works the other way around, if you add a feature to the array, the user instantly sees it as selected on the map. ## Conclusion We could go on with many more details, but the best way to explore all the different benefits of using a mapping library like OpenLayers or Leaflet with a modern JavaScript framework like Vue.js is to dig through the countless code examples and demo pages of the wrappers: A recent example implementation of the concepts discussed in this article is [eodash](https://github.com/eurodatacube/eodash), EOX's dashboard solution powering the [Rapid Action on Coronavirus and EO](https://race.esa.int) Dashboard for ESA and the European Commission, and the [Earth Observing Dashboard](https://eodashboard.org/) for ESA, NASA and JAXA. Vue Leaflet is used with various plugins for all its maps, which allowed us to bootstrap and publish this application within an extremely short time span. Read more about the dashboard(s) in [our other blog post](/2020/06/race-dashboard/). Did I convince you to try out a modern JavaScript framework for your next web app? *** References: * * * * [https://medium.com/@TechMagic/reactjs-vs-angular5-vs-vue-js-what-to-choose-in-2018-b91e028fa91dhttps://medium.com/@TechMagic/reactjs-vs-angular5-vs-vue-js-what-to-choose-in-2018-b91e028fa91d](https://medium.com/@TechMagic/reactjs-vs-angular5-vs-vue-js-what-to-choose-in-2018-b91e028fa91d) --- --- url: /2025/02/wild-whiskers-eox-hamsters-guest-blog-post.md --- *** The common hamster, once a familiar sight in European fields, is now teetering on the brink of extinction. Nature filmmaker and ecologist [David Cebulla's](https://www.davidcebulla.de/) documentary, ["The Last Common Hamsters" ("Die Letzten Feldhamster")](https://www.feldhamster.eu/en/film-the-last-common-hamsters/), explores this crisis. The documentary aired on the 17th of January 2025 at [David Cebulla’s YouTube channel](https://www.youtube.com/@davidcebulla). ## Looking for The Common Hamsters The film combines three years of fieldwork in central Germany with excursions to Hungary, France, Poland, and Austria, creating a portrait of this declining species. The team even journeyed into the Kazakh steppe to capture the first video record of a common hamster in its original steppe habitat. The observations and warnings in the documentary can be found in other literature as well for instance in the paper analysing the population developement of common hamster in Alsace, France: [*"Saving the common hamster (Cricetus cricetus) from extinction in Alsace (France): potential flagship conservation or an exercise in futility?", John O'Brien, (2015)*](http://dx.doi.org/10.4404/hystrix-26.2-11230). This particular area has been visited during the production of the documentary and gives the insight on how the sitiuation for the common hamster there is today. Sadly this is one of the examples of rather unfortunate developement and the struggle to keep the species in this region alive. The on-site interviews with local farmers and officials provide a comprehensive view of the challenges and potential solutions for common hamster conservation. These challenges notably include habitat loss and fragmentation due to agricultural practices, pesticide use that directly harms hamsters and reduces their food sources, and the overall simplification of agricultural landscapes. While the EU's Common Agricultural Policy (CAP) aims to support farmers and promote sustainable agriculture, some of its historical implementations have indirectly led to practices detrimental to biodiversity. However, the CAP is evolving, with increasing emphasis on biodiversity conservation and environmental protection through measures like agri-environmental schemes. These schemes encourage farmers to adopt practices that benefit wildlife, such as maintaining field margins, reducing pesticide use, and diversifying crops. These changes in agricultural landscapes, both past and present, have significantly affected the hamsters by reducing food sources and disrupting their natural burrows and breeding grounds, pushing the species toward extinction. ## Zooming in on Extinction: Using Satellite Imagery to Track Last Common Hamsters By using high-resolution satellite images from Sentinel-2, provided by EOX, David Cebulla created stunning animated maps visualizing the decline of the common hamster over the past century. Compiling the latest data, he produced a distribution map of the species as of summer 2024, revealing a 75% reduction in its European range since the mid-20th century. A separate animation showed the distribution of distinct genetic lineages within the species (see the banner image at the top of this article), offering insights into the hamster's evolutionary history. All data was meticulously referenced using QGIS to ensure accuracy before being integrated into the final animations. ## Small Species under Large Threat "The Last Common Hamsters" is a call to action. By combining science and storytelling, David Cebulla has created a tool for raising awareness about the plight of the common hamster. As we face a biodiversity crisis, it is important to find ways to monitor and protect vulnerable species. To raise awareness and support the protection of the common hamster, the filmmakers have made the film available free of charge. EOX provided free licenses for high-resolution satellite imagery, which helped to shed light on the need for conservation efforts. EOX thanks David Cebulla and his team for their dedicated work in raising awareness on this topic through their commitment to the conservation of species and their habitats. If you liked this excursion into the world of these little mammals, feel free to explore more: * [The Last Common Hamsters - Trailer](https://www.youtube.com/watch?v=R1E1wcmwxmw) * [The Last Common Hamsters - Full Documentary](https://www.youtube.com/watch?v=ajawWDkMIrw) * [David Cebulla Youtube Channel](https://www.youtube.com/@davidcebulla) * [The Last Common Hamster Website](https://www.feldhamster.eu/) * [David Cebula Naturfilmer & Ökologe Website](https://www.davidcebulla.de/) EOxCloudless: * [EOxCloudless - Sentinel-2 Cloudless Map View](https://s2maps.eu/) * [EOxCloudless Website](https://cloudless.eox.at) Attribution: * *Sentinel-2 cloudless - [https://s2maps.eu](https://s2maps.eu/) by EOX IT Services GmbH (Contains modified Copernicus Sentinel data 2023)* * *[Wild Whiskers Media – Cebulla & Moldenhauer GbR Impressum](https://www.feldhamster.eu/datenschutzerklaerung/)* *** David Cebulla, Wild Whiskers Media Email: Heide Moldenhauer, Wild Whiskers Media Email: Miriam Mabeya, EOX IT Services GmbH Email: Petr Ševčík, EOX IT Services GmbH Email: --- --- url: /2015/09/top5.md --- In the following I'm going to summarize and present my personal view on current trends in today's access and usage of Earth Observation (EO) data. As this is a subjective analysis I invite you, dear reader, to share your thoughts with me, best on Twitter. I might be convinced to write a follow-up post, although this one is going to be quite long anyway ;) ## tl;dr My personal top 5 trends in EO data access and usage are: * simple apps for single use cases are popular * apps are browser based and even run on mobile devices * more and more EO data is released as Open Data * the map becomes the central user interface element * there are more commercial opportunities also for smaller companies Let me know your top trends, best via Twitter, or read on to see examples of these trends and a more detailed analysis. ## User Interface Comparison Let me start by showing and comparing several user interfaces of apps for obtaining EO data. The comparison and analysis uses the freely available [Landsat data](http://landsat.usgs.gov/) and the images show this data whenever possible. Only apps providing access to or using Landsat data have been selected in order to prepare for the Sentinel missions, particularly for Sentinel 2, which also offer the data as Open Data like Landsat. Of course the list is far from being complete and I'd be interested to know which one you would add? Lets begin with some long-lived apps that have survived quite some Internet hypes, in particular ESA's good old **[EOLi-SA](https://earth.esa.int/web/guest/eoli)**, short for Earth Observation Link - Stand Alone. EOLi-SA is basically a classical catalogue client allowing also to order and sometimes download search results. In an attempt to be platform independent it is implemented as Java app. The main characteristics are the various forms and widgets to specify search criteria together with the big red Search Catalogue button at the bottom left. In general EOLi-SA is quite powerful and feature rich, although sometimes they are quite difficult to find. USGS provides [**GloVis**](http://glovis.usgs.gov/), a Java app, to access Landsat data. Take a look at the image and guess the first release date ;) Another dinosaur in the field is [**DataDoors**](http://www.datadoors.net/core/Documents/source/DataDoors_Brochure.pdf) a Silverlight app with lots of buttons at the top. It provides a complete management suite for data providers including management of users, products, orders, etc. Also from USGS is the [**EarthExplorer**](http://earthexplorer.usgs.gov/), another classic catalogue client with the need to explicitly specify area of interest, time of interest, and other search criteria. Something that these interfaces have in common, is that the user has to explicitly specify search criteria like area of interest before getting to see the data. Lets take a look at apps that don't have that need. I want to start with ESA's **ngEO client**. ngEO is short for next generation Earth Observation system. Area of interest and time of interest are automatically taken from the map extent and timeslider selection and searches are performed in the background transparent to the user. Browse images of the data are shown for user selected products. The ngEO system is not yet in operational use although preparations for Sentinel data are ongoing. But there is an [online demo](http://ngeo.eox.at) with Landsat data available. The ngEO client still allows the user to explicitly specify search criteria and list the search results, but this is not required. In my opinion the ngEO client is a great advancement from the apps above, but why not show the search results automatically on the map? In the frame of ESA's DREAM project we use the [EOxClient](https://github.com/EOX-A/EOxClient) as the **DREAM::ODA** (Online Data Access) client. The first image shows the available Landsat data in the current map view for the selected time interval, without any need for the user to specify search criteria or to explicitly perform a search. The second image shows the result list as a preparatory step for downloading the data. Unfortunately I can't provide a link to the app since we also had to use some restricted data. Ping me if you want further information. Did you note the background maps? Both apps are using our [EOX::Maps](http://maps.eox.at). Sorry for the shameless plug. NASA's latest EO data access client is named **[Earthdata](https://search.earthdata.nasa.gov)** and provides a nice interface to search in a rather classical way. In order to view latest Landsat data on a map, NASA provides its [**GIBS**](https://earthdata.nasa.gov/about/science-system-description/eosdis-components/global-imagery-browse-services-gibs) (Global Imagery Browse Services). GIBS is used in **[Worldview](https://worldview.earthdata.nasa.gov/)** as shown here in a polar projection. Note the functionality to scroll through the data in time at the bottom. Another example of GIBS usage is as map layer in [Leaflet](https://earthdata.nasa.gov/labs/gibs/examples/leaflet/webmercator-epsg3857.html), a popular web map client written in Javascript. A nice interface to browse and access Landsat data is [**Libra**](http://libra.developmentseed.org/), developed by [Development Seed](https://developmentseed.org/) and [Astro Digital](https://astrodigital.com/products/). By just zooming in on the map the user gets access to the full Landsat archive and can directly download individual scenes band by band or as different RGB variants. [Astro Digital](https://fetch.astrodigital.com/) adds the functionality to publish Landsat data as custom map ready to use in your application. A simple and for performance optimised interface is the **SM-Cat** which was detailed in a [previous post](https://eox.at/2015/08/sx-cat-part-one-the-static-map-catalogue/) by Fabian. Just take a look a the images to see how to get to the full Landsat data with just three clicks or [try it yourself](https://landsat-ds.eo.esa.int/smcat/LandsatETMCloudFreeCoverage/). USGS's [**LandsatLook**](http://landsatlook.usgs.gov/) is another modern app with zoom-to-data and mosaic download functionality. The full Landsat 8 data is available on **[Amazon's S3](https://aws.amazon.com/public-data-sets/landsat/)**. The data is provided via a simple HTTP API on top of which very basic interfaces are available as shown in the image below. Nevertheless this API is important as numerous apps are built on top of this offering. Some of them are shown below. A really interesting project is **[Remote Pixel](http://remotepixel.ca/webmapping/index.html)** where various small apps are provided to play with Landsat data. Note that the most recent data is always available due to the connection to Amazon's S3 offering. The following images show some of those apps. Finally I'd like to show some commercial uses of the open Landsat data. First there is [**Mapbox**](https://www.mapbox.com/), which besides their global satellite map derived from Landsat, offers a [Landsat Live](https://www.mapbox.com/blog/satellite-metadata/) map layer. This map shows the latest available Landsat image as well as the corresponding metadata for each location on Earth. At **CartoDB** I found another interesting usage of Landsat data. The provided animation shows which areas are acquired by Landsat over time. In addition links to Amazon's S3 are provided to preview and download the data. Another example of a great usage of Landsat data is by **[PlanetLabs](https://www.planet.com/)** where they use a Landsat 8 mosaic for geo-referencing their own data. Unfortunately they made me sign a pretty restrictive NDA before allowing access to their interfaces. Thus I can't provide an example view. To conclude the examples I include a sneak peek on an interface to access data from ESA's [Swarm](https://directory.eoportal.org/web/eoportal/satellite-missions/s/swarm) mission. Though this is not a classical Earth Observation mission, as it measures the Earth magnetic field at the satellite's position, the **VirES workspace** interface includes some interesting features. As you can see most of the concepts shown above can also be used for very specific data. I leave you with the images of the **VirES workspace** and invite you to check back for details provided in a future post, most probably by Daniel. ## Analysis In the examples shown only the classical search interfaces at the beginning use the map as support for the mandatory area of interest specification. All others automatically take the currently visible area. I see two classes of apps, those where search criteria need to be explicitly entered and searches are triggered by the user and those where searches are performed transparent to the user in the background. Further there are those apps with loads of buttons in the beginning compared to those with basically only the map later on. In my opinion those complex user interfaces take too long to learn for non-expert users. On the other side the single purpose apps with only a few buttons are easy to play with and invite also non-expert users for investigation. All apps, besides the first three, are browser based and thus more or less platform independent. Most of the apps are even running on mobile devices. As perspective for the future I see a large number of very specialised apps for single use cases. Non-expert users will not have to deal with the details of EO data but get answers to their specific questions form these single purpose apps right away. ## Conclusions Again, my personal top 5 trends in EO data access and usage are: * simple apps, in terms of user interface, for single use cases are popular and are more and more replacing the generic, monolithic, multi-purpose clients * apps are browser based, using latest Internet technologies like HTML5 or WebGL, for platform independence even including mobile devices * more and more EO data is released as Open Data for the benefit of numerous communities like OpenStreetMap * the map becomes the central user interface element to view the EO data instead of only supporting the area of interest specification * there are more commercial opportunities, also for smaller companies, coming up with the availability of Open Data like those from Landsat and the Sentinels, particularly from Sentinel 2 Thanks for reading this far. Do you agree with the analysis? Do you think I missed something important? What are your top trends? Please let me know your thoughts, best via Twitter. --- --- url: /_team/dolezalovat.md description: >- Excited about data engineering, modelling and data of all types. Passionate about introducing more women to the IT world as PyLadies Vienna Organizer. I really like to hike but also spend time with a large cup of coffee and a good book. --- --- --- url: /2015/12/understanding-sentinel-2-satellite-data.md --- This is the first entry in a series of posts about how to use data from Europe's Sentinel-2 satellites. Stay tuned as the remaining posts are going to be published over the next couple of months. As always, feedback is very much welcome. Please let us know which topics you want us to include. Today we start with an introduction to Copernicus and the Sentinel missions before diving into Sentinel-2 product levels and data concepts from granules to user products. How to download Sentinel-2 data and how to decode the SAFE file format is going to be described in the next posts. ## Introduction to Copernicus The [Sentinel satellites](https://sentinels.copernicus.eu) are part of Europe's ambitious [Copernicus programme](http://copernicus.eu/) (formerly known as Global Monitoring for Environment and Security (GMES)). The overall mission is composed of five constellations of satellites, each targeting a different application domain. The Sentinel missions implement two interesting and important concepts. First, they fly in constellations of multiple satellites which provide more frequent data. Second, the data is released under an [Open Data policy](https://scihub.copernicus.eu/) allowing free access to images and therefore fostering enhanced knowledge generation and application developments. * **Sentinel-1** (4 satellites) is a Synthetic Aperture Radar (C-band) mission designed to monitor sea ice zones, the Arctic environment, and the risk of land surface motion as well as generate forest, water, and soil mapping. * **Sentinel-2** (2 satellites) is an optical mission with a multi-spectral instrument mainly for agricultural applications such as crop monitoring and management, vegetation and forest monitoring (e.g. leaf area index, chlorophyll concentration, carbon mass estimations), monitoring land cover change for environmental monitoring, observation of coastal zones (marine environmental monitoring, coastal zone mapping), inland water monitoring, glacier monitoring, ice extent and snow cover mapping. * **Sentinel-3** (3 satellites) carries multiple instruments, mainly for sea-surface as well as sea and land ice topography, sea- and land-surface temperature, ocean- and land-surface colour, sea- and inland-water quality and pollution monitoring, land-use change monitoring, forest cover mapping, fire detection, marine and general weather forecasting, snow cover monitoring, measuring Earth's thermal radiation for atmospheric applications, and last but not least for overall environmental and climate monitoring and modelling. * **Sentinel-4** will be placed on geostationary Meteosat Third Generation satellites. * **Sentinel-5** plus Sentinel-5 Precursor are carried on the polar-orbiting MetOp Second Generation satellite. Both, Sentinel-4 and Sentinel-5, are dedicated to monitor the composition of the atmosphere for air quality, stratospheric ozone and solar radiation as well as climate monitoring. Beside the above mentioned domains all Sentinels will fully support natural disaster management (risk analysis, loss assessment, disaster management, etc.) and humanitarian aid in crisis situations. ## Sentinel-2 After this short overview of the Sentinel missions we are going to focus on Sentinel-2 (S-2) and provide some more details about data from this [recently started](https://eox.at/2015/07/sentinel-2/) optical mission. The Sentinel-2 constellation has currently one satellite in orbit, the second one is scheduled to start next year. The two satellites will fly at a height of 786 km and 180° degrees apart. The final constellation will provide a revisit time of five days at the Equator and proportionately more frequent in higher latitudes. It systematically covers all land and coastal areas between 84°N and 56°S. The data collection within this region includes islands greater than 100 km² in area, all islands in the European Union, all other islands within 20 km of a coastline, the whole Mediterranean Sea, all inland water bodies, and all closed seas. Each satellite carries a single multi-spectral instrument (MSI), which provides 13 bands in the visible, near infra-red, and short wave infra-red part of the spectrum, at different ground resolution. The Table below shows a summary of the bands, their resolution and their application. ## Processing Satellite data is provided in multiple levels typically ranging from 0 to 3, where 0 is unprocessed instrument and payload data at full resolution, and 3 means derived information, like a single percentage number. Level-0 and Level-1A processing are performed by the ground segment and are in general not accessible to users. So we skip them here. Level-1B and Level-1C processing is performed routinely by the S-2 ground segment and the results are available to users. We therefore provide a short overview on these products and go into more detail for Level-1C products later on. **Level-1B products** represent Top-Of-Atmosphere calibrated radiances (TOA) in sensor geometry, and include radiometric corrections and the refinement of the physical geometrical viewing model (which is not applied at this level). Products are available as JPEG2000 lossless compressed images plus associated metadata, all capsuled within a SAFE file container (basically a zip-file, more details about SAFE in a later post). Level-1B is composed of granules (see also Figure below), one granule represents a sub-image (one for each band) of the satellite track, with a size of 25 km across-track and approx. 23 km along-track on ground. Each Level-1B granule has a data volume of approximately 27 MB. Level-1B products require expert knowledge of orthorectification techniques. **Level-1C products** are radiometric and geometric corrected TOA products. The corrections include orthorectification and spatial registration on a global reference system (combined UTM projection and WGS84 ellipsoid) with sub-pixel accuracy. Level-1C images are delivered in tiles of 100x100 km (see also Figure below), each of which is approximately 500 MB. Products are available as JPEG2000 lossless compressed images plus associated metadata, all capsuled again within a SAFE file container. Currently, this type of data is the main target for almost all users. **Level-2 products** are Bottom-Of-Atmosphere (BOA) corrected orthophotos. In addition they contain an Aerosol Optical Thickness (AOT) map, a Water Vapour (WV) map, and a Scene Classification Map (SCM) together with Quality Indicators (QI) for cloud and snow probabilities at 60 m resolution. They are resampled and generated with an equal spatial resolution for all bands, based on the requested resolution (10 m, 20 m, or 60 m) Unfortunately, Level-2 products are currently not routinely generated by the ground segment but have to be generated by the users themselves (e.g. by using the [Sentinel toolbox](https://sentinel.esa.int/web/sentinel/toolboxes/sentinel-2), provided by ESA for free). However, the provisioning of Level-2 products, as a systematically generated core product of the ground segment, is currently assessed. ## Sentinel-2 Data Structure ### Datatakes & Datastrips OK, lets start from the beginning i.e. a satellite flies over an area and scans the Earth surface. For the Sentinels this is named a "datatake" – the continuous acquisition of an image from one Sentinel-2 satellite in a given MSI imaging mode. The maximum length of an imaging datatake is 15,000 km (e.g. the continuous observation from northern Russia to southern Africa). Since it is not possible for a single ground receiving station to cover such an area the term "datastrip" is introduced: within a given datatake, a portion of image downlinked during a pass to a given station is termed a "datastrip". If a particular orbit is acquired by more than one station, a datatake is composed of one or more datastrips. The maximum length of a datastrip downlinked to a ground station is approximately 5,000 km. ### Granules & Tiles Now back to Level-1C products. In the whole concept of providing orthorectified, spatially co-registered (to a global reference system) data to users a confusion has been introduced. Specifically by allowing the usage of the same expression for different things. In order to avoid this confusion we will strictly use the term "granules" for the 25x23 km sub-images at Level-1B and the term "tiles" for 100x100 km sub-images a Level-1C and higher. The figure below (Courtesy of ESA) shall provide an impression of the meaning of these two terms. The underlying datastrip (the satellite image) forms the basis. Since datatakes are usually composed of datastrips provided by multiple ground receiving stations (an exception would be if the data is transmitted via a geostationary satellite downlink) one question arises: “how are two adjacent ground stations handled?" The situation is illustrated in the figure below. Another question, we have frequently discussed here at EOX, is the relationship between a datatake, the UTM reference tiling  schema, the user defined Area-Of-Interest (AOI), and the actual product delivered to the user – the so called "User Product". The figure below provides a first impression of these relations, which will be explained in more detail together with the SAFE file format. Thanks for reading this far but let's stop for now. Come back to learn how to download Sentinel-2 User Products, how to decode the used SAFE file format, and how to further process the data. --- --- url: /2019/01/using-python-interface-of-vires-in-eox-jupyter-platform.md --- This blog post describes the experiences of using the freely available Python- and web-clients to the VirES for Swarm (more information [here](https://vires.services/)) interface that provides highly interactive data manipulation and retrieval interface for ESA’s Swarm constellation mission science products (more information [here](https://earth.esa.int/web/guest/missions/esa-operational-eo-missions/swarm)). The idea was to test the new service (for more details read a [blog post by Ashley Smith](/2018/10/building-a-python-interface-to-vires/)) provided by EOX that allows user very simple and flexible access to data, e.g., Swarm mission data, and perform fast visualisation of the different scientific products. ## ESA’s Swarm Constellation Mission Data Access All the three Swarm satellite are equipped with a set of identical instruments: absolute scalar and vector field magnetometers; star tracker; electric field instrument; GPS receiver; laser retro-reflector and accelerometer (more information [here](https://earth.esa.int/web/guest/missions/esa-eo-missions/swarm/instruments-overview)). This instrumentation allows achieving the main science objectives of the mission: studies of core dynamics, geodynamo processes and core-mantle interaction; mapping of the lithospheric magnetisation and its geological interpretation; determination of the 3D electrical conductivity of the mantle; investigation of electric currents flowing in the magnetosphere and ionosphere. The amount of scientific products is increasing from year to year and covers quite a wide range of measurements and models (more information [here](https://earth.esa.int/web/guest/missions/esa-eo-missions/swarm/data-handbook)). All these products/data are freely accessible to all users via anonymous access in accordance with ESA Earth Observation Data Policy via any HTTP browser ([http://swarm-diss.eo.esa.int](http://swarm-diss.eo.esa.int/)) and an ftp client ([ftp://swarm-diss.eo.esa.int](ftp://swarm-diss.eo.esa.int/)). ## VirES Web-Client Visualisation tools for Swarm Products The VirES web-client provides an excellent opportunity of Swarm data visualization in a fast and effective way and produces the publication-ready graphics and charts through an intuitive, powerful and customizable interface that immediately take effect on the display. The plot below is an example of such time-efficient plot production of Swarm orbits for one given day over the globe to illustrate the spatial coverage of all three satellites. ![](/images/swarm1.jpg) Screenshot from the talk given to the broad audience The “normal” way to produce this or other plots would be, first to access and download Swarm data (ftp or http); second to read, e.g., the CDF file information or data handbook (available [here](https://earth.esa.int/web/guest/missions/esa-eo-missions/swarm/data-handbook)) for the given product; third to write script or code to read and process the date and than as a last step visualize it (also very time consuming). And this was an “easy” example where no model calculations were involved that would make this process even longer. The VirES web-client, however, provides a tool for an efficient way for visualisation that does not request user knowledge on additional data included in the file to be visualised or how to combine different data files to get a final plot for publication or presentation purposes. Additionally, VirES allows data download that can be combined to fit various use cases. All this is achieved in a user-friendly environment with powerful and intuitive built-in visualisation tools. ## VirES Python-Client programmatic package for Swarm Products The VirES Python-client-module provides a good possibility of programmatic access to the VirES-server. Namely, this module serves as the application programming interface (API) and allows users to access VirES database of Swarm data products and offers opportunities for various model calculations. For this activity, I have used a Jupiter notebook environment, and tested the advantage to allow users to work in parallel and share codes and data. VirES offers several build in models, e.g., *MCO_SHA_2C* (use example [here](https://github.com/smithara/viresclient_examples/blob/master/model_residuals_and_cartopy.ipynb)), but for this case, it was important to import my user-defined models of the Earth’s magnetic field, e.g., core field and lithospheric field. In the VirES client, it is possible to import users’ own models. Therefore, the first step was to upload the spherical harmonic coefficients from external files. For this purpose, the models *AUX_CORE* and *AUX_LIT* (not available publicly, but accessible for DISC members) have been used. These models are also applied for calculating the Swarm Level 2 Ionospheric Radial Current (IRC) and Ionospheric Bubble Index (IBI) products (more information [here](https://earth.esa.int/web/guest/missions/esa-eo-missions/swarm/data-handbook)). I have used the following Python script to upload these models: ```py from viresclient import SwarmRequest from pylab import * import datetime as dt from eoxmagmod import load_model_shc corePath = '../share/jan.rauberg/SW_OPER_AUX_COR_2__20131126T000000_20190101T000000_0001.DBL' litPath = '../share/jan.rauberg/SW_OPER_AUX_LIT_2__00000000T000000_99999999T999999_0003.DBL' core_model = load_model_shc(corePath) lit_model = load_model_shc(litPath) ``` Next, I requested the magnetic field data for the given Swarm satellite (e.g., Swarm A on 5 July, 2017) from VirES server: ```py request = SwarmRequest(url="https://staging.viresdisc.vires.services/openows") request.set_collection("SW_OPER_MAGA_LR_1B") request.set_products(measurements=["B_NEC", "F"], sampling_step="PT1S") tss = dt.datetime(2017,7,5) tse = dt.datetime(2017,7,6) ds = request.get_between(start_time=tss, end_time=tse).as_xarray() ``` The following lines of code show an example how to evaluate the uploaded models, in this case *AUX_CORE* and *AUX_LIT*: ```py core = core_model.eval(mjd2k, pos, scale=[1, 1, -1]) #note the reversed radial component lit = lit_model.eval(mjd2k, pos, scale=[1, 1, -1]) ``` Note that these are only parts of the Python script, which uses designated VirES services. The following plot show the magnetic field residuals calculate as: ![](/images/swarm_image.png) ![](/images/swarm_image-1.png) Screenshot of residual magnetic field calculated with VirES client Based on these results, it was rather straightforward to extend the Python script for Ionospheric Radial Current (IRC) calculations for the single satellite case. However, these calculated IRCs are a reduced version of the official Swarm L2 IRC product since the model for the magnetospheric field is not considered. The comparison plot between the reduced model family to calculate IRC, one using the VirES client and other from original code (also without magnetospheric part), is shown below. ![](/images/swarm_image-2.png) Screenshot of the difference between the two codes used to calculate IRC The original amplitudes of the IRCs are about 0.01-10 mA/m^2. Hence, the differences between the two methods are very close to the noise level. This concludes that the implemented VirES tool provide reasonable and good results and can be offered for public use. --- --- url: /2020/01/vector-tiles-with-dynamically-updated-style-in-leaflet-lpvis.md --- TL; DR, Leaflet & [VectorGrid](https://github.com/Leaflet/Leaflet.VectorGrid) Demo app [LPvis](https://github.com/eurodatacube/LPvis) showing conformity of farmer crop declaration vs trained classification ML results using API-updated static vector tiles can be seen here: **[lpvis.hub.eox.at](https://lpvis.hub.eox.at/)** ### Why vector tiles? For a swift introduction into what a Vector Tile is, [Mapbox’s Vector Tiles](https://www.mapbox.com/vector-tiles/) (MVT) website states the following description: *Vector tiles make huge maps fast while offering full design flexibility. They are the vector data equivalent of image tiles for web mapping, applying the strengths of tiling — developed for caching, scaling and serving map imagery rapidly — to vector data.* There are a lot of resources already available about it, as Vector Tiles have been receiving attention for some years now, so I won’t delve into too much detail. It’s a vector data encoding in the form of binary data tiles, geared towards reduced data size, bandwidth reduction due to caching as much as possible and (potentially) improved rendering performance. A big advantage is that styling and raw data are separate in nature. Attribute changes of few selected already loaded features (bringing in a change of styling) can lead to modifying only very little low volume of information on client side, instead of the necessity to reload a whole tile. In the same way, global data can be styled on the client side in multiple ways on demand while only one set is cached. But none of that answers the question why we have started to be interested in the Vector Tiles ourselves! ### Motivation First some pieces of context. Land Parcel Identification System (LPIS) is an IT system based on high resolution satellite images for editing the graphical and descriptive information about agricultural land. It is very actively used to check payments made under the [Common Agricultural Policy](https://ec.europa.eu/info/food-farming-fisheries/key-policies/common-agricultural-policy/future-cap_en#a-new-way-of-working) (CAP) which were approximately [45.5 billion euro](https://op.europa.eu/en/publication-detail/-/publication/11049e0e-9a82-11e6-9bca-01aa75ed71a1) in total 2015. CAP is an economical ecosystem to sustain and support agriculture throughout Europe. From 2020 onward crop monitoring with help of remote sensing should be established across Europe, under wings of respective [paying agencies](https://enrd.ec.europa.eu/general-info/whos-who/implementing-authorities/paying-authorities/paying-agencies_en) as a part of modernization of the CAP ecosystem. These topics are however more thoroughly described in our previous posts: [EOxCloudless for Common Agricultural Policy](/2019/05/eoxcloudless-for-common-agricultural-policy-cap/) and [EOxCloudless Sentinel-1 and Sentinel-2 Analysis Ready Data](/2019/05/eoxcloudless-sentinel-1-and-sentinel-2-analysis-ready-data/). One of many tasks, which paying agencies have to face frequently in their agenda is monitoring if farmer declaration of what grows on their fields corresponds to the truth. Under normal circumstances, this means random field inspections, which increase labor costs unnecessarily. What if we could reduce those costs to a minimum by not having to send people to the field unless absolutely necessary? For example Machine Learning trained classification algorithms backed by collected ground truth data and openly accessible worldwide satellite data from Sentinel 1 and Sentinel 2 can yield very accurate estimates of crop type. These classification results should be distributed on parcel level to allow direct evaluation of conformity of farmer declaration. In order to allow a direct conformance, overview and verification tool for paying agencies, we have prepared a Leaflet demo application called [LPvis](http://github.com/eurodatacube/LPvis) - currently with Austrian LPIS data and inputs from [AgrarMarkt Austria](https://www.ama.at/Home) (AMA). Let's see, what powers the app, shall we? ![Visualization of attributes that lead to a final conformity map](/images/conformity_assesment-1.jpg) **Visualization of attributes that lead to a final conformity map** source: *KOETZ Benjamin, DEFOURNY* *Pierre, SEN4CAP output for monitoring, [link](https://ec.europa.eu/jrc/sites/jrcsh/files/14-sen4cap.pdf)* ### Leaflet VectorGrid to action Most of our web applications are based on either OpenLayers or Cesium libraries. As the borders and differences between OpenLayers and Leaflet are nowadays not very sharp, it is more a matter of taste, which one you choose, as differences between functionalities are mostly minor. In general, Leaflet is designed as easy-to-use and light-weight library, where extra features are added with numerous plugins, while OpenLayers offers most of functionality out of the box. For LPvis, we have decided to use Leaflet 1.5.1 with Leaflet [VectorGrid](https://github.com/Leaflet/Leaflet.VectorGrid) as a testing setup. Leaflet Vector Grid is one of the possible plugins how to display MVT in Leaflet. It displays gridded vector data - sliced [GeoJSON](http://geojson.org/), [TopoJSON](https://github.com/mbostock/topojson/wiki) or [protobuf MVT](https://github.com/mapbox/vector-tile-spec). Below is a code example loading such a layer to Leaflet map. ```js const map = L.map('map').setView([50.102223, 9.254419], 4) // create a basic map const AGRICULTURAL_PARCELS_URL_TEMPLATE = '/tiles/agricultural_parcels/{z}/{x}/{y}.pbf' // prepared local tiles or remote url const agricultural_parcels = L.vectorGrid.protobuf(AGRICULTURAL_PARCELS_URL_TEMPLATE, { rendererFactory: L.svg.tile, // canvas was pixelated with tooltip artifacts interactive: true, maxNativeZoom: 16, // minZoom must be higher than minNativeZoom minZoom: 14, vectorTileLayerStyles: { agricultural_parcels: properties => { return trafficLightStyle(properties.match, properties.accuracy, false) } }, getFeatureId: feature => feature.properties.AGRICULTURAL_PARCELS_UNIQUE_IDENTIFIER, // to allow further matching with classification results }).bindTooltip('', { sticky: true }).addTo(map) ``` LPvis currently expects LPIS data and administrative boundaries as vector tiles in uncompressed pbf format using an ZXY scheme. The important part of the setup is that styling is applied on all features. As we are working with parcels (field polygons), a sort of traffic light style based on the match of declaration and classification (*feature.match*) with a certain confidence threshold (*feature.accuracy*) was used. ```js // Simplified version of traffic light style without highlighting and hiding of features const parcel_style = { weight: 0.3, fill: true, fillOpacity: 0.6 } function trafficLightStyle (match, accuracy) { if (accuracy < 0.95) return { fillColor: 'yellow', // classification not confident enough color: 'yellow', ...parcel_style } else if (match === true) return { fillColor: 'green', // matches declaration with enough confidence color: 'green', ...parcel_style } else if (match === false) return { fillColor: 'red', // different than declaration with enough confidence color: 'red', ...parcel_style } else return { fillColor: 'grey', // no classification data color: 'grey', ...parcel_style } } ``` ### Dynamic updating vs dynamic serving There are multiple ways of providing vector tiles. The final choice heavily depends on multiple considerations. Firstly the number of parallel users and secondly the interval in which data are changing taking into account if only a single attribute (prediction) or whole geometries and all attributes changes. Thirdly, if users of the app will be working only on low tier devices or bad network environment, having to load & update several thousand polygons can be a huge problem. If that would be your case, a quite thorough talk from [Max Malynowsky (REACH Initiative) held on FOSS4G 2019](https://media.ccc.de/v/bucharest-133-user-generated-offline-maps-with-serverless-frameworks#l=eng) can be of much help. The main options are: **Dynamic MVT** with all parameters necessary created on the fly by a back end service (for example PostGIS and [ST_AsMVT](https://postgis.net/docs/ST_AsMVT.html) function) - ideal for attributes and geometries changing often or low amount of users at the same time. Even in this case, vector tiles still can and should be cached at least between data changes, unless that happens more than hourly lets say. Such an approach is well described [here](https://info.crunchydata.com/blog/dynamic-vector-tiles-from-postgis). **Static MVT** directly served to the user with all attributes being the originally planned use-case of vector tiles. This enables heavy caching and/or low price storage like s3. Ideal for not often changing data but also needs some extent of tailored pre-processing before. A decision needs to be made beforehand which hierarchy levels of data are displayed on which zoom to avoid extensive size increase of not well optimized geometries - for example by using a tool [Tippecanoe](https://github.com/mapbox/tippecanoe). **API updated Static MVT** served to a user with just a geometry, which changes rarely. Other attributes used for visualization are then fetched dynamically from back end and merged with features already drawn on the client for example by a shared *ID*. This kind of approach can be used well during development and for data with often changing attributes. In our case of LPIS conformance checking, it would be during the process of training and improving performance of the ML classification. Server side caching of used MVT is again a good idea. ### LPvis mocked data demo The initial step of the development was a front end only demo of [LPvis](https://github.com/EOX-A/LPvis) using static MVT of agriculture parcels with already included *match* and *accuracy* properties - mocking classification results, prepared by [Stefan Brand](https://github.com/StefanBrand) during his internship at EOX. Traffic light symbology shows conformity and a sidebar with mocked [NDVI](https://en.wikipedia.org/wiki/Normalized_difference_vegetation_index) timestack chart lets you dig deeper into data in case of non-conformance comparing different crop vegetation index timestacks. The demo itself features many custom Leaflet UI plugins and shows LPIS parcels with mocked results of most of Austria and can be viewed [on this link](http://lpvis-demo.s3-website.eu-central-1.amazonaws.com/). ### LPvis usecase of Euro Data Cube *The* [*European Space Agency (ESA)*](http://www.esa.int/) *has initiated a* [*European Data Cube Facility Service*](https://eo4society.esa.int/2019/05/21/european-data-cube-facility-service-an-eo-resource-factory/) *project to enable millions of people around the world getting EO and non-EO data in a convenient manner. It contributes to the strategy to bring EO data from the restricted Space Community to the larger GIS Community and finally to the end-users assisting them at various tasks. In order to reach this goal, EOX and other consortium partners will integrate and expand existing services to establish the one service for all kinds of processing.* To expand Euro Data Cube sample application deployments, LPvis was updated to utilize its back-end capabilities - namely Euro Data Cube geoDB. Here [eurodatacube](https://github.com/eurodatacube/LPvis)[/](https://github.com/eurodatacube/LPvis)[LPvis](https://github.com/eurodatacube/LPvis) ships bundled LPIS static vector tiles from a small part of Austria and fetches a geoDB stored ML crop type classification results to update the MVT styling on client side. It also accesses [Euro Data Cube Sentinel-Hub Service](https://hub.eox.at/marketplace?group=Euro%20Data%20Cube) - namely NDVI timestacks for concrete LPIS parcel in the format of the [Sentinel Hub FIS](https://www.sentinel-hub.com/develop/documentation/api/fis-request) as another dynamic source of validation data. Details of the classification itself will not be presented here yet but probably in one of next blog posts. ![Filtering out conforming parcels together with NDVI timestack sidebar](/images/summary_photo_lpvis.jpg) Filtering out conforming parcels together with NDVI timestack sidebar From the implementation point of view, this presented a challenge of updating the MVT parcels after classification results are received. A mapping parcel_id -> tile needed to be set up and updated after each fetch, also taking in consideration that a single parcel can be present on more tiles. We want to thank CAP Paying Agency Agrarmarkt Austria, represented by Bernhard Eder and Angela Trischak, for providing user requirements and practical insights into Austrian parcel data used in the demo example. *** #### More information Should you be interested in further developments in Euro Data Cube, please use the chatbot on [eurodatacube.com](https://eurodatacube.com) to subscribe to our newsletter. If you are interested in LPvis specifically, leave us a note. Any contributions are more than welcome. For questions, feedback and feature proposals open an [issue](https://github.com/eox-a/LPvis/issues). For minor bug fixes you can create a [PR](https://github.com/eox-a/LPvis/pulls) right away. Lubomír Bucek, EOX IT Services GmbH Email: lubomir.bucek@eox.at --- --- url: /2021/11/foss4g-2021-vector-tiles-that-would-be-great.md --- This blog post is about a talk that I held virtually at FOSS4G 2021 in Buenos Aires/Argentina. Our efforts are the continuation of work that we began with LPvis (more info in [this blog post](/2020/01/vector-tiles-with-dynamically-updated-style-in-leaflet-lpvis/)). ## Abstract End of last year we at EOX decided to go all-in on open source vector tiles (MVT). We had two use-cases: 1. An interactive Vue/Django web app visualizing millions of agricultural fields along with dynamic properties 2. Rendering vector and raster tiles from the same data source using the same style document and in multiple projections (web mercator and geodetic) Along the way, we found a consolidated Free and Open Source Software (FOSS) ecosystem that had lost many contributors to proprietary lands. However, we also encountered some real FOSS gems ([pg_tileserv], [django-vectortiles]) and a legacy tool ([TileServer GL]) that we "hacked" into rendering non-web-mercator raster tiles from MVT. ## The Talk The talk aims to be a reference for companies, organizations and individuals, who are looking into adopting FOSS for vector tiles. Participants can expect to learn about FOSS projects off the beaten paths of proprietary services. For both case studies, I introduce our requirements and then give an overview of the options that we evaluated. Finally I explain our solutions for each use-case. You can [➡ download the slides as PDF.](/content/FOSS4G2021_Vector_Tiles_EOX_Brand_Stefan.pdf) ## Case Studies ### 1. On-the-fly rendering of millions of parcels Our requirements were to 1. directly serve vector tiles from a database, 2. dynamically choose the set of attributes and 3. apply interactive styling in the frontend. In the following image you can see a filtering applied on the agricultural parcels of Austria. The software stack uses Django with the [django-vectortiles] extension for the backend and our FOSS [EOxElements](https://eox-a.github.io/elements/) based on VueLayers (Vue.js bindings for OpenLayers) for the frontend: ### 2. New [EOxMaps] overlay We basically strived to generate eight variations of our overlay, combining the following requirements: 1. Same style for vector/raster tiles 2. Same data source for Web Mercator and WGS84 tiles 3. Light/dark styles from the same source The outcome can be seen in the following image: It uses data from OpenStreetMap and Natural Earth and stores it in a PostGIS database. Since TileServer GL is not aware of other projections than Web Mercator, we implemented a custom FastAPI service that tricks TileServer GL into accepting WGS84 tiles as well (more info on our "hack" in [the talk from 15:21](https://www.youtube.com/watch?v=s_dWBOiuFiY\&t=921s)). Our custom service also provides a OGC-compliant WMS endpoint that can be accessed via a MapCache interface. ## Conclusion & Lessons Learnt In the course of the two projects presented above we hit some road blocks: * Mapbox Style JSON works well for static maps, but quickly becomes inefficient for high interactivity * `ST_AsMVT` of PostGIS cannot be modelled by Django (see [this StackOverflow question](https://stackoverflow.com/q/65508291/9778755)) * TileServer GL is not equipped for projections other than Web Mercator (EPSG:3857) In general, however, the very fact that we were able to employ the mentioned software packages for our advanced needs show one thing: The adoption of the `ST_AsMVT` function (PostGIS) by the FOSS community has led to the democratization of vector tile generation and distribution. We at EOX are in full support of this democratization and contribute to FOSS vector tiles where we can. Keep an eye out for upcoming blog posts where we go into detail about the update of [EOxMaps]. ## Other FOSS4G 2021 talks about Vector Tiles * [The state of Vector tile servers](https://www.youtube.com/watch?v=QzK9YXQ4uoY\&t=46s) by Pirmin Kalberer * [YouTube Search](https://www.youtube.com/results?search_query=foss4g+2021+Vector+tile) [TileServer GL]: https://github.com/maptiler/tileserver-gl [django-vectortiles]: https://github.com/submarcos/django-vectortiles [pg_tileserv]: https://github.com/CrunchyData/pg_tileserv [EOxMaps]: https://maps.eox.at/ --- --- url: /2015/11/videos-in-3d.md --- Video, on a globe? in 3D? interactive? in the browser? Yes we can! *Best in Chrome.* NASA last year released a [video](https://svs.gsfc.nasa.gov/cgi-bin/details.cgi?aid=11719) (Credit: NASA's Goddard Space Flight Center) showing how carbon dioxide in the atmosphere travels around the globe from an ultra-high-resolution computer model. It provides a stunning new look at plumes of carbon dioxide swirling and shifting as winds disperse the greenhouse gas away from its sources. Watch the video with the globe rotated to show the north pole. Astonishing new perspective compared to the [plain video](https://svs.gsfc.nasa.gov/cgi-bin/details.cgi?aid=11719), isn't it? By the way, you're looking at a sneak preview of our [mapalupa](http://mapalupa.com) service. Feel free to [sign up for early access](http://mapalupa.com/) if we got your interest. We make use of [Cesium](http://cesiumjs.org/) for the globe visualization. Many thanks to their team for the great Open Source software. --- --- url: /_team/farkasv.md description: >- Enthusiastic builder of frontends in the world of Earth observation. Hobbies include electronic music production, video games, cooking, rock climbing, camping and hiking. --- --- --- url: /2015/11/vires-swarm-and-magnetic-model-data-visualization.md --- As briefly anticipated by Stephan in [Top 5 trends in EO data usage](/2015/09/top5/) I'm happy to give an extended sneak preview on the prototype implementations we have done as part of the VirES project. Especially now that it has been decided to be extended for operations as an official service of the European Space Agency (ESA) for early 2016 where it will be openly accessible. ## About the project The VirES project, short for Virtual Workspaces for EO Scientists, is an [ESA Technology Research Project (TRP)](http://www.esa.int/Our_Activities/Space_Engineering_Technology/About_the_Basic_Technology_Research_Programme_TRP). The first phase started with collecting and analyzing user requirements from a variety of science communities for a generic "workspace" tool that could handle multiple missions, with emphasis on [ESA's Earth Explorers missions](http://www.esa.int/Our_Activities/Observing_the_Earth/The_Living_Planet_Programme/Earth_Explorers). For the second phase we implemented a prototype specifically for the [Swarm](http://www.esa.int/Our_Activities/Observing_the_Earth/The_Living_Planet_Programme/Earth_Explorers/Swarm/ESA_s_magnetic_field_mission_Swarm) mission but still based on the generic requirements collected. The implementation phase has been finalized and presented on various occasions such as at the [IUGG at Prague](http://www.iugg2015prague.com/) and the Swarm 5th Data Quality Workshop at IPGP (Paris) with great feedback so far. ## The prototype This is a very extensive subject from which I am going to cherry pick some visual examples in a second, but I just want to mention the rough architecture first. We make use of [EOxServer](http://eoxserver.github.io/) for servicing the registered [L1B Swarm Data](https://earth.esa.int/documents/10174/1514862/Swarm_L1b_CDF_Data_Format). It is not necessary to duplicate the data in any way, the EOxServer registers a reference to the CDF file and the data served through the interfaces is directly accessed. As interfaces we use [OGC Web Services](http://www.opengeospatial.org/), such as [WMS](http://www.opengeospatial.org/standards/wms) or [WCS](http://www.opengeospatial.org/standards/wcs), where appropriate. For specialized cases we use [WPS](http://www.opengeospatial.org/standards/wps) which allows us to do some processing (like calculating residuals, on which I will comment in a bit). The client runs directly in the browser without need of installation of any kind of plugin making use of WebGL and SVG (Scalable Vector Graphics) for visualization. ## Components ### Layer Selection As a first step we need to select which data we want to visualize/analyze. Through the Layer selection tool we can quickly select/de-select available Layers. The Layer concept can be considered a little extended to the normal understanding of layer as some of the data, such as the L1B Magnetic Swarm data is not the typical 2D "image" over a map but actually in-situ measurements, i.e. at the current position of the satellite. In addition to which collection is active we also need to know when data is available. Through several projects we have iterated on the [timeslider](https://github.com/EOX-A/d3.TimeSlider) idea to visualize this information. Layers have an assigned color and on the timeslider you can see the product availability for each active layer. The timeslider is interactive allowing zooming and panning and showing product identifiers on mouse over. There are other useful features for the timeslider which do not come into play with this type of data but I will try to describe this tool with more details in another post. ### Globe Visualization One of the two central visualizations tools is the virtual globe. It uses the awesome [Cesium](http://cesiumjs.org/) library to help create this beautiful visualizations directly on the browser using WebGL. It supports three different Views, as shown below, as 3D, 2.5D (called Columbus view), and 2D. The image above shows the Magnetic field intensity as measured by instrumentation onboard one of the Swarm satellites at around 450 km altitude in the three possible views. ### Analytics Viewer The second central visualization tool is what we call the Analytics Viewer. It provides a scatter plot tool for selected data at the top and histogram visualizations of available data parameters at the bottom. The whole visualization is done in SVG making use of another amazing library [D3.js](http://d3js.org/) as backbone. The possibility of saving the created plots as images was required by users so we worked on a method to render the SVG into a canvas element to provide it also as image. As can be seen the Swarm Data (MAG L1b) contains multiple parameters. The scatter plot tools allow multi-parameter selection in the y-axis and to be plotted against any other parameter on the x-axis through the use of both of the drop down selection elements. The Histogram visualization at the bottom allows multi-parameter filtering of the data by just dragging at the parameter range scales. This updates all histograms letting you know at which values the subsetted data is available. ## Data Explaining the meaning of the next visualizations without assuming knowledge of the data is a bit difficult so let me say some words about the data. ### Swarm L1B Firstly we have the Swarm L1B data, which measures, among many other things, Earth's magnetic field at the position of the satellite. This is what is represented in the previous images. ### Magnetic Model Additionally to the Swarm data we have integrated Magnetic Model Data. The model, specified as a set of coefficients, allows the computation of Earth's magnetic field for any point in space and time (where the model is applicable). A visual representation of the model at the approximate altitude of the Swarm Satellites is shown below. The model is actually computed in real time (pretty hefty computation) and served as WMS layer. It is even possible for a user to use their own local set of coefficients to create the visualization of the model (executed on the server). With the model calculation we can do other fancy stuff, such as calculating and displaying magnetic field lines. By selecting an area on the globe we use a 4x4 point grid as seed points, calculate the direction of the field for those points, "move" into that direction and do that until we get back to Earth, which generates fully interactive visualizations like these. ### Residuals The requirements collection also revealed the importance of residual calculation, which is the subtraction of the model from the measurements. The residuals are important for a multitude of fields. As the calculation of the model is "expensive" we do it directly on the server per request using the WPS interface, in this way the results can be visualized directly on the web client as seen below. Note that residuals have a direction in space and are therefore drawn as vectors. ### Auxiliary As a last point I also want to mention the introduction of auxiliary data ([kp-index](http://www.spaceweatherlive.com/en/help/the-kp-index) and [dst-index](https://www.ngdc.noaa.gov/stp/geomag/dst.html)) related to sun activity which has a great impact on the magnetic field. These fields are provided in the Analytics Viewer and can be used as filter criteria. Additionally, as these indices are global, we considered the best way to display them would be on the timeslider. So we extended it to request the data directly from EOxServer and display it as line plot on the time axis. This allows very quick browsing capabilities of the data where the user can navigate and analyze a time frame where strong or weak sun activity is present. ## Final Words I hope I could give you a clear and understandable overview of the VirES Prototype functionality. At the end it got a little out of hand as I just wanted to post some nice screenshots. Feel free to check here again for more information on the official service coming early next year as well as many other topics! --- --- url: /2020/09/vires-aeolus-workshop.md --- The Aeolus mission is one of ESA's Earth Explorer missions. It is the first satellite able to provide global wind profile information from space. We have been working in parallel and preparing the service prior to mission launch and it is great to be able to see the community evolving around the mission and the many great minds we hope to help do their work through this service. EOX has had the privilege of presenting the [VirES for Aeolus Service](https://aeolus.services/) as a dedicated in-depth workshop. The workshop was held twice to accommodate different time zones on the 26th of June and on the 6th of July. Thanks to the organization of ESA there was a large participation of ~80 people combined. We made one of the sessions available online on our channel. Thank to ESA for organizing the event and to all the participants for their time, inputs, and great discussion. We always welcome feedback on how your workflow could be improved using our services so please feel free to contact us on [feedback@vires.services](feedback@vires.services). The VirES for Aeolus Service is free to use, so if you don't know it yet head over to and have a look! --- --- url: /2021/09/vires-services-update-2021.md --- A few days ago a new release of the [VirES for Swarm](https://vires.services) service has been successfully deployed in production. The V3.6.0 is the first release of the service under the new ESA contract which assures the service operations for another two years, with a possible extension up to four years. Rather than just repeating the new features listed in the service [change log](https://vires.services/changelog), I would like to use this opportunity to look back at what has been achieved so far and outline where to evolve in the future. It is not possible to fit every detail in this short blog-post and thus I will highlight the points according to my own, inevitably subjective preference. ## VirES for Swarm VirES (Virtual workspace for Earth Observation Scientists) started approximately six years ago as a prototype of a web user interface (UI) for visualization of Swarm magnetic field measurements and their comparison with the models ([VirES prototype](/2015/11/vires-swarm-and-magnetic-model-data-visualization/)) The interactive visual web UI and snap on-the-fly evaluation of Spherical Harmonic geomagnetic models, as an alternative to tedious downloading of ZIP files and later in-house processing, attracted the stakeholders' attention and the prototype quickly turned into an [operational service](/2016/05/vires-services-launched/). Over time, VirES for Swarm has grown in size. New Swarm products, geomagnetic models, auxiliary data and UI features have been added to the service. Currently VirES for Swarm holds data from over 119k products in over 70 different collections and over 20 various geomagnetic models (not counting their possible combinations). The collaboration with scientists, primarily facilitated through the Swarm Data, Innovation, and Science Cluster [Swarm DISC](https://earth.esa.int/eogateway/activities/swarm-disc), helped immensely with steering of the service evolution in the right direction. ## VirES as a Data API One of the cardinal requirements coming from the scientific users' community was the opening of the VirES for Swarm server API, until then accessible solely by the web UI. While the on-line web interactive visualization and filtered data download are certainly convenient, they become too restricting for more advanced use cases when the data needs to be fed to users' custom processing which is very much at the core of any scientific research. In simple words, the users need to get data to their software, without any manual downloads, file manipulations or dealing with superfluous details of the download protocol. The abstraction of the data access has been the objective of the [VirES Python client](https://github.com/ESA-VirES/VirES-Python-Client) (see also Ashley Smith's [introduction of the Python client](/2018/10/building-a-python-interface-to-vires/)). This client allows retrieving of the data from the VirES server with just a few lines of Python code: ```py from viresclient import SwarmRequest request = SwarmRequest() request.set_collection("SW_OPER_MAGA_LR_1B") request.set_products( measurements=["B_NEC"], models=["CHAOS-Core"], auxiliaries=["QDLat", "QDLon"], sampling_step="PT15S" ) data = request.get_between( start_time="2021-08-01T00:00", end_time="2021-08-01T01:00" ) print(data.as_xarray()) ``` ``` Processing: 100%|████████████████| [ Elapsed: 00:01, Remaining: 00:00 ] [1/1] Downloading: 100%|██████████████| [ Elapsed: 00:00, Remaining: 00:00 ] (0.08MB) Dimensions: (NEC: 3, Timestamp: 240) Coordinates: * Timestamp (Timestamp) datetime64[ns] 2021-08-01 ... 2021-08-01T00... * NEC (NEC) you can now have a look at the new features or get to know the platform if you have not had the chance yet! If you find your work could be simplified or streamlined with an additional feature or some new configuration options or similar please don't hesitate to contact us under . All right, enough preamble here is a short overview of the new features. For all details (as well as previous releases) you can have a look at the [changelog](https://aeolus.services/accounts/changelog/). Multiple changes were introduced to the globe visualization to allow easier "publication-ready" image creation. The background color can now be changed and a new image export function allows higher quality saving. To make labels more readable independent of what is being shown a light white background was added to them. Finally two new layers were introduced, one to show latitude, longitude graticules and the other to show calibration and validation sites. When clicking a site a bounding box will be created automatically to help filtering Aeolus data for that region. Here is an image bringing all together! ![](/images/VirES-Aeolus_Screenshot-2.jpg) As for the analytics panel a new logarithmic colorscale has been implemented. This is also done directly in the web-gl shader, as the linear scale, so it should have very little performance impact. ![](/images/vires-aeolus-ALD_U_N_2B_wind-accumulation-result_2020-09-30T00_36_22_2020-09-30T02_02_21-2.png) All right, I think we are getting a bit long here, so briefly some other highlights: * A new unit conversion option was added for each parameter which allows to multiply the values by user defined power of 10 * The analytics panel has a new parameter mask functionality for L2A products * Multiple new parameters for L1B, L2B, and L2C products are now available through the client That's all for now, again we love your feedback, so feel free to contact us if you have comments related to VirES for Aeolus at . Keep an eye out for release information on the blog! --- --- url: /2022/04/vre-aeolus-teaser.md --- As we mentioned in a [previous blog post](/2022/01/vires-for-aeolus-hunga-tonga-volcano/) we have been working on implementing an extension to the [VirES for Aeolus](https://aeolus.services) service. The extension is called "Virtual Research Environment" (VRE) and since March it has been opened to the public. We see a great potential to help scientific activities with the VRE and are happy to be ready to make it available to everyone! ESA has created a great infografic (we have added as cover image to this story) providing a great insight into the tool. Here is the original source . If this peaked your interest we would like to invite you to visit to have a deeper dive into the capabilities of the service or if you want feel free to visit the [VRE](https://vre.aeolus.services/) directly! We are happy to have had the opportunity to present this tool during the [Aeolus 3rd Anniversary Conference](https://www.aeolus3years.org/). In the future we want to write a more in depth blog post about the tool and its capabilities, so keep your eyes peeled! --- --- url: /2016/10/vires-update-for-plasma-bubble-index-data.md --- We have launched **[VirES for Swarm 1.1](http://vires.services)** on 27 September 2016 and Daniel Santillan () presented it to core scientists and experts at ESA's 6th Swarm Data Quality Workshop held at Univ. Edinburgh. The new version gives also systematic access to the **plasma products** and **ionospheric bubble index products** of the [ESA Swarm mission](https://earth.esa.int/swarm), in addition to the complete sets of **magnetic vector and scalar data products**. VirES for Swarm is ESA's official Web-based interface to Swarm products. It includes tools for studying various geomagnetic models by comparing them to the Swarm satellite measurements at given space weather and ionospheric conditions. **Version 1.1 updates** include enhanced Swarm product layer handling and new rendering of color scale labels on the immersive virtual globe widget.  Multi-scaling control has been added to the Analytics Viewer. The Analytics viewer is the visual feedback and filter functionality offering scatter plot and multi-parameter histogram displays. Customization of the parameters a user wishes to download from multiple Swarm products is now possible. More details on features introduced with the update can be seen in the [changelog](https://vires.services/accounts/changelog/). We are also proudly informing that our company has been entrusted by ESA with the operations and maintenance of the VirES for Swarm service. Watch out for updates. --- --- url: /2016/05/vires-services-launched.md --- [VirES for Swarm](https://vires.services/) – the latest offspring from our geospatial web technology workbench. The online tool lets you dive into the most up-to-date and accurate science of the Earth’s magnetic field. The operational service launches officially today at [https://vires.services](https://vires.services/). The public announcement has gone out to the participants of the [Living Planet Symposium 2016](http://lps16.esa.int/) held these days in Prague, CZ. The VirES initiative builds on an awesome lot of talent, both on the customer as well as the developer/operator team sides. VirES for Swarm is required and implemented by the Earth Observation Directorate of the [European Space Agency (ESA)](http://esa.int) and designed and operated by EOX. As project manager at EOX I would like to say big thanks to all who contributed. We are proud to present today a highly interactive data manipulation and retrieval interface for the data shots taken by the magnetometers flying on a constellation trio of satellites named the [ESA magnetic field mission Swarm](https://earth.esa.int/swarm). In spite of the complexity of the science, the satellite orbital dynamics, the multi-dimensional analysis functionality, etc. the tool impresses due its simplicity of use. Citizen scientists are welcome to join as users just like the academic and professional workers in the field of geophysics and geomagnetics. We have implemented a world map and powerful analytics panel through which differentiated magnetic characterizations (models) provided by the world’s leading geomagnetists can be compared with the Swarm satellite measurements at given solar activity levels. I should better let the application engineers explain the details (see [Daniel’s blog](/2015/11/vires-swarm-and-magnetic-model-data-visualization/)), but I feel like a few extraordinary things deserve highlighting here: * The VirES Globe is the coolest utilization of the [Cesium - WebGL Virtual Globe and Map Engine](https://cesiumjs.org/), which I am aware of; * The VirES Analytics (scatterplot and histogram) filter and visual feedback panel is unprecedented in Earth observation data exploitation/mining; * Not obvious to end users because working invisibly behind the scenes is the powerful [EOxServer](http://eoxserver.org) framework which was largely enhanced to cope with VirES requirements (handling Swarm product formats; super-fast harmonics forward expansion model calculations); * The Open Source and Standards credo of EOX has been fully respected for VirES and [all code](https://github.com/ESA-VirES) is publicly available on GitHub. Note that community contributions are invited. Finally, just to mention a few of the many necessary steps and measures performed to evolve from prototype system to operational service: * Continuous testing and deployment set-up; * Multiple-server operations, failover, and load balancing configuration; * OAuth authentication with major social providers. The success of this release will not lead to laziness on our side. We are already going full speed with the implementation of further enhancements and next releases, couple of hundred items we are having in mind. Stay tuned and check back for announcements. VirES, an operational proof-of-concept for future Virtual Research Environments in geoscience. --- --- url: /vires.md --- ### About the VirES The VirES services, short for Virtual Workspaces for EO Scientists, are operated and maintained by EOX on behalf of the European Space Agency (ESA). They provide flexible and interactive access to the Earth Explorer missions Swarm and Aeolus directly in the web browser. #### Swarm Try out **VirES for Swarm** yourself for free. The video shows a brief impression of the available service. \