dwdGribExtractor: A tiny Python interface to request and extract NWP grib file data from opendata.dwd.de
========================================================================================================
.. image:: https://img.shields.io/pypi/v/dwdGribExtractor.svg
:target: https://img.shields.io/pypi/v/dwdGribExtractor
dwdGribExtractor is a smart package to easely extract data from `numerical weather prediction <https://www.dwd.de/EN/ourservices/nwp_forecast_data/nwp_forecast_data.html>`_
grib files provided by DWD.
The difference to other packages is the location flexibility. So every point
on the 2.2km x 2.2km grid (ICON-D2) can be retrieved for the next X forecast hours.
At the moment only ICON_D2 is supported.
Supported weather variables
===========================
For currently available weather variabels see: `ICON User Manual <https://www.dwd.de/SharedDocs/downloads/DE/modelldokumentationen/nwv/icon/icon_dbbeschr_aktuell.pdf?view=nasPublication&nn=495490/>`_
Chapter 6.1.4 table 6.4
Installation
============
Install with pip::
$ pip install dwdGrilbExtractor
Dependencies
============
To read the grib2 files `xarray <http://xarray.pydata.org/en/stable/>`_ with `cfgrib <https://github.com/ecmwf/cfgrib/>`_ engine is used.
The easiest way to install *cfgrib* and all its binary dependencies is via `Conda <https://conda.io/>`_::
$ conda install -c conda-forge cfgrib
alternatively, if you install the binary dependencies yourself, you can install the
Python package from *PyPI* with::
$ pip install cfgrib
Binary dependencies
-------------------
*cfgrib* depends on the `eccodes python package <https://pypi.org/project/eccodes>`_
to access the ECMWF *ecCodes* binary library,
when not using *conda* please follow the *System dependencies* section there.
Windows
-------
It is strongly recommended to use Unix enironment running *dwdGrib2location*. For
windows several issues may appear. At the moment *dwdGrib2location* is tested
with Windows10 and works if following requirements are satiesfied.
To build eccodes on windows by your own should be avoided. The easiest way to
use eccodes on windows is to install it in an `MSYS <https://www.msys2.org/>`_ environment.
1. Install MSYS
2. Install `eccodes <https://packages.msys2.org/base/mingw-w64-eccodes>`_. Depending on your system run in the MSYS cli::
$ pacman -S mingw-w64-ucrt-x86_64-eccodes
3. Set environment variables::
ECCODES_DIR = <path_to_ecccodes_install_dir> e.g (C:\msys64\ucrt64)
ECCODES_DEFINITION_PATH = <path_to_eccodes_definitions> (e.g C:\msys64\ucrt64\share\eccodes\definitions)
4. Add eccodes to path. This is the folder inside the MSYS environment where the .exe files are located::
e.g. C:\msys64\ucrt64\bin
5. Install ecCodes::
$ pip install eccodes
6. Install cfgrib::
$ pip install cfgrib
7. Check if cfgrib is working::
$ python -m cfgrib selfcheck
Found: ecCodes v2.20.0.
Your system is ready.
Linux
-----
1. Install eccodes with apt or `build it by your own <https://gist.github.com/MHBalsmeier/a01ad4e07ecf467c90fad2ac7719844a>`_
TODO
Example
============
.. code-block:: python
locationList = {
"Vienna": {
"lat": 48.20,
"lon": 16.37
},
"Graz": {
"lat": 47.07,
"lon": 15.43
}
}
variables = ["aswdir_s", "aswdifd_s", "t_2m"]
forecast = ICON_D2(locations = locationList, forecastHours = 3)
data = forecast.collectData(varList = variables, cores = None) # Disable multiprocessing
#data = forecast.collectData(varList = variables, cores = 4)
#### Indexing one location
loc = "Graz"
result = data.loc[loc]
result = data.loc[loc, "2021-09-06 06:15:00"]
#### Indexing multiple locations
loc = ["Graz", "Vienna"]
result = data.loc[loc]
#### Indexing one location with datetime condition
loc = "Graz"
mask = data.loc[loc].index.get_level_values(0) > np.datetime64('2021-09-06T06:15:00')
mask = data.loc[loc].index.get_level_values(0).hour == 8
result = data.loc[loc][mask]
#### Indexing multiple locations with datetime condition
loc = ["Graz", "Vienna"]
mask = data.loc[loc].index.get_level_values(1) > np.datetime64('2021-09-06 06:15:00')
mask = data.loc[loc].index.get_level_values(1).hour == 8
result = data.loc[loc][mask]
Knwon Issues
============
Windows
-------
- Multiprocessing on some windows machines may not work. Disable it by setting ``forecast.collectData(varList = variables, cores = None)``
- `Spyder IDE <https://www.spyder-ide.org/>`_ does not produce print outputs if multiprocessing is enabled.
- Dont run the code in Spyder with F5 or debug mode. This calls runfile() and sometimes crashes memory.
Author
======
Manuel Strohmaier
License
=======
Code license
------------
Licensed under the MIT license. See `LICENSE <https://github.com/panodata/dwdGrib2location/blob/master/LICENSE>`_ for details.
Data license
------------
The DWD has information about their terms of use policy in
`German <https://www.dwd.de/DE/service/copyright/copyright_node.html>`_
and
`English <https://www.dwd.de/EN/service/copyright/copyright_node.html>`_.