Adding Data

The console application data is stored in ~/rw/rw/data. If you look in the directory, you will see subdirectories for each of the river stretches and also a tools directory.

Very briefly, the steps to adding data are:

  1. Add a dataset to one of the directories
  2. Make a soft link to it
  3. Regenerate the derived data

Let's examine each of these steps more closely.

The ASC format

The console application accepts data in the ASC format. This is an ASCII (text) format that describes a raster dataset (an image). The format is very simple. It has a 6 line header followed by scalar values. The header describes the location, extent, and spacing of the data. This defines the uniform 2D grid of points at which the scalar values are stored. The scalars are then assigned to the points in the grid in order.

Note: If you have point data (such as the locations of buoys or fish), this data can be added as individual 3D objects. Go to the Add Content section.

To see the header for an ASC dataset, you can use head:

head -n 6 somefile.asc

This simply prints out the first 6 lines in the file. Here is an example:

cd ~/rw/rw/data/stlouis/
head -n 6 dem.asc

and the output is:

ncols         1180
nrows         1911
xllcorner     727110
yllcorner     4263880
cellsize      30
NODATA_value  -9999

There are actually two formats for the header that are acceptable. Here is an example of the other format:

cd ~/rw/rw/data/stlouis/
head -n 6 WSE_max.asc

and the output is:

north: 4308790
south: 4276540
east: 751620
west: 741360
rows: 1075
cols: 342



Types of Data

The types of data that the console application is expecting in this format are:

elevation data - a grid of elevations

land usage/cover data - a grid of integer values according to the following classification:
  1 - Urban or Built-Up Land
  2 - Agricultural Land
  3 - Rangeland
  4 - Deciduous Forest
  5 - Evergreen Forest
  6 - Water
  7 - Forested Wetland
  8 - Non-Forested Wetland
  9 - Barren Land

bathymetry data - a grid of river bottom elevations

flow data - for each flow condition, the following datasets are required:
  vx - x component of velocity vectors
  vy - y component of velocity vectors
  wse - water surface elevation (not needed for overland flow)

The console application supports the loading of one set of overland flow datasets and any number of sets of channel flow datasets.


Soft Links

In each of the river stretch subdirectories, the console application expects the following filenames:

dem.asc - the elevation data
lulc.asc - the land usage/cover data
bath.asc - the bathymetry data
VX_land.asc - vx for overland flow
VY_land.asc - vy for overland flow
VX_i.asc - vx for flow condition i (0,1,2,3,...)
VY_i.asc - vy for flow condition i
WSE_i.asc - wse for flow condition i

You could simply rename your datasets to these names. However, it is better to create soft links. By using soft links, it is much easier to keep track of dataset versions. You can keep several datasets around with their original filenames and simply link to the one that you want to use.

Here is how it is done:

rm dem.asc
ln -s somefile.asc dem.asc

This removes the current soft link and creates a new one (to somefile.asc). To see what a soft link is linked to, use "ls -l":

ls -l dem.asc


Regenerating Derived Data

From these few ASC datasets, a bunch of data is generated for use with the console application. This data is stored in each river stretch subdirectory in the genData directory. Since this data is derived from the ASC datasets, it can be destroyed at any time and regenerated.

The reason why it is generated outside of the console application is that generating this data can take a long time. So, rather than having to generate this data each time the application is run, it only has to be done when the source data (the ASC files) have changed.

To regenerate the data for a stretch of the river, go to the appropriate directory and run the regenData script. Here is an example:

cd ~/rw/rw/data/stlouis
regenData

That's it. Running the script will regenerate all of the derived data for that stretch. This can take anywhere from 5 to 30 minutes.


About the Derived Data

The genData directory contains two types of data: maps and tiles. Maps are texture data (images) that are for the console map and are also used for texturing terrain. The tiles are geometry that has been generated from the elevation data and any bathymetry or water surface elevation data.

These datasets are generated using two tools: makeMap and rwTerrain. These tools are found in ~/rw/rw/data/tools/.

To understand why the terrain is broken up into tiles and why it takes a long time to generate, go to the code section.


Changing Land Usage Textures

The land usage textures used in the console application for each river stretch are stored in the lcTex directory. Each texture corresponds to a land usage as described in the classification found higher up on this page.

You can modify or replace these land usage textures. However, after doing so, you should regenerate the data. This is because these textures are used to generate some of map data.


Log files

In each river stretch data directory, there is a log.txt file. This file is a log of all the times that the application has been restarted. It also has information about runs of the river pilot simulator.