RapidMiner Module 2: Filtering & Sorting

EXPLANATION

Fun fact: in RapidMiner, rows are called examples, columns are called attributes, and data tables are called example sets. It is good to know this beforehand, because when we work on rows, we will be calling operators that work on examples, attributes, or the whole examples set.

ACTIVITY

1. Search for the Filter Examples operator using the search box at the top of the Operator panel.
2. Drag Filter Examples into the Process panel.
3. Connect the output port of Retrieve Altoona Crime Rates with the input port of Filter Examples.
4. Click on Filter Examples to select it; then in the Parameters panel click Add Filters to define a filter, and set it to Juvenile Total, >, and 0.00.

EXPLANATION

Remember, rows are called examples, so the Filter Examples operator is basically saying “filter rows by some criteria”, and its settings in the Parameters panel say “the criteria is all rows with Juvenile Total being greater than 0”.

When you add an operator to your process, you should immediately connect it to the previous operators in the process. Remember that data flows between operators, so an operator’s connection can influence its parameters. For instance, how could the Filter Examples operator “know” about the column Juvenile Total, if it is not connected to the data source?

ACTIVITY

1. Search for the Sort operator, and then drag it into the Process.
2. Connect the output port of Filter Examples with the input port of Sort.
   
3. Click on Sort to select it; then in the Parameters panel set attribute name to Juvenile Total, and sorting direction to decreasing.
4. Connect the output port of Sort to the result port (“res” port) on the right of the Process panel.
5. Click    Run to execute the process.

EXPLANATION

Remember, columns are called attributes, so the Sort operator above is saying “sort the data by column Juvenile Total in a decreasing order”.

CHALLENGE

1. Looking at the resulting data, what is the most common crime that male juvenile offenders committed in one month? What about the female juvenile offenders?
2. In what month did both of the above common crimes occur?

ACTIVITY

1. Return to Design view, and disconnect operator Sort from the “res” port.
2. Search for the Aggregate operator, and then drag it into the Process.
3. Connect the output port of Sort with the input port of Aggregate.
4. Click on Aggregate to select it; then in the Parameters panel set aggregation attributes to Juvenile Total
   (for aggregation attribute), and sum (for aggregation functions); also set group by attributes to Offense Code.
   
5. Connect the output port of Aggregate to the “res” port.
6. Click    Run to execute the process.

EXPLANATION

The Aggregate operator above is saying “sum the data in column Juvenile Total by column Offense Code”. As a result, we got the sum of all juvenile crimes broken down by offense code, i.e. the type of crime they committed, across all months in our dataset. Also, note the Aggregate operator dropped all other attributes.

CHALLENGE

3. Looking at the resulting data, what is the most common crime that juvenile offenders committed across all months? How many times was it committed?
4. Change the process so it looks at adult offenders rather than juvenile offenders, and answer questions 1-3 for them.