We built this project from a [data science competition](https://www.datascience.net/fr/challenge/9/details) hosted by [datascience.net](https://www.datascience.net/).
<br/>

**You can explore this sample project in your own Dataiku DSS instance.**  
- From the Dataiku DSS home page, click **+New project > Sample Projects > Use Cases**.
- If your Dataiku DSS server doesn't have Internet access, you can [download this sample](https://update.dataiku.com/dss/5.0/tutorials/DKU_ENERGY_CONSUMPTION/dss-tutorial-DKU_ENERGY_CONSUMPTION.zip) and import it manually.  Click **+New project > Import project**.



<br/>
# Project Goal

The goal of the project is to model the energy consumption of various sites and predict consumption peaks. The original data is from more than 40 French consumer sites.
The data in this project has been simplified and contains records for only 3 sites.

<br/>

# How we do this

To build this project we have two different data sources you can have a look at: 

<p class="text-center">
<a href="/projects/DKU_ENERGY_CONSUMPTION/flow/"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;Flow</a><br/><br/>
</p>

- **[sites](datasets/sites/explore/)**: this dataset contains information about the three consumer sites from which the data was collected 
- **[data](datasets/data/explore/)**: this dataset contains energy-use records from the different sites. This dataset gives us energy-use records for every 10 minutes. 

In that dataset you'll find these features:

- **DATE_LOCAL**: date of the record 
- **ID01**, **ID18**, **ID31**:  for each site, a column valued 1 if the record if for the location #1, 0 if not 
- **consomation**: energy consumption (in kW) 
- **temperature**: temperature (in Celcius) 

# Explore the project

1/ start by taking a look at the charts we built on the dashboard.

<p class="text-center">
<a href="/projects/DKU_ENERGY_CONSUMPTION/dashboards/IQVACLV_sites-consumption/view/"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;Dashboard</a><br/><br/>
</p>

You can start seeing differences between the 3 sites :

* ID01 has a very high energy consumption, all the time, even during nights and weekends. It could be an industrial site that works 24/7.
* ID18 and ID31 seem to have a consumption peak at night.
* ID18 has a higher consumption when the temperature is low, ID31 has a higher consumption when the temperature is high.


2/ Have a look at the flow to understand how the site data is prepared, then modeled and scored. 

<p class="text-center">
<a href="/projects/DKU_ENERGY_CONSUMPTION/flow/"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;Flow</a><br/><br/>
</p>


3/ [The first recipe](recipes/compute_data_prepared/) combines the two datasets (LEFT JOIN) and clean some variables.

<p class="text-center">
<a href="/projects/DKU_ENERGY_CONSUMPTION/recipes/compute_data_prepared/"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;Data preparation</a><br/><br/>
</p>

4/ We use a [groupby](recipes/compute_data_prepared_by_day_and_site/) followed by a [window recipe](recipes/compute_data_prepared_lagged/) to do some feature engineering. We want to recover the average of temperatures during the previous week, as well as the values for the temperature and consumption on the previous day. We also added the consumption on the same day, but a week before. We use a [join recipe](recipes/compute_data_prepared_final/) to create the final prepared dataset:

<p class="text-center">
<a href="datasets/data_prepared_final"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;Prepared data</a><br/><br/>
</p>


5/ Then, the data is split into two datasets: records for the year 2011 and records for the year 2012. We do this because we want to train a model on historic data of 2011 and deploy it on the data from 2012 to make predictions. 

<p class="text-center">
<a href="/projects/DKU_ENERGY_CONSUMPTION/recipes/split_data_prepared_final/"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;Splitting data</a><br/><br/>
</p>

6/ A Ridge Regression algorithm is trained and used for the prediction. Note that to train on the year 2011 we [split the data](/projects/DKU_ENERGY_CONSUMPTION/analysis/X7qGHMZu/ml/p/GZt3fYls/settings/) into a train set (dates before september) and a test set (rest of the year).

<p class="text-center">
<a href="/projects/DKU_ENERGY_CONSUMPTION/savedmodels/4wUI1vp8/versions/"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;Model</a><br/><br/>
</p>

7/ Then, an [R recipe calculates the MAPE](/projects/DKU_ENERGY_CONSUMPTION/recipes/compute_scores/), (Mean Average Percentage Error) score globally and individually for each industries/locations.

<p class="text-center">
<a href="/projects/DKU_ENERGY_CONSUMPTION/datasets/scores/explore/#filters"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;MAPE results</a><br/><br/>
</p>

The model performs better for ID01. this could be expected since this ID has a consumption that is more consistent over time and temperature.

Finally, have a look at the dashboard displaying the results of the model:

<p class="text-center">
<a href="/projects/DKU_ENERGY_CONSUMPTION/dashboards/CjL20Ee_model-results/view/"  class="btn btn-datasets-color btn-cta-big-mod"><i class="icon-dku-sample_project" class="btn-cta-big-mod-icon" />&nbsp;Dashboard</a><br/><br/>
</p>

<br/>
#Related content

To learn more, see these Dataiku Academy courses:

-  [Deploying predictive models](https://academy.dataiku.com/real-time-deployment). 
-  [Forecasting time series with R](https://knowledge.dataiku.com/latest/ml-analytics/time-series/tutorial-forecasting-r.html).