Skip to content

Fix profiled detection issued with custom columns and non-mandatory features in input data #8

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
abadiabosch wants to merge 2 commits into
base: main
Choose a base branch
from
+10 −6
Open

Fix profiled detection issued with custom columns and non-mandatory features in input data #8

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b39bb5e
Fixed issue #6. Not supporting custom column names
abadiabosch Aug 9, 2022
e02c3c4
Fixed issue #7. Expecting non-mandatory features in input data
abadiabosch Aug 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions R/preparation.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -345,15 +345,19 @@ detect_ts_calendar_model_outliers_window <- function(data,
#' @param data timeSeries Data time serie with potential profiled content
#' @return list <date> List of dates with profiled data

detect_profiled_data <- function(data){
detect_profiled_data <- function(data, valueColumn="value", localTimeColumn="localtime") {
if(period("PT1H")<=detect_time_step(data)){
# Create wide data frame with aggregated hourly demand per
# each of the days
data_wide <- data %>%
data_wide <- data %>%
rename(Qe = !!valueColumn) %>%
rename(localtime = !!localTimeColumn) %>%
mutate(
"hourly_lt"=as.POSIXct(format(localtime,"%Y-%m-%d %H"),
format="%Y-%m-%d %H",
tz = attr(localtime[1],"tzone"))
hourly_lt=as.POSIXct(format(localtime,"%Y-%m-%d %H"),
format="%Y-%m-%d %H",
tz = attr(localtime[1],"tzone")),
date = lubridate::as_date(hourly_lt),
hour = lubridate::hour(hourly_lt)
) %>%
group_by(hourly_lt) %>%
summarise(Qe=sum(Qe),date=first(date),hour=first(hour)) %>%
Expand Down Expand Up @@ -432,7 +436,7 @@ detect_ts_calendar_model_outliers <- function(data,
autoDetectProfiled = T) {
if(autoDetectProfiled==T)
# Ignore days if are profiled
daysThatAreOutliers <- c(daysThatAreOutliers, detect_profiled_data(data))
daysThatAreOutliers <- c(daysThatAreOutliers, detect_profiled_data(data, valueColumn, localTimeColumn))
if(class(window)=="numeric"){
start <- as.integer(seconds(data$time[1]))
# Windowize data in order to obtain outliers in data samples
Expand Down