Overall workflow for qctimeseries

Overview

This vignette shows the end‑to‑end workflow for window‑by‑window QC of large time‑series:

1. Import your own data as a data frame with a time column and numeric variables.
2. Initialize flag columns with qc_add_flags().
3. Review & flag values interactively with qc_window_app().
4. Save progress to disk regularly (recommended each variable).
5. Summarize progress with qc_progress().
6. (Optional) Visual check with qc_check_plot().
7. Apply flags and export cleaned data.

Flag encoding: 1 = approved, 0 = unchecked, -1 = original NA, -2 = manual flag, -3 = unsure.

0) Install the package (first time only)

# Not run in vignette:
# install.packages("remotes")
# remotes::install_github("anthonydn/qctimeseries", build_vignettes = TRUE)

1) Start or resume a session

You can resume from prior work, or start fresh with your own source file.

# Loads an R object you saved earlier (see Section 5)
# This should define an object like `ibutton_example_qc` in your workspace
# load("ibutton_example_qc.RData")

ibutton_example <- read.csv(
  "https://raw.githubusercontent.com/anthonydn/snowmelt_belowground/master/data/ibuttons.csv")

# subset rows
ibutton_example <- ibutton_example[
  ibutton_example$block == 1 &
  ibutton_example$treatment == "C.N" &
  ibutton_example$tussock == "I",]

# convert datetime to POSIXct
ibutton_example$datetime <- as.POSIXct(ibutton_example$datetime)

# keep only the columns you want
ibutton_example <- ibutton_example[, c("datetime", "temp")]

ibutton_example_qc <- qc_add_flags(ibutton_example, vars = "temp")

2) Assess data checking progress

qc_progress() reports per‑variable completion. Use hide_complete = TRUE to focus on what remains.

qc_progress(ibutton_example_qc, hide_complete = FALSE)

Pick any variable with pct_checked < 100, then proceed to the app.

3) Review & flag each variable

qc_window_app() opens an interactive plotly graphical environment. Explanations of plotly graphical controls and the buttons and navigation are at the bottom of this document, after step 6.

# Change y_col to the variable you are about to check
ibutton_example_qc <- qc_window_app(ibutton_example_qc, time_col = "datetime",
  y_col   = "temp", # <- EDIT THIS
  win_hrs = 10000) # default is 168 (one-week windows)

The app returns the entire data frame when you click Done / Return, with the temp_qcflag column updated in place. Thus to record work done in the app, it must be assigned to an object, typically the same as the input data frame, but it could also be a new copy like ibutton_example_qc_copy

Tip: it’s common to work through variables one by one: run the app, finish a variable, save (next section), then move to the next variable.

Warning: When you hide flagged variables, they can still sometimes have flags changed if for example you click “Reset Window → Unchecked” thus it is a good idea to turn them back on to check before you are done working on that window.

4) Save progress (recommended after each window app session)

Save after each variable and before quitting RStudio, so you can resume later.

save(ibutton_example_qc, file = "ibutton_example_qc.RData")

Next session, load("ibutton_example_qc.RData") and continue.

5) Optional: Visual overview of a cleaned series

qc_check_plot(ibutton_example_qc, "temp", time_col = "DateTime")

6) Apply flags and export cleaned data

When you’re done, mask flagged values (set to NA) and, if you like, drop flag columns.

df_clean <- qc_apply_flags(ibutton_example_qc, drop_flags = TRUE)

Export

CSV (always available):

write.csv(df_clean, "cleaned_output.csv", row.names = FALSE)

Excel (requires a writer; two options):

# Option A: writexl (simple)
# install.packages("writexl")
writexl::write_xlsx(df_clean, "cleaned_output.xlsx")

# Option B: openxlsx (more control)
# install.packages("openxlsx")
openxlsx::write.xlsx(df_clean, "cleaned_output.xlsx")

Plotly controls (modebar) used in the app

Note: the app removes Plotly’s Autoscale and Reset axes buttons. Use the app’s Home zoom button to restore the default x‑range for the current window.

Selections you make (box/lasso) drive the Flag/Approve Selected buttons.

App controls (buttons & inputs)

Selections are made with Plotly Box Select or Lasso Select tools; the “Selected” actions below operate on those points.

Summary of Workflow

# 1. Import
data <- read.csv("path/to/input.csv", stringsAsFactors = FALSE)
data$DateTime <- as.POSIXct(data$DateTime, tz = "UTC")

# 2. Add QC flags
vars_to_qc <- c("temp", "hum")
data_qc <- qc_add_flags(data, vars = vars_to_qc)

# 3. Work variable by variable
data_qc <- qc_window_app(data_qc, y_col = "temp", time_col = "DateTime")
save(data_qc, file = "data_qc.RData")

# 4. Progress and optional plot
qc_progress(data_qc, hide_complete = TRUE)
# qc_check_plot(data_qc, "temp", time_col = "DateTime")

# 5. Apply and export
data_clean <- qc_apply_flags(data_qc, drop_flags = TRUE)
write.csv(data_clean, "data_clean.csv", row.names = FALSE)

Click-to-run QC (R Markdown template)

You can set up a simple QC environment so helpers with little to no R experience can review data by opening an editable R Markdown file and pressing “Run” on each chunk. The only thing they change is the variable name as they move through each variable that needs QC checking.

# 1) Import your data and add QC flags
dat <- read.csv("your_data.csv")  # or readxl::read_excel(...)
dat_qc <- qctimeseries::qc_add_flags(dat) # adds *_qcflag columns

# 2) Save the working object for checkers
save(dat_qc, file = "your_data_qc.RData")

# Via Rstudio Menu:
# File -> New File -> R Markdown -> From Template -> "Data checker - qctimeseries"

# Or create directly in your working directory:
rmarkdown::draft("your_data_checker.Rmd",
  template = "data-checker", package = "qctimeseries")