Add SVG outputs; save final plots; move repayment rate data

Author: jimjam-slam

.devcontainer/devcontainer.json

@@ -18,11 +18,11 @@
 		/* add apt packages (eg. system requirements for r packages) here.
 		   note that they are not frozen */
 		"ghcr.io/rocker-org/devcontainer-features/apt-packages:1": {
-			"packages": "libudunits2-dev,libxtst6,libxt6,libmagick++-dev,librsvg2-dev"
+			"packages": "libudunits2-dev,libxtst6,libxt6,libmagick++-dev,librsvg2-dev,libproj-dev"
 		},
 		/* add additional r packages here */
 		"ghcr.io/rocker-org/devcontainer-features/r-packages:1": {
-			"packages": "github::rstudio/renv,tidyverse,here,httpgd,janitor,github::360-info/themes360info"
+			"packages": "github::rstudio/renv,tidyverse,here,httpgd,janitor,ggalt,github::360-info/themes360info"
 		}
 	},
 

.gitignore

@@ -16,7 +16,8 @@ Thumbs.db
 /.quarto/
 
 # no raw data (this cannot be shared!)
-/data/raw
+/data/raw/*
+!/data/raw/repayment-*.csv
 
 # no exploratory plots that may expose individuals
 /out/explore

README.md

@@ -1,19 +1,94 @@
-# 360info Quarto scaffold
 
-Use this repo as a template to quickly start a 360info project in [Quarto](https://quarto.org).
 
-This template is designed to be used as a [dev container](https://containers.dev). You can work inside the container [locally using Docker and VScode](https://code.visualstudio.com/docs/devcontainers/containers), or you can open the project in GitHub Codespaces:
+# HELP debts
 
-![Screenshot of the code menu on a GitHub repository, showing the Codespaces tab](https://github.com/360-info/quarto-scaffold/assets/6520659/dc1afe15-2c1a-4a5a-8861-e3e131dacc02)
+### [📋 Add graphic to your story](blob/main/out/synth-heatmap.png?edshare) • [📊 Get the data](data) • [💻 Reproduce the analysis](#-reproduce-the-analysis)
 
-Add dependencies, such as R package or other tools, using the `.devcontainer/devcontainer.json` file.
+Visualises the impact of changing repayment rates and indexation on
+Australian student debt
 
-The project's `README.md` will be generated from `README.qmd`, not this file. Fill the frontmatter in there to customise the README. When the project is ready for publication, overwrite this file with the rendered `out/README.md`.
+## ♻️ Use + Remix rights
 
-## Adding analyses or embeds
+![[Creative Commons Attribution
+4.0](https://creativecommons.org/licenses/by/4.0)](https://mirrors.creativecommons.org/presskit/buttons/80x15/png/by.png)
 
-This template just includes the project scaffolding to et started. To create an analysis or embed document, use [`360-info/quarto-templates`](https://github.com/360-info/quarto-templates):
+These charts, as well as the analyses that underpin them, are available
+under a Creative Commons Attribution 4.0 licence. This includes
+commercial reuse and derivates.
 
-```bash
-quarto use templaye 360-info/quarto-templates/360-embed
+<!-- Do any of the data sources fall under a different licence? If so, describe the licence and which parts of the data fall under it here! if most of it does, change the above and replace LICENCE.md too -->
+
+Data in these charts comes from:
+
+<ul>  
+<li>  
+<a href="https://www.ato.gov.au/about-ato/research-and-statistics/in-detail/taxation-statistics/taxation-statistics-previous-editions/taxation-statistics-2016-17/statistics/individuals/individuals-sample-files">Australian
+Taxation Office</a>  
+</li>  
+</ul>
+
+**Please attribute 360info and the data sources when you use and remix
+these visualisations.**
+
+> [!WARNING]
+>
+> Please note that the source data for this analysis is not included
+> with this repository. It is available from the ATO at the above link
+> on request and upon submission of a confidentiality deed.
+
+## 💻 Reproduce the analysis
+
+### Quickstart: use the dev container
+
+This project comes with a ready-to-use [dev
+container](https://code.visualstudio.com/docs/remote/containers) that
+includes everything you need to reproduce the analysis (or do a similar
+one of your own!), including [R](https://r-project.org) and
+[Quarto](https://quarto.org).
+
+[![Open in GitHub
+Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/360-info/report-helpdebt?quickstart=1)
+
+If you have Docker installed, you can build and run the container
+locally:
+
+- Download or clone the project
+- Open it in [Visual Studio Code](https://code.visualstudio.com)
+- Run the **Remote-Containers: Reopen in Container** command
+
+Once the container has launched (it might take a few minutes to set up
+the first time), you can run the analysis scripts with:
+
+``` sh
+quarto render
 ```
+
+Or look for the `.qmd` files to modify the analysis.
+
+### Manual setup
+
+To setup a development environment manually,
+
+You’ll need to:
+
+- [Download and install Quarto](https://quarto.org/docs/get-started)
+- [Download the install R](https://www.r-project.org)
+- Satisfy the R package dependencies. In R:
+  - Install the [`renv`](https://rstudio.github.io/renv) package with
+    `install.packages("renv")`,
+  - Then run `renv::restore()` to install the R package dependencies.
+  - (For problems satisfying R package dependencies, refer to [Quarto’s
+    documentation on virtual
+    environments](https://quarto.org/docs/projects/virtual-environments.html).)
+
+Now, render the `.qmd` files to the `/out` directory with:
+
+``` sh
+quarto render
+```
+
+## ❓ Help
+
+If you find any problems with our analysis or charts, please feel free
+to [create an
+issue](https://github.com/360-info/report-helpdebt/issues/new)!

README.qmd

@@ -1,14 +1,13 @@
 ---
 format: gfm
 360readme:
-  title: Title here
-  summary: One line summary here
-  repo: report-example
-  share-url: https://example.360visuals.org/example
+  title: HELP debts
+  summary: Visualises the impact of changing repayment rates and indexation on Australian student debt
+  repo: report-helpdebt
+  share-url: blob/main/out/synth-heatmap.png
   data-sources: |
     <ul>
-      <li><a href="https://example.com">Example 1</a></li>
-      <li><a href="https://example.com">Example 2</a></li>
+      <li><a href="https://www.ato.gov.au/about-ato/research-and-statistics/in-detail/taxation-statistics/taxation-statistics-previous-editions/taxation-statistics-2016-17/statistics/individuals/individuals-sample-files">Australian Taxation Office</a></li>
     </ul>
 ---
 
@@ -31,6 +30,10 @@ Data in these charts comes from:
 
 **Please attribute 360info and the data sources when you use and remix these visualisations.**
 
+:::{.callout-warning}
+Please note that the source data for this analysis is not included with this repository. It is available from the ATO at the above link on request and upon submission of a confidentiality deed.
+:::
+
 ## 💻 Reproduce the analysis
 
 ### Quickstart: use the dev container

analysis/help/index.qmd

@@ -14,6 +14,7 @@ code-fold: true
 #| label: setup
 library(tidyverse)
 library(RColorBrewer)
+library(ggalt)
 library(here)
 dir.create(here("out", "explore"), showWarnings = FALSE, recursive = TRUE)
 ```
@@ -193,7 +194,7 @@ Let's tidy up the repayment rate tables. Note that the top rate for each year is
 
 ```{r}
 #| label: get-repayment-rates
-here("data", "repayment-rates-2004-2019.csv") |>
+here("data", "raw", "repayment-rates-2004-2019.csv") |>
   read_csv(col_types = "ccc") |>
   pivot_longer(-Year, names_to = "rate", values_to = "threshold") |>
   mutate(
@@ -208,7 +209,7 @@ here("data", "repayment-rates-2004-2019.csv") |>
   filter(!is.na(threshold)) ->
 repayment_rates_oldest
 
-here("data", "repayment-rates-2019-2021.csv") |>
+here("data", "raw", "repayment-rates-2019-2021.csv") |>
   read_csv(col_types = "ccc") |>
   pivot_longer(-Rate, names_to = "Year", values_to = "threshold") |>
   mutate(
@@ -248,7 +249,7 @@ tidy_ratetables <- function(path, suffix = c("*", "+")) {
 }
 
 tibble(
-  path = list.files(here("data"),
+  path = list.files(here("data", "raw"),
     pattern = glob2rx("repayment-rates-FY20??.csv"),
     full.names = TRUE)) ->
 rate_table_files_newest
@@ -275,7 +276,8 @@ repayment_rates_oldest |>
   rename(repay_rate = rate, repay_threshold = threshold) |>
   group_by(year_end) |>
   mutate(repay_threshold =
-    if_else(repay_rate == max(repay_rate, na.rm = TRUE), Inf, repay_threshold)) ->
+    if_else(repay_rate == max(repay_rate, na.rm = TRUE), Inf, repay_threshold)) |>
+  write_csv(here("data", "repayment-rates.csv"))->
 repayment_rates
 ```
 
@@ -945,6 +947,8 @@ stats_alldebtors_plot_scaled
 
 save_360plot(stats_alldebtors_plot_scaled,
   here("out", "stats-alldebtors-scaled-final.png"))
+save_360plot(stats_alldebtors_plot_scaled,
+  here("out", "stats-alldebtors-scaled-final.svg"))
 ```
 
 ### Wide for Datawrapper/Flourish
@@ -972,32 +976,24 @@ I'd like to make a heatmap that really shows where these categories lie. Instead
 ```{r}
 #| label: synthetic-data
 expand_grid(
-  year_end = c("2023*", "2023+"),
+  year_end = "2018",
   income = seq(0, 150000, by = 1000),
   help_debt = seq(0, 100000, by = 500)) ->
 synthetic
 
 # get the repayment rates under both scenarios for just 2024
 
-repayment_rates_new |>
-  filter(year_end == "2023*") |>
+repayment_rates_oldest |>
+  filter(year_end == "2018") |>
   mutate(repay_threshold =
     if_else(rate == max(rate, na.rm = TRUE), Inf, threshold)) |>
   select(year_end, repay_rate = rate, repay_threshold) ->
 latest_repayment_rates_nopolicy
 
-repayment_rates_new_alppolicy |>
-  filter(year_end == "2023+") |>
-  mutate(repay_threshold =
-    if_else(rate == max(rate, na.rm = TRUE), Inf, threshold)) |>
-  select(year_end, repay_rate = rate, repay_threshold) ->
-latest_repayment_rates_alppolicy
-
-latest_repayment_rates <-
-  bind_rows(latest_repayment_rates_nopolicy, latest_repayment_rates_alppolicy)
+latest_repayment_rates <- latest_repayment_rates_nopolicy
 
 indexation_rates |>
-  filter(str_detect(year_end, "2023")) ->
+  filter(str_detect(year_end, "2018")) ->
 latest_indexation_rates
 
 # now join them all
@@ -1019,18 +1015,15 @@ synthetic |>
       repay_amount < 1                 ~ "No repayment (low income)",
       net_reduction < -200              ~ "Debt grew despite payment",
       between(net_reduction, -200, 200) ~ "Indexation and repayment balanced",
-      net_reduction > 200               ~ "Debt reduced"),
-    scenario = case_match(year_end,
-      "2023*" ~ "Before 2024 pre-budget policy: 7.1% indexation",
-      "2023+" ~ "With 2024 pre-budget policy: 3.2% indexation")) ->
+      net_reduction > 200               ~ "Debt reduced")) ->
 synthetic_categories
 
 # visualise
 
 ggplot(synthetic_categories) +
   aes(x = income, y = help_debt) +
   geom_raster(aes(fill = help_category)) +
-  facet_wrap(vars(scenario)) +
+  # facet_wrap(vars(scenario)) +
   scale_fill_manual(
     values = c(
       "Debt grew despite payment" = "#c7321c",
@@ -1058,15 +1051,106 @@ ggplot(synthetic_categories) +
     x = "Income",
     y = "HELP loan balance",
     fill = NULL,
-    title = "HELP loans going backwards in 2023",
-    subtitle = "Debts that \"go backwards\" to remain last year for some people earning $50,000–70,000 despite new policy",
+    title = "HELP loans didn't go backwards before 2020",
+    subtitle = "Before 2020, the combination of repayment rates and indexation levels ensured that if you were making mandatory payments, your balance never rose.",
+    # subtitle = "Debts that \"go backwards\" to remain last year for some people earning $50,000–70,000 despite new policy",
     caption = paste(
       "**CHART:** James Goldie, 360info",
       "**DATA:** Australian Taxation Office",
       sep = "<br>")) ->
 synthetic_heatmap
 
 save_360plot(synthetic_heatmap,
-  here("out", "synth-heatmap.png"), shape = "sdtv-landscape")
+  here("out", "synth-heatmap-2018.png"), shape = "sdtv-landscape")
 ```
 
+### Repayment rates over time
+
+Although indexation is a big part of this, the changing repayment rates are also a big component. Let's visualise those too.
+
+
+```{r}
+repayment_rates |>
+  filter(!str_detect(year_end, fixed("+"))) |>
+  rename(threshold_upper = repay_threshold) |>
+  arrange(year_end, repay_rate) |>
+  group_by(year_end) |>
+  mutate(
+    year_end = as.integer(str_remove(year_end, fixed("*"))),
+    threshold_lower = lag(threshold_upper, 1)) |>
+  replace_na(list(threshold_lower = 0)) ->
+repayment_ranges
+
+pal_YlGnBu <- brewer.pal(7, "YlGnBu") |> colorRampPalette()
+year_count <- repayment_ranges |> pull(repay_rate) |> unique() |> length() 
+repay_rate_pal <- pal_YlGnBu(year_count)
+
+repayment_ranges |>
+  ungroup() |>
+  mutate(
+    repay_rate_fct = factor(repay_rate,
+      levels = seq(0, 0.1, by = 0.005),
+      labels = seq(0, 0.1, by = 0.005) |> scales::percent()),
+    repay_rate_fct = fct_drop(repay_rate_fct),
+    threshold_upper =
+      if_else(is.infinite(threshold_upper), 200000, threshold_upper)
+  ) |>
+  ggplot() +
+    aes(x = year_end, fill = repay_rate_fct) +
+    geom_ribbon(aes(ymin = threshold_lower, ymax = threshold_upper),
+      colour = "#00000044",
+      stat = "stepribbon") +
+    scale_fill_manual(values = c("white", repay_rate_pal)) +
+    scale_x_continuous(expand = expansion()) +
+    scale_y_continuous(expand = expansion(), labels = scales::label_currency()) +
+    coord_cartesian(ylim = c(NA, 160000)) +
+    annotate_360_glasslight(
+      x = 2006.5, y = 20000, hjust = 0,
+      label = paste(
+        "Until 2019, HELP loan repayments",
+        "**started at 4%.** This meant anyone",
+        "repaying a debt would see their",
+        "balance go down.",
+        sep = "<br>"
+      ),
+      size = 3
+    ) +
+    annotate_360_glasslight(
+      x = 2023.5, y = 25000, hjust = 1,
+      label = paste(
+        "The **lower rates** introduced since",
+        "then have meant people making small",
+        "payments end up going backwards,",
+        "as indexation overtakes their repayment.",
+        sep = "<br>"
+      ),
+      size = 3
+    ) +
+    theme_360(base_size = 14) +
+    theme(
+      panel.ontop = TRUE,
+      panel.grid.major.x = element_blank(),
+      panel.grid.minor.x = element_blank(),
+      panel.grid.major.y = element_line(colour = "#00000022"),
+      panel.grid.minor.y = element_blank(),
+    ) +
+    labs(
+      x = NULL, y = "HELP loan balance",
+      fill = "Repayment rate",
+      title = "HELP loan repayment rates",
+      subtitle = paste(
+        "The introduction of lower repayment rates in recent years has produced a new class of people",
+        "whose **debt repayments are overshadowed by indexation**",
+        sep = "<br>"),
+      caption = paste(
+        "**CHART:** James Goldie, 360info",
+        "**DATA:** Australian Parliament House, Australian Taxation Office",
+        sep = "<br>")) ->
+repayment_rates_plot
+
+save_360plot(repayment_rates_plot,
+  here("out", "repayment-rates.png"), shape = "sdtv-landscape")
+save_360plot(repayment_rates_plot,
+  here("out", "repayment-rates.svg"), shape = "sdtv-landscape")
+
+```