STA 9750: Tips and Tricks

Miscellaneous RStudio and Git Advice

1. When you are writing quarto documents in RStudio, do not use the “Visual” editor mode. It will seem tempting at first, but it will lead to difficult and hard to debug errors down the road. You should disable it in your RStudio settings (if possible) and include editor: source in the header of quarto documents to keep it from opening automatically.

2. git supports a huge variety of programming workflow and was originally designed to allow large teams of programmers to work together. This flexibility leads to a few “sharp edges”, however.

   You will have the best experience with git in this course if:

   1. You commit early and often. git commits are essentially free (40 bytes - less than a text!) and give you the ability to roll back changes if you get into a bad situation.

   2. You do all your work on a single computer. Keeping files and git synchronized across multiple machines is possible, but trickier. If, for whatever reason, you need to use multiple computers, reach out to course staff in office hours for advice.

      In particular, even though it may be tempting to edit files directly through the GitHub web interface, resist! You will get into situations where your local copy and the copy on GitHub are out of sync and you will have issues pushing your work onto GitHub.

      (It is possible to fix this, but you’ll have to learn a lot more about how git works, far beyond what is expected of you in this course.)

   3. You git push at the end of every work session. This is the stage when things are most likely to go wrong, so you want to be pushing early and often, giving you the best chance to spot and fix an issues early.

   4. You work in a “regular” file system. Synchronization and back up tools like DropBox or OneDrive perform magic behind the scenes that doesn’t always play well with git. You don’t need to use these tools with git since git (used properly and in conjunction with GitHub) will give you automatic history and backups.

   5. You avoid anti-virus issues. If you are on a Windows machine, tell your anti-virus software to ignore the directory containing your work for this course. Both git and quarto create and delete lots of little files as they execute; certain anti-virus tools interpret this as evidence of a cyber attack and attempt to lock the file system, leading to a variety of subtle and hard to explain errors.

   6. You are careful about what files you add in git. You should generally only include your qmd files and the rendered outputs in the docs folder. The build_site script included in Mini-Project #00 is helpful, but not bullet-proof, here.

3. Use the file names and directory structures specified in the course instructions. I use a variety of automated processing tools to coordinate this course and if you do not follow instructions closely, your assignments will not be processed. I will typically manually handle your submissions, but with a penalty applied to your grade. It is better for all parties if you avoid this.

   The helper scripts below can be used to confirm whether your submissions are properly formatted.

4. Avoid using file or directory names with spaces in them. Some software, particularly on Windows computers, does not handle spaces in directory or file names properly.

Helper Scripts

The following helper scripts can be used to help automate several students tasks in this course. To use any of them, run

source("https://michael-weylandt.com/STA9750/load_helpers.R")

and then run the name of the relevant script.

Mini-Projects

Start a New Mini-Project

This function will create a new file for a mini-project and pre-populate it with a basic skeleton and some licensing information. Note that this function should only be used for the graded Mini-Projects (MP#01 to MP#04) and not for MP#00.

mp_start <- function(N, github_id){
    if(!require("yaml")) install.packages("yaml"); library(yaml)
    if(!require("gh")) install.packages("gh"); library(gh)
    if(!require("tidyverse")) install.packages("tidyverse"); library(tidyverse)
    if(!require("rvest")) install.packages("rvest"); library(rvest)
    if(!require("glue")) install.packages("glue"); library(glue)
    if(!require("httr2")) install.packages("httr2"); library(httr2)
    
    if(missing(N)){
        N <- menu(title="Which Mini-Project would you like to submit on GitHub?", 
                  choices=c(1, 2, 3, 4))
    }
    
    `%not.in%` <- Negate(`%in%`)
    
    if(N %not.in% c(1, 2, 3, 4)){
        stop("Mini-Project Number not Recognized")
    }
    
    
    fname <- glue("mp0{N}.qmd")
    
    
    if(file.exists(fname)){
        stop(paste0("A ", sQuote(fname), " file already exists in this directory."))
    }
    
    
    mp_url <- glue("https://michael-weylandt.com/STA9750/miniprojects/mini0{N}.html")
    
    if(missing(github_id)){
        github_id <- readline("What is your GitHub ID? ")
    }
    
    variables <- read_yaml("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/_variables.yml")
    course_repo  <- variables$course$repo
    course_short <- variables$course$short
    
    if(!endsWith(getwd(), course_repo)){
        stop(paste("This command should be run in your", course_repo, "directory only."))
    }
    
    mp_skeleton_url <- "https://michael-weylandt.com/STA9750/mp_template.qmd"
    mp_instructions_url <- glue("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/miniprojects/mini0{N}.qmd")
    
    mp_instructions <- readLines(mp_instructions_url)
    
    dash_lines <- which(str_equal(mp_instructions, "---"))
    
    mp_header <- mp_instructions[(dash_lines[1] + 1):(dash_lines[2]-1)]
    
    mp_variables <- yaml.load(mp_header)
    
    mp_skeleton <- mp_skeleton_url |> 
        readLines() |> 
        str_replace_all(fixed("{GITHUB_ID}"), github_id) |>
        str_replace_all(fixed("{N}"), N) |>
        str_replace_all(fixed("{MP_TITLE}"), mp_variables$mp_title) |>
        str_replace_all(fixed("{MP_SKILLS}"), mp_variables$mp_skills |> str_split(", ")  |> pluck(1) |> paste(collapse = "/")) |>
        str_replace_all(fixed("{MP_DATA}"), mp_variables$mp_application |> str_split(", ")  |> pluck(1) |> paste(collapse = "/"))
    
    writeLines(mp_skeleton, fname)
    
    if("tools:rstudio" %in% search()){
        cat("Opening miniproject file")
        invisible(rstudioapi::documentOpen(fname))
    } else {
        cat("You can now open", sQuote(fname), "and begin the mini-project.")
    }
}

Push Mini-Project to GitHub

TODO

Check Mini-Project Prior to Submission

When you are about ready to submit a mini-project, the following script can be used to make sure all relevant files have been uploaded. In particular, it verifies that the source mp0N.qmd and the rendered docs/mp0N.html files are on GitHub, then it also checks for the css, js and image files, alerting if these cannot be loaded. Note that this check is only ‘first-order’ (so a css imported by another css file will not be verified) but this is enough to catch most problems.

mp_submission_ready <- function(N, github_id){
    if(!require("yaml")) install.packages("yaml"); library(yaml)
    if(!require("gh")) install.packages("gh"); library(gh)
    if(!require("tidyverse")) install.packages("tidyverse"); library(tidyverse)
    if(!require("rvest")) install.packages("rvest"); library(rvest)
    if(!require("glue")) install.packages("glue"); library(glue)
    if(!require("httr2")) install.packages("httr2"); library(httr2)  
    
    if(missing(N)){
        N <- menu(title="Which Mini-Project would you like to check was properly submitted on GitHub?", 
                  choices=c(0, 1, 2, 3, 4))
    }
    
    mp_url <- glue("https://michael-weylandt.com/STA9750/miniprojects/mini0{N}.html")
    
    mp_text <- read_html(mp_url) |> html_element("#submission-text") |> html_text()
    
    if(missing(github_id)){
        github_id <- readline("What is your GitHub ID? ")
    }
    
    variables <- read_yaml("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/_variables.yml")
    course_repo  <- variables$course$repo
    course_short <- variables$course$short
BASE_URL <- glue("https://{github_id}.github.io/{course_repo}/")

page_resp <- request(BASE_URL) |>
    req_url_path_append(glue("mp0{N}.html")) |>
    req_error(is_error = \(r) FALSE) |> 
    req_perform()

if(resp_is_error(page_resp)){
    stop(glue("I was unable to access your submission at {page_resp$url}. Please make sure that your content has been successfully uploaded to GitHub."))
}

cat(glue("I was able to access your submission at {page_resp$url}, now checking JS and CSS files."), "\n")

SUPPORT_FILES <- page_resp |> resp_body_html() |> html_elements("link") |> html_attr("href")

SUPPORT_STATUS <- map_lgl(SUPPORT_FILES, function(s){
    supp_resp_error <- request(BASE_URL) |> 
        req_url_path_append(s) |>
        req_error(is_error = \(r) FALSE) |>
        req_perform() |>
        resp_is_error()
    
    if(supp_resp_error){
        cat(glue("- I could not find the file {s} that your site attempts to load.\n  Please make sure docs/{s} has been uploaded via Git to your GitHub.\n"))
        cat("\n")
    }
    !supp_resp_error
}) 

if(!all(SUPPORT_STATUS)){
    stop("Necessary support files could not be verified. Please visually inspect all elements of your site before submission.")
    browseURL(page_resp$url)
}

    IMAGE_FILES <- page_resp |> resp_body_html() |> html_elements("img") |> html_attr("src")
    
    IMAGE_STATUS <- map_lgl(IMAGE_FILES, function(s){
        img_resp_error <- request(BASE_URL) |> 
            req_url_path_append(s) |>
            req_error(is_error = \(r) FALSE) |>
            req_perform() |>
            resp_is_error()
        
        if(img_resp_error){
            cat(glue("- I could not find the image file {s} that your site attempts to load.\n  Please make sure docs/{s} has been uploaded via Git to your GitHub.\n"))
            cat("\n")
        }
        !img_resp_error
    }) 
    
    if(!all(IMAGE_STATUS)){
        stop("Necessary image files could not be verified. Please visually inspect all elements of your site before submission.")
        browseURL(page_resp$url)
    }
    
    if(N == 0){
        raw_url <- glue("https://raw.githubusercontent.com/{github_id}/{course_repo}/refs/heads/main/index.qmd")
    } else {
        raw_url <- glue("https://raw.githubusercontent.com/{github_id}/{course_repo}/refs/heads/main/mp0{N}.qmd")
    }
    
    resp_raw <- request(raw_url) |> 
        req_error(is_error = \(r) FALSE) |>
        req_perform()
    
    if(resp_is_error(resp_raw)){
        ## Try again with 'master' instead of 'main' branch
        raw_url_master <- str_replace(raw_url, "main", "master")
        resp_raw <- request(raw_url_master) |> 
            req_error(is_error = \(r) FALSE) |>
            req_perform()
    }
    
    if(resp_is_error(resp_raw)){
        cat("I cannot find the source qmd document at", raw_url, "or", 
            raw_url_master, ".\n",
            "Please confirm it was correctly submitted and try again.\n",
            "This document is needed for automated code quality checks.\n")
        
        stop("Miniproject not ready for submission.")
    }
    
    cat("Congratulations! Your mini-project appears to be ready for submission!\n")
    cat("Visually verify it one more time before submission.\n")
    cat("Then run the following command to submit on GitHub.\n")
    cat("\n")
    cat(glue('source("https://michael-weylandt.com/STA9750/load_helpers.R"); mp_submission_create(N={N}, github_id="{github_id}")'), "\n")
    cat("Then upload a PDF of your submission to Brightspace.\n")
    invisible(TRUE)
}

Submit Mini-Project via GitHub Issue

The following functon will create the GitHub issue necessary to submit a mini-project:

mp_submission_create <- function(N, github_id){
    if(!require("yaml")) install.packages("yaml"); library(yaml)
    if(!require("gh")) install.packages("gh"); library(gh)
    if(!require("tidyverse")) install.packages("tidyverse"); library(tidyverse)
    if(!require("rvest")) install.packages("rvest"); library(rvest)
    if(!require("glue")) install.packages("glue"); library(glue)
    if(!require("httr2")) install.packages("httr2"); library(httr2)
    
    if(missing(N)){
        N <- menu(title="Which Mini-Project would you like to submit on GitHub?", 
                  choices=c(0, 1, 2, 3, 4))
    }
    
    mp_url <- glue("https://michael-weylandt.com/STA9750/miniprojects/mini0{N}.html")
    
    mp_text <- read_html(mp_url) |> html_element("#submission-text") |> html_text()
    
    if(missing(github_id)){
        github_id <- readline("What is your GitHub ID? ")
    }
    
    variables <- read_yaml("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/_variables.yml")
    course_repo  <- variables$course$repo
    course_short <- variables$course$short
    
    title <- glue("{course_short} {github_id} MiniProject #0{N}")
    
    body <- mp_text |> str_replace("<GITHUB_ID>", github_id)
    
    r <- request("https://github.com/") |>
        req_url_path_append("michaelweylandt") |>
        req_url_path_append(course_repo) |>
        req_url_path_append("issues/new") |>
        req_url_query(title=title, 
                      body=body) 
    
    browseURL(r$url)
}

Verify Mini-Project Submission

The following function will confirm that the submission issue is properly formatted.

mp_submission_verify <- function(N, github_id){
    if(!require("yaml")) install.packages("yaml"); library(yaml)
    if(!require("gh")) install.packages("gh"); library(gh)
    if(!require("tidyverse")) install.packages("tidyverse"); library(tidyverse)
    if(!require("rvest")) install.packages("rvest"); library(rvest)
    if(!require("glue")) install.packages("glue"); library(glue)
    if(!require("httr2")) install.packages("httr2"); library(httr2)  
    
    if(missing(N)){
        N <- menu(title="Which Mini-Project would you like to check was properly submitted on GitHub?", 
                  choices=c(0, 1, 2, 3, 4))
    }
    
    mp_url <- glue("https://michael-weylandt.com/STA9750/miniprojects/mini0{N}.html")
    
    mp_text <- read_html(mp_url) |> html_element("#submission-text") |> html_text()
    
    if(missing(github_id)){
        github_id <- readline("What is your GitHub ID? ")
    }
    
    variables <- read_yaml("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/_variables.yml")
    course_repo  <- variables$course$repo
    course_short <- variables$course$short
    
    title <- glue("{course_short} {github_id} MiniProject #0{N}")
    
    body <- mp_text |> str_replace("<GITHUB_ID>", github_id) |> str_squish()
    
    page <- 1
    
    issues <- list()
    
    while((length(issues) %% 100) == 0){
        new_issues <- gh("/repos/michaelweylandt/{repo}/issues?state=all&per_page=100&page={page}", 
                         repo=course_repo, 
                         page=page)
        
        issues <- c(issues, new_issues)
        page <- page + 1
    }
    
    issue_names <- vapply(issues, function(x) str_squish(x$title), "")
    
    name_match <- which(issue_names == title)
    case_insensitive_match <- which(tolower(issue_names) == tolower(title))
    
    if((length(name_match) == 0) & (length(case_insensitive_match) == 0)){
        cat("I could not find a unique issue with the title:\n", 
            "    ", sQuote(title),"\n",
            "The issues I found had the following titles:\n",
            paste(c("", issue_names), collapse="\n - "), "\n",
            "If something on that list looks correct, please check",
            "capitalization and punctuation.\n")
        stop("MINIPROJECT NOT SUBMITTED CORRECTLY.")
    } else if((length(name_match) == 0) & (length(case_insensitive_match) >= 0)) {
        cat("I have found a partial match (differing by case only).\n",
            "- Searching for: ", title, , "\n", 
            "- Found: ", issue_names[case_insensitive_match], ".\n",
            "Attempting to proceed with partial match issue")
        name_match <- case_insensitive_match
    } else if(length(name_match) > 1){
        cat("I found multiple issues with the title:\n", 
            "    ", sQuote(title),"\n",
            "Please change the names of issues so that there is only", 
            "issue with the desired name. (Note that closing an issue is",
            "not sufficient.)\n")
        
        stop("MINIPROJECT NOT SUBMITTED CORRECTLY.")
    }
    
    issue <- issues[[name_match]]
    
    issue_num <- issue$number
    issue_url <- issue$html_url
    issue_body <- issue$body |> str_squish()
    
    cat("Identified Issue #", issue_num, "at", issue_url, "as possible candidate.\n")
    
    if(issue$state != "open"){
        cat("Issue does not appear to be in 'open' status. Please",
            "confirm the issue is open and try again.\n")
        stop("MINIPROJECT NOT SUBMITTED CORRECTLY.")
    }
    
    if(!str_equal(issue_body, body, ignore_case = TRUE)){
        cat("Issue does not appear to exactly match the correct body text.",
            "This is not necessarily an issue since the expected link is present",
            "but you may want to confirm all relevant",
            "information is included.\n")
    }
    
    # This isn't quite general enough, but I think it covers everything
    # we'll see in this course. 
    URL_REGEX <- "http[A-Za-z0-9:/\\-\\.]*"
    
    urls <- str_match_all(paste(issue_body, "\n Thanks!\n", sep=""), 
                          URL_REGEX)[[1]] |> unique()
    expected_url <- str_extract(body, URL_REGEX)
    
    if(length(urls) > 1){
        cat("The following URLs were found in the issue text:",
            paste(c("", urls), collapse="\n - "), 
            "Only one URL was expected; please remove any extras.\n")
        stop("MINIPROJECT NOT SUBMITTED CORRECTLY.")
    }
    
    submitted_url <- urls[1]
    
    if(!str_equal(submitted_url, expected_url, ignore_case = TRUE)){
        cat("The submitted URL does not match the Mini-Project instructions.\n", 
            "Expected:\n - ", expected_url, "\n", 
            "Submitted:\n - ", submitted_url, "\n", 
            "Please correct any differences and try again.\n")
        stop("MINIPROJECT NOT SUBMITTED CORRECTLY.")
    }
    
    resp <- request(submitted_url) |> 
        req_error(is_error = \(r) FALSE) |>
        req_perform()
    
    if(resp_is_error(resp)){
        cat("Something appears to be incorrect at the URL: ", 
            submitted_url, 
            "\n Please confirm that it is working as expected and try again.")
        browseURL(submitted_url)
        stop("MINIPROJECT NOT SUBMITTED SUCCESSFULLY.")
    }
    
    if(N == 0){
        raw_url <- glue("https://raw.githubusercontent.com/{github_id}/{course_repo}/refs/heads/main/index.qmd")
    } else {
        raw_url <- glue("https://raw.githubusercontent.com/{github_id}/{course_repo}/refs/heads/main/mp0{N}.qmd")
    }
    
    resp_raw <- request(raw_url) |> 
        req_error(is_error = \(r) FALSE) |>
        req_perform()
    
    if(resp_is_error(resp_raw)){
        ## Try again with 'master' instead of 'main' branch
        raw_url_master <- str_replace(raw_url, "main", "master")
        resp_raw <- request(raw_url_master) |> 
            req_error(is_error = \(r) FALSE) |>
            req_perform()
    }
    
    if(resp_is_error(resp_raw)){
        cat("I cannot find the source qmd document at", raw_url, "or", 
            raw_url_master, ".\n",
            "Please confirm it was correctly submitted and try again.\n",
            "This document is needed for automated code quality checks.\n")
        
        stop("MINIPROJECT NOT SUBMITTED SUCCESSFULLY.")
    }
    
    cat("Congratulations! Your mini-project appears to have been submitted correctly!\n")
    invisible(TRUE)
}

Peer Feedback

Find Peer Feedback Assigned to Me

mp_feedback_locate <- function(N, github_id){
    if(!require("yaml")) install.packages("yaml"); library(yaml)
    if(!require("gh")) install.packages("gh"); library(gh)
    if(!require("tidyverse")) install.packages("tidyverse"); library(tidyverse)
    if(!require("rvest")) install.packages("rvest"); library(rvest)
    if(!require("glue")) install.packages("glue"); library(glue)
    if(!require("httr2")) install.packages("httr2"); library(httr2)
    
    if(missing(N)){
        N <- menu(title="Which Mini-Project's Peer Feedback cycle is it currently?", 
                  choices=c(0, 1, 2, 3, 4))
    }
    
    if(missing(github_id)){
        github_id <- readline("What is your GitHub ID? ")
    }
    
    page <- 1
    issues <- list()
    
    variables <- read_yaml("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/_variables.yml")
    course_repo  <- variables$course$repo
    course_short <- variables$course$short
    
    while((length(issues) %% 100) == 0){
        new_issues <- gh("/repos/michaelweylandt/{repo}/issues?state=all&per_page=100&page={page}", 
                         repo=course_repo, 
                         page=page)
        
        issues <- c(issues, new_issues)
        page <- page + 1
    }
    
    pf_urls <- issues |> 
        keep(~ str_detect(.x$title, glue("#0{N}"))) |>
        keep(~ !str_equal(.x$user$login, github_id, ignore_case=TRUE)) |>
        map(~data.frame(html=.x$html_url, comments=.x$comments_url, body=.x$body)) |>
        keep(~any(str_detect(map_chr(gh(.x$comments), "body"), github_id) | 
                  str_detect(.x$body, github_id)
                  )) |>
        map("html") |>
        list_c()
    
    cat(glue("I have found several MP#0{N} issues that may be assigned to you.\n"),
        "Please review the following:\n")
    for(pf in pf_urls){
        cat(" - ", pf, "\n")
    }
    
    to_browser <- menu(title="Would you like me to open these in your web browser?", 
                       choices=c("Yes", "No"))
    
    if(to_browser == 1){
        for(pf in pf_urls){
            browseURL(pf)
        }
    }
    invisible(TRUE)
}

Create Peer Feedback Comment

mp_feedback_submit <- function(N, peer_id){
    if(!require("yaml")) install.packages("yaml"); library(yaml)
    if(!require("gh")) install.packages("gh"); library(gh)
    if(!require("tidyverse")) install.packages("tidyverse"); library(tidyverse)
    if(!require("rvest")) install.packages("rvest"); library(rvest)
    if(!require("glue")) install.packages("glue"); library(glue)
    if(!require("clipr")) install.packages("clipr"); library(clipr)
    if(!require("httr2")) install.packages("httr2"); library(httr2)
    
    if(missing(N)){
        N <- menu(title="Which Mini-Project's Peer Feedback would you like to check was properly submitted on GitHub?", 
                  choices=c(0, 1, 2, 3, 4))
    }
    
    if(missing(peer_id)){
        peer_id <- readline("What is your Peer's GitHub ID? ")
    }
    
    variables <- read_yaml("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/_variables.yml")
    course_repo  <- variables$course$repo
    course_short <- variables$course$short
    
    title <- glue("{course_short} {peer_id} MiniProject #0{N}")
    
    page <- 1
    
    issues <- list()
    
    while((length(issues) %% 100) == 0){
        new_issues <- gh("/repos/michaelweylandt/{repo}/issues?state=all&per_page=100&page={page}", 
                         repo=course_repo, 
                         page=page)
        
        issues <- c(issues, new_issues)
        page <- page + 1
    }
    
    issue_names <- vapply(issues, function(x) str_squish(x$title), "")
    
    name_match <- which(issue_names == title)
    
    if(length(name_match) != 1){
        cat("I could not find a unique issue with the title:\n", 
            "    ", sQuote(title),"\n",
            "I'm afraid I can't verify whether the peer feedback was submitted properly\n", 
            "but the issue likely lies with the submittor, not with your feedback.")
        stop("PEER FEEDBACK NOT VERIFIED.")
    } 
    
    issue <- issues[[name_match]]
    
    issue_url <- issue$html_url
    
    template_url <- "https://michael-weylandt.com/STA9750/miniprojects.html"
    
    template_text <- read_html(template_url) |> 
        html_element("#peer-feedback-template") |> 
        html_text() |>
        str_trim() 
    
    template_categories <- c(
        "Written Communication", 
        "Project Skeleton", 
        "Formatting & Display", 
        "Code Quality", 
        "Data Preparation", 
        "Extra Credit"
    )
    
    overall <- readline("Overall Comments: (Hit enter if no comments) ")
    
    template_text <- template_text |> str_replace("OPTIONAL TEXT", overall)
    
    for(category in template_categories){
        has_score <- FALSE
        
        while(!has_score){
            score <- readline(glue("On a scale of 0 to 10, how many points does {peer_id} deserve for {category}? "))
            
            score <- as.integer(score)
            
            if(!is.na(score) & (score >= 0) & (score <= 10)){
                has_score <- TRUE
            }
        }
        
        has_score <- FALSE
        
        text <- readline(glue("Why did you give {peer_id} {score} points for {category}? "))
        
        template_text <- template_text |>
            str_replace("NN", as.character(score)) |>
            str_replace("TEXT TEXT TEXT", as.character(text))
    }
    
    write_clip(template_text)
    
    readline(paste0(
        "After you hit Enter / Return, you will be taken to a GitHub page.\n",
        "Paste from the clipboard into the comment box at the bottom to\n",
        "pre-populate your peer feedback. Then hit 'Comment' to finish your\n",
        "submission. "))
    
    browseURL(issue_url)
    
    cat(glue("You can now use `mp_feedback_verify({N}, peer_id={peer_id})`\n",
             "to confirm your submission was properly formatted.\n"))
}

Verify Peer Feedback Properly Formatted

mp_feedback_verify <- function(N, github_id, peer_id){
    if(!require("yaml")) install.packages("yaml"); library(yaml)
    if(!require("gh")) install.packages("gh"); library(gh)
    if(!require("tidyverse")) install.packages("tidyverse"); library(tidyverse)
    if(!require("rvest")) install.packages("rvest"); library(rvest)
    if(!require("glue")) install.packages("glue"); library(glue)
    
    if(missing(N)){
        N <- menu(title="Which Mini-Project's Peer Feedback would you like to check was properly submitted on GitHub?", 
                  choices=c(0, 1, 2, 3, 4))
    }
    
    if(missing(github_id)){
        github_id <- readline("What is your GitHub ID? ")
    }
    
    if(missing(peer_id)){
        peer_id <- readline("What is your Peer's GitHub ID? ")
    }
    
    variables <- read_yaml("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/_variables.yml")
    course_repo  <- variables$course$repo
    course_short <- variables$course$short
    
    template_url <- "https://michael-weylandt.com/STA9750/miniprojects.html"
    
    template_text <- read_html(template_url) |> 
        html_element("#peer-feedback-template") |> 
        html_text() |>
        str_trim() |>
        str_replace_all("OPTIONAL TEXT", "(.*)") |>
        str_replace_all("NN", "(.*)") |>
        str_replace_all("TEXT TEXT TEXT", "(.*)")
    
    title <- glue("{course_short} {peer_id} MiniProject #0{N}")
    
    page <- 1
    
    issues <- list()
    
    while((length(issues) %% 100) == 0){
        new_issues <- gh("/repos/michaelweylandt/{repo}/issues?state=all&per_page=100&page={page}", 
                         repo=course_repo, 
                         page=page)
        
        issues <- c(issues, new_issues)
        page <- page + 1
    }
    
    issue_names <- vapply(issues, function(x) str_squish(x$title), "")
    
    name_match <- which(issue_names == title)
    
    if(length(name_match) != 1){
        cat("I could not find a unique issue with the title:\n", 
            "    ", sQuote(title),"\n",
            "I'm afraid I can't verify whether the peer feedback was submitted properly\n", 
            "but the issue likely lies with the submittor, not with your feedback.")
        stop("PEER FEEDBACK NOT VERIFIED.")
    } 
    
    issue <- issues[[name_match]]
    
    issue_url <- issue$html_url
    issue_comment_url <- issue$comments_url
    
    comments <- gh(issue_comment_url)
    
    commenters <- vapply(comments, function(x) x$user$login, "")
    
    comment_num <- which(commenters == github_id)
    
    if(length(comment_num) != 1){
        cat("I cannot identify your comment on the issue at", 
            sQuote(issue_url), ". Please verify that this is the correct link.\n")
        
        if(length(comment_num) == 0){
            cat("You do not appear to have commented on this issue.")
        } else {
            cat("You have left multiple comments on this issue.",
                "Please consolidate your feedback")
        }
        stop("PEER FEEDBACK NOT VERIFIED.")
    }
    
    comment_body <- comments[[comment_num]]$body
    
    cat("I have found your comment, with the following text:\n-----\n")
    
    cat(comment_body)
    
    cat("\n-----\n")
    
    comment_body  <- comment_body |> str_squish()
    template_text <- template_text |> str_squish()
    
    matches <- str_match_all(comment_body, template_text)[[1]][-1]
    
    if(anyNA(matches)){
        cat("I couldn't match the template to your comment. Please modify and try again.")
        stop("PEER FEEDBACK NOT VERIFIED.")
    }
    
    cat(glue("Congratulations! Your peer feedback on {peer_id}'s MP #0{N} appears properly formatted.\nThank you for your contributions to {course_short}!\n"))
    
    invisible(TRUE)
    
}

Count Words

The following script can be used to count words on a Quarto-generated web page.

count_words <- function(url){
    library(rvest)
    library(stringi)
    
    # Note that this includes code inside an inline block, but omits
    # full-sized code blocks
    read_html(url) |>
        html_elements("main p") |>
        html_text() |>
        stri_count_words() |>
        sum()
}

Lint Submission

The following will automatically run the lintr package on a submitted qmd document.

lint_submission <- function(N, peer_id){
    if(!require("yaml")) install.packages("yaml"); library(yaml)
    if(!require("gh")) install.packages("gh"); library(gh)
    if(!require("tidyverse")) install.packages("tidyverse"); library(tidyverse)
    if(!require("rvest")) install.packages("rvest"); library(rvest)
    if(!require("glue")) install.packages("glue"); library(glue)
    if(!require("lintr")) install.packages("lintr"); library(lintr)
    if(!require("httr2")) install.packages("httr2"); library(httr2)
    
    if(missing(N)){
        N <- menu(title="Which Mini-Project submission would you like to lint?", 
                  choices=c(1, 2, 3, 4))
    }
    
    if(missing(peer_id)){
        peer_id <- readline("What is your Peer's GitHub ID? ")
    }
    
    variables <- read_yaml("https://raw.githubusercontent.com/michaelweylandt/STA9750/refs/heads/main/_variables.yml")
    course_repo  <- variables$course$repo
    course_short <- variables$course$short
    
    raw_url <- glue("https://raw.githubusercontent.com/{peer_id}/{course_repo}/refs/heads/main/mp0{N}.qmd")
    
    cat("Attempting to access qmd source at", raw_url, ".\n")
    
    resp <- request(raw_url) |> 
        req_error(is_error = \(r) FALSE) |>
        req_perform()
    
    if(resp_is_error(resp)){
        cat("I could not access the raw qmd document. Attempting to open in browser...\n")
        browseURL(resp)
        return(FALSE)
    } else {
        cat("I was able to access the raw qmd document. Beginning to lint.\n")
        
        tf <- tempfile(pattern=glue("mp0{N}_{peer_id}_lint_document.qmd"))
        
        writeLines(resp_body_string(resp), tf)
        
        lintr::lint(tf)
    }
}

Advice for Learning to Program

Learning a programming language is in many ways like learning a new natural (human) language. We can look at best practices for learning spoken languages and use them to inform how you should best approach learning R in this course:

1. Immersion Learning: Immersion programs-intensive environments where students have to live their entire life in the language they are learning-are the best way to learn a language. Similarly, there really is no more effective way to learn to program than forcing yourself to use the tools of this course to do something. Small random isolated exercises will not build speed or fluency, just like speaking a single sentence is a poor substitute for actual conversation. In this course, I attempt to replicate this ‘immersive’ experience through realisitic mini-projects. You can get more experience by just committing to use R for every day tasks outside this class: say, e.g., you need to rename a bunch of files on your computer. Rather than doing this manually, force yourself to use R to do the same task. It may take a bit longer the first few times, but you will quickly develop additional programming fluency that will serve you well.

2. Don’t Memorize the Dictionary. New programmers often try to sit down and learn every function and every detail before attempting a task. This is impractical - there are tens of thousands of functions in just the small set of R we cover in this course - and honestly rather dull. It is far more efficient to take on a task and look up functions as you need them. (You can always Google “How do I XYZ in R?”) This strategy will naturally focus your attention on the most useful functions - since you will look them up many more times - and let you learn the additional bells-and-whistles of each function as you need it. In computer science education, this is often called JIT (Just-in-Time) learning.

   You wouldn’t sit down to learn French by reading the dictionary front-to-back and you shouldn’t try to do it for R either.

3. Accept that You Will Feel Foolish. You are learning a new language, one that comes with its own grammar (syntax), nouns and pronouns (variables), verbs (functions), and idioms. You won’t get it all right the first time-no matter how hard you work, developing fluency simply takes time. Thankfully for you, you have a ‘conversation partner’ who is willing to spend as much time with you as you want twenty-four hours a day, seven days a week. And unlike a human conversation partner, R won’t tell anyone if you say anything embarrassing.

4. Native Speakers Can be a Bit Judgmental R is picky about getting the grammar right. This can be frustrating - why can’t it just know what you want! - but it is ultimately to your benefit as it gives you an opportunity for immediate feedback and improvement. R’s error message are systematic, but are not always perfectly clear. (Writing clear error messages is actually quite difficult, as they only occur when something has already unexpectedly gone wrong.) As you start seeing the same error messages over and over, you will learn to make sense of them, but whenever you get a new one, simply Google it. You are almost certainly not the first person to have made any mistake. And never be embarrassed to ask the instructor or other course staff for help making sense of an error: you would never judge someone for using the English past progressive tense incorrectly after only 10 weeks of speaking English - give yourself the same charity.

5. You Won’t Think in the Foreign Language at First. When facing an analytics question, don’t immediately think “how can I code this in R?” Take a moment, figure out the steps you need to execute, and then translate your problem step by step from English to R. You will develop fluency quickly, but always make sure you know what you want to do and the steps you need to take before you jump straight into writing code.

   If you take a moment to clarify for yourself - in English or any other language - what you want the computer to do first, then the only “R-thinking” you have to do is translating your steps into R. This is often much easier than trying to write in R on a “blank page.” Compare the difficulty of translating a sentence you already have written from one language to another word-by-word to the much harder task of writing an essay from scratch.

Computers are Very Fast and Very Dumb

Your computer is a powerful tool, but it is fundamentally a very simple one. It executes millions or billions of simple steps every second, but it cannot read your mind. You have to be very precise in giving commands, as small omissions or errors will immediately lead to errors and other problems. As you fix these, you will often feel a bit silly for having made those mistakes in the first place. Don’t be embarrassed - working with a computer is like giving instructions to an enthusiastic toddler: if something goes wrong, it’s almost always because you assumed the toddler has more background knowledge and common sense than is actually there.

There are really only two fundamental programming problems you will come across in this course:

• You don’t know what you want the computer to do.
• You know what you want the computer to do, but you told it incorrectly.

Technologically, we are at a point in time where errors of that second category are becoming less common, but they are certainly still very present and you’ll encounter them repeatedly in this course. That said, whenever you’re facing a problem, try to categorize it as one of these two classes (or maybe a bit of each) and address them appropriately. The best way to deal with a “What should I do?” question is not the same as the best way to deal with a “How do I do it?” question. Being able to distinguish which type of question you are answering is the first step to being able to answer it effectively.