Assignment 5, finally!

assignment5.Rmd

@@ -1,5 +1,5 @@
 ---
-title: "Principle Component Aanalysis"
+title: "Vidya Madhavan Assignment 5 -Principle Component Aanalysis"
 output: html_document
 ---
 ## Data
@@ -16,7 +16,7 @@ The data you will be using comes from the Assistments online intelligent tutorin
 
 ## Start by uploading the data
 ```{r}
-D1 <- 
+D1 <- read.csv("assistments-confidence.csv", header=TRUE)
 
 ```
 
@@ -29,7 +29,9 @@ library(ggplot2)
 library(GGally)
 
 ggpairs(D1, 2:8, progress = FALSE) #ggpairs() draws a correlation plot between all the columns you identify by number (second option, you don't need the first column as it is the student ID) and progress = FALSE stops a progress bar appearing as it renders your plot
+```
 
+```{r}
 ggcorr(D1[,-1], method = c("everything", "pearson")) #ggcorr() doesn't have an explicit option to choose variables so we need to use matrix notation to drop the id variable. We then need to choose a "method" which determines how to treat missing values (here we choose to keep everything, and then which kind of correlation calculation to use, here we are using Pearson correlation, the other options are "kendall" or "spearman")
 
 #Study your correlogram images and save them, you will need them later. Take note of what is strongly related to the outcome variable of interest, mean_correct. 
@@ -38,7 +40,8 @@ ggcorr(D1[,-1], method = c("everything", "pearson")) #ggcorr() doesn't have an e
 ## Create a new data frame with the mean_correct variable removed, we want to keep that variable intact. The other variables will be included in our PCA.
 
 ```{r}
-D2 <- 
+library(dplyr)
+D2 <- select(D1,-id,-mean_correct)
 
 ```
 
@@ -67,21 +70,29 @@ plot(pca, type = "lines")
 ```
 
 ## Decide which components you would drop and remove them from your data set.
+Since P6 has a very small proportion of variance, I would remove it 
+
 
 ## Part II
 
 ```{r}
 #Now, create a data frame of the transformed data from your pca.
 
-D3 <- 
+D3 <- data.frame(pca$x)
+```
 
 #Attach the variable "mean_correct" from your original data frame to D3.
 
-
+```{r}
+D3 <- data.frame(D3,D1$mean_correct)
+```
 
 #Now re-run your correlation plots between the transformed data and mean_correct. If you had dropped some components would you have lost important infomation about mean_correct?
-
-
+```{r}
+ggpairs(D3, progress = FALSE)
+ggcorr(D3, method = c("everything", "pearson"))
+```
+Even though PC6 represented a very small portion of the correlation, it was strongly related to mean_correct. So yes, we would have lost valuable information about the variable of interest.
 
 ```
 ## Now print out the loadings for the components you generated:
@@ -91,24 +102,116 @@ pca$rotation
 
 #Examine the eigenvectors, notice that they are a little difficult to interpret. It is much easier to make sense of them if we make them proportional within each component
 
-loadings <- abs(pca$rotation) #abs() will make all eigenvectors positive
+loadings <- abs(pca$rotation) 
+#abs() will make all eigenvectors positive
 
 #Now examine your components and try to come up with substantive descriptions of what some might represent?
+#Answers
+#1) PC1 represents how much effort or how many attempts students took, since mean hint, mean attempt and problems attempted have high loading scores
+#2)PC2 represents prior knowledge of students to some extent, since the loading score for prior problem correct and prior problem count are high
+#3)PC3 shows high mean confidence of students, probably related to higher correctness in prior problems.
+#4)PC4 represents how much prior attempts (practice) students have with their confidence levels
+#5)PC5 represents students’ attempts in a general sense
+#6)PC6 represents mean hints or help needed?
 
 #You can generate a biplot to help you, though these can be a bit confusing. They plot the transformed data by the first two components. Therefore, the axes represent the direction of maximum variance accounted for. Then mapped onto this point cloud are the original directions of the variables, depicted as red arrows. It is supposed to provide a visualization of which variables "go together". Variables that possibly represent the same underlying construct point in the same direction.  
 
 biplot(pca)
-
-
 ```
 # Part III  
 Also in this repository is a data set collected from TC students (tc-program-combos.csv) that shows how many students thought that a TC program was related to andother TC program. Students were shown three program names at a time and were asked which two of the three were most similar. Use PCA to look for components that represent related programs. Explain why you think there are relationships between these programs.
 
 ```{r}
+library(ggplot2)
+library(GGally)
+library(dplyr)
 
+DF1 <- read.csv("tc-program-combos.csv", header=TRUE)
 ```
 
+```{r}
+DF2 <- DF1[,-1]
+```
 
+```{r}
+ggcorr(DF2, method = c("everything", "pearson"))
+pca2 <- prcomp(DF2, scale. = TRUE)
+```
 
+```{r}
+pca2$sdev
+```
+```{r}
+pca2$sdev^2
+```
+
+```{r}
+summary(pca2)
+```
+
+```{r}
+plot(pca2, type = "lines")
+```
 
+```{r}
+pca2$rotation
+```
+
+```{r}
+pca2loadings <- abs(pca2$rotation)
+```
+
+```{r}
+biplot(pca2)
+```
 
+#Answers
+From the summary, we can see that there are 67 components out of which we can make logical assumptions with more confidence until PC4. It is observed that PCs indicate relatability in terms of the area of specialization amongst the courses listed as having highest loading.
+
+>PC1
+     Change.Leadership
+     Economics.and.Education
+     Education.Policy
+     Arts.Administration
+     Politics
+     School.Principals
+     Social.Organizational.Psychology
+     Private.School.Leadership
+     Cooperation.and.Conflict.Resolution
+     Leadership
+Students probably think these programs are related as they all appear to specialize in subjects like administration, leadership and policy, which fall under the general umbrella of courses that focus on developing leadership skills.
+
+
+>PC2
+     Clinical.Psychology
+     Neuroscience
+     Kinesiology
+     Physiology
+     Psychology
+     Health.Education
+     Behavior.Analysis
+     Nursing
+     Physical.Education
+Students probably think these programs are related as they all appear to specialize in subjects related to health, psychology, behaviour and generally focussed on the developing specialized knowledge about the human body and behaviour.
+
+
+>PC3
+     Design.and.Development.of.Digital.Games
+     Cognitive.Science
+     Mathematics
+     Learning.Analytics
+     Education.Technology
+     Creative.Technologies
+     Instructional.Technology.and.Media
+     Measurement..Evaluation.and.Statistics
+     Communication.Media.and.Learning.Technologies
+Students are probably able to quite easily group these programs together as they characteristically appear to include quantitative, analytical and tech related modules compared to other programs. They would all come under the general umbrella of specialization in a tech-related skill.
+
+>PC4
+     Linguistics
+     English.Education
+     Teaching.English
+     Literacy
+     Deaf.and.Hard.of.Hearing
+     Bilingual.Bicultural.Education
+Students think these programs are related as they all seem to include a language and education component, grouping them under the general umbrella of linguistics.