Assignment 3 - Social Network Analysis
+Xing Yixie
+2020/10/10
+ +Part I
+Start by installing the “igraph” package. Once you have installed igraph, load the package.
+Now upload the data file “comment-data.csv” as a data frame called “D1”. Each row represents a comment from one student to another so the first line shows that student “28” commented on the comment of student “21”. It also shows the gender of both students and the students’ main elective field of study (“major”").
+D1 <- read.csv("comment-data.csv", header = TRUE)Before you proceed, you will need to change the data type of the student id variable. Since it is a number R will automatically think it is an integer and code it as such (look at the list of variables by clicking on the data frame arrow in the Data pane. Here you will see the letters “int”" next to the stid variable, that stands for integer). However, in this case we are treating the variable as a category, there is no numeric meaning in the variable. So we need to change the format to be a category, what R calls a “factor”. We can do this with the following code:
+D1$comment.to <- as.factor(D1$comment.to)
+D1$comment.from <- as.factor(D1$comment.from)igraph requires data to be in a particular structure. There are several structures that it can use but we will be using a combination of an “edge list” and a “vertex list” in this assignment. As you might imagine the edge list contains a list of all the relationships between students and any characteristics of those edges that we might be interested in. There are two essential variables in the edge list a “from” variable and a “to” variable that descibe the relationships between vertices. While the vertex list contains all the characteristics of those vertices, in our case gender and major.
+So let’s convert our data into an edge list!
+First we will isolate the variables that are of interest: comment.from and comment.to
+library(dplyr)
+
+D2 <- select(D1, comment.to, comment.from) #select() chooses the columnsSince our data represnts every time a student makes a comment there are multiple rows when the same student comments more than once on another student’s video. We want to collapse these into a single row, with a variable that shows how many times a student-student pair appears.
+EDGE <- count(D2, comment.to, comment.from)
+
+names(EDGE) <- c("from", "to", "count")EDGE is your edge list. Now we need to make the vertex list, a list of all the students and their characteristics in our network. Because there are some students who only recieve comments and do not give any we will need to combine the comment.from and comment.to variables to produce a complete list.
+#First we will separate the commenters from our commentees
+V.FROM <- select(D1, comment.from, from.gender, from.major)
+
+#Now we will separate the commentees from our commenters
+V.TO <- select(D1, comment.to, to.gender, to.major)
+
+#Make sure that the from and to data frames have the same variables names
+names(V.FROM) <- c("id", "gender.from", "major.from")
+names(V.TO) <- c("id", "gender.to", "major.to")
+
+#Make sure that the id variable in both dataframes has the same number of levels
+lvls <- sort(union(levels(V.FROM$id), levels(V.TO$id)))
+
+VERTEX <- full_join(mutate(V.FROM, id=factor(id, levels=lvls)),
+ mutate(V.TO, id=factor(id, levels=lvls)), by = "id")
+
+#Fill in missing gender and major values - ifelse() will convert factors to numerical values so convert to character
+VERTEX$gender.from <- ifelse(is.na(VERTEX$gender.from) == TRUE, as.factor(as.character(VERTEX$gender.to)), as.factor(as.character(VERTEX$gender.from)))
+
+VERTEX$major.from <- ifelse(is.na(VERTEX$major.from) == TRUE, as.factor(as.character(VERTEX$major.to)), as.factor(as.character(VERTEX$major.from)))
+
+#Remove redundant gender and major variables
+VERTEX <- select(VERTEX, id, gender.from, major.from)
+
+#rename variables
+names(VERTEX) <- c("id", "gender", "major")
+
+#Remove all the repeats so that we just have a list of each student and their characteristics
+VERTEX <- unique(VERTEX)Now we have both a Vertex and Edge list it is time to plot our graph!
+#Load the igraph package
+
+library(igraph)
+
+#First we will make an object that contains the graph information using our two dataframes EDGE and VERTEX. Notice that we have made "directed = TRUE" - our graph is directed since comments are being given from one student to another.
+
+g <- graph.data.frame(EDGE, directed=TRUE, vertices=VERTEX)
+
+#Now we can plot our graph using the force directed graphing technique - our old friend Fruchertman-Reingold!
+
+plot(g,layout=layout.fruchterman.reingold)
#There are many ways to change the attributes of the graph to represent different characteristics of the newtork. For example, we can color the nodes according to gender.
+
+plot(g,layout=layout.fruchterman.reingold, vertex.color=VERTEX$gender)
#We can change the thickness of the edge according to the number of times a particular student has sent another student a comment.
+
+plot(g,layout=layout.fruchterman.reingold, vertex.color=VERTEX$gender, edge.width=EDGE$count)
Part II
+In Part II your task is to look up in the igraph documentation and modify the graph above so that:
+-
+
- Ensure that sizing allows for an unobstructed view of the network features (For example, the arrow size is smaller) +
- The vertices are colored according to major +
- The vertices are sized according to the number of comments they have recieved +
#Load the igraph package
+library(igraph)
+g <- graph.data.frame(EDGE, directed=TRUE, vertices=VERTEX)
+class(g)## [1] "igraph"vertex.attributes(g)## $name
+## [1] "3" "28" "6" "11" "15" "17" "7" "5" "16" "10" "27" "4" "2" "20" "26"
+## [16] "13" "19" "21" "1" "24" "23" "29" "18" "12" "9" "22" "25" "8" "14"
+##
+## $gender
+## [1] 2 2 2 2 1 2 1 1 2 2 1 2 1 1 1 2 1 2 2 2 1 1 1 1 2 1 2 2 2
+##
+## $major
+## [1] 2 4 3 3 4 4 4 3 2 2 4 2 4 2 2 4 3 4 1 2 1 3 3 4 4 2 2 1 3edge.attributes(g)## $count
+## [1] 1 1 5 1 1 1 1 5 1 2 3 1 5 1 1 1 3 1 1 1 1 3 1 1 1 1 1 2 1 3 2 1 1 1 1 1 1 1
+## [39] 1 1 1 1 1 1 1 1 2 1 2 2 1 1 1 1 1 1summary(g)## IGRAPH 020702b DN-- 29 56 --
+## + attr: name (v/c), gender (v/n), major (v/n), count (e/n)g1 <- igraph::graph_from_data_frame(d= EDGE,vertices = VERTEX, directed = T)
+plot(g1)
+
+# isolated nodes
+detach(package:igraph)
+library(statnet)
library(intergraph)
+g2 <- asNetwork(g1)
+length(isolates(g2))## [1] 0plot(g2)
plot(g,edge.width=0.1)
plot(g,vertex.color=VERTEX$major,edge.size=0.01)
plot(g,edge.size=0.1,vertex.color=VERTEX$major,vertex.size=EDGE$count)
g %>%
+ plot()
g1 %>%
+ plot()
plot(g2)
degree(g2)## [1] 2 4 4 3 6 4 8 3 7 2 6 5 3 4 4 1 3 6 2 6 5 5 5 3 1 2 4 2 2closeness(g2)## [1] 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0# Betweenness centrality
+# B(ni) = ΣG(jk)(ni)/G(jk)
+# where G(jk) is the geodesic between nodes j and k.
+# G(jk)(ni) is the number of geodesics between nodes j and k that contain node i.
+betweenness(g2)## [1] 0.000000 16.000000 17.000000 0.000000 47.583333 0.000000
+## [7] 127.666667 0.000000 119.916667 15.833333 61.333333 6.500000
+## [13] 0.000000 79.166667 65.833333 0.000000 4.500000 27.916667
+## [19] 0.000000 71.833333 80.333333 6.583333 54.000000 12.000000
+## [25] 0.000000 0.000000 0.000000 0.000000 0.000000####Model
+Network_model <- ergm(g2 ~ edges +
+ nodefactor("gender")+
+ nodefactor("major"))
+summary(Network_model)##
+## ==========================
+## Summary of model fit
+## ==========================
+##
+## Formula: g2 ~ edges + nodefactor("gender") + nodefactor("major")
+##
+## Iterations: 5 out of 20
+##
+## Monte Carlo MLE Results:
+## Estimate Std. Error MCMC % z value Pr(>|z|)
+## edges -2.8016 0.7256 0 -3.861 0.000113 ***
+## nodefactor.gender.2 -0.2801 0.2043 0 -1.371 0.170403
+## nodefactor.major.2 0.3179 0.3903 0 0.815 0.415333
+## nodefactor.major.3 0.1243 0.4096 0 0.304 0.761474
+## nodefactor.major.4 0.3254 0.3863 0 0.842 0.399542
+## ---
+## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
+##
+## Null Deviance: 1125.7 on 812 degrees of freedom
+## Residual Deviance: 404.4 on 807 degrees of freedom
+##
+## AIC: 414.4 BIC: 437.9 (Smaller is better.)Part III
+Now practice with data from our class. This data is real class data directly exported from Qualtrics and you will need to wrangle it into shape before you can work with it. Import it into R as a data frame and look at it carefully to identify problems.
+Please create a person-network with the data set hudk4050-classes.csv. To create this network you will need to create a person-class matrix using the tidyr functions and then create a person-person matrix using t(). You will then need to plot a matrix rather than a to/from data frame using igraph.
Once you have done this, also look up how to generate the following network metrics:
+-
+
Betweeness centrality and dregree centrality. Who is the most central person in the network according to these two metrics? Write a sentence or two that describes your interpretation of these metrics
+Color the nodes according to interest. Are there any clusters of interest that correspond to clusters in the network? Write a sentence or two describing your interpetation.
+
dd1 <- read.csv("hudk4050-classes.csv", stringsAsFactors = FALSE, header= TRUE)
+D1 <- dd1
+dd1 <- dd1[-1,-2,]
+colnames(D1) <- D1[1,]#Data cleaning #transform into long data
+library(tidyr)
+library(dplyr)
+library(stringr)
+library(janitor)
+D1 <- slice(D1,3:49)
+#Remove last column
+D1 <- select(D1,1:8)
+#Merge name columns
+D1 <- unite(D1,"name",`First Name`, `Last Name`, sep="")
+#Remove unpredictable characters from names
+D1$name <- str_replace(D1$name, "`", "")
+#Make all name captalized first letters only
+D1$name <- str_to_title(D1$name)
+#Make all class letters capitals
+D1 <- D1 %>% mutate_at(2:7, list(toupper))
+#Remove whitespace between letters and numbers in class
+D1 <- D1 %>% mutate_at(2:7, str_replace_all, " ", "")Data restructuring
+# Create a dataframe with two variables, student and class
+D2 <- D1 %>%gather(labe, class, 2:7, na.rm = TRUE, convert = FALSE) %>% select(name, class)
+#Create a new variable containing 1s that will become our counts
+D3 <- D2 %>% tabyl(name,class)
+rownames(D3) <- D3$name
+D3 <- select(D3, -name, -HUDK4050)#Matrix operations
+#Convert to matrix
+D3 <- as.matrix(D3)
+#Create person-person matrix
+D3 <- D3 %*% t(D3)#Graphing
+library(igraph)## Warning: package 'igraph' was built under R version 3.6.3##
+## Attaching package: 'igraph'## The following object is masked from 'package:tidyr':
+##
+## crossing## The following objects are masked from 'package:sna':
+##
+## betweenness, bonpow, closeness, components, degree, dyad.census,
+## evcent, hierarchy, is.connected, neighborhood, triad.census## The following objects are masked from 'package:network':
+##
+## %c%, %s%, add.edges, add.vertices, delete.edges, delete.vertices,
+## get.edge.attribute, get.edges, get.vertex.attribute, is.bipartite,
+## is.directed, list.edge.attributes, list.vertex.attributes,
+## set.edge.attribute, set.vertex.attribute## The following objects are masked from 'package:dplyr':
+##
+## as_data_frame, groups, union## The following objects are masked from 'package:stats':
+##
+## decompose, spectrum## The following object is masked from 'package:base':
+##
+## uniong <- graph.adjacency(D3, mode="undirected", diag = FALSE)
+plot(g,layout=layout.fruchterman.reingold,
+ vertex.size = 4,
+ vertex.label.cex =0.8,
+ vertex.label.color="black",
+ vertex.color="yellow")
Network <- g %>%
+ simplify(remove.multiple = TRUE,remove.loops = TRUE) %>%
+ delete.vertices(.,which(degree(.)==0)) %>%
+ intergraph::asNetwork()
+
+plot(Network)
# degree centrality
+degree(g)## Abdul Malik Muftau Ali Al Jabri Amandaoliveira Berjakian
+## 452 452 236 461
+## Chriskim Danlei Dannyshan Fangqiliu
+## 560 132 1 453
+## Feiwang Guoliangxu Hangshijin He Chen
+## 236 261 261 342
+## Hyungoolee Jiaao Qi Jiacongzhu Jiahaoshen
+## 343 261 261 261
+## Jiashengyu Jieyao Kaijie Fang Mahshaddavoodifard
+## 458 11 351 452
+## Nicoleschlosberg Qianhuiyuan Rongsang Ruoyi Zhang
+## 240 560 124 464
+## Saravasquez Shuyingxiong Stanley Si Hengzhao Tianyuchang
+## 560 343 19 354
+## Vidyamadhavan Wenningxiao Wenqigao Xiaojialiu
+## 560 346 261 252
+## Xijiawang Xiyun Zhang Xueshiwang Yifeizhang
+## 242 261 244 254
+## Yingxinxie Yingxinye Yixiongxu Yuchengpan
+## 261 232 240 349
+## Yunzhaowu Yuruiwang Yutingzhou Yuxuange
+## 242 230 246 138
+## Zachfriedman Zhixin Zheng Zhoudawu
+## 241 250 130# Closeness centrality
+# The inverse of the sum of all the distances between node i and all the other nodes in the network.
+closeness(g)## Abdul Malik Muftau Ali Al Jabri Amandaoliveira Berjakian
+## 0.02040816 0.02040816 0.02127660 0.02083333
+## Chriskim Danlei Dannyshan Fangqiliu
+## 0.02040816 0.02127660 0.01075269 0.02040816
+## Feiwang Guoliangxu Hangshijin He Chen
+## 0.02083333 0.02040816 0.02040816 0.02040816
+## Hyungoolee Jiaao Qi Jiacongzhu Jiahaoshen
+## 0.02040816 0.02040816 0.02040816 0.02040816
+## Jiashengyu Jieyao Kaijie Fang Mahshaddavoodifard
+## 0.02127660 0.01190476 0.02083333 0.02040816
+## Nicoleschlosberg Qianhuiyuan Rongsang Ruoyi Zhang
+## 0.02083333 0.02040816 0.02083333 0.02040816
+## Saravasquez Shuyingxiong Stanley Si Hengzhao Tianyuchang
+## 0.02040816 0.02040816 0.01298701 0.02083333
+## Vidyamadhavan Wenningxiao Wenqigao Xiaojialiu
+## 0.02040816 0.02083333 0.02040816 0.02040816
+## Xijiawang Xiyun Zhang Xueshiwang Yifeizhang
+## 0.02040816 0.02040816 0.02083333 0.02083333
+## Yingxinxie Yingxinye Yixiongxu Yuchengpan
+## 0.02040816 0.02040816 0.02127660 0.02127660
+## Yunzhaowu Yuruiwang Yutingzhou Yuxuange
+## 0.02083333 0.02040816 0.02083333 0.02040816
+## Zachfriedman Zhixin Zheng Zhoudawu
+## 0.02083333 0.02040816 0.02127660# Betweenness centrality
+# B(ni) = ΣG(jk)(ni)/G(jk)
+# where G(jk) is the geodesic between nodes j and k.
+# G(jk)(ni) is the number of geodesics between nodes j and k that contain node i.
+betweenness(g) ## Abdul Malik Muftau Ali Al Jabri Amandaoliveira Berjakian
+## 0.00000000 0.00000000 5.44828100 2.82468533
+## Chriskim Danlei Dannyshan Fangqiliu
+## 0.00000000 5.14222776 0.00000000 0.00000000
+## Feiwang Guoliangxu Hangshijin He Chen
+## 4.03593833 0.00000000 0.00000000 0.00000000
+## Hyungoolee Jiaao Qi Jiacongzhu Jiahaoshen
+## 0.00000000 0.00000000 0.00000000 0.00000000
+## Jiashengyu Jieyao Kaijie Fang Mahshaddavoodifard
+## 10.89656199 0.03219697 2.11851400 0.00000000
+## Nicoleschlosberg Qianhuiyuan Rongsang Ruoyi Zhang
+## 1.43587208 0.00000000 2.01796916 0.00000000
+## Saravasquez Shuyingxiong Stanley Si Hengzhao Tianyuchang
+## 0.00000000 0.00000000 0.23960081 2.11851400
+## Vidyamadhavan Wenningxiao Wenqigao Xiaojialiu
+## 0.00000000 2.11851400 0.00000000 0.00000000
+## Xijiawang Xiyun Zhang Xueshiwang Yifeizhang
+## 0.00000000 0.00000000 2.82468533 1.55916270
+## Yingxinxie Yingxinye Yixiongxu Yuchengpan
+## 0.00000000 0.00000000 6.92668057 8.19623102
+## Yunzhaowu Yuruiwang Yutingzhou Yuxuange
+## 1.41234267 0.00000000 2.82468533 0.00000000
+## Zachfriedman Zhixin Zheng Zhoudawu
+## 45.00000000 0.00000000 4.82733694df.prom <- data.frame(
+ deg = degree(g),
+ cls = closeness(g),
+ btw = betweenness(g))
+plot(df.prom$deg,df.prom$btw)
which(df.prom$deg>500)## [1] 5 22 25 29summary(df.prom)## deg cls btw
+## Min. : 1.0 Min. :0.01075 Min. : 0.000
+## 1st Qu.:240.0 1st Qu.:0.02041 1st Qu.: 0.000
+## Median :261.0 Median :0.02041 Median : 0.000
+## Mean :295.5 Mean :0.02008 Mean : 2.383
+## 3rd Qu.:352.5 3rd Qu.:0.02083 3rd Qu.: 2.119
+## Max. :560.0 Max. :0.02128 Max. :45.000which(df.prom$btw>40)## [1] 45df.prom[45,]## deg cls btw
+## Zachfriedman 241 0.02083333 45df.prom <- df.prom[-45,]
+summary(df.prom)## deg cls btw
+## Min. : 1.0 Min. :0.01075 Min. : 0.000
+## 1st Qu.:240.0 1st Qu.:0.02041 1st Qu.: 0.000
+## Median :261.0 Median :0.02041 Median : 0.000
+## Mean :296.7 Mean :0.02007 Mean : 1.457
+## 3rd Qu.:353.2 3rd Qu.:0.02083 3rd Qu.: 2.119
+## Max. :560.0 Max. :0.02128 Max. :10.897which(df.prom$btw>10)## [1] 17df.prom[17,]## deg cls btw
+## Jiashengyu 458 0.0212766 10.89656Answer: I think Jia Shengyu is the most central person in the network. She has the second highest betweenness, which is very important.
+#creating a variable for the number of classes so I can use it as the vertex size
+plot(g,vertex.color=factor(dd1$Q18), vertex.label=NA,main="PERSON-NETWORK")
#I think common interest has to do with a person’s major and it is likely that students in the same major have the same classes.
+To Submit Your Assignment
+Please submit your assignment by first “knitting” your RMarkdown document into an html file and then comit, push and pull request both the RMarkdown file and the html file.
+
