Social Capital in Networks

Variation Testing

2009-10-09T16:59:00.003-06:00

What effect do subtle differences have upon website pages? There is growing interest in this area across a variety of groups. Here is a sample of sites/services that take a stab at it:

http://www.whichtestwon.com
http://www.google.com/websiteoptimizer
http://www.splittestaccelerator.com
http://www.abtests.com

What do you use to test and measure your page variations and conversion?

Social Media Revolution

2009-10-06T09:43:00.002-06:00

Here is a fun little video that highlights some of the the current trends in the new ways people are communicating...

Gary King

2009-10-01T17:32:00.001-06:00

Recommended by Elkan as one who is doing interesting research in Political Science using the latest technology.

in reference to: Gary King (view on Google Sidewiki)

Search Engine Optimization

2009-06-24T10:29:00.005-06:00

I read an interesting article this morning about Search Engine Optimization (SEO) and keywords from a former classmate Nick on his blog Nick's Traffic Tips. For the most part, he explained the standard SEO tactics (and provided some nice case studies and links to go along with it):

SEO Checklist
Learn the basics of HTML
Choose your keywords
Find what other sites are competing for those keywords
Put keyword phrase in title, h1, and h2 tags
Include an image on the page with the file name & the alt attribute containing your keyword phrase
Make sure your keyword phrase is found at the very bottom of your page.
Embed your keyword phrase throughout your page at a rate of 1% to 2%.
Have your keyword phrase in domain and in the page’s file name.
Create your robots.txt file

Additionally, he mentioned the technique of creating content about what you predict will be searched for in the future. Although, this means you need to be a bit of a visionary, it seems like a fun challenge to engage in.

Statistical Design

2009-03-26T13:13:00.006-06:00

Here are a few notes on Statistical Design from class today...

As you design your study it is important to realize that statistical significance is different than practical significance. Your results can be statistically significant, but not practically important.

Determining the Sample size
The following four pieces of information are needed to determine the sample size:

Acceptable type on error (alpha) - standard value is .05
Acceptable type two error (beta) - standard value is .2 (power of .8)
Standard deviation - pooled estimate of the standard deviation (sq. rt. of the error mean squared error), includes block effects and covariates
Meaningful difference you want to detect

There are various "rules of thumb" that can help you figure this out based on having the above information. For instance, the following is a basic formula that could be used:

n=16/(delta/sigma)^2

where delta is the change in means between the two groups you might be comparing.

Here is another rough regression rule of thumb:

n=15*p

Where p is the number of parameters being used.

Finally, here are some steps to cover when designing your statistical experiments.

Steps in Design

Question of interest
Score of inference
Understand the system
Response
Other factors
Planning
Statistical anaysis
Sample size

The often overlooked considerations are very important as new studies are designed.

Tweeting Blog Posts

2009-03-24T12:20:00.003-06:00

I just automated the process of making a tweet on Twitter whenever I make a new post on this blog. I set this up using a free service called TwitterFeed. As a side note, it allows you to do this fo multiple blogs.

If all goes well, then you will see this post (and all future posts) on my Twitter stream: @smithworx

Things that Make People Happier

2009-03-24T11:30:00.007-06:00

Martin Seligman spoke at the BYU forum today in the Marriott Center. Listening from my office I found it quite interesting and relevant, particularly with the doom and gloom ever present in the news. Here are a few tips that he mentioned (they had be learned primarily through controlled experiments):

Write down three things that went well today and why they went well. (After six months on average you will be happier.)
People that deploy their greatest strengths will be happy.
"Active Constructive Responding" helps marriages succeed.
Do something philanthropic and your whole day go better.

Finally, he directed everyone to his Authentic Happiness website for more information on being happy.

Spring Research Conference at BYU

2009-03-23T09:51:00.006-06:00

Here are the slides form the presentation that I gave at the Spring Research Conference at BYU last weekend.

Finding Gold in Waste

2009-03-16T14:53:00.004-06:00

I first learned about this here, but wanted to share it again. In short, before you discard all of your junk, you might want to be sure that it is indeed worthless. In the video below, these guys are finding gold in the sewage.

Tolerance and Acceptance

2009-02-03T12:25:00.006-07:00

Dr. Barbara Culatta gave an excellent devotional today on being tolerant and accepting people around you. She effectively discussed the seven tips below:

Look for Commonalities
Accept People Where They Are
Reflect on Goodness in Other Religions
See Differences as Part of Life
Value What We Can Learn From Others
Serve People from Different Faith and Backgrounds
Give a Soft Response When Criticized

Each of these suggestions (with the exception of #1) encourages us to build "bridging" relationships with people different from us. Tip #1 reminds us that in order to effectively build bridging relationships, we need at least some common ground. When it gets right down to it, we have something in common with everyone (e.g., we are all human beings, we all need food and water). Bridging social capital increases as we establish these types of relationships. Bridging capital is usually harder to obtain as it requires more effort to establish, yet, it is often more rewarding since it gives connecting parties access to resources and information that they otherwise would not have.

These useful tips that can help us be more effective at building these bridges, which improve our social networks.

Using Color in Latex

2009-02-02T10:54:00.010-07:00

Have you ever wanted to use color in latex? It is actually pretty simple. All you need to do is include the color package:\usepackage[usenames]{color}Then you can use simple commands like:\textcolor{Red}{I like Red.}To produce:
Here is a list of the 68 standard colors that you could use:

Apricot Aquamarine Bittersweet Black Blue BlueGreen BlueViolet BrickRed Brown BurntOrange CadetBlue CarnationPink Cerulean CornﬂowerBlue Cyan Dandelion DarkOrchid Emerald ForestGreen Fuchsia Goldenrod Gray Green GreenYellow JungleGreen Lavender LimeGreen Magenta Mahogany Maroon Melon MidnightBlue Mulberry NavyBlue OliveGreen Orange OrangeRed Orchid Peach Periwinkle PineGreen Plum ProcessBlue Purple RawSienna Red RedOrange RedViolet Rhodamine RoyalBlue RoyalPurple RubineRed Salmon SeaGreen Sepia SkyBlue SpringGreen Tan TealBlue Thistle Turquoise Violet VioletRed White WildStrawberry Yellow YellowGreen YellowOrange

Furthermore, you can define custom colors by specifying the RGB values as follows:

\definecolor{MyCrazyColor}{cmyk}{0.2,0.8,0,0.1}

and then using it as you would the other colors, like so:

\textcolor{MyCrazyColor}{My text in my crazy color.}

Read more about it at Bent Peterson's site.

Social Capital in Startup Networks

2009-01-30T19:07:00.005-07:00

Social capital in action surrounds us. Below is a list of sites focused on the niche of starting up new businesses. In order for such ventures to be successful, it is often necessary to build a team of individuals that have complementary attributes and abilities. We are currently working on ways that we can quantify some of these behaviors given the rich data that is generated online everyday.

Social dimensions of social media

2009-01-28T08:50:00.004-07:00

Below is the slide presentation that Marc Smith gave at ICWSM. I met Marc at the CIKM 2008 Workshop on Search in Social Media (SSM 2008) in Napa Valley, CA. He is a creative thinker that is always interesting to watch and listen to.

2008 - ICWSM - Marc Smith - Some Dimensions Of Social Media

View more presentations or upload your own. (tags: telligent marc)

Also, you can watch the video of this presentation by clicking the picture below:

Social dimension of social media (Marc Smith)

Enjoy!

Family Reunion on Facebook

2009-01-15T23:56:00.010-07:00

Reunion sites are used primarily to "get the word out" about the reunion (e.g., location, events, cost), gather people's information (e.g., family member names), collect dues, and share family media (e.g., stories, pictures, videos).

Traditionally, I've been the guy that gets asked to build the reunion website --- as I'm involved in the planning meetings. Through the years, I've built a few high school reunion sites and a family reunion site. Through the process I have reused much of the code, yet they have all been "custom" websites. Although I had big plans to make my code generic enough that anyone could create a reunion site, I never took the time to do it. Since creating my last reunion site there have been a number of others that have taken the time to do this. For example, Ning and MyEvent are among a host of such sites that people have used to create reunion sites.
Currently, I believe that the quickest and most efficient way to do a reunion website is on Facebook. Here are a few reasons why I currently believe it to be most efficient solution:

It is free
No coding is necessary (saving time and money)
Lots of your family is likely already using Facebook, so they will not have to register for another account or re-enter information
People using Facebook already have media shared and can easily share more, which makes it easy to "catch-up" with family members
It is easy to create a Group specifically for you family reunion allowing family members to join
It is easy to create a "Related Event" for the group (i.e., the reunion activities) that members can easily indicate whether or not they plan to attend
In general, it is much more interactive than any of the other reunion sites
It may even be "one of the most fulfilling experiences of [your] life"
The Network will continue on, even after your planned event is over

So, to all that may wish to utilize my services to build them a reunion website... I'm still at your service, so long as we do it on Facebook. ;)

Recording Skype Calls

2008-12-27T13:25:00.005-07:00

Here are the basic settings that I used on a Mac to record Skype conversations using SoundFlower, Skype, and Audacity.

SoundFlower
Default Input: SoundFlower (2ch)
Default Output: Built-in Output

Skype
Audio output: SoundFlower (2ch)
Audio input: SoundFlower (2ch) (if you wan't to record both sides of the Skype call)
Built-in Microphone (if you only wish to record the other side of the conversation)

Audacity
Recording Device: SoundFlower (2ch)
Playback Device: Core Audio: Built-in Output

You could record many other things using similar configurations. Happy Recording!

Update: Since writing this I happened to come across another tutorial on doing this with pictures, that you might wish to follow.

Network Roles

2008-12-16T13:35:00.007-07:00

Social networks tend to have people that fill positions within the community. For example, within an academic community the roles of being a professor or a student can sometimes be identified solely by using the directed interactions among the individuals. To perform such an analysis, some measure of equivalence is used. In Social Network Analysis by Wasserman and Fauts, the following definitions of equivalence are reviewed (each with a note):

Structural Equivalence - requires identical ties to other identical actors
Automorphic and Isomorphic Equivalence - requires identical ties to other actors
Regular Equivalence - actors have identical ties to and from equivalent actors
Local Role Equivalence - actors are role equivalent if they have the same role sets
Ego Algebra Equivalence - based on algebra of relational structuresSo, why is knowing how to use this important? Well, say you would like to better understand the network surrounding your blog by learning which other blogs are similar in their ties as you, then this is how you could do it.

Blog Posts Increasing and Cyclic

2008-11-28T18:21:00.006-07:00

During the span of a year, the over 200 blogs aggregated above show very cyclic behavior. It shows very prominently that these bloggers post significantly more posts during the week rather than on the weekends.

Additionally, as time went on, these blogs as a group posted more frequently.

I would guess that the sink during December was caused by the Christmas holiday.

Christmas Gift Giving Tool

2008-11-25T09:53:00.011-07:00

Christmas is a time for gift-giving --- sometimes, families and friends need a quick way to determine who gives to who. So, I created this little tool that allows families and friends to quickly enter the names of people (that will be giving and receiving gifts) to have gift-giving list automatically (and randomly) generated.

It is nice to have a tool like this so the lists can be generated early and gift-giving becomes that much easier --- you don't have to wait until somebody gets the old hat out and writes down everyone's name. Additionally, it seems to be more fair and is less complicated. ;)

The program was built using Python and Google App Engine.

LaTex: Vertical Text

2008-11-20T11:26:00.007-07:00

Scientific publications in computer science are often created using LaTeX. Here is a little tip for making text appear vertically in Latex.
First, you need to include the following library:

\usepackage{rotating}

Next, you can use the sideways environment, as follows:

\begin{sideways}YOURTEXT\end{sideways}

Finally here is an example used within a table:

\begin{tabular}{|r|r|c|}\hline
\begin{sideways}Letter\end{sideways} & \begin{sideways}Frequency\end{sideways} & Words \\
\hline
A & 0.05 & Apple, Algebra, Altruistic, Angel \\
B & 0.45 & Basketball, Ballroom, Bear, Bountiful \\
\hline
\end{tabular}

Which produces:

Obama: 44th US President

2008-11-04T23:51:00.004-07:00

Barack Obama has been elected as the next President of the United States of America. I enjoyed following the election coverage this season as it was particularly exciting. Although my first choice for President was Mitt Romney, I believe that Obama will work to unite the country and improve International relations.

PIKM - rough notes

2008-10-30T10:13:00.003-06:00

Session 1: Chair - Prasan Roy
A SQL Database System for Solving Constraints
An interesting take on enhancing SQL to solve constraint problems
Acquiring Advanced Properties in Ontology Mapping
Using ontologies to improve knowledge management
Social Capital in Online Communities
This was my presentation. ;)

Session 2: Chair - Aparna Varde
Concept Search in Urdu - interesting challenges. He proposes to write a language specific stemmer to be used for the Urdu language
Topic Models and a Revisit of Text-related Applications
An Extended Cooperative Transaction Model for XML

Session 3: Chair - Anisoara Nica
The Benefit of additional Semantics in Folksonomy Systems
Exploiting additional context in folksonomies (abel@L3S.de)
Ideas: GroupMe, Social HITS, Automating MOAT, relations between tag assignments
MOAT - Meaning of a Tag (automatic MOAT using context of resource), MOAT server, DBPedia
A Microscopic View on Community Detection in Complex Networks - No Show
Towards Privacy-Preserving Integration of Distributed Heterogeneous Data
Privacy-Perserving Data Sharing Architecture. Pawel Jurczyk presented a fairly complex system which hopes to solve the problem of preserving privacy when sharing data. This approach could be applied to hospitals that wish to share data in order to make use of one another's data.
Concurrency Control and Recovery for Multiversion Database Structures
Tuukka Haapasalo (thaapasa@cs.hut.fi) proposes a solution for multiversion databases. Propositions: 1) extend B-Trees: TSBT, Transactional MVBT, or 2) Two-dimensional R-tree.

Web Startup Group Meets Thursday Night

2008-10-22T22:39:00.005-06:00

The first official Web Startup Group meeting is tomorrow night. It should be a fun event that good things will come from. It will be at 7:00 PM in the TMCB at Brigham Young University. All interested are welcome to attend.

the updated logo

Security Analysis of Reputation Systems

2008-10-15T11:08:00.004-06:00

I came across this report on reputation-based systems today which I found at a reputation based social capital blog. It highlights the security threats against current reputation systems, use cases, and even ten recommendations to combat these threats.

Snapshot of the some of the recommendations

Information Pathways in Social Networks

2008-10-10T12:00:00.015-06:00

The first talk presented in the social network session of KDD 2008 was for an interesting paper by G. Kossinets, J. Kleinberg, and D. Watts titled The Structure of Information Pathways in a Social Communication Network (PDF). Although I was not at KDD I was able to watch it online at videolectures.net.
Kleinberg, the presenter, made some interesting observations having to do with our "rhythmic" everyday conversations. The approach to analyzing communication within these social networks is focused on the frequency of correspondence, rather than the content conveyed.

They measure "distance" between individuals by measuring the minimum time required for information to pass from one node to another. A methodology based on Lamport's work and vector clocks in the area of distributed computing.

Using this metric they are able to filter a busy network (one having edges for all communication packets) in a simplified network that contains only the edges that are minimum-delay paths between a pair of nodes. They call this simplified network view the network backbone. Below is an example of such a network (along with the caption) taken from the paper.
The nodes further outside of the center of the graph are more "out-of-date" with respect to node v, since they communicate less frequently.

I found the approach to be novel and useful. As with nearly any analysis technique, caution should be used in selecting the time-period and group size to be studied. Recency and frequency issues come into play as correspondence is aggregated. However, this pursuit offers another approach for more fully understanding information flow.

Revision Control

2008-10-06T15:19:00.005-06:00

If you worked on software in collaboration with multiple developers, then you've probably used (or wished you used) some sort of revision control system. The Google Search Volume Index plot below suggests some trends surrounding the currently available tools.
(Note: by no means is this very scientific, due to the fact that people searching with these terms could have been searching for something entirely different.)
CVS, although huge in its time, is on the decline, while SVN, Git, and Mercurial are on the rise. I have used plenty of CVS and SVN to be ready for change. I am now using Git which I have really liked so far. If you have already been using SVN as I had, I would recommend the Git-SVN Crash course to get started quickly.

Facebook growth rising past MySpace

2008-10-02T15:34:00.011-06:00

From my local perspective Facebook has been on the rise --- I've noticed that many of my less computer savvy friends have now joined Facebook. I wondered if this trend was global, so I decided to investigate...

During the past few years MySpace has been the dominant social network, however, Facebook has continued to grow much quicker and is expected to become the leading social network. The first plot below (Figure 1) shows a comparison of searches for the keywords "facebook" and "myspace". Lately, for most of 2008, Facebook has been getting a little more attention in the news (lower portion of Figure 1) and has achieved a significantly higher search volume index.

Figure 1. Search Volume Index Comparison of 'facebook' and 'myspace'
(source: Google Trends)

Figure 2 shows the massive popularity of MySpace which began late in 2004, peaked in the middle of 2006, and has since declined --- possibly in part due to the rise of Facebook.

Figure 2. Search Volume Index of 'facebook.com' and 'myspace.com'
(source: Google Trends)

Finally, Figure 3 shows the number of daily unique visitors to Facebook as being more than that of MySpace as far back as November of 2007. (I'm not sure, but I would guess these figures to be based upon Google search result click-thrus)

Figure 3. Daily Unique Visitors of 'facebook' and 'myspace'
(source: Google Trends)

I find it very interesting to see how quickly social networks grow and evolve. As an aside, I think that Facebook is doing things more efficiently and currently providing a better service.

The Web Startup Group

2008-09-24T15:29:00.010-06:00

Today, we founded the Web Startup Group to bring together people interested in creating new sites and services online. Group members include web developers (programmers and designers), marketing and business-minded individuals, creative idea people, and others with technology related skills. The group intends to meet regularly to discuss and make these ideas come to life.

If you are here at BYU and are interested in making a difference online then come join us!

Social Capital, Nan Lin

2008-09-22T12:06:00.010-06:00

This is a great book that provides a sociologist's perspective on social capital.

Google Releases a Browser

2008-09-14T22:53:00.003-06:00

Google Chrome is the new Web browser that was just released for Windows. It is much faster, elegant, and easy to use than Firefox, IE, and Safari. After using it for a day, the I love everything about it except for the following:

It is not available for Mac yet (currently only for Windows)
Nifty plug-ins are not yet available (Firefox wins here)

I would predict that the above shortcomings will be quickly overcome and Google will end up having the dominant browser. If interested, learn more about the features of Google Chrome or why they decided to build a new browser.

Update: as you probably guessed, development of Mac and Linux versions is underway. Sign up to be notified when Mac or Linux versions are ready for download.

Google's Picassa 3 (Beta)

2008-09-14T22:48:00.004-06:00

Picassa 3 is now available which has some pretty nice updates. Watch the video above to hear about what has been added. I just wish they had a Mac version available as it is superior to iPhoto. (Note: The current release is still Beta, which means that there will likely be a few minor bugs here and there.)

Eclipse for Web Development

2008-08-02T00:49:00.006-06:00

Well, I've been doing quite a bit of programming lately, however, it has been in a number of different languages. Naturally, I have considered Eclipse as a possible editor. Vanilla Eclipse is usually geared to toward Java development and can become tricky and tedious to setup for Web Development (hence the rise of Aptana). Eclipse can be great to work with, but only when you can get it working how you'd like. The following update sites may be helpful to get Eclipse setup how you hope to have it:

JavaScript
http://download.macromedia.com/pub/labs/jseclipse/autoinstall (JSEclipse)

Python
http://pydev.sourceforge.net/updates/ (PyDev)

PHP
http://update.phpeclipse.net/update/nightly (PHPEclipse)

Java Tapestry
http://m2eclipse.sonatype.org/update (Maven2)
http://jettylauncher.sourceforge.net/updates (Jetty)

Currently, all of these plugins can be loaded into a single installation of Eclipse Europa. However, I'm not sure that they are all compatible with Ganymede (latest version of Eclipse).

Google Visualization API

2008-07-08T12:02:00.000-06:00

Here is another great presentation from Google I/O.

Monetizing Social Application Traffic

2008-07-08T11:58:00.002-06:00

This is a presentation from Google I/O that was done by a company called SocialMedia.

Nifty Data Technique

2008-05-24T22:41:00.004-06:00

Google Spreadsheets now added some nifty ways to auto-fill data. For instance, rather than typing all of the days or months, you can simply type two or three, select them, and then drag the little blue square in the bottom right corner of the selection. Then, the rest of the days or months will be populated below. That is nice, but what I think is much more interesting, is that you can click and drag holding while down Ctrl (Windows and Linux) or Option (Mac) to pull data from Google Sets. So, in the image below, I only filled in the first three rows of each column. Then, I used the former technique to auto-fill the first three columns and the latter technique (holding down Ctrl or Option) to auto-fill the extra twelve rows.

Nowadays, software developers, such as Google, have a great opportunity to utilize the ginormous pile of data available online. The data that individuals generate is ever increasing and can be extraordinarily useful.

Save R Plot in EPS format

2008-05-24T16:35:00.004-06:00

Here is a code example of how to save an R plot in EPS (instead of PS):

postscript(file="testplot.eps",
           paper="special",
           width=10,
           height=10,
           horizontal=FALSE)

yvalues = runif(50)
plot(yvalues)

dev.off()

The variation is adding paper="special" and horizontal=FALSE.

Java Programming Notes

2008-05-16T15:49:00.006-06:00

Java Programming Notes is a handy Java reference by Fred Swartz. In his words, he explains:

These Java programming notes are written to fill in missing or weak topics in textbooks that I've taught from. Many pages are useful for reference, but not as an ordered tutorial. Some pages are still rough drafts, but I'm slowly working on fixing them.

Walmart Visualization

2008-05-05T22:45:00.002-06:00

Here is an interesting animation of Walmart Store growth overtime. Below is a snapshot of the movie in progress (1991).

Virtual Host Setup

2008-05-04T01:09:00.006-06:00

To add a virtual host on your local machine (running apache), do the following two things:

1. Add a virtual host definition to your apache configuration file, like this:


<VirtualHost *:80>
  ServerName sitename
  DocumentRoot "/location/of/your/site/"
</VirtualHost>

2. Add a corresponding line to your HOSTS file (on my Mac, it is located at /etc/hosts).

127.0.0.1 sitename

You should then be able to access your site in any Web browser by going to:

http://sitename

This then allows you to develop locally in an environment nearer to how it will likely be deployed.

Abstract classes and Interfaces

2008-05-02T16:18:00.005-06:00

In response to some of the questions asked in class today, I compiled some properties of interfaces and abstract classes that should help guide your choice when deciding when to use an Abstract class or an Interface as a parent class.

Neither an Interface nor an Abstract class can be instantiated. Both can be used to as a template for concrete (implemented) child classes.

Interfaces

example interface definition:

public interface Monkey {
  public double getWeight();
  public void setWeight(double w);
  public void walk();
  public void talk();
}

fields (i.e., members, variables) are not allowed
all methods are implicitly abstract
a child class can implement many interfaces in Java
child classes must implement all methods

Abstract Classes

example abstract class definition:

public abstract class Monkey {
  private double weight;
  public Monkey(){
  }
  public double getWeight(){
      return weight;
  }
  public void setWeight(double w){
      weight = w;
  }
  public abstract void walk();
  public abstract void talk();
}

may have members (e.g., weight)
may have implemented methods (e.g., getWeight, setWeight) and abstract methods (e.g., walk, talk)
a child class can only extend a single parent class in Java (multiple inheritance is not allowed)
child classes must implement all of the parent's abstract methods

Section 4.4 in Data Structures and Problem Solving in Java discusses this more extensively.

This talks more about when you might use one, the other, or both. Furthermore, I found some questions and answers about the two that interviewers like to use. ;)

Java Tutorials

2008-05-02T09:15:00.005-06:00

Sun provides some excellent tutorials that cover most aspects of programming in Java. Learning the Java language is a set of tutorials, or "trails", on the following fundamental topics:

The content of these trails is also available as a book, called The Java Tutorial, Fourth Edition.

ForwardTrack

2008-04-23T22:01:00.006-06:00

ForwardTrack is an open source tool (now entirely written in php) that allows email campaigns to be tracked and mapped as they are forwarded from person to person. This is definitely useful as it reveals the spread of information and some of the underlying social network.

Political Campaign Contributions

2008-04-23T16:44:00.013-06:00

The Federal Election Commission (FEC) requires that all campaign contributions over $200 (per donor) be reported publicly. The reported information includes the donor's name, job title, zip code, and even address. All of it, since 2001 is available electronically via FTP at ftp://ftp.fec.gov/FEC/electronic/.

In collaboration with Political Scientists here at BYU we have been performing record linkage (aka. entity resolution) on this data, so that they will be able to more accurately perform their studies.

Fundrace
On a related note, fundrace.org has created an interesting mashup (shown below) that maps donors on a Google map colored by the party or candidate donated to. It also, reveals donor information and appears do do some coarse record linkage.

FEC Maps
Additionally, the FEC itself has started to produce maps both for the Presidential Election and House and Senate Elections. The maps they provide aggregate the donated funds by state, party, and candidate.

Duncan Watts Downplays Viral Marketing Hype

2008-04-22T10:30:00.014-06:00

A while back I quickly saw Clive Thompson's article entitled Is the Tipping Point Toast?, but didn't have the time to read it all nor investigate it any further --- until today.

Thompson's article pits Malcolm Gladwell's thesis (in The Tipping Point) against the recent research of Duncan Watts (cited below). I thought the article was well-written and adequately presented both sides of the issue. In short, Watt's claims that spending time and money marketing to influential individuals is no better than marketing to the masses.

Through all of this, Watts makes some important points such as (quoted from Thompson's article):

The problem of popular viral marketing talk is that it is "incredibly vague"; "how an influential actually influences is not explained." "Precision matters when trying to explain highly social epidemics"
"Influentials don't govern person-to-person communication. We all do."
"Common sense is misleading"
Thompson writes that Watts found the "rank-and-file citizen [to be] far more likely to start a contagion"

So, today I finally took the time to learn more about Watts' recent research, available at Collective Dynamics Group website (at Columbia University) as a Working paper in the Papers section. Through the years, I had previously read some of Watts' work, so I was excited to see his recent findings. In this paper he presents an approach they call "Big Seed Marketing", which in essence combines a traditional mass marketing model with a viral propagation.

The idea that there is "no free lunch" in viral marketing is useful to point out, as "there are many more unsuccessful attempts that one never hears about." He also, points out that it is "hard, if not impossible" to predict which of attempts will succeed.

The take-home message in the conclusion is that effective marketing campaigns can be produced without identifying "influentials", but simply by adding a mechanism of peer-to-peer sharing to propagate the message. (As an aside, the formalism presented in the paper is useful for discussing the problem and easily evaluating the results.)

Watts makes some good points, however, I would still argue that people with high social capital (you might call "highly influential") can heighten the network effect. This is even evidenced in Duncan's paper --- as one of Tom Mauser's 'friend' was StopTheNRA, who, in turn sent a large email blast (Table 1, footnote 1). So, Tom Mauser, had a significant enough relationship with StopTheNRA that they used their resources (their large email list) to forward his message.

Although, there is an element of hype in the presentation of "Big Seed Marketing", I find it useful as it presents a nice way of making the issue sticky and bringing to light these more subtle points. The desired effect of propagating these ideas seems to be occurring.

Update (4/23): Podcast with Duncan Watts on Buzz Marketing (mp3)

Looking for a Job?

2008-04-15T10:01:00.006-06:00

There are a lots of places to search for jobs online these days including:

SimplyHired - easy to use job search
HotJobs - Yahoo's job site
Monster - you've probably heard of Monster
Indeed - newer site that allows you to filter by salary (for instance, here is an example salary search on computer science and data mining)
LinkedIn - use your social network to find your next job
erecruiting - for new college graduates

An interesting approach to finding your next job might be to leverage your social connections to match you with a good employer with needs inline with your skills. Of course, as nice that sounds in theory, I would bet it could be challenging in practice.

Although, I won't be needing a full-time job for another couple years, it is always interesting to see what jobs are available (and what skills are in demand) by quickly searching on your skills and interests.

SIP Recap - Thursday

2008-04-01T15:19:00.003-06:00

Here is a recap from the Social Information Processing Symposium:

Brian Skyrms (UCI), Signaling Games: Some Dynamics of Evolution and Learning
John Nicholson (USU), The Blind Leading the Blind: Toward Collaborative Online Route Information
~~Cosma Shalizi (CMU), Social Media as Windows on the Social Life of the Mind~~
Gustavo Glusman (Systems Biologist), Users, photos, groups, words: Analyzing mixed networks on Flickr
Luc Steels (Vrije U), Social tagging in community memories
Aram Galstyan (USC/ISI), Influence Propagation in Modular Networks
Adam Anthony (UMBC), Generative Models for Clustering: The Next Generation
Peter Pirolli (PARC), A Probabilistic Model of Semantics in Social Information Foraging
Hak-Lae Kim (DERI), int.ere.st: Building a Tag Sharing Service with the SCOT Ontology
Yu Zhang (Zhejiang U), Mining Target Marketing Groups From Users' Web of Trust on Epinions
Andrei Broder (Yahoo), Reviewing the Reviewers: Characerizing Biases and Competencies using Socially Meaningful Attributes (see Sihem Amer-Yahia)

The Wednesday talks were excellent. In particular, I really enjoyed:

The subtleties of the blind leading the blind (see 2 above)
Gustavo's unique way of analyzing Flickr relationships (see 4)
Adam Anthony's overview of generative models that can be used in clustering (see 7)
Pirolli's analysis of Lostpedia using LDA (see 8)
Hak-Lae Kim's tag aggregator application (see 9)
The use of socially meaningful attributes as presented by Yahoo's Andrei Broder (see 11)

Freemium Business Model

2008-04-01T12:18:00.004-06:00

The freemium model is something that I've been telling start-up businesses to do for quite a while. It allows community --- where social capital resides --- to build around your service at a fundamental level. After a captive community has been established, premium features (or services) can be offered, which effectively converts social capital into profit.

Freebase

2008-03-31T14:39:00.004-06:00

At the SIP Symposium (at Stanford), there were some guys from the Freebase development crew. This was my first exposure to Freebase and I was intrigued with the idea. According to Kurt Bollacker (the Chief Scientist) the reason they were at the symposium was to "get people using our data". I also learned from him that they were VC funded and currently had about 60 developers.

They have built upon the existing data sources, such as Wikipedia, and have added structurally typed data to go along with it. The resulting data repository is then easily accessible via the Freebase API.

It'll be interesting to see what happens with it.

SIP Recap - Wednesday

2008-03-26T23:43:00.007-06:00

I'm here in Palo Alto, California attending the AAAI Spring Symposium at Stanford. So far, the Social Information Processing Symposium has been very interesting and exciting. So far, I've met some people doing some neat research. Today's presentations were the following (I've added links to those I could find online):

Bernardo Huberman (HP Labs), Social Dynamics in the Age of the Web
Ed Chi (PARC), Augmented Social Cognition
Tad Hogg (HP Labs), Solving the organizational free riding problem with social networks
Riley Crane (ETH), Viral, Quality, and Junk Videos on YouTube: Separating Content From Noise in an Information-Rich Environment
Yi-Ching Huang (NTU), You Are What You Tag
Julia Stoyanovich (Columbia), Leveraging Tagging to Model User Interests in del.icio.us
Steve Whittaker (Sheffield), Temporal Tagging: Implicit Behaviour Identifies Points of Interest in Complex Event
Georg Groh, Implicit Social Network Construction and Expert User Determination in Web Portals
Elizeu Santos-Neto, Content Reuse and Interest Sharing in Tagging Communities
Matt Smith (BYU), Social Capital in the Blogosphere: A Case Study (this was our presentation, of course)

I enjoyed all of the presentations, in particular I liked Bernardo's address which covered a variety of interesting topics, Ed's comments, Riley's trend analysis, Julia's talk analyzing del.icio.us hotlist generation and tag, Steve's flamboyant presentation, Georg's work (as it had some thoughts related to our work on Implicit Affinity Networks).

(Oh, and I lost my cell phone today.)

I'm looking forward to another great day tomorrow!

Firefox 3 Extensions

2008-03-20T16:22:00.003-06:00

As Firefox 3 nears graduation from the Beta phase, many of the great extensions are finally becoming available. Here are links to the extensions that I find essential:

What extensions do you use?

Building Community around a Blog

2008-03-17T16:03:00.005-06:00

At the spring research conference I presented the following:

Building Community around a Blog (PDF)

Emptying a File in the Terminal

2008-03-09T00:53:00.003-07:00

Here is a handy trick that I find useful from time to time:

$ echo "here is some jibberish" > somefile.txt
$ cat somefile.txt
here is some jibberish
$ cat /dev/null > somefile.txt
$ cat somefile.txt
$

Essentially by calling "cat /dev/null > somefile.txt" we redirect nothing (/dev/null) to the file 'somefile.txt', which in effect empties it. I like to use this for emptying log files as it saves time recreating files and possibly resetting the right permissions.

If interested, you can read more on I/O redirection.

Social Science Data

2008-02-28T14:37:00.005-07:00

Amidst my search for social science data (to perform social capital experiments), I have discovered the following data:

Social Science Data on the Internet - seems to have lots of links to data, however the search is limited and results pages are clunky.
Social capital datasets - presents some information on data sources that are specifically related to investigating social capital.
INSNA - has the data used in Wasserman and Faust book on Social Network Analysis.
ICIPSR - Inter-university consortium for political and social research
SDA - web-based software available for accessing much of the social science data. Additionally, I noticed some archived data available.

There appears to be plenty of data. Now on to the task of filtering it down to the most relevant...

Social Network Graphing Tools

2008-02-27T12:00:00.007-07:00

Here is a write-up on various social network graphing tools that I posted on the Data Mining Lab wiki: Social Network Graphing Tools

Most recently, I have enjoyed using Cytoscape and the Network Workbench Tool for larger networks and GraphViz for small example networks.

What tools do you like to use?

Google Chatback

2008-02-26T16:02:00.004-07:00

Google just released a new gadget that will allow people visiting your blog or website to chat with you (via Google Talk). It is like the live chat and call-back buttons that have been available previously. To set it up, all you need to do is add the chatback badge code to your blog. Then, it'll look something like this...

Quick Mac and Windows Keyboard Switching

2008-02-25T23:56:00.006-07:00

Since Mac OS X doesn't save profiles for multiple keyboards (which I hope will change in subsequent versions), I had to find a solution for quickly changing between a Windows style keyboard and a Mac style keyboard. After spending some time with Google, I found some AppleScript that seems to provide an adequate solution. For convenience, I made two scripts: one called keyboard_win.scpt that switches the modifier keys on a Windows style keyboard to behave Mac-like, and one called keyboard_mac.scpt that restores the default keyboard behavior for a Mac style keyboard.

(1) keyboard_win.scpt:
(2) keyboard_mac.scpt:
Hopefully, this will save you some time.

Mining "Stories" from Blogs

2008-02-21T23:13:00.004-07:00

I just finished reading a paper called, Mining Blog Stories Using Community-Based and Temporal Clustering, which I found very interesting as it is extremely related to the work that we have been doing. As I read, I had the following thoughts and questions.

Thoughts

It is refreshing to discover others that are pursuing similar research paths as us.
Their work gives more fuel to the approaches we have been taking.
Some of the formalism is elegant and useful (particularly, for talking about blogs and entries), however, some of it gets cumbersome (the lookup table is definitely necessary).
Using Lucene library to index the data is an idea that we could consider (we have been storing the data in a MySQL database of our own make)

Questions

I'd be curious to know how many degrees of separation they crawled from their seed blogs? I would guess that they did not get too far since the blogs in our study appear to be less sparse on average. They reported over 2,000 blogs having 1 million entries (on average, 50 entires per blog per month). In Social Capital in the Blogosphere we retrieved blog content just two degrees away from Scoble (our single seed blog) and obtained over 38,000 blogs having 13 million entries (on average, 28.5 entries per blog per month).
How did they perform blog entity resolution? (here is an approach that we used)

(Note: Thanks to both Christophe who sent me the paper and Robbie Haertel for sending it to Christophe --- Social capital in action. ;) )

Bandwidth

2008-02-21T22:19:00.001-07:00

The Bandwidth Place is a nice site that tests the bandwidth available with you current network connection.

Escaping JavaScript and PHP

2008-02-21T22:05:00.003-07:00

Here are some references for encoding and decoding in JavaScript and PHP, so that passing information via ajax or urls can be done cleanly.

The Art of Web - useful testing area for JavaScript and PHP.
W3Schools JavaScript Reference - just JavaScript

What is Uplift Modeling?

2008-02-21T11:26:00.006-07:00

Uplift modeling is an approach to predicting the incremental impact a marketing campaign has on a customer through controlled experimentation. It measures the variation in the difference between a treated group and a control group segmenting customers into the following groups:

1. Those that buy only when treated
2. Those that would buy or not buy regardless of whether or not they were treated
3. Those that do not buy when treated, but do buy when not treated

I've have yet to use Uplift Modeling, but it sounds rather interesting.

Would you like to read more?
DM Review article
Using Control Groups to Target on Predicted Lift

Star Rating Widget

2008-02-20T20:43:00.007-07:00

Here is a rating widget and how to use it. It can be used for collecting ratings from website visitors. Typically, it will be used for five-star rating, but you can pass it any number, so that it can be used as an n-star rating widget.

Notes on Social Capital Views

2008-02-19T12:51:00.008-07:00

In Social Capital, by Nan Lin, presents some differing perspectives on social capital. I have made some notes of his assessment below, and have added some additional comments.

Bordieu - Social capital consists of social obligations or connections. It can be reduced to economic capital as it is viewed as collective asset that endows members with credits. The collective asset can be obtained through group membership.
Burt - Social capital is based on the number of people that an individual is connected to, the strength of these relationships, and the location the individual reside within this structure (see Flap's perspective). Individuals that bridge non-redundant groups of people have more social capital. Social capital accrues through bridging structural holes.
Coleman - Social capital is an aspect of social structure, and it facilitates certain actions of individuals within the structure. Social capital accrues through bonding.
Flap - Social capital consists of (1) the number of persons within one's social network who "are prepared or obliged to help you when called upon to do so," (2) the strength of the relationship indicating readiness to help, and (3) the resources of these persons.
Lin - Resources are divided into two types: personal and social. Personal resources include material objects (e.g, an airplane) and symbolic objects (e.g., a diplomas and degrees). Social resources, on the other hand, are resources accessed through an individual's social connections. Social resources, in both quantity and quality, far outweigh personal resources.
Putnam - Group level theory which tends to measure social capital collectively.

These notes are in no way conclusive or complete. However, they serve as a quick reminder to the various views.

Social Capital Explanations

2008-02-19T12:02:00.006-07:00

In Nan Lin's book, Social Capital, he provides the following explanations as to why social capital works, or more clearly, why embedded resources in social networks enhance the outcomes of actions. They are as follows with an example of each (see pages 19-20):

information flow is facilitated - word of mouth effect (e.g., being connected to people with useful information is a benefit)
social ties may exert influence - some ties carry more weight (e.g., son of the President)
social credentials - resources beyond an a single individuals' are available if necessary (e.g., politician endorsement)
reinforcement identity and recognition - can provide support and public acknowledgment of one's claim of resources (e.g., maintenance of mental health and entitlement of resources)

Lin suggests that these four elements "may explain why social capital works in instrumental and expressive actions not accounted for by forms of personal capital such as economic or human capital."

I would agree and add that relationships among people add an additional layer of complexity that cannot be accounted using only individual centered metrics.

Social Capital Simulation Updated

2008-02-16T02:19:00.007-07:00

The social capital simulation has been improved! The usability has been improved, preset examples have been added, and additional information in now reported.

Social Connections in Decline

2008-02-13T13:39:00.002-07:00

Robert Putnam, an influential social capital researcher, visited BYU nearly two years ago to discuss how social connections are on the decline. Here is good summary of Putnam's talk on BYU NewsNet. His research during the past decade has shown a negative trend in that people are socially connecting less these days. The speech gave fuel to the research on social networks that I had been involved in and has been a strong motivation to our current work on social capital.

Figure 1. "The TV Connection" shows that group membership tends to decline as television viewing increases among those having twelve or more years of education. (see The Strange Disappearance of Civic America)

Empirical studies on group membership, like the study shown in the plot above contribute to the evidence which Putnam uses to support this claim.

Social Capital Simulation (Online)

2008-02-13T00:48:00.007-07:00

The past couple days I have been working on an online social capital simulation that was created primarily with Javascript. Currently, it calculates social capital in the same manner as the excel version, however, it is more powerful as it allows you to set how many nodes you would like in the network, it dynamically creates a visual graph of the network, and it is accessible online.
I used Walter Zorn's High Performance JavaScript Vector Graphics Library to draw the network (i.e., nodes, lines, and text). This is an impressive library, which makes drawing with Javascript more pleasant than I originally expected. Also, to facilitate this project, I extended Zorn's library by adding getColor, getOpacity, and setOpacity methods. Furthermore, Michael Deardeuff and other Data Mining Lab members used their keen pattern finding skills to develop the mathematical equation for node placement in the graph.

Let me know how it works for you. I want to make this available so that it is easy to for people to get a feel for how we calculate social capital, which will allow us to refine our method.

Burt's Views

2008-02-08T12:00:00.002-07:00

I have been reading Ronald Burt's book called "Brokerage and Closure" to gain a better understanding of his view of social capital. Here are a few points that I have found interesting as I have read:

The value of a relationship is not defined inside the relationship; it is defined by the social context around the relationship" (Brokerage and Closure, pg. 11).

The "working definition" of a structural hole is the relationship between two people is a hole-spanning bridge when there is no effective indirect connection between the people. However, Burt does not wish to imply that the concept of a structural hole has an absolute meaning. He explains that structural holes could come and go by simply changing the population size of the network being analyzed. He argues that the same definitional issue exists for the absolute meaning of a relationship, a fundamental element of network theory (Brokerage and Closure, pg. 24).So, as an example, the structural hole in the network above is the relationship marked blue, while the value of that relationship is defined by the social context, or the other nodes in the network (i.e., A, B, G, E, F, H, and I).

Gmail Temporarily Down

2008-02-07T15:44:00.001-07:00

Dependence on Google is not always good. Here is the error message:

Update: Gmail is back up! The duration of the "Temporary Error" lasted about half an hour.

Book List: Social Capital

2008-02-05T22:00:00.000-07:00

In my quest to better understand how sociologists view social capital, I visited the library and checked out the following books:

Structural holes : the social structure of competition Burt, Ronald S.
Brokerage and closure : an introduction to social capital Burt, Ronald S.
Applied network analysis : a methodological introduction Burt, Ronald S.
Social capital : theory and research Lin, Nan
New social ties : contemporary connections in a fragmented society Chambers, Deborah.
Complex social networks Vega-Redondo, Fernando.
Effective small group and team communication Hoover, Judith D. (Judith Davis)
The role of social capital in development : an empirical assessment Grootaert, Christiaan
Social capital : a theory of social structure and action Lin, Nan
The well-connected community : a networking approach to community development Gilchrist, Alison.
The Analysis of social interactions : methods, issues, and illustrations Cairns, Robert B.

Of course, I won't have time to read all of these books from cover to cover. However, I do plan on reading enough to understand how they define, calculate, and show the presence of social capital.

Social Capital Simulation

2008-02-04T13:00:00.000-07:00

Our recent work has explored the concept of social capital, which I have discussed previously. Our social capital metrics, namely bonding and bridging (popularized by Robert Putnam), utilize the hybrid network methodology that we have developed for online communities.

To understand our metrics, I have created a basic social capital simulation (an excel spreadsheet) having five nodes. The simulation allows for you to change the connection strengths in both the implicit affinity network (IAN) and the explicit social network (ESN). Changing these values will give you an idea of how social capital fluctuates as the social network changes.

The figure above shows the initial configuration of the simulation. The dashed blue lines represent the IAN and the solid pink lines represent the ESN. The thicker the lines the stronger the connection. The weights for the IAN were randomly assigned, while the ESN weights were all set to one, thus creating a clique.

Initially, the bonding and bridging social capital are both 1, since everyone in the network is connected. To see how the social capital fluctuates, change the blue and/or pink values, again representing the IAN and the ESN weights respectively, in the spreadsheet.

Social Graph API by Google

2008-02-04T12:10:00.000-07:00

Google's Social Graph API allows developers to utilize the public connections among people on the web. The idea is simple, yet it could make it easier for people to connect across sites. Of course, the data all comes through Google, which yet increases our dependence on them. No doubt, other search engines could easily create the same API.

Since Google already saves a copy of all of the web pages that it spiders for search, the task of extracting the annotated links is somewhat trivial. Currently, this will only work as web developers annotate user links with XHTML Friends Network (XFN) and Friend of a Friend (FOAF). It is a great idea, but may take some time before web developers start annotating.
Furthermore, the easy access to people's connections is a nice data source for some new applications and experiments.

Measuring Social Capital (Weekly Update)

2008-01-29T17:00:00.000-07:00

During this week, I have been reading and thinking more about how to measure social capital. In the literature, there is no agreed upon definition of social capital nor is there any agreed upon method for measuring it. Therefore, proximal indicators are used to measure some believably related contributor to social capital.

For instance, Putnam has used the density of voluntary organizations (VOs) in a community to measure social capital. So, in this case, the density of VOs is a proximal indicator for social capital.

Nan Lin uses the number of connections an individual has weighted by the importance of the person.

Social Capital Survey Reading:
Social Capital: One or Many? by Martin Paldam (Economist)
Social Capital Survey by Tristan Claridge (in particular, see measurement)

Topic Tool

2008-01-29T14:00:00.001-07:00

Today, I put together a web page topic tool by using the web service that Nathan Davis made available. The tool takes in one or more web pages (i.e., a list of URLs) and then extracts the topics given the text on the web pages. The topics, or more accurately, the most likely topic components are extracted using an algorithm called Latent Dirichlet Allocation (LDA).

One potential use is for quickly generating blogger profiles to be used for implicit affinity networks. You can try it out at:

http://dml.cs.byu.edu/matthewsmith/tools/topictool/

Nathan uses his web service to make the query expansion service for Google searches called GooEgg.

Social Capital Measurement

2008-01-24T12:35:00.000-07:00

There are countless methods for measuring social capital. Below is a list of papers, studies, and resources that discuss some of these techniques:

Paldam, M. (2000), Social Capital: One Or Many? Definition And Measurement, in Journal of Economic Surveys, 14, 5, pp. 629-653.
Stone, W. (2001), Measuring Social Capital, Melbourne, Australian Institute of Family Studies, Research Paper No. 24/2001.
Li, Y, Pickles, A., Savage, M. (2003), Conceptualizing and measuring social capital: a new approach, Paper for BHPS 2003, Centre for Census and Survey Research, University of Manchester.
Grootaert, C., van Bastelaer, T. (2002), Understanding and Measuring Social Capital: A Multi-Disciplinary Tool for Practitioners, Washington DC, The World Bank.
Bullen, P., Onyx, J. (2000), Measuring Social Capital in Five Communities in NSW, Onyx and Bullen - Journal of Applied Behavior Science, Vol 36 No 1 March 2000 pp23-42. (Summary Report)
More Readings on Measuring Social Capital

It is clear that there are many ways to measure social capital. The techniques vary from discipline to discipline and researcher to researcher, whether it be Putnam or Burt, sociology or political science, the approaches tend to be different. Is there a chance for standardizing the definition? (Update: see more recent post for more on this)

DML Research Collaboration

2008-01-23T01:06:00.000-07:00

In effort to collaborate and refine our research at a faster pace, it might be fun to (1) blog about our research ideas each week and then (2) visit each lab member's blog to comment on what they have written to encourage good ideas and weed out the bad.

What do you think?

Weekly Update

2008-01-22T10:58:00.000-07:00

The tasks that I was involved (and some thoughts) during this week include the following:

Outlined a preliminary schedule and brainstormed for the "Making a Blog Big" project with Nathan Purser

Read more about Structural Holes and Network Closure, including:

Ronald Burt's Structural Holes versus Network Closure (PDF). Notes:

Some more quickly become prominent; some enjoy higher incomes; some lead more important projects; the interests of some are better served than interests of others. Better connected people enjoy higher returns. (These are some things that we could use to validate our social capital metrics.)
The Social Capital Metaphor is "people that are better connected do better", whereas the Human Capital Metaphor is "people who do better are more able individuals; they are more intelligent; more attractive; more articulate; more skilled."
A generic research finding in sociology and social psychology is that information circulates more within than between groups
Social capital can be argued to exist in structural holes (bridging) and network closures (bonding). However, the studies he performs and shows indicate that structural holes are the source of social capital.
Networks of densely interconnected contacts are systematically associated with substandard performance (bonding); networks that span structural holes are associated with creativity and innovation, positive evaluations, early promotions, high compensation and profits (bridging).
There remains an important role for closure. It can be critical to realizing the value buried in structural holes.
The mechanisms remain distinct. Closure describes how dense or hierarchical networks lower the risk associated with transaction and trust, which can be associated with performance. The hole argument describes how structural holes are opportunities to add value with brokerage across the holes, which is associated with performance.

Bruce Hoppe's blog post, Reputation and Trust (aka "Network Closure")

Uses a good personal example of Network Closure

I commented, "Virtual communities, no doubt, have an impact on real communities. However, I would argue that Putnam's work stands as does Burt's. The main issue seems to be that social capital is challenging to define precisely. Are there any agreed upon mathematical definitions of social capital? (It seems that there is still too much ambiguity.)"

Reading that I like to get to:

Found some other interesting resources, along the way:

Institute for Social Network Analysis of the Economy (ISNAE)
Social Network Researchers By Author (at ISNAE)
Introduction to Networks Mathematics by Bruce Hoppe
Bruce Hoppe's blog
Social Network Researchers:

Top Social Networking and Blogging Sites

2008-01-21T16:26:00.000-07:00

Nielsen Online released the top Social Networking sites (ranked by unique audience), as follows:MySpace still leads the pack this year, yet Facebook grew at an astounding rate of 72%, proving to be the "hottest" of the Top 5.

Top Blogging sites were as follows, as of December 2007:Google backed Blogger continues to have and add the most unique audience.

For more information, try this or either of these related reports.

Finding an Important Problem

2008-01-21T13:33:00.000-07:00

There seems to be infinitely many problems in the world. The trick is finding one that is sufficiently interesting to focus a dissertation on. The hope is that it has the following qualities:

Should impact a diverse audience (more than just the geeks in the computer lab)
Should be scientific yet have a host of business applications
Would be nice if it were related, at least in part, to my previous work
To be continued...

Generating Research

2008-01-16T15:43:00.000-07:00

As I am amidst the decision of what to focus my PhD work on, my thoughts take me back to a talk given nearly a year ago by Dan Olsen. In this presentation he mentioned the following steps for generating research:

Find an important problem
Generate lots of ideas
Filter them down
Make them real

This week I'd like to work on the step 1 and find an important problem in the realm of data mining and likely social networks. I'll let you know what I come up with.

Let It Live

2008-01-16T14:00:00.000-07:00

This is a little presentation that I gave to the DML today.

What is a Blog?

2007-12-12T18:42:00.000-07:00

Much or our recent work has been focused on the Blogosphere, which refers to the growing social network of people that write blogs, or web logs. We explore the social capital found in these networks of bloggers. Here is a little screencast that explains what blogs are and their relevance in today's world.

Javascript Word Jumper

2007-12-04T23:33:00.000-07:00

Here is a little JavaScript I wrote that can be used to move the cursor between words within a input fields.

WITS '07 in Montreal December 8-9

2007-12-01T01:46:00.001-07:00

The Seventeenth Annual Workshop on Information Technologies and Systems (WITS'07) will be held next weekend in Montreal, Canada. Despite the fact that Montreal is a very cold place, above average temperatures are expected on the days of the workshop (the white line on the plot below indicates the average temperature), which will be nice.

The workshop will be covering the following topics:

Social Networks
Security, Privacy and Risk Management
Collaboration and Blogging
Learning and Classification
Software Strategies
Knowledge Management
Ecommerce Systems
Workflow and Business Process Management
Recommender Systems
Data Management
Modeling and Evaluation of IS
Data Mining

They have me scheduled to present first in the "Social Networks" session, which happens to be the first session of the conference. I'm excited to share the work that we have been doing and collaborate with those that attend.

Becoming More Meaningful

2007-11-26T08:05:00.000-07:00

Since 2004, I've been posting occasionally to this blog --- however much of the content has been inaccessible for many visitors. This problem is evident in the low number of people that have commented on previous posts. ;)

In the future, I plan to make posts more meaningful and accessible for readers that have no special knowledge of the issue under discussion. This will give an opportunity for anyone to contribute and provide insight through comments. I should hope to increase the number of comments attached with future posts, including this one. :)

Methods to Identify Cell Architecture and Dynamics

2007-10-11T12:28:00.000-06:00

Notes and summary from Chapter 11 of Szallasi:

Regulatory networks consist of both network topology and dynamics. The term "network" emphasizes the topology, whereas the term "regulatory" emphasizes the dynamic interactions within the network, also called kinetics.

Experimental data can be used to capture both the topology, or architecture, and the dynamics of a cell. In the preceding chapter (and my corresponding write-up) data acquisition was discussed, whereas this chapter (and this write-up) focuses on using this data to model the topology and dynamics.

Szallasi mentions that, "engineering approaches have been instrumental in the reverse engineering effort". Reverse engineering or network inference in this context refers to identification of cellular networks from experiments. Various approaches are discussed in this chapter including Bayesian networks, iterative modeling, dynamic flux balance analysis, and

The reverse engineering of cellular networks is very complex. The kinetics/dynamics within the network are changing in time, which makes modeling incredibly challenging. The interactions change in complex manner that are often difficult to model by collecting data.

Szallasi identifies three challenges (within reverse engineering cellular networks) that will allow efficient and accurate dynamical modeling of networks:
(i) to improve the signal-to-noise ratio in the measurements
(ii) to develop new tools for measuring the cellular concentrations, fluxes, and interactions in both space and time
(iii) to incorporate model-based design of experiment protocol

Biological Data Acquisition

2007-10-09T12:21:00.000-06:00

My thoughts and comments are in between the points, labeled A-D, that Szallasi discusses in chapter 10 of "System Modeling in Cellular Biology":

A) "The overall size and complexity of intracellular networks"

From my experience working with social networks, I easily accept this to be a challenge. The complexity of a network grows quickly as the number of possible interactions grows
quadratically as each node is added.

In biological systems the complexity abounds as scientists have created models for everything form human social behavior on down to the an atom's behavior. Szallasi discusses the modular approach to Systems Biology and that can alleviate some of this complexity. He mentions that estimating the size of intracellular networks is the number of active genes in a given cell. (Of course, the number of genes is slightly in flux as new research continues to be done.)

B) "The general principles of biological measurements --- their technical and conceptual limitations."

When working with data there are always limitations based on your measurements. In some cases we often are not even measuring the right thing. Other times we are limited by the precision of our measurement devices.

I find it quite interesting that measurement devices often do not (and sometimes cannot) account for various environmental changes, which cause irregularities in the collected data.

C) "Concentration measurement versus kinetic parameter measurements"

Parameter estimation is important to get right, yet it is extremely challenging. In fact, both measurement techniques build upon the uncertainty acquired through data measurement.

D) "The actual target of the measurements"

Biological data can analyzed and measured using individual-centric or community-centric approaches, each of which are useful for in their own right. In each view, different problems exist and different interactions occur. Researchers should take great care to justify which measurements is being used to be how they are being viewed.

In conclusion, researchers must understand and accept the many limitations of biological data. As with any research, assumptions must be made and justified.

MicroArray Data Links

2007-10-04T15:45:00.000-06:00

Gene Expression Omnibus (GEO)
Stanford MicroArray Data

Improved Algorithm for Learning of Gene Regulatory Network Connectivity from Time Series Data

2007-10-01T12:35:00.000-06:00

Barker et al. presents the GeneNet algorithm designed to learn genetic regulatory network connectivity from time series data. The GeneNet algorithm is similar to work by Yu et al (2004), however, it takes a new approach by computing ratios of conditional probabilities and accumulating votes to determine influence between species. The approach taken by Yu et al uses Dynamic Bayesian Networks (DBN) and a cumulative distribution function (cdf) to determine a score for each species that may influence a gene. GeneNet approaches the problem differently by searching for differences between time points.

The pseudocode of the GeneNet algorithm is as follows:

GeneNet(Species S, Expts E, Influences I, Thresholds T, Levels L)
L:=DetermineLevels(S,E,L)
foreach c element of S:
Y:=CreateInfluenceVectorSet(c,S,E,I,T,L)
Y:=CombineInfluenceVectors(c,S,E,I,T,L,Y)
I(c):=CompeteInfluenceVectors(c,S,E,T,L,Y)
return I

Due to the lack of time series data available, synthetic data sets were generated for comparison. Empirical studies were performed which pitted GeneNet versus Yu's DBN algorithm on these synthetic datasets. GeneNet had significantly better precision, recall, and runtime for the majority of experiments.

(See the paper in Transactions on Computation Biology and Bioinformatics, No. 8, March 2007.)

Metabolite

2007-09-20T13:17:00.000-06:00

Quoted from the Columbia Encyclopedia:

metabolite, organic compound that is a starting material in, an intermediate in, or an end product of metabolism. Starting materials are substances, usually small and of simple structure, absorbed by the organism as food. These include the vitamins and essential amino acids. They can be used to construct more complex molecules, or they can be broken down into simpler ones. Intermediary metabolites are by far the most common; they may be synthesized from other metabolites, perhaps used to make more complex substances, or broken down into simpler compounds, often with the release of chemical energy. For example, glucose, perhaps the single most important metabolite, can be synthesized in a process called gluconeogenesis, can be polymerized to form starch or glycogen, and can be broken down during glycolysis in order to obtain chemical energy. End products of metabolism are the final result of the breakdown of other metabolites and are excreted from the organism without further change; they usually cannot be used to synthesize other metabolites.

2007-09-20T13:09:00.000-06:00

The Wikipedia article on Bioinformatics is worth scanning to get a feel for the research area.

Bioinformatics Reading Thoughts

2007-09-13T14:16:00.000-06:00

I'm taking a Bioinformatics class and have been reading "System Modeling in Cellular Biology" by Szallasi et. al. Here are the thoughts that I have had while reading the first few chapters.

Data-driven versus hypothesis-driven research
The world is very complex. Science has been used to understand how things work. Science has often been driven by a hypothesis followed by experimentation which then increases our understanding of the problem --- these questions were based on what we observe or maybe a few researchers have observed. Recently, we continue to gather more and more data which also can be used to drive research --- these questions are based not only on what we might observe in life, but additionally on what the data suggests, in some instances of millions of people. Both ways of attacking the problem can lead us to the same truth, however, it seems that the later has more potential of getting us there quicker.

Modeling
Modeling is constantly used in biological research. Szallasi mentions a couple reasons why models might be useful (1) testing whether a model is accurate and relect known facts, and (2) models can help us to understand which parts of the system contribute most to some desired properties of interest.

Robustness
I love how robust and resilient biological processes are. I would love to be able to create a computer program that is a fraction as robust as, say the body is at healing itself.

Modularity
Many biological processes are modular, much like how good programmers would make a function or class. For example, the human kidney can be substituted into another person and it can work successfully in them. Likewise, in programming, code that connects to a database can be used interchangeably withing multiple programs.

Bottom-up versus Top-down approaches
Bottom-up approaches typically build on existing biological knowledge, whereas, top-down approaches leverage the enormous amount of biological data to find something important to then delve into.

Export data from Postgres

2007-05-03T13:09:00.000-06:00

To export data from Postgres to an output file of your choice can be done by following the simple steps below:

1. Start psql with the database that you'd like to export from...

$ psql [DATABASE]

2. Toggle the output mode to unaligned (\a toggles between unaligned and aligned output mode)

=# \a

3. Turn "tuples only" off (\t toggles between tuples on and off)

=# \t

4. Set the output file (replace [FILE] with what you'd like to call your output file). It will send all query results to the file or |pipe.

=# \o [FILE]

5. Run whatever query you'd like to send to the output file. For example,

=# SELECT * FROM [TABLE];

In summary:

\a
\t
\o /tmp/outputfile.txt
SELECT ......
\o

Personalized Marketing

2006-08-07T13:58:00.000-06:00

Personalized marketing as a four phase process:

identifying potential customers
determining their needs and their lifetime value to the company
interact with customers so as to learn about them
customize products, services, and communications to individual customers

From Wikipedia, “Personalized marketing,” (Cited: Peppers, D. and Rogers, M. 1993)

Dimensionality Reduction Notes

2006-06-28T11:07:00.000-06:00

Principal Components Analysis (PCA)

How do you choose how many and which eigenvalues/eigenvectors to use?

Kaiser Criterion
This says to retain only factors with eigenvalues greater than 1. In other words, if a factor does not extract at least as much as the equivalent of one original variable then it is discarded. This criterion is named after Kaiser as he proposed it in 1960. It seems used quite frequently.

The Scree Test
This is a graphical test used to decide how many factors to keep. To perform this test, first, plot the eigenvalues in decreasing order. Next, Cattell suggests to find the place where the smooth decrease of eigenvalues appears to level off (to the right) similar geological scree (loose rock debris at the bottom of a rocky slope).

Here are some other useful terms and definitions from the dictionary:

Multicollinearity refers to linear inter-correlation among variables. Simply put, if nominally "different" measures actually quantify the same phenomenon to a significant degree -- i.e., wherein the variables are accorded different names and perhaps employ different numeric measurement scales but correlate highly with each other -- they are redundant.

K-Means Clustering

2006-06-23T16:26:00.000-06:00

Basic Algorithm
1. Choose k cluster centers at random
2. Assign each point to nearest cluster center
3. Compute the new cluster centers based on the assigned points
4. Repeat until cluster centers converge

Shortcomings
Finds local minima
The random placement of cluster centers affects the outcome

Here is a nice K-Means Demo

Taking Genealogy to the Common Person

2006-05-01T15:07:00.000-06:00

Subversion

2006-03-18T14:12:00.000-07:00

Handy Links for Subversion:
Download Subversion Client
http://subversion.tigris.org/faq.html

Google and data

2006-03-17T08:10:00.000-07:00

Here is an interesting article about Google and all their data.

GEDCOM file format information

2006-01-31T20:23:00.000-07:00

The following links discuss the how the GEDCOM format is defined:
Cyndi's List
The GEDCOM Standard Release 5.5
GEDCOM: The Next Generation

Networked Data File Types

2006-01-20T18:02:00.000-07:00

Here are reference links to common network data file types used in Link Mining and Social Network Analysis:
Pajek Net File
UCINet DL Files and VNA

Machine Learning Topics

2005-12-09T13:43:00.000-07:00

Particle Swarm Optimization
wikipedia
Swarm Intelligence

Ant Algorithms
ant colony optimization

Reinforcement Learning
wikipedia
Q-learning
Q-learning definition
Markov decision process

Computational Learning Theory
wikipedia
VC dimension
Principle of maximum entropy

Ensembles, Bagging and Boosting
Boosting

Meta-Learning
METAL KDD
Christophe Giraud-Carrier

HMMs
Hidden Markov model