warning: Creating default object from empty value in /var/www/drupal/modules/taxonomy/taxonomy.pages.inc on line 33.

irc

Django IRC Contributions Graph

Django IRC D3 CONTRIBUTIONS GRAPH

This graph represents The number of posts in the Django IRC logs. The lighter green squares represent days with less posts than the darker green squares. Months go from left to right and are separated by the darker lines. Days go by columns from left to right.

SQL Code

SELECT YEAR( date_of_entry ) , MONTH( date_of_entry ) , DAY( date_of_entry ) , COUNT( `type` )
FROM `django_irc`
WHERE TYPE = 'message'
GROUP BY 1 , 2, 3
ORDER BY YEAR( date_of_entry ) ASC, MONTH( date_of_entry ) ASC,DAY( date_of_entry ) ASC

Source code

This visualization was created using D3 and JavaScript. To view similar source code on GitHub head over to the FLOSSmole Github repositories

Want to see more examples of how to use FLOSSmole data? Examples

Ubuntu IRC Contributions Graph

Ubuntu IRC D3 CONTRIBUTIONS GRAPH

This graph represents The number of posts in the UbuntuIRC logs. The lighter green squares represent days with less posts than the darker green squares. Months go from left to right and are separated by the darker lines. Days go by columns from left to right.


SQL Code

SELECT YEAR( date_of_entry ) , MONTH( date_of_entry ) , DAY( date_of_entry ) , COUNT( `type` )
FROM `ubuntu_irc`
WHERE TYPE = 'message'
GROUP BY 1 , 2, 3

Source code

This visualization was created using D3 and JavaScript. To view similar source code on GitHub click here

Bitcoin IRC D3 Contributions Graph

BITCOIN IRC D3 CONTRIBUTIONS GRAPH

This graph represents The number of posts in the Bitcoin IRC logs. The lighter green squares represent days with less posts than the darker green squares. Months go from left to right and are separated by the darker lines. Days go by columns from left to right.

SQL Code

SELECT YEAR( date_of_entry ) , MONTH( date_of_entry ) , DAY( date_of_entry ) , COUNT( `type` )
FROM `bitcoindev_irc`
WHERE TYPE = 'message'
GROUP BY 1 , 2, 3

Source code

This visualization was created using D3 and JavaScript. To view the source code on GitHub click here

Bitcoin-dev, Ubuntu, Perl6, Django, Puppet IRC logs are updated

Thanks to the work of my two summer research assistants Evan Ashwell & Greg Batchelor, the IRC channels for #bitcoin-dev, perl6, #ubuntu, #django, and puppet (#gen, #dev, and #razor) have been updated.

Things to know:

  • These IRC chats are only available on the FLOSSmole MySQL database server (how to get access) and not as flat files. Why? Well, they started out as flat files, so we don't want to just re-host flat archives. The original flat files are available for Puppet (puppetlogs.com), Bitcoin-dev (bitcoinstats.com), Ubuntu (Ubuntu Logs), Perl6 (Perl6 logs), and Django (Django IRC logs)
  • The data model is one day = one datasource id
  • The chat logs have been divided into the following columns (some logs have fewer columns):
    • datasource_id
    • line_num
    • line_message
    • type
    • send_user
    • date_of_entry
    • time_of_entry
    • unix_time
    • last_updated
  • An example row looks like the following:
    • 61835
    • 42
    • ah thanks. I'll search.
    • message
    • arubi
    • 2016-05-28
    • 21:38:00
    • l1464471492.0
    • 2016-06-02 13:03:3

Software Archaeology: GNUe IRC data & summaries

Back in the 2000's, the GNU Enterprise (GNUe) project chat logs (and human-created chat log summaries!) were used by several papers in the area of text summarization, especially dialogue summarization.

The reason the GNUe chat logs and summaries were used is that the logs were accompanied by summaries that were compiled periodically (manually) by a human. The summarized chat logs can thus be considered a kind of "gold standard" for what kind of summary a machine summarizer should produce.

Here are some papers that reference the GNUe chat logs or the summaries:
--Zhou & Hovy (2005) Digesting virtual "geek" culture: the summarization of technical internet relay chats
--Elliott & Scacchi (2007) Free Software Development: Cooperation and Conflict in a Virtual Organizational Culture
--Ulthus & Aha. Multiparticipant chat analysis: A survey
--Sood, Mohamed, & Varma. Topic-focused summarization of chat conversations

Unfortunately, the group that put together the summaries ("Kernel Traffic") no longer has a web presence, and the summaries and original log files are no longer available at any of the locations those papers link to.

FLOSSmole to the rescue! Here are the files that have been reconstructed, using what we could find on the Wayback Machine (archive.org):
-1-original chat logs for GNUe
-2-original Kernel Traffic GNUe chat summaries
-3-text list of URLs for chat logs, taken from Archive.org (used to build #1 above)
-4-text list of URLs for XML summaries of the chat logs, taken from Archive.org (used to build #2 above)
-5-all source code for how I collected and parsed this data on Github
-6-all data loaded into the FLOSSmole database interface on MySQL server (get your username and password)

In the database (#6 in the list above), there are 6 tables:
--GNUeIRCLogs (the log files themselves)
--GNUeSummaryItems (the text & metadata for the weekly summary)
--GNUeSummaryMentions (all the people mentioned in each summary)
--GNUeSummaryPara (the paragraph summary text, links removed)
--GNUeSummaryParaQuote (the quoted text from the logs that made it into the summary itself as quoted text)
--GNUeSummaryTopic (the topic that the summarizer classified each summary into)

IRC log updates: perl6, ubuntu, django

Hi moles! New IRC chat logs now cleaned and stored in the irc database on the FLOSSmole mysql server, thanks to Andrea Black, one of our intrepid FLOSSmole research assistants. This data is part of an overall IRC collection started by another student, Becca Gazda, last summer.

We now have the following IRC chat histories available:

Apache
--activemq
--aries
--camel
--cxf
--kalumet
--karaf
--servicemix

Wordpress
--bbpress
--buddypress
--buddypress-dev
--coreplugins
--dev
--events
--gsoc
--meta
--mobile
--polyglots
--sfd
--themes
--ui
--general irc

Openstack
--devdns
--infra
--meeting3
--meetingalt
--meeting
--irc

Django
Perl6 -- NEW
Ubuntu

Coming soon : Bitcoin

FLOSS as a source for insults

FLOSSmole is hosting the data from a new paper: FLOSS as a source for profanity and insults: Collecting the data

Get the data

Get the slides

New schema for IRC data

In my continuing quest to be organized, I've created a new schema to hold just the IRC log data. On the database server (access instructions here), there is a new schema called 'irc' and it includes (for now) Ubuntu logs, Django logs, 7 Apache projects, and the topic lines from Freenode for all channels with 3+ users.

Coming soon: email updates, including Linux Kernel Mailing List (LKML) and more IRC (Wordpress, etc).

Enjoy!

New Apache project IRC data

Hello moles! Happy January. Here are some fresh new data sources for your mining pleasure:

1. Freenode channel list and topics (all public channels with 3 or more users). The table is called "fn_irc_channels".
2. Apache Activemq IRC logs (one datasource_id per day, one row per message).
3. Apache Aries IRC logs
4. Apache Camel IRC logs
5. Apache CXF IRC logs
6. Apache Karaf IRC logs
7. Apache Kalumet IRC logs
8. Apache Servicemix IRC logs

here is a sample of what the structure looks like for 2-8:

CREATE TABLE `apache_servicemix_irc` (
`datasource_id` int(11) NOT NULL,
`line_num` int(11) NOT NULL,
`full_line_text` varchar(500) NOT NULL,
`line_message` varchar(500) NOT NULL,
`date_of_entry` date NOT NULL,
`time_of_entry` varchar(5) NOT NULL,
`type` enum('action','system','message') NOT NULL,
`about_user` varchar(50) NOT NULL,
`last_updated` datetime NOT NULL,
PRIMARY KEY (`datasource_id`,`line_num`)
)

These are available on the live MySQL connection.

Syndicate content