Comments on: Cleaning up a set of tags, part 1 http://www.drmaciver.com/2009/01/cleaning-up-a-set-of-tags-part-1/ Wed, 10 Feb 2010 18:19:37 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: david http://www.drmaciver.com/2009/01/cleaning-up-a-set-of-tags-part-1/comment-page-1/#comment-591 david Wed, 28 Jan 2009 17:09:41 +0000 http://www.drmaciver.com/?p=373#comment-591 Also I'm glad you're finding it interesting. :-) I don't expect that much of what I'm doing will see use on these sites - it's not really for that. It's more for building data analysis on top of a noisy tag set. Also I’m glad you’re finding it interesting. :-) I don’t expect that much of what I’m doing will see use on these sites – it’s not really for that. It’s more for building data analysis on top of a noisy tag set.

]]> By: david http://www.drmaciver.com/2009/01/cleaning-up-a-set-of-tags-part-1/comment-page-1/#comment-590 david Wed, 28 Jan 2009 17:08:55 +0000 http://www.drmaciver.com/?p=373#comment-590 It's true that stripping off pluralisation can change the meaning of the tag - my expectation is that that these cases will be sufficiently far outliers compared to the number of cases where this removes that I can live with the slight loss of useful information. Generally the meaning will be close enough to preserved that it's tolerable. I'm going to do some analysis on usage later when I try to clean up things further, and when I do I'll see if I can spot any cases where it's actually breaking things. It’s true that stripping off pluralisation can change the meaning of the tag – my expectation is that that these cases will be sufficiently far outliers compared to the number of cases where this removes that I can live with the slight loss of useful information. Generally the meaning will be close enough to preserved that it’s tolerable.

I’m going to do some analysis on usage later when I try to clean up things further, and when I do I’ll see if I can spot any cases where it’s actually breaking things.

]]> By: Joshua Drake http://www.drmaciver.com/2009/01/cleaning-up-a-set-of-tags-part-1/comment-page-1/#comment-589 Joshua Drake Wed, 28 Jan 2009 17:00:10 +0000 http://www.drmaciver.com/?p=373#comment-589 One comment on pluralization. Although it sounds like this did not apply to the data set you normalized, it is possible that Statistic applied to a single fact or piece of information, while Statistics referred to the field of, and their may be additional cases where the plural and singular are different usefully different entities. That said I appreciate your work on this, especially the details given on your approach. I hope that because of work like yours, sites will consider cleaning up their tags. Only two sites I can think of attempt to control their tags to any degree. Amazon, which presents previously used tags and Stack Overflow which handles this with tag auto-completion and a reputation requirement for creating new tags. One comment on pluralization. Although it sounds like this did not apply to the data set you normalized, it is possible that Statistic applied to a single fact or piece of information, while Statistics referred to the field of, and their may be additional cases where the plural and singular are different usefully different entities.

That said I appreciate your work on this, especially the details given on your approach. I hope that because of work like yours, sites will consider cleaning up their tags. Only two sites I can think of attempt to control their tags to any degree. Amazon, which presents previously used tags and Stack Overflow which handles this with tag auto-completion and a reputation requirement for creating new tags.

]]> By: david http://www.drmaciver.com/2009/01/cleaning-up-a-set-of-tags-part-1/comment-page-1/#comment-588 david Wed, 28 Jan 2009 06:39:04 +0000 http://www.drmaciver.com/?p=373#comment-588 I don't mind at all. :-) It was an interesting read. Thanks. I will however probably continue using Ruby for these tasks. In particular I think you're going to be completely unable to port the second one to awk because that's where it stops being remotely line oriented and where I start making use of more general purpose libraries. I could start out with awk and switch to Ruby when that happens, but frankly I'm much more comfortable keeping it in the same language throughout. I don’t mind at all. :-) It was an interesting read. Thanks.

I will however probably continue using Ruby for these tasks. In particular I think you’re going to be completely unable to port the second one to awk because that’s where it stops being remotely line oriented and where I start making use of more general purpose libraries. I could start out with awk and switch to Ruby when that happens, but frankly I’m much more comfortable keeping it in the same language throughout.

]]> By: Porges http://www.drmaciver.com/2009/01/cleaning-up-a-set-of-tags-part-1/comment-page-1/#comment-587 Porges Wed, 28 Jan 2009 03:28:54 +0000 http://www.drmaciver.com/?p=373#comment-587 As you can see from the pingback I've written a reply semi-tutorial on how to use Awk to do the same task. Hope you don't mind :) As you can see from the pingback I’ve written a reply semi-tutorial on how to use Awk to do the same task. Hope you don’t mind :)

]]>