2,054,148 Pages

This is a page set up to centralize information relevant to the work undertaken by bots on LyricWiki.

If you encounter difficulties, this is the place to discuss it.
If you are currently seeking a solution to a problem, you may find your answer here.
If you have an idea or suggestion for a future project, please propose it on this page.

The page has been split into five sections:

Please provide details of any major project that you are considering undertaking in the Ideas and Suggestions section, in order that it can be discussed. This is recommended to avoid problems and conflicts with other users and in other areas of the site.


Adding all orphaned pages Special:Lonelypages to their respective artist pages' Other Songs lists. (Active July 2009)
Threading through links to redirects. (Active July 2009)
Adding language to song pages. (Active July 2009)
UmatBot / Umat 
Creating artist/album/lyrics pages from various sources. The source code is available. (Last active 23rd March 2009)
S2E2 / ∃cho⚡ierr∀ () 
Complete list: click here (Last active 20th March 2009)
Replacing all Category:Review Me's with star=Green (Last active 5th January 2009)
The Notorious B.O.T. / team a 
Adding Green Stars to Album pages (Last active 3rd January 2009)

NOTICE: Login process change on April 14th, 2010

Due to an emergency security upgrade of MediaWiki for version 1.15.3, both Wikipedia and Wikia have instituted a new login process that unfortunately is not backward-compatible with existing bots or MediaWiki API tools. If you use pywikipediabot, please upgrade to the newest version - it has the fixes in it. ÜberBot is broken too, but the fixes are in a parallel branch for his bot-framework so hopefully I can pull those changes in tonight. If you don't use pywikipediabot, here are two links which will hopefully help you in making the transition on your own:

If you have any other helpful tips or an example of code that you have updated, please leave them here to help the other Botters. Thank you! :)
-Sean Colombo 21:31, April 15, 2010 (UTC)

Code Discussion

If you think you might have a solution to any of the following problems, please offer it here. Once a solution has been found, please detail this to assist others who may at some point encounter the same problem. Thanks!

-start...or not

I can't run anything using the "-start" option, not for lack of trying! The exact same code (as Red has given on the preceding page) gives me an error, the last line of it being:

'NoneType' object has no attribute 'query'

Waah! I also tried


same result. ∃cho⚡ierr∀ () October 2008 (UTC)

You are using it like this, right? -start:START_PAGE "FIND" "REPLACE"
I know this came up awhile ago, but I figured I'd comment since it's on this page, which was created recently team atalkctrb 05:35, 25 October 2008 (UTC)

Long delay, sorry. Yes that's how I use it and it errors, here is another correctly syntaxed, non working (here) example:

python -start:"Train:Cab" "Help" "help"

∃cho⚡ierr∀ () 18:36, 5 November 2008 (UTC)

Edit Error: Blacklist Entry

Changing page [[en:The Blues Brothers:Opening: I Can't Turn You Loose]]
Cannot change The Blues Brothers:Opening: I Can't Turn You Loose because of blacklist entry 

This exact problem comes up often with other pages as well. Is it possible to allow bots to edit such pages? tia ∃cho⚡ierr∀ () 20:17, 5 November 2008 (UTC)

The fix for blacklist entry (which stops bots from editing the page) is to remove all instances of "[" & "]" from within lyrics tags, in a seperate edit.
Another item that would cause bots to skip is when a previous editor (not necessarily the last editor) inserted text outsie of tags. So after a bot run, whatever pages remain that should've been edited but weren't, most likely contains some sort of spam that needs a human editor. ∃cho⚡ierr∀ () 14:47, 1 January 2009 (UTC)

Searching for Languages

What would be the syntax for searching the entire lyrics for foo, then inserting a value in a parameter based on foo? Is this correct conditional syntax for bot:

if-then-else: (?(condition)yes-pattern|no-pattern)

so...looking for "ñ" (Unicode 00F1) anywhere in lyrics, and if found, set the language param to Spanish, otherwise exit. (Aqua's idea)

First, look for "ñ" within the lyric tags



"(?(/<lyric.*/>.*ñ.*<//lyric.*/>)Insert Spanish in Language|Do Nothing)"

∃cho⚡ierr∀ () October 2008 (UTC)

While the fancy-pants (like Sean) write their own bots from scratch, there is a way to do this with pywikipedia:
"(.*?)\<lyric(.*?)ñ(.*?)\<\/lyric(.*?)language\s?\=\s?([\r\n])" "\1<lyric\2ñ\3</lyric\4language = Spanish\5"
I would also do it -recursive and on -cat:Songs_Needing_Language_Identification. I ran this and it seemed to work, though there were no Spanish songs near the front of Category:Songs Needing Language Identification.--Aquatiki - T - E October 2008 (UTC)
Just one note: how about such songs as Jethro Tull:El Niño, having English lyrics with one or several Spanish (French, etc.) words? Senvaikis October 2008 (UTC)
You could maybe get a list of all pages that have the ñ, and I could go through, or you could batch edit by artist, once you found out if a particular artist sings in Spanish. - King_Nee1114 (talk pagecontributionsdeletions) 14:19, 21 October 2008 (UTC)
Thanks Aqua! Ran your code over the Spanish category, no hits, no hits in SNLI either. King, point well taken, this was just a concept, not one lyric page was modified. The idea of 100% figuring out lyric language based on a single character is not workable in Spanish (it's more usable on non Latin script systems though). Ultimately imho, all non English lyric pages need to be eyeballed by humans. If you've seen/used the translator dashboard widget ... a nifty solution would be passing a paragraph of the lyrics to a translation site with "?" as the current lang param, otherwise pass the chunk with a list of possible suspected source languages. The hit with me was the conditional example, not this particular implementation *|* ∃cho⚡ierr∀ () 04:47, 22 October 2008 (UTC)
I can also think of some examples: words like "het" would probably be Dutch, "le" or "ça" in French, stuff in Cyrillic probably Russian, in Greek letters Greek, etc. In what remains, languages like French, Spanish, Dutch, .. could be filtered out by looking for |de| (with | indicating word boundaries). --Mischko Talkicon EsperanzaIcon 16:24, 23 October 2008 (UTC)
Great idea, but with one problem- a lot of the non-English stuff has the little boxes instead of the proper accent marks. I'd almost suggest searching for common words- for example, a lot of Spanish songs would have "canción", "corazón", "amor", words like that, though I'm not coming up with the world's greatest examples here, since canción and corazón have the same accent problem that searching on the ñ would, and amor is multilingual. Still, it'd cut down the Songs Needing Language Identification, which I wouldn't mind.  :D NYCScribbler 02:23, 1 November 2008 (UTC)
This is an almost impossible quandry, and I don't think we can automate it well enough to never make mistakes. I think we're better off just making everything English and fixing things after that! --Aquatiki - T - E 02:57, 1 November 2008 (UTC)
It might be better to err on the side of non-English, rather than English. It's probably not meaningful for a page to be in the English category, but might be meaningful for another language. That is, native speakers can look over non-English pages, but the English language category will probably never really be sorted through. Also, those non-English pages can be searched for very complex 4 or 5 syllable English words, which everyone here knows, in order to add the artist to the English category. However, it's doubtful if the same complexities of every single other language will be explored by someone fluent, committed, and bot-knowledgeable enough to search through the English category. team atalkctrb09:03, 3 November 2008 (UTC)
While I agree with everything you said, "non-English" is not a language and doubt we could get away with picking a language at random. Maybe we should pick on Sev and use Russian! haha. Mainly I just wanted to point out that this is not the kind of problem we can automate ourselves out of. But how is picking just any other language better than leaving it unknown? Perhaps it's not. There! I've talked myself out of it! --Åqüã†ìкíƒΔΣ§ 16:48, 3 November 2008 (UTC)
By "non-English", I don't mean an actual category, but whatever language category any page would be put in as a result of a dictionary-based find-and-replace (i.e. ñ for Spanish). What I mean to say is, it is preferable for any dictionary-based searches to not search for English words ("the"), but for words in any other language. I'm with NYCScribbler on looking for longer words, but I'd suggest that the non-accented version of the word is likely to also be a globally unique language identifier (even though the non-accented version isn't officially part of the language, I know). There's got to be some Wikipedia page on globally unique words for languages (should be easier for Asian languages). Rather than looking for one word that's likely to appear on nearly all the pages using a language ("the" for English), we should focus on multiple words that are very unlikely to appear on pages using other languages than the one we're working with. Using complex words rather than just symbols gives us the benefit of a high probability of assigning the page to the right language, at least cutting the pages down to songs with no globally unique language identifiers. Artists (and their albums and songs) that can't be easily identified by this method will be left alone (possibly most of them). As far as the limits on automation, I think bots should be used to find pages with these words, but users should examine what these bots are finding, and add all pages by that artist to the language in question (I think this is the method EchoSierra is using). I agree that we can't rely on bots alone to solve this problem, but we can bot ourselves out of maybe 1/4 of these, which is a huge number. team atalkctrb 00:10, 4 November 2008 (UTC)
Aside from someone's Ph.D thesis on algorithms for unique language identification, I see no path to clarity. I know next to nothing of Japanese Pop, but I've heard them say things like
Nihonko watake the beat is hot chote moshu to

or similar such weirdness. I don't we can trust any computer with this task. I'm not depressed by this, though, since apart from page ranking details, I think this is the only human-dependent task we have. We can use bots in this task, though. Some are already going through and finding artists and marking all their songs "English" and the like. This seems to be the safest method. --Åqüã†ìкí ƒΔΣ § 00:35, 4 November 2008 (UTC)
You're probably right. It's tempting to think that we can figure even part of this out with a complex enough formula, but even if we could, it might be more trouble than it's worth. team atalkctrb 02:26, 4 November 2008 (UTC)

Way back in the war (the hot one and the following Cold one) they had this same problem..with ciphertext. They knew it would all be in one language, so they came with frequency tables for languages; apparently characters show up at different frequencies in different languages, of course geographic location names wouldn't skew the results too much (unlike a few Spanish words in an English song). It eventually led to what we now call search engines and NSA. There is a reason google has human editors!

However, I do it the low tech way. I look at SNLI one artist at a time, figure out the primary language of the artist (via musicbrainz, itunes store(s) or wikipedia) and go from there, if English, all unlabeled songs get English, or Swedish or what have you. If artist did a mix of lots of songs in two languages like Celine Dion or Gloria Estefan, I tag the minority in language 2 semi automatic (like a AK-47, lol) and the rest in bullet time. In special cases I paste a couple of verses in translator widget (yes I got to automate that, anything that shows in a dashboard widget has an API). I think it's better to err on the side of caution; once the English songs in SNLI is out of there (and that is the #1 priority for me), it will become the repository of unidentified languages and will be much smaller and easier to deal with. If a few Spanish songs end up in the Portuguese category or Bulgarian in the Russian category, native readers of such texts, going thru said language categories or artist of that native tongue, will either remove the incorrect language and shoot it back to SNLI, or correct the language label (one hopes). I'm happy the bot is doing some useful work towards a solution while I ramble on here, fwiw ∃cho⚡ierr∀ () 07:50, 4 November 2008 (UTC)

Looking for Neo

Calling the Oracle! There are tens of thousands of pages that don't have a language label or language data at all. Can you help with a regex that looks for "|language" in SongFooter, and if none exists, inserts it? Do I have to parse out all the existing fields (label+data), and reassemble them with a newly inserted lang, or is it possible by mere search and insert in bullet time? Has anyone been successful in doing simple "If Then" regex looking for a spoon that's not there? TIA ∃cho⚡ierr∀ () 03:20, 23 October 2008 (UTC)

I don't know if this is the same question as Echo is asking above but can someone correct this code for me please:
python -start:A_Klana_Indiana "language = " "language = Austrian"
And just in case you was wondering, no I wouldn't select a[ll] ;)  ♫Яєdxx Actions Words 16:45, 25 October 2008 (UTC)
I already did that ( that artist lang is done) and here be the code I used: -ref:"A_Klana_Indiana" -excepttext:"English" -regex "language\s*=\s*\n" "language = Austrian\n"
-excepttext means exclude all pages that contain the text "English", I use -ref because -start doesn't work for me. Try it slowly on another artist, like those remaining 79 ;)
excepttext is the answer to my (IF Then) question ∃cho⚡ierr∀ () 19:20, 25 October 2008 (UTC)
Thanks Echo! (I knew you'd know but thought I best post question here) And yes that was why I wanted to know. Is your -ref more specific to a particular group of pages? i.e. would it actually be better than -start for which I will either have to hit yes for each change, or be ready to quick fire control + C. ♫Яєdxx Actions Words 21:19, 25 October 2008 (UTC)
Sorry, I didn't have this page on watch list Duh! -ref (Works on all pages that link to a certain page. Argument can also be given as "-ref:referredpagetitle". ) -start works from allpages. When you look at the options for replace, they all help narrow down your search. A Good exercise would be writing the regex to scan thru SNLI, look for the word Instrumental in the page and if found, set the language of the page to Instrumental. so you can use -excepttext "French|English|German|Swedish" and -nocase to do case insensitive regex. hth -cheers ∃cho⚡ierr∀ () 11:36, 26 October 2008 (UTC)
Not that easy, need to look for
tag within lyrics so here's my 1st try on that:
regex: "\<lyric*\>.*\{\{Instrumental\}\}.*\<lyric*\>"
blah, back to the "If Then" again, so If Instrumental tag shows within lyrics tag, now how do we define the "Then" of the regex? dunno yet ma'am! ∃cho⚡ierr∀ () 11:46, 26 October 2008 (UTC)
Thanks again. Think I'll pass on the Instrumental thing for now (lol), but if I'm understanding correctly, it seems like -ref is probably best for confining to one artist.
I was also looking at the end bit, i.e. the \n. I use Windows. Is there sometimes differences in the way arguments are written if the bot is being run on Windows as opposed to Mac? (If there is, maybe we should incorporate that in our instructions/solutions section.)  ♫Яєdxx Actions Words 15:21, 26 October 2008 (UTC)
I think the line ending would be [\n\r] (character class of Mac/Unix endings), but I'm not certain. Shouldn't it be the line ending that the server uses, Aqua? I've not used \r yet, though I'm using Mac OS ∃cho⚡ierr∀ () 16:53, 26 October 2008 (UTC)
It's not either/or. Windows is \r + \n. Mac/Unix is \n. They do get converted, but if the page is made by a bot (i.e. Sean's Uberbot) then they might retain the automatically uploaded text. So, best to use [\r\n]. --Aquatiki - T - E 22:38, 26 October 2008 (UTC)

Artist Name = Reserved Word?

There are two artists that cause problems when being edited by bot (one showed up during the monobook skin switchover, hence my false alaram on that). One is Foundation, the other Toyah. Any edit on those artists stops the bot. fwiw ∃cho⚡ierr∀ () 18:26, 5 November 2008 (UTC)


Toyah !! Every page in Toyah namespace throws an exception and all goes dead...

No changes were necessary in [[The Soviettes:LP II (2004)]]
Getting [[Category:Albums L]] starting at The+Soviettes:LP+III+(2005)...
Target link 'Toyah:Love Is The Law (1983)' has different family 'toyah'
Exception in Page constructor
site=lyricwiki:en, title=Toyah:Love Is The Law (1983), insite=lyricwiki:en, defaultNamespace=0
Traceback (most recent call last):
  File "pywikipedia/", line 751, in __iter__ for page in self.wrapped_gen:
  File "pywikipedia/", line 684, in DuplicateFilterPageGenerator for page in generator:
  File "pywikipedia/", line 299, in CategorizedPageGeneratorfor page in category.articles(recurse = recurse, startFrom = start):
  File "pywikipedia/", line 332, in articles for tag, page in gen(recurse=recurse, startFrom=startFrom):
  File "pywikipedia/", line 160, in _getContentsNaive for tag, page in self._parseCategory(startFrom=startFrom):
  File "pywikipedia/", line 252, in _parseCategory yield ARTICLE, wikipedia.Page(, title)
  File "pywikipedia/", line 430, in __init__ % (title,, familyName))
NoPage: Toyah:Love Is The Law (1983) is not a local page on lyricwiki:en, and the toyah family is not supported by PyWikipediaBot!
Toyah:Love Is The Law (1983) is not a local page on lyricwiki:en, and the toyah family is not supported by PyWikipediaBot!


Target link 'Toyah:Obsession' has different family 'toyah'
Exception in Page constructor
site=lyricwiki:en, title=Toyah:Obsession, insite=lyricwiki:en, defaultNamespace=0
Traceback (most recent call last):
  File "pywikipedia/", line 751, in __iter__ for page in self.wrapped_gen:
  File "pywikipedia/", line 684, in DuplicateFilterPageGenerator for page in generator:
  File "pywikipedia/", line 299, in CategorizedPageGenerator for page in category.articles(recurse = recurse, startFrom = start):
  File "pywikipedia/", line 332, in articles for tag, page in gen(recurse=recurse, startFrom=startFrom):
  File "pywikipedia/", line 160, in _getContentsNaive for tag, page in self._parseCategory(startFrom=startFrom):
  File "pywikipedia/", line 252, in _parseCategory yield ARTICLE, wikipedia.Page(, title)
  File "pywikipedia/", line 430, in __init__ % (title,, familyName))
NoPage: Toyah:Obsession is not a local page on lyricwiki:en, and the toyah family is not supported by PyWikipediaBot!
Toyah:Obsession is not a local page on lyricwiki:en, and the toyah family is not supported by PyWikipediaBot!

∃cho⚡ierr∀ () 06:18, 18 November 2008 (UTC)

and my kludgy solution is to change the fLetter of the offending page to X to get it out of the bot's way.... ∃cho⚡ierr∀ () 02:31, 22 November 2008
So there is a method to the madness I was kinda hoping that my recent edits to Toyah had resolved this problem for you.  ♫Яєdxx Actions Words 03:44, 22 November 2008 (UTC)
All Toyah pages must be human edited... ∃cho⚡ierr∀ () 05:03, 22 November 2008 (UTC)
moved all Toyah to Toyah Wilcox, left on on her page as well cheers ∃cho⚡ierr∀ () 19:04, 2 January 2009 (UTC)

Ideas, Suggestions and Requests

Thoughts For This Page

Wanted to get everyone's thought on what the Bot Portal should be. Besides a place to talk about bot code, I am thinking that it could be a place to start discussing some issues before they get so scattered on different talk pages. The way I see it, this site requires quite a bit of automation, and using fixes for just the pages that a user works on isn't what this is all about, even if every user has a bot and looks at this page. Of course, I don't want to overload the server either, but I'm hoping that problems can be pointed out early before we have problems like the language problem, even if a user who spots something doesn't have the time, knowledge or even interest to fix a problem. That is, I'm envisioning that part of it can be considered a global to-do list of bot edits, and maybe a place to keep track of long-term editing that each user's bot is doing, so they don't conflict or produce redundant results.

I think Redxx was right in removing the temporary "problems" section from the main page, as it isn't really the place for discussion, which would have ended up on this page anyway. However, I think the Bot Portal should probably also have subpages, which it clearly links to, about users' thoughts about various problems and ideas about what needs fixing. Does anyone have thoughts about this? team atalkctrb 17:56, 14 October 2008 (UTC)

Let's see what kind of stuff we want to keep here. I think most of it'll be discussions, where once the question gets answer, people won't care to keep it. --Aquatiki - T - E 23:46, 14 October 2008 (UTC)
I agree with your thoughts for this page and I'm sure that is exactly what Echo had in mind too when he suggested it. I also agree with Aqua about the sub pages. I don't see the need for any atm. Like Aqua has indicated, I think we should just keep everything on this one page for time being and see where it leads...  ♫Яєdxx Actions Words 01:36, 16 October 2008 (UTC)


Many artists have [[Category:Genre/GENRE_NAME]] wikitext on their page, instead of using the {{Artist}} template to list genres. Many of these use a slash to list different genres incorrectly, i.e. [[Category:Genre/GENRE_1/GENRE_2/GENRE_3]] team atalkctrb October 2008 (UTC)

Commented on this on my talk page  ♫Яєdxx Actions Words 12:29, 18 October 2008 (UTC)

Duplicate iTunes

I'll probably end up doing it (since I seem to be the only one with the patience to do stuff site wide!) but there seem to a be a interspersed plethora of double iTunes links, one completed; one not. It's annoying. --Aquatiki - T - E 13:41, 29 October 2008 (UTC)

That still isn't fixed? I thought that had been around for awhile. team atalkctrb 17:38, 29 October 2008 (UTC)
Aqua, what is the best way (server side) to determine all records (songs) that contain a certain keyword? In this case, all songs with iTunes field label, but no iTunes data? Would it only be possible site wide ONLY by adding allpages to the job queue (given that this would have to be reckoned from SongFooter data?). In this case it may as well be done server side since currently we have a iTunes Category for Album pages, but none for Song pages!
Songs With Empty iTunes can be made into a (temp) category as well as all songs with filled iTunes for the permanent "Songs With Itunes Link" Category. SWEI can be traveresed with a bot in no time, rather than traversing allpages. my 2¢ cheers ∃cho⚡ierr∀ () 20:23, 29 October 2008 (UTC)
Even if you made a new category by saying #if {{{iTunes|}}} NOT then cat:SWEI, it wouldn't catch the one's I'm seeing since they do have a filled in one later on (i.e., the template wouldn't think them blank). As far as server-side doing, I know of no regex search tool for it. Even Replace Text can't handle multi-line or regex. --Aquatiki - T - E 20:57, 29 October 2008 (UTC)
All the Songs with duplicate iTunes have one blank and one filled, so...once Category Songs With iTunes is created, traversing that category will be sufficient to remove the duplicates, rather than the much lengthier job of traversing allpages. Does that compute? ;) cheers ∃cho⚡ierr∀ () 21:03, 29 October 2008 (UTC)
Sorry, I must be being thick: how are we going to create this category without traversing all pages? --Aquatiki - T - E 21:10, 29 October 2008 (UTC)
Sorry Aqua, I was not clear, instead of traversing allpages merely for dupe itunes (as your opening post suggests), traverse allpages for iTunes, so we get the category we need, and as collateral benefit, a subset of that category will be the dupe itunes songs. All clear? :) ∃cho⚡ierr∀ () 21:23, 29 October 2008 (UTC)
You mean to first edit {{SongFooter}} with {{#if | {{{iTunes|}}} | [[Category:Songs with iTunes]] | [[Category:Songs missing iTunes]]}} to create Category:Category:Songs with iTunes, then -subcat:Songs_with_iTunes? team atalkctrb 06:55, 31 October 2008 (UTC)
That looks suspiciouly close to what I am talking about.... If(itunes≠"", "Toons", "No Toons") . But given that SNLI still contains songs that Do have a language, maybe at the same time a fix can be produced and applied for both Language & iTunes identification for the price of one ~650K job queue... ∃cho⚡ierr∀ () 08:30, 31 October 2008 (UTC)
Sorry for the confusion, but I was trying to restate what you said for clarification, not propose something else (note the "?"). I don't know if it's possible to edit all pages that fall within two categories, but one longer regex could do both. I haven't kept up with SNLI and the language problems... are you referring to pages that have some kind of caching error that causes them to not be sorted properly, or that have two language parameters? team atalkctrb 08:55, 31 October 2008 (UTC)
I don't understand your comment about editing overlapping categories. recall that any mods to the SongFooter will have to propagate across all song pages, once the job queue after that mod is completed, We should have the categories for toons & no toons, and then we can edit them as necessary/ SNLI problem is that some songs that do have a language param entered still show up in SNLI, but then if you open one of those songs for edit, and save it without any mods, it gets removed from SNLI (good thing). I can't tell if it's caching problem or not. I've been working on killing SNLI, I need another 4-6 weeks..LOL ∃cho⚡ierr∀ ()09:35, 31 October 2008 (UTC)
In that case, fixing the duplicate iTunes parameter, using the new iTunes category, could not help but fix the songs with the language problem. As for what I wrote about editing both, I think I was writing something else but meant to delete all of it so I could ask you about it first. I need to go to sleep. team atalkctrb 09:41, 31 October 2008 (UTC)

YA: dupe iTunes

S2E2 has been removing double iTunes (not by scanning Cat:Songs), so If you come across any song with double itunes, please leave a note to let S2E2 know to clean up the Artist's songs. ∃cho⚡ierr∀ () 13:02, 19 December 2008 there ya go woking ;)

Thanks for response Echo. Will do!  ♫Яєdxx Actions Words 13:11, 19 December 2008 (UTC)

Bot: Resave

The same problem with SNLI has now started happening with Category:NofLetter (songs that do have valid fLetter are showing up there), solution seems to be opening page for edit, and save without mods, then it disappears from Category, Can this sort of bogus editing be done with bot? ∃cho⚡ierr∀ () 13:29, 31 October 2008 (UTC)

And then the guy answered his own question and done it Thanks Echo!  ♫Яєdxx Actions Words 00:18, 5 November 2008 (UTC)
Thanks to Aqua for providing the answer: ∃cho⚡ierr∀ () 07:13, 5 November 2008 (UTC)


I'm collecting ideas on doing some mass corrections to song pages, any ideas, suggestions, floating around out there? I intend to do as many corrections as possible in one pass.

(`) -->(')
[\r\n]* --> (\n)
([\r\n]_) --> (\n)

∃cho⚡ierr∀ () 07:25, 5 November 2008 (UTC)

Lines within >lyric(s)< tags that begin with a space (old style). --Åqüã†ìкí ƒΔΣ 07:40, 5 November 2008 (UTC)
This is contentious, but multiple iTunes ='s, typically with the first one blank. --Åqüã†ìкí ƒΔΣ 07:45, 5 November 2008 (UTC)
Should have {{Song}} on the first line. I like it when the lyric tags are on the line immediately above and below the lyrics themselves. Some pages also need a blank line between the end lyric tag and {{SongFooter}}. The thing is, the ringtone ads have screwed things up so the infoboxes (like wikipedia) don't line up with the upper right corner of the lyrics box anymore, so you might want to wait and see what happens, and maybe you can format all those infobox templates too. team atalkctrb 21:43, 5 November 2008 (UTC)
\si\s with  I --Åqüã†ìкí ƒΔΣ 05:15, 6 November 2008 (UTC)
What I like for song pages is Song, blank line, Lyrictag, lyrics, close Lyrictag, SongFooter. I usually format pages as much as I can stand.
How about {{SongFooter|, {{Album|, and down the line?
At this point we would probably want to roll in a fix for the fLetter change that just went through.
I'll agree with the double iTunes thing.
King_Nee1114 (talk pagecontributionsdeletions) 05:46, 6 November 2008 (UTC)
& amp ; --> &, curing doublets of type: &amp; , &quot; , &lt; , &gt; in lyrics (example).--Senvaikis (talk) 22:43, 6 November 2008 (UTC)

What about:
down within {{SongFooter ?... whitespace?
Outside lyrics: one blank line before <lyric*>and one after </lyric*>?
Within lyrics: Multiple (≥3) of [\r\n] --> (\n\n)?
Within lyrics: Verse - Chorus - Instrumental?
Within lyrics: all lines capped?

On the list:
Leading \s* after ^
Leading space within entire page
Cap all lonely "i"s in lyrics
double iTunes
html entities: & - ` - “ - > - <
Any more ideas? tia ∃cho⚡ierr∀ () 06:30, 7 November 2008 (UTC)

Just removing 'chorus' and 'verse' wouldn't be a good idea. Some pages are set up, granted they are wrong, where the chorus is explicit once, and then implied later. I would say there are too many different things that could happen with that, needs to be done by human hands.
King_Nee1114 (talk pagecontributionsdeletions) 18:36, 7 November 2008 (UTC)
Could you throw a category on those pages, possibly?
Also: I have found out that swedish has lonely 'i's that should not be capitalized. I would say that to get around that, check for english in the language, if english fix, else skip.
Another category that would help me is all pages with '==External Links==' explicitly stated, ie getting rid of these still floating around.
King_Nee1114 (talk pagecontributionsdeletions) 05:37, 8 November 2008 (UTC)
Here: Category:Need Footer Attention
As for typography, I can't figure out the case change code, Senv you have any examples?
 -regex "[\r\n](.)" "\1\U" 
or \u instead should work but doesn't (inserts literal \U \u instead) ∃cho⚡ierr∀ () 21:50, 16 November 2008 (UTC)
Woosh! ∃cho⚡ierr∀ () 18:57, 12 December 2008 (UTC)
Rfl. Is that song in SNLI Echo? I can't sing in code  ♫Яєdxx Actions Words 03:09, 13 December 2008 (UTC)
Not anymore, but I caught it in "Review Me", and now the source matches the (entity rendered) text. ∃cho⚡ierr∀ () 04:00, 13 December 2008 (UTC)

& -> And

Here's a job for anyone up for a brain workout. Redirect all of Nick_Cave_And_The_Bad_Seeds#Other_Songs that have an & in them to the "And" counterpart. All the "And" counterparts seem to be in pretty good nick. The {{mergefrom}} can be removed from the destination too. If your feeling really special, you can get it to compile a list of all the songs that still have {{mergefrom}}'s in them and I'll say thanks by manually doing them. What have I gotten myself into?? -- Humbug 15:11, 8 November 2008 (UTC)

Pages Without any Language Parameter

Request by user.  ♫Яєdxx Actions Words 19:52, 26 December 2008 (UTC)

Mass move request

Can someone with a bot move all pages on Caesar to Caesars (resp. remove if the song already exists)? Too much to do by hand ;) (is there a special page for such requests? If so, sorry I couldn't find it) --Singsangsung 23:36, 10 January 2009 (UTC)

done ∃cho⚡ierr∀ () 00:48, 11 January 2009 (UTC)
And the best place to make such requests is on Category:Requests For Moves. --Hfs·· 16:40, 11 January 2009 (UTC)

Unformatted Album Pages

Is there a quick solution/bot fix to get rid of album pages like this which have no {{Album}} template? I just stumbled on a few when looking at The Clarks' albums, so I don't know how many exist, but I can't imagine these are the only ones. --WillMak050389 04:25, 10 February 2009 (UTC)

sounds like walking thru all album categories looking for "[[Category:Albums.*" text and replace with default album template withe MAGIC words ∃cho⚡ierr∀ () 04:40, 10 February 2009 (UTC)
Sounds good to me, if it is plausible and not too time consuming. Also, keep in mind that some have album art that may not correspond to the current naming conventions. If the programmer want to earn bonus points with me, if they could get a list by doing this bot edit of the album art that doesn't fit the current conventions ("Image:<<artist name>> - <<album name, no year!>>.ext") I would be totally stoked. And also discouraged at how many need fixing, but terrifically happy in having the list. --WillMak050389 04:57, 10 February 2009 (UTC)
since this means going thru every album, what about appearance of these texts:
  1. "[[Category:Compilation"
  2. "[[Category:Genre.*"
  3. "[[Category:Label.*"
so I think maybe I should look for "[[Category:" then add "[[Category:Albums Needing Attention]]" to the album pages? ∃cho⚡ierr∀ () 06:41, 10 February 2009 (UTC)
and also album pages like this: A Tribute To The Priest, Vol. 1 (2002) ∃cho⚡ierr∀ () 14:04, 10 February 2009 (UTC)
I don't know of any category that should be explicitly placed on an album page. I see no problem with creating "[[Category:Albums Needing Attention]]" and having a bot populate it with these type pages. As for the ones which have unspecified parameters that are required in templates, have the templates take care of that. Add something like {{#if:{{{artist|}}}||[[Category:Albums Needing Footer Attention]]}} to albumfooter (this should be done with all required parameters). That's what I think. --WillMak050389 22:04, 10 February 2009 (UTC)
Yes excellent idea on detection by template, looks like 6 already took care of, so bot will look for any appearance of "[[Category" in album pages and flag them. ∃cho⚡ierr∀ () 00:02, 12 February 2009 (UTC)


Hi everyone, I talked about this topic with User_talk:6_times_9/Archive/2009-05#Multi-Lang and he said I could ask for it here. Is it possible to create a bot that moves translations of songs to a page named "artist:title/locale" and put {{Multiple Languages}} on it? Those songs can be found Category:Translated_songs... Reasons on the linked talk-page. Chris 22:52, 13 May 2009 (UTC)

To Do

Bot project - Multiple Song templates

I don't know how feasible this actually is but it would tidy and update pages and also avoid songs being wrongly categorised due to differences in the star parameter:

  1. Look for multiple instances of {{Song}} and place these in a category, e.g. Category:Songs needing Additional Albums attention
  2. Look for those in that category with up to 3 {{Song}} templates per page. Combine into the one Song template by adding album2 parameter and album3, type2, type3 where applicable. Remove S.N.A.A.A. category link.
  3. Replace all but one Song template on the remaining pages left in the S.N.A.A.A. category with the {{Additional Albums}} template.

Bot project - No 1

Extract the countries from the Category:Number-One Hits in links at the bottom of each song page in Category:Number-One Hits and add these to the {{No.1}} template by way of using this parameter. After which the Category:Number-One Hits in links can be deleted. If necessary, move No.1 template between {{Song}} and lyrics tag.

Bot Project - Page Ranking

Change everyalbum = Black and allsongs = Black on both the {{Artist Info}} and on {{Album Info}} to everyalbum = Green, allsongs = Green.  ♫Яєdxx Actions Words 04:27, 12 February 2009 (UTC)

hih? black? ∃cho⚡ierr∀ () 16:38, 13 February 2009 (UTC)
Yes those two parameters in the PR certificate on existing talk pages. I've seen quite a few that are everyalbum = Black and allsongs = Black where they should now be Green.  ♫Яєdxx Actions Words 18:44, 14 February 2009 (UTC)
Errr… What about albums/artists where some songs are still red links? How can all song pages be Green if some of them don't even exist yet??? — 6x9 (Talk) 18:58, 14 February 2009 (UTC)
Trust you to complicate the issue 6, when for once I was actually trying to K.I.S.S. Admittedly I'm a bit out of practice.Cyclops1  ♫Яєdxx Actions Words 20:21, 14 February 2009 (UTC)
Examples Echo >>>> Talk:Paramore Talk:Bad Company  ♫Яєdxx Actions Words 23:56, 18 February 2009 (UTC)

Uncategorized files

Dunno if this is even possible… Could a bot go through Special:UncategorizedImages, find all images that have an album page link to them and add a completed {{Albumcover}} template? Possibly adding albums with more than one colon to a special category for humans to review… — 6x9 (Talk) 19:06, 14 February 2009 (UTC)

Janitor has just started doing this.
- teknomunk (talk,E,,A) 18:36, 11 July 2009 (UTC)
[1] Great! Thanks! I'm sure Will and notime will be happy as well. — 6×9 (Talk) 20:07, 11 July 2009 (UTC)
There was a problem that came up, however. Many of these album covers are not on album pages at all, just on the artist's page. That will eventually need to be handled. If somebody else does not get to if first, I will do it eventually (not today).
- teknomunk (talk,E,,A) 20:49, 11 July 2009 (UTC)

"Length = 00:00"

According to a Google search there are about 200 albums that have length parameter set to "00:00" (probably because people copied the example template from the help page and forgot to replace it). Could a bot delete these, so the pages can join the ~30,000 others in Category:Albums missing length? — 6x9 (Talk) 23:17, 15 February 2009 (UTC)

This obviously was done by S2E2 the next day, just no confirmation. Example - Chris 15:57, 11 July 2009 (UTC)


Could some bot go through all talk pages in this category and delete those whose content pages have been turned into redirects? — 6x9 (Talk) 17:13, 19 March 2009 (UTC)

Other Songs – Spring Cleaning

Is the following procedure possible with a bot? Go through all artist pages with Other Songs sections and…

  1. if there's more than one such section, merge them;
  2. check if any of them are redirects*, if yes, update the links;
  3. sort alphabetically and weed out duplicates;
  4. remove those OS already linked to elsewhere on the page;
  5. put those artist pages which have OS with different artist spellings (e.g. songs by Faithfull Marianne on the Marianne Faithfull page) in a special category for humans to sort out?

(*) For those who haven't seen it yet, there's now a special page listing links to redirects. — 6x9 (Talk) 19:36, 30 March 2009 (UTC)

Tuning toons

Would be nice if a bot could go through all pages in Category:iTunes/Artist and strip the "id=" bit off those iTunes parameters that still have it… — 6×9 (Talk) 23:50, 13 May 2009 (UTC)


iTunes determination

From Album Footer: code to determine whether album has itunes and if so, will categorise the album in

{{#if:{{{iTunes|}}}|<span class="plainlinks"> 
iTunes: {{ITunes|title={{{album}}}|id={{{iTunes}}} }}[[Category:ITunes]] </span><br/>}}

Artist fLetter

There was a few Artists Needing fLetter, here's how I fixed them, extremely simple and dangerous!

  1. working on Category:Artists Needing fLetter only!
-log:S2E2 -v -cat:"Artists_Needing_fLetter" -regex "(\{\{Artist[\r\n])" "\1|fLetter = {{SUBST:ARTISTFLETTER}}"

looks for "{{Artist" followed by a line break; either of [\r\n] then appends "|fLetter = ARTISTFLETTER". hth ∃cho⚡ierr∀ () 00:00, 13 November 2008 (UTC)

Thanks Echo!  ♫Яєdxx Actions Words 00:11, 13 November 2008 (UTC)
Also, all Artist fLetters Corrected, and removed explicit artist category from artist pages ;)

∃cho⚡ierr∀ () 17:46, 13 November 2008 (UTC)

Good work, Echo. I bet you didn't know that you won the award for most consecutive hits on the server, at 47 per minute at 13:03, November 13, 2008. Also for making me laugh. You doing drive-by regex too, now? team atalkctrb 05:09, 14 November 2008 (UTC)
methinks SandBot has been impersonating S2E2, can't conjure up any other explanation... ∃cho⚡ierr∀ () 06:12, 14 November 2008 (UTC) and the volume was set to 11 ;)
Nope you can't get away with that >>>>>Special:Contributions/SandBot (she's had a cold since the 12th  ♫Яєdxx Actions Words 01:06, 17 November 2008 (UTC)

Pywikipedia ""

Does anyone know how to add entries to It seems that this file (located in default pywikipedia directory) is a list of replacements to be performed simultaneously, which can be used with the -fix: switch of pywikipedia's command. It looks fairly simple, but I was wondering if anyone has screwed with it yet. team atalkctrb 02:32, 14 October 2008 (UTC)

It screwed up pages so badly, that I just got mad and haven't even tried again! --Aquatiki - T - E 04:18, 14 October 2008 (UTC)
Using already existing "fixes", or when you tried to add custom ones? team atalkctrb 04:25, 14 October 2008 (UTC)
The existing "fixes" were so obnoxious, I quit trying to sort through them all and just gave up >-( --Aquatiki - T - E 06:33, 14 October 2008 (UTC)
Thanks Team a for bringing up fixes, here is a custom version. Basis for my typography fixes, I wonder if janitor uses a similar housekeeping file...
# -*- coding: utf-8  -*-
""" File containing my custom fixes """
__version__ = '$Id: 6068 2008-11-25 03:48:48Z EchoSierra $'

help = """
                       * Custom  - fixes for lyricwiki

fixes = {
    'songpages': {
        'regex': True,
        'msg': {
            'en': u'Robot: Fixing Whatever ',
        'replacements': [
            (u'foo1',  u'bar1'),
            (u'foo2',  u'bar2'),
            (u'foo3',  u'bar3'),

# Load the user fixes file.

import config

    execfile(config.datafilepath(config.base_dir, ""))
except IOError:

cheers ∃cho⚡ierr∀ () 03:35, 27 November 2008 (UTC)

Score! I'm a PHP guy, not Python, so this really hits the spot. You rock! --Åqüã†ìкí ƒΔΣ 05:31, 27 November 2008 (UTC)

Other Business

Useful Sites

Monty Python seems to be a good place to look for regex synatx specific to python. ∃cho⚡ierr∀ () October 2008 (UTC)

Yes thanks Echo!  ♫Яєdxx Actions Words 01:36, 16 October 2008 (UTC)
Two More: Tutorial 1 Tutorial 2

New Picture

Love it. Good thinking, Aqua! team atalkctrb 02:26, 14 October 2008 (UTC)

Awww...a bot family! Thank you Aqua!  ♫Яєdxx Actions Words 13:49, 14 October 2008 (UTC)

Broken signatures

After some messing around with my signature, I am sure some broken signatures are hanging around here. And not only mine, I also found one here, for example. Assuming the bots see the Wiki markup and not the generated HTML, maybe one of you bots can locate pages with <img or &lt;img and put them in some category so I can go through them and clean them up. I know, not really priority #1, but I like to leave things tidy. --Mischko Talkicon EsperanzaIcon (wishing happiness, health and heaps-of-fun for 2009 to all my friends here at LyricWiki!) 15:08, 3 November 2008 (UTC)

There are only a few such broken things, and since they are talk pages, they might be very big and just putting them in a category wouldn't help that much. Why not just give us the full search and replace? --Åqüã†ìкíƒΔΣ§ 16:52, 3 November 2008 (UTC)
Green check  ♫Яєdxx Actions Words 00:16, 5 November 2008 (UTC)
I'd like to give you a full search/replace expression, but the problem is that I don't know any :) For example, I can try cook up some regex which will match all or most of my broken sigs, but probably it won't catch for example Kiefer's on the page linked earlier. Maybe the best solution is just to go about and only fix them when I run into them. --Mischko Talkicon EsperanzaIcon (wishing happiness, health and heaps-of-fun for 2009 to all my friends here at LyricWiki!) 18:35, 8 November 2008 (UTC)

Important change: all skins going to "LyricWiki" instead of "Monobook"

To allow us to upgrade versions of MediaWiki easily in the future, we forked the skin and now maintain it under the name LyricWiki. In order to allow users to disable the ringtone ads, I've made some changes but they would be confusing as heck if some users are on Monobook and don't see the changes, so I'm about to migrate everyone to the new LyricWiki skin. In the future there is only going to be the LyricWiki skin (since it has important/essential functionality... and also ads :P).

To make the transition easier, I'm not going to migrate the bots automatically... but you might want to move your bot over to use the new LyricWiki skin as soon as possible so that any future bots you write will be ready to rock (and I'm not even going to be testing other skins, so it's possible they'll break in future upgrades).
-Sean Colombo (talk|contribs) 22:56, 4 November 2008 (UTC)

How do we go about migrating the bots from monobook skin to Lyricwiki skin? tia ∃cho⚡ierr∀ () 00:33, 5 November 2008 (UTC)
Log in as your bot and go to Preferences-> Skins. --Åqüã†ìкí ƒΔΣ 01:57, 5 November 2008 (UTC)
Problem is LyricWiki skin is the only choice now! Nothing else :( S2E2 02:03, 5 November 2008 (UTC)
Bedeebeedeebee SandBot 02:04, 5 November 2008 (UTC)
Ok, let me clarify: Bots will break when trying to do bot jobs with LyrocWiki skin, they can log in via browser like humans with LyricWiki skin. As Aqua said, it's best to keep Monobook as a choice for bots (or even make it the default for bots only, there's only less than 10 bots) tia ∃cho⚡ierr∀ () 02:19, 5 November 2008 (UTC)
Irony - SandBot works.  ♫Яєdxx Actions Words 02:55, 5 November 2008 (UTC)
Well, I seem to stand corrected. I seems LW skin is close enough to Monobook skin as not to matter for bots. False alarm (so far). --Åqüã†ìкí ƒΔΣ 03:35, 5 November 2008 (UTC)
Sheepish Grin! ok, it's a bug in me. very very sorry! ∃cho⚡ierr∀ () 04:31, 5 November 2008 (UTC) bad filename or command, press any key to continue seems to work, but is horribly broken. Bots don't view ads anyway, so can't monobook be made to work somehow? team atalkctrb 01:07, 6 November 2008 (UTC) rofl @ SandBot - so that's what she sounds like...

Good news!

It seems that Janitor has made a full recovery!  ♫Яєdxx Actions Words 13:44, 12 November 2008 (UTC)

Congratulations! *Mischko cancels the order for the fruit basket.* --Mischko Talkicon EsperanzaIcon (wishing happiness, health and heaps-of-fun for 2009 to all my friends here at LyricWiki!) 15:29, 18 November 2008 (UTC)

Another Take on SNLI

Hi. I’m new to the bot business and I have yet to apply for a bot account with Sean. I have written a bot with pywikipedia to do language classification which I would like to run on SNLI. It’s a naïve Bayesian classifier (similar to how many learning spam filters work) working on the 3-grams of the lyrics.

I have already trained it on the existing languages with more than a few songs and about 1/20 of the English songs. It’s tuned to be very conservative about classification. For each language a probability from 0.0 = low over 0.5 = uncertain to 1.0 = certain is calculated. Only if one language has a high probability and all other languages have a low one, a song will be changed to this language. Else the song will be left unchanged. I works well for big languages like English, German, French, but has it’s problems e.g. to discern Portuguese from Spanish. It should still be acceptable as those cases will be left untouched.

What do you think? --Hfs·· 14:03, 24 November 2008 (UTC)

Hi Hs
imho, Let it tag the English songs only, that will bring down SNLI considerably for humans to deal with the non-English ones. cheers ES (Talk) 14:47, 24 November 2008 (UTC) Hs of ASUD list?
Sounds great! I'd been hoping we'd be able to use something like Naïve Bayes for this. I like the concept of not classifying it unless we have a pretty deterministic rating too. Sometimes language classification can be... iffy. Thanks for writing this code, it'll be really helpful :)
I put more here about bot-flags and stuff: User_talk:Sean_Colombo#Request_for_bot_account.
As for the stuff of just rating English, I personally trust classifiers pretty well regardless of the language as long as the scores for that language are significantly higher than all other languages (we worked with that a bit at my day-job). It'll be interesting to see how accurate it is or isn't in the wild.
-Sean Colombo (talk|contribs) 15:13, 24 November 2008 (UTC)
Excellent stuff Hs..Welcome to the family!  ♫Яєdxx Actions Words 16:55, 24 November 2008 (UTC)
Ok, the bot is now running as User:Man-Machine (contribs), slowly at 1 edit per minute. It’s set up to work on every 20th song of the category, such that all of the alphabet gets some love. Let’s see how it works out. --Hfs·· 23:05, 24 November 2008 (UTC)

After moving to wikia

Most bot accounts won't work anymore because the bots were renamed as well as users. I still don't like the account-system of wikia We'll all have to got to search our bots new names. After the bot was renamed, the user-config and the family have to be changed to get it running again. I'll see if I somehow can make it. - Chris 22:14, September 10, 2009 (UTC)

I have already updated the Pywikipedia settings that worked for me. I wasn't able to login, though: I always get the "wrong password or captcha" message. A workaround was to login with the bot account, list the cookie contents for in Firefox and transfer them manually to login-data/ Hfs 19:17, September 12, 2009 (UTC)
I got this too when I tried to log in with Ex-Botanic (have to find me a new name -_-), I used the wrong master for my settings. Then I used this and then it worked. - Chris 21:08, September 12, 2009 (UTC)
Thanks for the hint! I have merged the missing path() method in the Pywikipedia settings. Login works for me again, too. Hfs 14:58, September 13, 2009 (UTC)
Guys I'm not having any luck with this. These settings (which I already had) don't work since the move so I tried the Pywikipedia settings you kindly put up on the preceding page Hfs (* copied and pasted) but I simply get "SyntaxError: invalid syntax". Any ideas?  Яєdxx Actions Words 00:34, October 29, 2009 (UTC)
I'm not sure but I think it'd be sensible to change to - Chris 15:01, October 29, 2009 (UTC)
I already did that CHris as Hfs included this in his settings. This is my So where am I going wrong?  Яєdxx Actions Words 23:47, November 2, 2009 (UTC)
When I tried your file from pastebin I got an IndentationError. It may sound weird if you're not used to Python, but whitespace is significant. When I fixed the indentation your file worked for me. Please try to reformat the file such that the number of leading whitespaces is a multiple of 4 [2]. --Hfs·· 18:41, November 5, 2009 (UTC)
Gosh, of course - I could've thought about that one! *sigh* - I'm trying to learn Python atm, and indention is what forms the script-language: instead of brackets like ( { and [ for block-opening and -closing, Python uses indention per line to find the lines belonging to the blocks. - Chris 20:22, November 5, 2009 (UTC)
Thanks Hfs. I was really hopeful, but alas... :(  Яєdxx Actions Words 01:41, November 6, 2009 (UTC).
Or to put it another way I get the following error message: "Incorrect export page". Any suggestions guys?  Яєdxx Actions Words 07:50, November 13, 2009 (UTC)
Community content is available under Copyright unless otherwise noted.