* [gentoo-commits] gentoo-projects commit in forums/scripts/projectSCAS/scripts: rebuildSearchindex.pl
@ 2007-09-07 21:07 Tom Knight,,, (tomk)
0 siblings, 0 replies; 4+ messages in thread
From: Tom Knight,,, (tomk) @ 2007-09-07 21:07 UTC (permalink / raw
To: gentoo-commits
tomk 07/09/07 21:07:15
Modified: rebuildSearchindex.pl
Log:
Removed some stopwords
Revision Changes Path
1.16 forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl
file : http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?rev=1.16&view=markup
plain: http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?rev=1.16&content-type=text/plain
diff : http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?r1=1.15&r2=1.16
Index: rebuildSearchindex.pl
===================================================================
RCS file: /var/cvsroot/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl,v
retrieving revision 1.15
retrieving revision 1.16
diff -u -r1.15 -r1.16
--- rebuildSearchindex.pl 3 Sep 2007 14:22:27 -0000 1.15
+++ rebuildSearchindex.pl 7 Sep 2007 21:07:14 -0000 1.16
@@ -167,7 +167,7 @@
Text::Iconv->raise_error(0); # don't call croak on errors
# - defines >
-my @stopwords = qw(afaik iirc ive lol rotf rotflmao ymmv aber able about above access actually add after again ago all almost along alot already also always amp and another answer any anybody anybodys anyone anything anyway anywhere are arent around ask askd auch auf available back bad because been before being believe best better between big bit both box btw bug build but can cannot cant card case change che check code come command compile compiled compiling computer con configuration correct could couldnt course create das day days days default den der desktop did didnt die different does doesnt doing done dont down drive each edit either else emerged end enough errors etc even ever every everybody everybodys everyone everything exactly example failed far few file files find fine first fix fixed following for forum forums found from function gentoo get getting give going gone good got gotten great guess had hard hardware has have have havent having help her here hers him h
is home hope how however hows href ich idea ideas ill info ini install installation installed installing instead into isnt issue ist its its ive just keep know large last latest least less let lib like liked line link linux list little load local log lol long look looked looking looking looks lot machine made mal man many may maybe mean message might mit mode more most much must mustnt name near need net network never new news next nice nicht non none not nothing now off often old once one only oops open option options org other our ours out output over own package packages page part pas people per play please point possible post pretty probably problem problems program put que question questioned questions quite quot quote rather read really reason recent remember right run said same saw say says screen script see seem seems sees server set setting settings setup she should since sites small software solution some someone something sometime somewhere soon sorry source start
started still stuff such support sure take tell than thank thanks that thatd thats the their theirs them then there theres these they theyd theyll theyre thing things think this those though thought thread through thus time times too tried true try trying two type und under until untrue update upon use used user users using usr version very via want was way well went were werent what whats when where which while who whom whose why wide will wink with with within without wont work worked working works world worse worst would wrong wrote www yes yet you youd youll your youre yours);
+my @stopwords = qw(afaik iirc ive lol rotf rotflmao ymmv aber able about above access actually add after again ago all almost along alot already also always and another answer any anybody anyone anything anyway anywhere are around ask auch auf available back bad because been before being believe best better between big bit both box btw build but can cannot cant card case change che code come compile compiled compiling computer con configuration correct could course create das day days days default den der did didnt die different does doesnt doing done dont down each edit either else end enough errors etc even ever every everyone everything exactly example failed far few file files find fine first fix fixed following for forum forums found from function gentoo get getting give going gone good got gotten great guess had hard has have have having help her here him his home hope how however ich idea ideas info install installation installed installing instead into issue ist its
its ive just keep know large last latest least let lib like line link linux list little load local log lol long look looked looking looking looks lot machine made mal many may maybe mean message might mit mode more most much must name near need net network never new news next nice nicht non none not nothing now off often old once one only oops open option options other our out output over own package packages page part pas people per play please point possible post pretty probably problem problems put que question questions quite quote rather read really reason recent remember right run said same saw say says see seem seems server set setting settings setup she should since small software solution some someone something somewhere soon sorry source start started still stuff such support sure take tell than thank thanks that thats the their them then there these they thing things think this those though thought thread through thus time times too tried true try trying two type
und under until update upon use used user users using usr version very via want was way well went were what whats when where which while who why will wink with with within without wont work worked working works worse would wrong wrote yes yet you your yours);
my %synonyms = ("abcense" => "absence", "abridgement" => "abridgment", "accomodate" => "accommodate", "acknowledgment" => "acknowledgement", "airplane" => "aeroplane", "allright" => "alright", "andy" => "andrew", "anemia" => "anaemia", "anemic" => "anaemic", "anesthesia" => "anaesthesia", "appologize" => "appologise", "archean" => "archaean", "archeology" => "archaeology", "archeozoic" => "archaeozoic", "armor" => "armour", "artic" => "arctic", "attachment" => "attachement", "attendence" => "attendance", "barbecue" => "barbeque", "behavior" => "behaviour", "biassed" => "biased", "biol" => "biology", "buletin" => "bulletin", "calender" => "calendar", "canceled" => "cancelled", "car" => "automobile", "catalog" => "catalogue", "cenozoic" => "caenozoic", "center" => "centre", "check" => "cheque", "color" => "colour", "comission" => "commission", "comittee" => "committee", "commitee" => "committee", "conceed" => "concede", "creating" => "createing", "curiculum" => "curriculum", "
defense" => "defence", "develope" => "develop", "discription" => "description", "dulness" => "dullness", "encyclopedia" => "encyclopaedia", "enroll" => "enrol", "esthetic" => "aesthetic", "etiology" => "aetiology", "exhorbitant" => "exorbitant", "exhuberant" => "exuberant", "existance" => "existence", "favorite" => "favourite", "fetus" => "foetus", "ficticious" => "fictitious", "flavor" => "flavour", "flourescent" => "fluorescent", "foriegn" => "foreign", "fourty" => "forty", "gage" => "guage", "geneology" => "genealogy", "grammer" => "grammar", "gray" => "grey", "guerilla" => "guerrilla", "gynecology" => "gynaecology", "harbor" => "harbour", "heighth" => "height", "hemaglobin" => "haemaglobin", "hematin" => "haematin", "hematite" => "haematite", "hematology" => "haematology", "honor" => "honour", "innoculate" => "inoculate", "installment" => "instalment", "irrelevent" => "irrelevant", "irrevelant" => "irrelevant", "jeweler" => "jeweller", "judgement" => "judgment", "labeled
" => "labelled", "labor" => "labour", "laborer" => "labourer", "laborers" => "labourers", "laboring" => "labouring", "licence" => "license", "liesure" => "leisure", "liquify" => "liquefy", "maintainance" => "maintenance", "maintenence" => "maintenance", "medieval" => "mediaeval", "meter" => "metre", "milage" => "mileage", "millipede" => "millepede", "miscelaneous" => "miscellaneous", "morgage" => "mortgage", "noticable" => "noticeable", "occurence" => "occurrence", "offense" => "offence", "ommision" => "omission", "ommission" => "omission", "optimize" => "optimize", "organise" => "organize", "pajamas" => "pyjamas", "paleography" => "palaeography", "paleolithic" => "palaeolithic", "paleontological" => "palaeontological", "paleontologist" => "palaeontologist", "paleontology" => "palaeontology", "paleozoic" => "palaeozoic", "pamplet" => "pamphlet", "paralell" => "parallel", "parl" => "parliament", "parlt" => "parliament", "pediatric" => "paediatric", "pediatrician" => "paediatr
ician", "pediatrics" => "paediatrics", "pedodontia" => "paedodontia", "pedodontics" => "paedodontics", "personel" => "personnel", "practise" => "practice", "program" => "programme", "psych" => "psychology", "questionaire" => "questionnaire", "rarify" => "rarefy", "reccomend" => "recommend", "recieve" => "receive", "resistence" => "resistance", "restaraunt" => "restaurant", "savior" => "saviour", "sep" => "september", "seperate" => "separate", "sept" => "september", "sieze" => "seize", "summarize" => "summarise", "summerize" => "summarise", "superceed" => "supercede", "superintendant" => "superintendent", "supersede" => "supercede", "suprise" => "surprise", "surprize" => "surprise", "synchronise" => "synchronize", "temperary" => "temporary", "theater" => "theatre", "threshhold" => "threshold", "transfered" => "transferred", "truely" => "truly", "truley" => "truly", "useable" => "usable", "valor" => "valour", "vigor" => "vigour", "vol" => "volume", "whack" => "wack", "withold"
=> "withhold", "yeild" => "yield");
my %stopword = ();
--
gentoo-commits@gentoo.org mailing list
^ permalink raw reply [flat|nested] 4+ messages in thread
* [gentoo-commits] gentoo-projects commit in forums/scripts/projectSCAS/scripts: rebuildSearchindex.pl
@ 2007-09-07 21:59 Tom Knight,,, (tomk)
0 siblings, 0 replies; 4+ messages in thread
From: Tom Knight,,, (tomk) @ 2007-09-07 21:59 UTC (permalink / raw
To: gentoo-commits
tomk 07/09/07 21:59:58
Modified: rebuildSearchindex.pl
Log:
removed a few more stopwords
Revision Changes Path
1.17 forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl
file : http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?rev=1.17&view=markup
plain: http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?rev=1.17&content-type=text/plain
diff : http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?r1=1.16&r2=1.17
Index: rebuildSearchindex.pl
===================================================================
RCS file: /var/cvsroot/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl,v
retrieving revision 1.16
retrieving revision 1.17
diff -u -r1.16 -r1.17
--- rebuildSearchindex.pl 7 Sep 2007 21:07:14 -0000 1.16
+++ rebuildSearchindex.pl 7 Sep 2007 21:59:57 -0000 1.17
@@ -167,7 +167,7 @@
Text::Iconv->raise_error(0); # don't call croak on errors
# - defines >
-my @stopwords = qw(afaik iirc ive lol rotf rotflmao ymmv aber able about above access actually add after again ago all almost along alot already also always and another answer any anybody anyone anything anyway anywhere are around ask auch auf available back bad because been before being believe best better between big bit both box btw build but can cannot cant card case change che code come compile compiled compiling computer con configuration correct could course create das day days days default den der did didnt die different does doesnt doing done dont down each edit either else end enough errors etc even ever every everyone everything exactly example failed far few file files find fine first fix fixed following for forum forums found from function gentoo get getting give going gone good got gotten great guess had hard has have have having help her here him his home hope how however ich idea ideas info install installation installed installing instead into issue ist its
its ive just keep know large last latest least let lib like line link linux list little load local log lol long look looked looking looking looks lot machine made mal many may maybe mean message might mit mode more most much must name near need net network never new news next nice nicht non none not nothing now off often old once one only oops open option options other our out output over own package packages page part pas people per play please point possible post pretty probably problem problems put que question questions quite quote rather read really reason recent remember right run said same saw say says see seem seems server set setting settings setup she should since small software solution some someone something somewhere soon sorry source start started still stuff such support sure take tell than thank thanks that thats the their them then there these they thing things think this those though thought thread through thus time times too tried true try trying two type
und under until update upon use used user users using usr version very via want was way well went were what whats when where which while who why will wink with with within without wont work worked working works worse would wrong wrote yes yet you your yours);
+my @stopwords = qw(afaik iirc ive lol rotf rotflmao ymmv aber able about above actually add after again ago all almost along alot already also always and another answer any anybody anyone anything anyway anywhere are around ask auch auf available back bad because been before being believe best better between big bit both box btw build but can cannot cant card case change che code come computer con correct could course das day days days default den der did didnt die different does doesnt doing done dont down each edit either else end enough etc even ever every everyone everything exactly example failed far few file files find fine first fix fixed following for forum forums found from function gentoo get getting give going gone good got gotten great guess had hard has have have having help her here him his hope how however ich idea ideas info install installed instead into issue ist its its ive just know last latest least let lib like line link linux list little local lol long
look looked looking looking looks lot machine made mal many may maybe mean message might mit mode more most much must name near need never new news next nice nicht non none not nothing now off often old once one only oops open other our out output over own page part pas people per play please point possible post pretty probably problem problems put que question questions quite quote rather read really reason recent remember right run said same saw say says see seem seems server set setup she should since small solution some someone something somewhere soon sorry start started still stuff such support sure take tell than thank thanks that thats the their them then there these they thing things think this those though thought thread through thus time times too tried true try trying two type und under until update upon use used user users using usr version very via want was way well went were what whats when where which while who why will wink with with within without wont wor
k worked working works worse would wrong wrote yes yet you your yours);
my %synonyms = ("abcense" => "absence", "abridgement" => "abridgment", "accomodate" => "accommodate", "acknowledgment" => "acknowledgement", "airplane" => "aeroplane", "allright" => "alright", "andy" => "andrew", "anemia" => "anaemia", "anemic" => "anaemic", "anesthesia" => "anaesthesia", "appologize" => "appologise", "archean" => "archaean", "archeology" => "archaeology", "archeozoic" => "archaeozoic", "armor" => "armour", "artic" => "arctic", "attachment" => "attachement", "attendence" => "attendance", "barbecue" => "barbeque", "behavior" => "behaviour", "biassed" => "biased", "biol" => "biology", "buletin" => "bulletin", "calender" => "calendar", "canceled" => "cancelled", "car" => "automobile", "catalog" => "catalogue", "cenozoic" => "caenozoic", "center" => "centre", "check" => "cheque", "color" => "colour", "comission" => "commission", "comittee" => "committee", "commitee" => "committee", "conceed" => "concede", "creating" => "createing", "curiculum" => "curriculum", "
defense" => "defence", "develope" => "develop", "discription" => "description", "dulness" => "dullness", "encyclopedia" => "encyclopaedia", "enroll" => "enrol", "esthetic" => "aesthetic", "etiology" => "aetiology", "exhorbitant" => "exorbitant", "exhuberant" => "exuberant", "existance" => "existence", "favorite" => "favourite", "fetus" => "foetus", "ficticious" => "fictitious", "flavor" => "flavour", "flourescent" => "fluorescent", "foriegn" => "foreign", "fourty" => "forty", "gage" => "guage", "geneology" => "genealogy", "grammer" => "grammar", "gray" => "grey", "guerilla" => "guerrilla", "gynecology" => "gynaecology", "harbor" => "harbour", "heighth" => "height", "hemaglobin" => "haemaglobin", "hematin" => "haematin", "hematite" => "haematite", "hematology" => "haematology", "honor" => "honour", "innoculate" => "inoculate", "installment" => "instalment", "irrelevent" => "irrelevant", "irrevelant" => "irrelevant", "jeweler" => "jeweller", "judgement" => "judgment", "labeled
" => "labelled", "labor" => "labour", "laborer" => "labourer", "laborers" => "labourers", "laboring" => "labouring", "licence" => "license", "liesure" => "leisure", "liquify" => "liquefy", "maintainance" => "maintenance", "maintenence" => "maintenance", "medieval" => "mediaeval", "meter" => "metre", "milage" => "mileage", "millipede" => "millepede", "miscelaneous" => "miscellaneous", "morgage" => "mortgage", "noticable" => "noticeable", "occurence" => "occurrence", "offense" => "offence", "ommision" => "omission", "ommission" => "omission", "optimize" => "optimize", "organise" => "organize", "pajamas" => "pyjamas", "paleography" => "palaeography", "paleolithic" => "palaeolithic", "paleontological" => "palaeontological", "paleontologist" => "palaeontologist", "paleontology" => "palaeontology", "paleozoic" => "palaeozoic", "pamplet" => "pamphlet", "paralell" => "parallel", "parl" => "parliament", "parlt" => "parliament", "pediatric" => "paediatric", "pediatrician" => "paediatr
ician", "pediatrics" => "paediatrics", "pedodontia" => "paedodontia", "pedodontics" => "paedodontics", "personel" => "personnel", "practise" => "practice", "program" => "programme", "psych" => "psychology", "questionaire" => "questionnaire", "rarify" => "rarefy", "reccomend" => "recommend", "recieve" => "receive", "resistence" => "resistance", "restaraunt" => "restaurant", "savior" => "saviour", "sep" => "september", "seperate" => "separate", "sept" => "september", "sieze" => "seize", "summarize" => "summarise", "summerize" => "summarise", "superceed" => "supercede", "superintendant" => "superintendent", "supersede" => "supercede", "suprise" => "surprise", "surprize" => "surprise", "synchronise" => "synchronize", "temperary" => "temporary", "theater" => "theatre", "threshhold" => "threshold", "transfered" => "transferred", "truely" => "truly", "truley" => "truly", "useable" => "usable", "valor" => "valour", "vigor" => "vigour", "vol" => "volume", "whack" => "wack", "withold"
=> "withhold", "yeild" => "yield");
my %stopword = ();
--
gentoo-commits@gentoo.org mailing list
^ permalink raw reply [flat|nested] 4+ messages in thread
* [gentoo-commits] gentoo-projects commit in forums/scripts/projectSCAS/scripts: rebuildSearchindex.pl
@ 2007-09-09 10:29 Tom Knight,,, (tomk)
0 siblings, 0 replies; 4+ messages in thread
From: Tom Knight,,, (tomk) @ 2007-09-09 10:29 UTC (permalink / raw
To: gentoo-commits
tomk 07/09/09 10:29:05
Modified: rebuildSearchindex.pl
Log:
Set explicit charset for updates
Revision Changes Path
1.18 forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl
file : http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?rev=1.18&view=markup
plain: http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?rev=1.18&content-type=text/plain
diff : http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?r1=1.17&r2=1.18
Index: rebuildSearchindex.pl
===================================================================
RCS file: /var/cvsroot/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl,v
retrieving revision 1.17
retrieving revision 1.18
diff -u -r1.17 -r1.18
--- rebuildSearchindex.pl 7 Sep 2007 21:59:57 -0000 1.17
+++ rebuildSearchindex.pl 9 Sep 2007 10:29:04 -0000 1.18
@@ -304,6 +304,9 @@
if (defined $description && defined $fname)
{
$dbh=DBI->connect($connectString, '', '', { RaiseError => 1, AutoCommit => 1 }) || die("Could not connect to database!");
+
+ executeSQL_charset(\$dbh, "utf8");
+
$sth = $dbh->prepare(qq{update phpbb_forums set forum_desc=?, forum_name=? where forum_id=?});
$ext = $sth->execute($description,$fname,$forumId);
$sth->finish();
@@ -800,6 +803,8 @@
# - is this word a stopword? >
if (!$stopword{$thisword})
{
+ executeSQL_charset(\$dbh, "utf8");
+
$sth = $dbh->prepare(qq{select word_id from scas_search_wordlist where word_text=?});
$ext = $sth->execute($thisword);
($word_id)=$sth->fetchrow_array;
@@ -816,6 +821,8 @@
}
else
{
+ executeSQL_charset(\$dbh, "utf8");
+
# - No word found; insert >
$sth = $dbh->prepare(qq{insert into scas_search_wordlist (word_text,word_id) values (?,?)});
$ext = $sth->execute($thisword,'NULL');
@@ -858,12 +865,16 @@
{
if ($sql_buf{$this_table_suffix})
{
+ executeSQL_charset(\$dbh, "utf8");
+
$sth = $dbh->prepare("insert into scas_search_wordmatch_".$this_table_suffix." (post_id,word_id,word_count,word_inSubject,forum_id) values ".substr($sql_buf{$this_table_suffix},0,length($sql_buf{$this_table_suffix})-1));
$ext = $sth->execute();
}
}
if ($sql_buf{'others'})
{
+ executeSQL_charset(\$dbh, "utf8");
+
$sth = $dbh->prepare("insert into scas_search_wordmatch (post_id,word_id,word_count,word_inSubject,forum_id) values ".substr($sql_buf{'others'},0,length($sql_buf{'others'})-1));
$ext = $sth->execute();
}
--
gentoo-commits@gentoo.org mailing list
^ permalink raw reply [flat|nested] 4+ messages in thread
* [gentoo-commits] gentoo-projects commit in forums/scripts/projectSCAS/scripts: rebuildSearchindex.pl
@ 2007-09-11 7:05 Tom Knight,,, (tomk)
0 siblings, 0 replies; 4+ messages in thread
From: Tom Knight,,, (tomk) @ 2007-09-11 7:05 UTC (permalink / raw
To: gentoo-commits
tomk 07/09/11 07:05:32
Modified: rebuildSearchindex.pl
Log:
fixed for the PM part of the script
Revision Changes Path
1.19 forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl
file : http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?rev=1.19&view=markup
plain: http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?rev=1.19&content-type=text/plain
diff : http://sources.gentoo.org/viewcvs.py/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl?r1=1.18&r2=1.19
Index: rebuildSearchindex.pl
===================================================================
RCS file: /var/cvsroot/gentoo-projects/forums/scripts/projectSCAS/scripts/rebuildSearchindex.pl,v
retrieving revision 1.18
retrieving revision 1.19
diff -u -r1.18 -r1.19
--- rebuildSearchindex.pl 9 Sep 2007 10:29:04 -0000 1.18
+++ rebuildSearchindex.pl 11 Sep 2007 07:05:31 -0000 1.19
@@ -457,7 +457,7 @@
# - do update >
$sth = $dbh->prepare(qq{update phpbb_privmsgs_text set privmsgs_text=? where privmsgs_text_id=?});
- $ext = $sth->execute($pm_subject, $pmId);
+ $ext = $sth->execute($pm_text, $pmId);
}
else
{
--
gentoo-commits@gentoo.org mailing list
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2007-09-11 7:12 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-09-07 21:07 [gentoo-commits] gentoo-projects commit in forums/scripts/projectSCAS/scripts: rebuildSearchindex.pl Tom Knight,,, (tomk)
-- strict thread matches above, loose matches on Subject: below --
2007-09-07 21:59 Tom Knight,,, (tomk)
2007-09-09 10:29 Tom Knight,,, (tomk)
2007-09-11 7:05 Tom Knight,,, (tomk)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox