[gentoo-dev] [dRFC] slotted mysql ready

[gentoo-dev] [dRFC] slotted mysql ready

[Francesco Riosa]

Background, to make life easyer for people that use more versions of
mysql (and just for fun) MySQL is going to be slotted.

Currently the slot enabled versions are mysql-4.1.16-r30
mysql-5.0.18-r30 mysql-5.1.4_alpha-r30, in short those with "-r30".

The few patches needed are already included in current ~ mysql-4.1.16
and mysql-5.0.18 but hidden due to SLOT=0, starting from 2005-11

An eselect-mysql module has been prepared to create simlinks for the
desired version of mysql making easy to switch between them (hopefully)

The libraries (libmysqlclient & co) don't follow this logic and the
higher version is always the default (similarly to sys-libs/db).

The rc-scripts have been modified to use the logic of
sys-apps/baselayout net.ethx => net.lo symlink to be able to start more
servers at once (this is working from 2005-11 but reworked today)

Additionally the ebuilds code has been moved to two new eclasses,
mysql_fx.eclass and mysql.eclass, the first own support functions, the
last own the src_* pkg_* functions.
The initial idea is that when a specific ebuild need to marked stable
the code is moved from the mysql.eclass to the ebuild itself, to froze it.

Documentation on:
o switch to slotted
o upgrade with no downtime using this new capabilities
need to be written from scratch (this is going to be the more difficult
part for me).

there is a bash construct the eclass use to retrieve the latest two
chars of a string, in the form: MYVAR="${MYARRAY[3]:(-2)}" , is this
acceptable?

The intention is to made mysql-5.0.18-r30 stable on 2005-02-15 at
maximum on the first archs.

AFAIK I'm the only one who have seen this stuff until now, any comment,
any suggestion is highly apreciated.


Francesco Riosa
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [dRFC] slotted mysql ready

[Kalin KOZHUHAROV]

Francesco Riosa wrote:
> Background, to make life easyer for people that use more versions of
> mysql (and just for fun) MySQL is going to be slotted.
> 
> Currently the slot enabled versions are mysql-4.1.16-r30
> mysql-5.0.18-r30 mysql-5.1.4_alpha-r30, in short those with "-r30".
> 
> The few patches needed are already included in current ~ mysql-4.1.16
> and mysql-5.0.18 but hidden due to SLOT=0, starting from 2005-11
> 
> An eselect-mysql module has been prepared to create simlinks for the
> desired version of mysql making easy to switch between them (hopefully)
> 
> The libraries (libmysqlclient & co) don't follow this logic and the
> higher version is always the default (similarly to sys-libs/db).

So how does this affect packages that use say dev-perl/DBD-mysql ?

If they can only use mysql-5, then the will be broken if they connect to a mysql-4 server using non
UTF-8 encoding. (bad wording, but generally speaking, there is an inconsistence in character
encoding betweet mysql-4 and mysql-5 that DOES break things on non ISO-8859-1 encoded databases)

> The rc-scripts have been modified to use the logic of
> sys-apps/baselayout net.ethx => net.lo symlink to be able to start more
> servers at once (this is working from 2005-11 but reworked today)
> 
> Additionally the ebuilds code has been moved to two new eclasses,
> mysql_fx.eclass and mysql.eclass, the first own support functions, the
> last own the src_* pkg_* functions.
> The initial idea is that when a specific ebuild need to marked stable
> the code is moved from the mysql.eclass to the ebuild itself, to froze it.
> 
> Documentation on:
> o switch to slotted
> o upgrade with no downtime using this new capabilities
> need to be written from scratch (this is going to be the more difficult
> part for me).
> 
> there is a bash construct the eclass use to retrieve the latest two
> chars of a string, in the form: MYVAR="${MYARRAY[3]:(-2)}" , is this
> acceptable?
> 
> The intention is to made mysql-5.0.18-r30 stable on 2005-02-15 at
> maximum on the first archs.
> 
> AFAIK I'm the only one who have seen this stuff until now, any comment,
> any suggestion is highly apreciated.

Kalin.

-- 
|[ ~~~~~~~~~~~~~~~~~~~~~~ ]|
+-> http://ThinRope.net/ <-+
|[ ______________________ ]|

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [dRFC] slotted mysql ready

[Francesco Riosa]

Kalin KOZHUHAROV wrote:
[...]
>>
>> An eselect-mysql module has been prepared to create simlinks for the
>> desired version of mysql making easy to switch between them (hopefully)
>>
>> The libraries (libmysqlclient & co) don't follow this logic and the
>> higher version is always the default (similarly to sys-libs/db).
> 
> So how does this affect packages that use say dev-perl/DBD-mysql ?
> 
> If they can only use mysql-5, then the will be broken if they connect to a mysql-4 server using non
> UTF-8 encoding. (bad wording, but generally speaking, there is an inconsistence in character
> encoding betweet mysql-4 and mysql-5 that DOES break things on non ISO-8859-1 encoded databases)
> 
[...]

At the moment 4.0 is not slotted, only >= 4.1 are.

The fact that is effectively possible to use a non slotted 4.0 together
 with 4.1 and 5.0 does not mean that it should be used that way.

Does this answer your question ?

-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [dRFC] slotted mysql ready

[Kalin KOZHUHAROV]

Francesco Riosa wrote:
> Kalin KOZHUHAROV wrote:
> [...]
> 
>>>An eselect-mysql module has been prepared to create simlinks for the
>>>desired version of mysql making easy to switch between them (hopefully)
>>>
>>>The libraries (libmysqlclient & co) don't follow this logic and the
>>>higher version is always the default (similarly to sys-libs/db).
>>
>>So how does this affect packages that use say dev-perl/DBD-mysql ?
>>
>>If they can only use mysql-5, then the will be broken if they connect to a mysql-4 server using non
>>UTF-8 encoding. (bad wording, but generally speaking, there is an inconsistence in character
>>encoding betweet mysql-4 and mysql-5 that DOES break things on non ISO-8859-1 encoded databases)
>>
> 
> [...]
> 
> At the moment 4.0 is not slotted, only >= 4.1 are.
> 
> The fact that is effectively possible to use a non slotted 4.0 together
>  with 4.1 and 5.0 does not mean that it should be used that way.
> 
> Does this answer your question ?
Well, not exactly :-)
Let me try to make it clearer.

Now this is in my mind, a mind that looks for bugs and troubles before they come out.

If mysql is slotted (and you said it it will be soon), what will happen to software that use mysql
bindings, for example dev-perl/DBD-mysql?

"{{{ A hypothetical example:

I used mysql-4.1.16-r30 and I have a locally encoded (say Shift_JIS) database. I have DBD-mysql
compiled against that and a package FOO using that works as expected.

Some package wants >=mysql-5.0.0 and DBD-mysql, and because mysql is SLOTed, portage says ok,
emerges mysql-5.0.18-r30 and rebuilds DBD-mysql against it.

The probable result is that package FOO will be silently broken because of incompatible changes
between mysql-4.1 and mysql-5.0 ...

"}}}

Actually reading on http://dev.mysql.com/doc/refman/5.0/en/upgrading-from-4-1.html there are quite
many incompatible changes.

Although this may be a problem for DBD-mysql, it will be exposed after mysql is SLOTed...
Or shall we SLOT dev-perl/DBD-mysql as well?
And what about ruby, python (any other language bindings?)?

Kalin.
-- 
|[ ~~~~~~~~~~~~~~~~~~~~~~ ]|
+-> http://ThinRope.net/ <-+
|[ ______________________ ]|
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [dRFC] slotted mysql ready

[Francesco Riosa]

Kalin KOZHUHAROV wrote:
> Francesco Riosa wrote:
>> Kalin KOZHUHAROV wrote:
[...]
>
> Now this is in my mind, a mind that looks for bugs and troubles before
they come out.

That's exactly what I'm looking for ;-)

>
> If mysql is slotted (and you said it it will be soon), what will
happen to software that use mysql
> bindings, for example dev-perl/DBD-mysql?

will happen that the bindings need to be fixed at first,
fex see bug #114925 "dev-perl/DBD-mysql-3.0002_p4 fails to compile"
where Malcolm Lashley give a patch to add backward compatibility.

TODO: threat include files like libraries, also mind about "mysql_config"

>
> "{{{ A hypothetical example:
>
> I used mysql-4.1.16-r30 and I have a locally encoded (say Shift_JIS)
database. I have DBD-mysql
> compiled against that and a package FOO using that works as expected.
>
> Some package wants >=mysql-5.0.0 and DBD-mysql, and because mysql is
SLOTed, portage says ok,
> emerges mysql-5.0.18-r30 and rebuilds DBD-mysql against it.

Oops *you* need to tell portage to rebuild DBD-mysql.
Difficult that a package will want mysql >= something, but take this is
an example, so it's ok.

>
> The probable result is that package FOO will be silently broken
because of incompatible changes
> between mysql-4.1 and mysql-5.0 ...
>
> "}}}

Not so probable, or at least at a lesser degree than what you think.
The major part of incompatibilityes are absorbed from the additional
layer offered by DBD-mysql.
The client app will still work with DBD-mysql, not directly with the server.
Ah and by the way remember that the server running will continue to be
4.1, and that default datadir will be different for 4.1 and 5.0 (it's
insane to make two different versions work with the same data)

Charset side there is additional support in MySQL 5.0 to support UTF-16,
but the big jump has been done between 4.0 and 4.1, now it should be
just more easy.

>
> Actually reading on
http://dev.mysql.com/doc/refman/5.0/en/upgrading-from-4-1.html there are
quite
> many incompatible changes.

Take a look at the changes:

Server Changes:
- The indexing order for end-space in TEXT  columns ...
  Unlikely to affect many apps.
- For BINARY columns, the pad value and how ...
  Unlikely to affect many apps.
- MyISAM and InnoDB  tables created with DECIMAL ...
  Database only change, not visible outside (well not exatly but with a
good approx.)
- Incompatible change: As of MySQL 5.0.3, the server by default no
longer loads user-defined functions...
  Unlikely to affect many apps.
- Incompatible change: The update log was removed in MySQL 5.0 ...
  Don't affect apps, learn to use "mysqlbinlog"
- Support for the ISAM storage engine has been removed ...
  Well it's real time to switch to myisam
- Support for RAID options in MyISAM tables has been removed ...
  Don't affect apps.

SQL Changes:
- Previously, a lock wait timeout caused InnoDB to roll ...
  Look like a bug fixed
- The namespace for triggers has changed in MySQL 5.0.10. ...
  Don't affect 4.1
- As of MySQL 5.0.15, the CHAR() function returns a binary string rather
than ...
  Filtered from DBD-mysql layer
- Beginning with MySQL 5.0.12, natural joins and joins with USING,
including outer join variants, are processed according to the SQL:2003
standard. The changes include elimination of redundant output columns
for NATURAL joins and joins specified with a USING clause and proper
ordering of output columns. The precedence of the comma operator also
now is lower compared to JOIN.
  LIKELY TO AFFECT MANY APPS THAT USE SQL, however the fixes are cheap
as this nonsense
  1 + 2 - 3 ==> ( 1 + 2 ) * 3
  there is an operator that changed precedence.

>
> Although this may be a problem for DBD-mysql, it will be exposed after
mysql is SLOTed...
> Or shall we SLOT dev-perl/DBD-mysql as well?
> And what about ruby, python (any other language bindings?)?

No at all, every one of these must be able to compile on whatever
version of mysql,
examples of how this could be done are at bugs 85783, 85810, 85829,
85843, 85844, 114925.

Better ?
-- 
gentoo-dev@gentoo.org mailing list

Re: [gentoo-dev] [dRFC] slotted mysql ready

[Kalin KOZHUHAROV]

Francesco Riosa wrote:
>> Kalin KOZHUHAROV wrote:
>> 
[...]
>> "{{{ A hypothetical example:
>> 
>> I used mysql-4.1.16-r30 and I have a locally encoded (say Shift_JIS)
>> database. I have DBD-mysql compiled against that and a package FOO using
>> that works as expected.
>> 
>> Some package wants >=mysql-5.0.0 and DBD-mysql, and because mysql is
>> SLOTed, portage says ok, emerges mysql-5.0.18-r30 and rebuilds
>> DBD-mysql against it.
> 
> 
> Oops *you* need to tell portage to rebuild DBD-mysql. Difficult that a
> package will want mysql >= something, but take this is an example, so
> it's ok.

Yup, /me bad. The usual way for a package using a language binding is to
depend on that bindig (dev-perl/DBD-mysql-VVV).

All my worries were for the case when the latest stable dev-perl/DBD-mysql
is compiled against the latest SLOTed mysql-5.1.x and is being used to
access data on an old (SLOTed or not; localhost or some remote host) server.
But I guess then the problem will be entirely in dev-perl/DBD-mysql being
unable to connect to older servers :-)

At this moment I should say "Sorry for the wasted bandwidth/time" :-|

[...]

> Better ?

Times better! Hypothetical problem is solved, thank you for your time!

Kalin.

-- 
|[ ~~~~~~~~~~~~~~~~~~~~~~ ]|
+-> http://ThinRope.net/ <-+
|[ ______________________ ]|
-- 
gentoo-dev@gentoo.org mailing list