[gentoo-user] [OT] - Code translation tools?

[gentoo-user] [OT] - Code translation tools?

[Mark Knecht]

Hello,
   I'm wondering if there are any generic sorts of code translation
tools in portage wherein I could translate from an 'uncommon' language
no one here is likely to use (EasyLanguage) into C?

   As an example I've attached a little EL function that takes
buy/sell command data an puts it away in an array for safe keeping.
What tools are out there, if any, that might allow me to describe how
EL works and then the tool does the conversion?

Thanks,
Mark


Inputs:
Array1[A1,AA1](NumericArrayRef),
EType(StringSimple),
EQty(NumericSimple),
ESPrice(NumericSimple),
ELPrice(NumericSimple),
MyDate(NumericSimple),
MyTime(NumericSimple)
;

Switch (EType)
	Begin
		Case "Buy":			Value1 = 1;
		Case "Sell":		Value1 = 2;
		Case "SellShort":	Value1 = 3;
		Case "BuyToCover":	Value1 = 4;
	End
;

If ((EQty >= 1) and (ESPrice = 0) and (ELPrice = 0)) then //Implies Market order
	Begin
		Array1[Value1,1] = Value1-1;
		Array1[Value1,2] = EQty;
		Array1[Value1,7] = MyDate;
		Array1[Value1,8] = MyTime;		
	End
else if ((EQty >= 1) and (ESPrice > 0) and (ELPrice = 0)) then
//Implies Stop order
	Begin
		Array1[Value1,1] = Value1-1;
		Array1[Value1,3] = EQty;
		Array1[Value1,4] = ESPrice;
		Array1[Value1,7] = MyDate;
		Array1[Value1,8] = MyTime;
	End
else if ((EQty >= 1) and (ESPrice = 0) and (ELPrice > 0)) then
//Implies Limit order
	Begin
		Array1[Value1,1] = Value1-1;
		Array1[Value1,5] = EQty;
		Array1[Value1,6] = ELPrice;
		Array1[Value1,7] = MyDate;
		Array1[Value1,8] = MyTime;
	End	
;

MWK.ADE_SaveSysTraderTrades3_ = 0;

[gentoo-user] Re: [OT] - Code translation tools?

[walt]

On 01/24/2011 02:34 PM, Mark Knecht wrote:
> Hello,
>     I'm wondering if there are any generic sorts of code translation
> tools in portage wherein I could translate from an 'uncommon' language
> no one here is likely to use (EasyLanguage) into C?
>
>     As an example I've attached a little EL function...

Can't give you a real answer, but that code looks very much like Pascal.
Reminds me of a misspent youth.

Is there a definite reason for choosing C instead of something else, e.g.
python?  Just idle curiosity, nothing more.

If I knew a wee bit more about m4 I'm sure I could do it with that alone.
But I'd need to do actual work to learn m4, which is not likely to happen...

Re: [gentoo-user] Re: [OT] - Code translation tools?

[Mark Knecht]

On Mon, Jan 24, 2011 at 4:35 PM, walt <w41ter@gmail.com> wrote:
> On 01/24/2011 02:34 PM, Mark Knecht wrote:
>>
>> Hello,
>>    I'm wondering if there are any generic sorts of code translation
>> tools in portage wherein I could translate from an 'uncommon' language
>> no one here is likely to use (EasyLanguage) into C?
>>
>>    As an example I've attached a little EL function...
>
> Can't give you a real answer, but that code looks very much like Pascal.
> Reminds me of a misspent youth.
>

In 1983 I certainly wasn't a turbo user of Turbo Pascal either... ;-)

> Is there a definite reason for choosing C instead of something else, e.g.
> python?  Just idle curiosity, nothing more.
>

I'm interested in learning a bit about CUDA as an evaluation platform
and the tools from NVidia are based around C. (As best I understand.)

> If I knew a wee bit more about m4 I'm sure I could do it with that alone.
> But I'd need to do actual work to learn m4, which is not likely to happen...
>

Thanks!

- Mark

Re: [gentoo-user] [OT] - Code translation tools?

[Michael Orlitzky]

On 01/24/2011 05:34 PM, Mark Knecht wrote:
> Hello,
>    I'm wondering if there are any generic sorts of code translation
> tools in portage wherein I could translate from an 'uncommon' language
> no one here is likely to use (EasyLanguage) into C?
> 
>    As an example I've attached a little EL function that takes
> buy/sell command data an puts it away in an array for safe keeping.
> What tools are out there, if any, that might allow me to describe how
> EL works and then the tool does the conversion?

Since no one else has given you the bad news, this is basically
impossible if you care that the two programs behave the same.

For any particular program, the best you can do is rewrite it by hand
after creating a battery of unit tests. The alternative is to compile
your source language to a common low-level language, and then decompile
back to your target language.

Unfortunately, unless your common low-level language is some sort of
bytecode with additional metadata (you can translate between .NET
languages for example), the output from the decompiler is going to look
like garbage.

Re: [gentoo-user] [OT] - Code translation tools?

[Mark Knecht]

On Tue, Jan 25, 2011 at 5:15 PM, Michael Orlitzky <michael@orlitzky.com> wrote:
> On 01/24/2011 05:34 PM, Mark Knecht wrote:
>> Hello,
>>    I'm wondering if there are any generic sorts of code translation
>> tools in portage wherein I could translate from an 'uncommon' language
>> no one here is likely to use (EasyLanguage) into C?
>>
>>    As an example I've attached a little EL function that takes
>> buy/sell command data an puts it away in an array for safe keeping.
>> What tools are out there, if any, that might allow me to describe how
>> EL works and then the tool does the conversion?
>
> Since no one else has given you the bad news, this is basically
> impossible if you care that the two programs behave the same.
>
> For any particular program, the best you can do is rewrite it by hand
> after creating a battery of unit tests. The alternative is to compile
> your source language to a common low-level language, and then decompile
> back to your target language.
>
> Unfortunately, unless your common low-level language is some sort of
> bytecode with additional metadata (you can translate between .NET
> languages for example), the output from the decompiler is going to look
> like garbage.

Michael,
   Thanks for the inputs. It gives me more to think about.

   In this case the input language is interpreted, not compiled. The
trading platform interprets the program and internally turns it into
buy & sell operations. (Not the piece of code I supplied - that was
just a small function.) Unfortunately, as the language is proprietary
to the trading platform there isn't a way to go to any common
low-level language.

   The 'battery of tests' would be, I think, the trading program being
executed on a certain market, producing a certainly list of buy & sell
operations and a specific gain or loss. It would be quite easy to
compare the outcome because it's nothing more than a list of trades.
If the translated code generates the same list then it works. If not,
I dig into why. This part of the task seems relatively straight
forward to me.

   I was mainly hoping to find a tool that might generate _reasonable_
C code, even if it's not perfect. If the C code compiles and runs then
I could determine what works, what doesn't, and start fixing things.
I'm not a C programmer and haven't touched that language in at least
15 years so anything that moves me forward would be helpful.

   Again, I do appreciate your inputs. If this extra info gives you
any new ideas please let me know.

Cheers,
Mark

Re: [gentoo-user] [OT] - Code translation tools?

[Michael Orlitzky]

On 01/26/2011 12:56 PM, Mark Knecht wrote:
> Michael,
>    Thanks for the inputs. It gives me more to think about.
> 
>    In this case the input language is interpreted, not compiled. The
> trading platform interprets the program and internally turns it into
> buy & sell operations. (Not the piece of code I supplied - that was
> just a small function.) Unfortunately, as the language is proprietary
> to the trading platform there isn't a way to go to any common
> low-level language.
> 
>    The 'battery of tests' would be, I think, the trading program being
> executed on a certain market, producing a certainly list of buy & sell
> operations and a specific gain or loss. It would be quite easy to
> compare the outcome because it's nothing more than a list of trades.
> If the translated code generates the same list then it works. If not,
> I dig into why. This part of the task seems relatively straight
> forward to me.
> 
>    I was mainly hoping to find a tool that might generate _reasonable_
> C code, even if it's not perfect. If the C code compiles and runs then
> I could determine what works, what doesn't, and start fixing things.
> I'm not a C programmer and haven't touched that language in at least
> 15 years so anything that moves me forward would be helpful.
> 
>    Again, I do appreciate your inputs. If this extra info gives you
> any new ideas please let me know.

If you don't even have a common low-level language, what you're
essentially doing is creating a compiler for EasyLanguage. Take a small
example, adding two integers in E.L. I won't pretend to know the syntax,
but let's just assume that you have two integers (or numbers or
whatever) 'a' and 'b' declared.

How do you translate "a+b" to C code? You can declare two ints 'a' and
'b' in C, of course. But these are 32- or 64-bit integers. So if 'a' and
'b' are large, "a+b" will overflow. Do EasyLanguage integers work that
way? Probably not...

How about "a/b"? Does EasyLanguage do integer division, or does it treat
them like floats? If it does treat them like floats, do the rounding and
precision agree with C floats? Probably not, so floats are out too.

If you try to fix all of these problems, what you'll end up with is
something like a struct EasyLanguageInteger {...} with associated
functions add_easylanguage_integers, divide_easylanguage_integers, etc.
Then, you can translate "a+b" into add_easylanguage_integers(a, b) where
'a' and 'b' are now structs instead of just ints or floats.

Then, you'll have to write a parser that understands the rules of
precedence, looping constructs, functions, and everything else so that
they can be converted into the appropriate structs and function calls.
At the end, if it works, you'll have an EasyLanguage compiler.

Without a spec (the language is proprietary?), you'd have to guess at
most of that stuff anyway, so the chances you'd get it all right are
about zero.

Your best bet[1] is to create a ton of test data, and feed it to the
E.L. program. Make sure the test data triggers any edge cases. Then you
can attempt to rewrite the code in C, and compare the output. You as a
human who understands what the code does can take a lot of shortcuts
that a translator couldn't.


[1] I'm assuming you want to do this for a relatively small number of
programs, and that writing a compiler would not actually be less
time-consuming.

Re: [gentoo-user] [OT] - Code translation tools?

[Mark Knecht]

On Wed, Jan 26, 2011 at 10:47 AM, Michael Orlitzky <michael@orlitzky.com> wrote:
> On 01/26/2011 12:56 PM, Mark Knecht wrote:
>> Michael,
>>    Thanks for the inputs. It gives me more to think about.
>>
>>    In this case the input language is interpreted, not compiled. The
>> trading platform interprets the program and internally turns it into
>> buy & sell operations. (Not the piece of code I supplied - that was
>> just a small function.) Unfortunately, as the language is proprietary
>> to the trading platform there isn't a way to go to any common
>> low-level language.
>>
>>    The 'battery of tests' would be, I think, the trading program being
>> executed on a certain market, producing a certainly list of buy & sell
>> operations and a specific gain or loss. It would be quite easy to
>> compare the outcome because it's nothing more than a list of trades.
>> If the translated code generates the same list then it works. If not,
>> I dig into why. This part of the task seems relatively straight
>> forward to me.
>>
>>    I was mainly hoping to find a tool that might generate _reasonable_
>> C code, even if it's not perfect. If the C code compiles and runs then
>> I could determine what works, what doesn't, and start fixing things.
>> I'm not a C programmer and haven't touched that language in at least
>> 15 years so anything that moves me forward would be helpful.
>>
>>    Again, I do appreciate your inputs. If this extra info gives you
>> any new ideas please let me know.
>
> If you don't even have a common low-level language, what you're
> essentially doing is creating a compiler for EasyLanguage. Take a small
> example, adding two integers in E.L. I won't pretend to know the syntax,
> but let's just assume that you have two integers (or numbers or
> whatever) 'a' and 'b' declared.
>
> How do you translate "a+b" to C code? You can declare two ints 'a' and
> 'b' in C, of course. But these are 32- or 64-bit integers. So if 'a' and
> 'b' are large, "a+b" will overflow. Do EasyLanguage integers work that
> way? Probably not...
>
> How about "a/b"? Does EasyLanguage do integer division, or does it treat
> them like floats? If it does treat them like floats, do the rounding and
> precision agree with C floats? Probably not, so floats are out too.
>
> If you try to fix all of these problems, what you'll end up with is
> something like a struct EasyLanguageInteger {...} with associated
> functions add_easylanguage_integers, divide_easylanguage_integers, etc.
> Then, you can translate "a+b" into add_easylanguage_integers(a, b) where
> 'a' and 'b' are now structs instead of just ints or floats.
>
> Then, you'll have to write a parser that understands the rules of
> precedence, looping constructs, functions, and everything else so that
> they can be converted into the appropriate structs and function calls.
> At the end, if it works, you'll have an EasyLanguage compiler.
>
> Without a spec (the language is proprietary?), you'd have to guess at
> most of that stuff anyway, so the chances you'd get it all right are
> about zero.
>
> Your best bet[1] is to create a ton of test data, and feed it to the
> E.L. program. Make sure the test data triggers any edge cases. Then you
> can attempt to rewrite the code in C, and compare the output. You as a
> human who understands what the code does can take a lot of shortcuts
> that a translator couldn't.
>
>
> [1] I'm assuming you want to do this for a relatively small number of
> programs, and that writing a compiler would not actually be less
> time-consuming.
>
>

OK - this is probably WAY too off topic for this list. If folks
strongly want me to stop the thread I will but I appreciate the
technical adeptness of folks on this list quite a lot and the eventual
outcome would be the use of this thing on Gentoo, so I hope folks
won't mind too much if we continue a little further.


REALLY great points about the math issues. Thanks.

As for testing it _may_ be a slight bit easier than having to get to
that level. There is a library in portage called ta-lib which
implements lots of standard technical analysis constructs. After it's
installed I don't seem to have the C code for the actual functions
anymore. What I have is a compiled library as well as some header
files to look at. I suspect I can install the library again using
portage bt not getting rid of the functions which I could then use as
an example for my coding.

I have not used this library myself, but I've read enough on the web
to be reasonably sure it's results are very consistent with what
TradeStation functions of the same type do.

For a simple function call like a moving average EasyLanguage would write:

Inputs:
Price(Close),
Length(10)
;
variable:
MA1(0)
;

MA1 = Average(Price,Length);

Here is what I see for a similar moving average function from ta-lib:

/*
 * TA_MA - Moving average
 *
 * Input  = double
 * Output = double
 *
 * Optional Parameters
 * -------------------
 * optInTimePeriod:(From 1 to 100000)
 *    Number of period
 *
 * optInMAType:
 *    Type of Moving Average
 *
 *
 */
TA_RetCode TA_MA( int    startIdx,
                  int    endIdx,
                  const double inReal[],
                  int           optInTimePeriod, /* From 1 to 100000 */
                  TA_MAType     optInMAType,
                  int          *outBegIdx,
                  int          *outNBElement,
                  double        outReal[] );

TA_RetCode TA_S_MA( int    startIdx,
                    int    endIdx,
                    const float  inReal[],
                    int           optInTimePeriod, /* From 1 to 100000 */
                    TA_MAType     optInMAType,
                    int          *outBegIdx,
                    int          *outNBElement,
                    double        outReal[] );

int TA_MA_Lookback( int           optInTimePeriod, /* From 1 to 100000 */
                  TA_MAType     optInMAType );


   Maybe I could use the definitions of these functions as a basis for
understanding what might be a reasonable set of guesses? From my point
of view, a good start at translation would be to use whatever is
available in ta-lib whenever possible and only have to deal with other
stuff like If/Else, Switch, etc.

   Just thinking.

Thanks,
Mark

Re: [gentoo-user] [OT] - Code translation tools?

[Michael Orlitzky]

On 01/26/2011 02:40 PM, Mark Knecht wrote:
> 
> REALLY great points about the math issues. Thanks.
> 
> As for testing it _may_ be a slight bit easier than having to get to
> that level. There is a library in portage called ta-lib which
> implements lots of standard technical analysis constructs. After it's
> installed I don't seem to have the C code for the actual functions
> anymore. What I have is a compiled library as well as some header
> files to look at. I suspect I can install the library again using
> portage bt not getting rid of the functions which I could then use as
> an example for my coding.

Use FEATURES="noclean" and the patched source will be left under
/var/tmp/portage. The unpatched source is probably in your
/usr/portage/distfiles already. (I don't know if there *are* any patches
for ta-lib, but if there are, you usually want them applied).


> I have not used this library myself, but I've read enough on the web
> to be reasonably sure it's results are very consistent with what
> TradeStation functions of the same type do.
> 
> For a simple function call like a moving average EasyLanguage would write:
> 
> ...
>
> TA_RetCode TA_MA( int    startIdx,
>                   int    endIdx,
>                   const double inReal[],
>                   int           optInTimePeriod, /* From 1 to 100000 */
>                   TA_MAType     optInMAType,
>                   int          *outBegIdx,
>                   int          *outNBElement,
>                   double        outReal[] );
>
> ...
> 
>    Maybe I could use the definitions of these functions as a basis for
> understanding what might be a reasonable set of guesses? From my point
> of view, a good start at translation would be to use whatever is
> available in ta-lib whenever possible and only have to deal with other
> stuff like If/Else, Switch, etc.
> 
>    Just thinking.

If you can figure out what all those parameters mean -- that will be the
hard part. What type of moving average is EasyLanguage doing? Which
TA_MAType does it match up to? Can the E.L. version fail if it runs off
the end of the prices array, or does it just add zeros at the end? Is
there a way to make TA_MA do the same?

Sorry I have nothing but discouragement to offer =) This isn't an easy
problem.

Re: [gentoo-user] [OT] - Code translation tools?

[Mark Knecht]

On Wed, Jan 26, 2011 at 12:32 PM, Michael Orlitzky <michael@orlitzky.com> wrote:
> On 01/26/2011 02:40 PM, Mark Knecht wrote:
<SNIP>
>> As for testing it _may_ be a slight bit easier than having to get to
>> that level. There is a library in portage called ta-lib which
>> implements lots of standard technical analysis constructs. After it's
>> installed I don't seem to have the C code for the actual functions
>> anymore. What I have is a compiled library as well as some header
>> files to look at. I suspect I can install the library again using
>> portage bt not getting rid of the functions which I could then use as
>> an example for my coding.
>
> Use FEATURES="noclean" and the patched source will be left under
> /var/tmp/portage. The unpatched source is probably in your
> /usr/portage/distfiles already. (I don't know if there *are* any patches
> for ta-lib, but if there are, you usually want them applied).
>

Actually, the tar.gz file in distfiles was easily expanded and I found
the code within for all the functions. I hadn't considered patches but
fortunately there aren't any applied so it seems I get off lucky this
time.

<SNIP>
>
> If you can figure out what all those parameters mean -- that will be the
> hard part. What type of moving average is EasyLanguage doing? Which
> TA_MAType does it match up to? Can the E.L. version fail if it runs off
> the end of the prices array, or does it just add zeros at the end? Is
> there a way to make TA_MA do the same?
>
> Sorry I have nothing but discouragement to offer =) This isn't an easy
> problem.

I don't consider this discouragement at all. In fact it's _very_ helpful.

Documentation:

I've found this much:

Online at the following links:
http://ta-lib.org/d_api/d_api.html
http://ta-lib.org/function.html

I haven't yet found a nice PDF that tells me the exact meaning of
every parameter but I suspect it's out there somewhere. There's a
forum for the library where people ask questions. I just noticed
someone asking about writing documentation to be put in a Wiki so
there's something going on. I'm not overly worried about matching up
ta-lib functionality with what I use today in EL.

Functionality:

Software for real-time stock trading tends to be a little more
restrictive than more general programming

1) First, we are operating on price data that  has date & time
attached to every value. Think of a price chart for any stock. This
stuff (in general) just operates across time in a forward direction.

2) When doing a moving average, for example, we are calculating from
the current bar _backwards_. The only way to run off the end of the
array is to try to reference too far back, so a 20 period moving
average cannot be calculated before bar #20. As I go through a price
data array, I just start at the length parameter into the array N[20]
and keep going until the end.

The way I see this is there's an upper layer that reads the price data
in, puts it in an array and the calls the strategy bar-by-bar.
(row-by-row in the array) When it gets to the last bar of data (last
row with date and time) then it knows it's done. Pretty much all this
stuff seems to work that way.

At this point I feel like I've imposed on gentoo-user far too much. I
was hoping maybe there was some program in portage for doing
translations that might work but I guess there isn't. I'll let this
thread drift asleep unless someone responds back. I think I've got
enough info to sort of play around at this point, and I suspect my
next set of questions are better addressed at someplace like
StackOverflow as I can hardly program beyond Hello World at this
point! ;-)

Again, THANKS for your help and insights!

Anyone interested in this subject matter is always welcome to contact
me off-list.

Cheers,
Mark