From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1QBPiK-0002hi-4p for garchives@archives.gentoo.org; Sun, 17 Apr 2011 11:00:24 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 5358F1C0AE; Sun, 17 Apr 2011 10:58:32 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id E33611C0AD for ; Sun, 17 Apr 2011 10:58:31 +0000 (UTC) Received: from pelican.gentoo.org (unknown [66.219.59.40]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 8F22B1B408B for ; Sun, 17 Apr 2011 10:58:31 +0000 (UTC) Received: from localhost.localdomain (localhost [127.0.0.1]) by pelican.gentoo.org (Postfix) with ESMTP id EC2908006A for ; Sun, 17 Apr 2011 10:58:30 +0000 (UTC) From: "Petteri Räty" To: gentoo-commits@lists.gentoo.org Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Petteri Räty" Message-ID: <4299999e3ac1b881484db6fb9795dd3da66a17b5.betelgeuse@gentoo> Subject: [gentoo-commits] proj/libbash:master commit in: bashast/, bashast/gunit/, bashast/features_script/ X-VCS-Repository: proj/libbash X-VCS-Files: bashast/bashast.g bashast/features_script/features.sh.ast bashast/gunit/brace.gunit bashast/gunit/compound.gunit bashast/gunit/expansions.gunit bashast/gunit/simp_command.gunit X-VCS-Directories: bashast/ bashast/gunit/ bashast/features_script/ X-VCS-Committer: betelgeuse X-VCS-Committer-Name: Petteri Räty X-VCS-Revision: 4299999e3ac1b881484db6fb9795dd3da66a17b5 Date: Sun, 17 Apr 2011 10:58:30 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: quoted-printable X-Archives-Salt: X-Archives-Hash: e6d468f30a7b51fe36f380435e0df437 commit: 4299999e3ac1b881484db6fb9795dd3da66a17b5 Author: Petteri R=C3=A4ty petteriraty eu> AuthorDate: Sat Apr 16 11:56:33 2011 +0000 Commit: Petteri R=C3=A4ty gentoo org> CommitDate: Sun Apr 17 10:44:40 2011 +0000 URL: http://git.overlays.gentoo.org/gitweb/?p=3Dproj/libbash.git;a= =3Dcommit;h=3D4299999e Parser: fix multiple variable expansions in an arg The parser now properly supports things like dodir ${foo}/${bar}. This was fixed by not special casing variable references and command substitutions. The AST is a little noisier now but if that's a problem then it can be simplified later. --- bashast/bashast.g | 9 +-------- bashast/features_script/features.sh.ast | 2 +- bashast/gunit/brace.gunit | 2 +- bashast/gunit/compound.gunit | 4 ++-- bashast/gunit/expansions.gunit | 2 +- bashast/gunit/simp_command.gunit | 1 + 6 files changed, 7 insertions(+), 13 deletions(-) diff --git a/bashast/bashast.g b/bashast/bashast.g index f67013e..9619d64 100644 --- a/bashast/bashast.g +++ b/bashast/bashast.g @@ -124,14 +124,7 @@ simple_command variable_definitions : var_def (BLANK!+ var_def)*; bash_command - : fname_no_res_word (BLANK+ arg)* -> ^(COMMAND fname_no_res_word arg*); -//An argument to a command -//fname can also match var_ref and command_sub but that gives a noisier = AST -arg -options{backtrack=3Dfalse;} - : (var_ref) =3D> var_ref - | (command_sub) =3D> command_sub - | fname; + : fname_no_res_word (BLANK+ fname)* -> ^(COMMAND fname_no_res_word fnam= e*); redirect: BLANK!* here_string_op^ BLANK!* fname | BLANK!* here_doc_op^ BLANK!* fname EOL! heredoc | BLANK* redir_op BLANK* redir_dest -> ^(REDIR redir_op redir_dest) diff --git a/bashast/features_script/features.sh.ast b/bashast/features_s= cript/features.sh.ast index 23ac261..62a969c 100644 --- a/bashast/features_script/features.sh.ast +++ b/bashast/features_script/features.sh.ast @@ -1 +1 @@ -(LIST (function lots_o_echo (CURRSHELL (LIST (COMMAND (STRING echo) (STR= ING (DOUBLE_QUOTED_STRING The number of tests that have faile= d : (VAR_REF failedtests)))) (COMMAND (STRING echo) (STRING (SINGLE_QUO= TED_STRING $ failedtests))) (COMMAND (STRING echo) (VAR_REF failedtests))= ))) (function do_some_arith (CURRSHELL (LIST (COMPOUND_ARITH (* 5 4)) (CO= MPOUND_ARITH (** 5 4)) (COMPOUND_ARITH (+ (VAR_REF (VAR_REF failedtests))= (/ 5 4))) (COMPOUND_ARITH (+ (VAR_REF (VAR_REF z)) (MINUS_SIGN 3)))))) (= function arrays (SUBSHELL (LIST (=3D asdf (ARRAY (STRING a) (STRING b) (S= TRING c) (STRING d))) (COMMAND (STRING echo) (VAR_REF (asdf 3))) (=3D foo= (ARRAY (STRING (COMMAND_SUB (COMMAND (STRING echo) (STRING 6)))) (STRING= b) (STRING c) (STRING d))) (=3D (arr (VAR_REF foo)) (STRING 3)) (=3D bar= (ARRAY (STRING a) (STRING b) (=3D 5 (STRING c))))))) (COMMAND (STRING ec= ho) (STRING (BRACE_EXP (STRING a) (STRING b)))) (COMMAND (STRING echo) (S= TRING (BRACE_EXP (.. a d)))) (COMMAND (STRING echo) (STRING (BRACE_EXP (BRACE_EXP (STRING a) (STRING b)) (STRING c) (STRING= d)))) (COMMAND (STRING echo) (STRING a (BRACE_EXP (STRING b) (STRING c))= )) (COMMAND (STRING (COMMAND_SUB (COMMAND (STRING echo) (STRING foobar)))= )) (| (COMMAND (STRING ls)) (COMMAND (STRING grep) (STRING gunit) (REDIR = >> (STRING filelist)))) (case (COMMAND_SUB (COMMAND (STRING echo) (STRING= asdf))) (CASE_PATTERN (STRING gz) (LIST (COMMAND (STRING echo) (STRING y= ay)))) (CASE_PATTERN (STRING bzip) (LIST (COMMAND (STRING echo) (STRING t= hree)))) (CASE_PATTERN (STRING *) (LIST (COMMAND (STRING echo) (STRING wo= o))))) (for each (STRING (COMMAND_SUB (| (COMMAND (STRING ls)) (COMMAND (= STRING grep) (STRING log))))) (LIST (COMMAND (STRING echo) (VAR_REF each)= ) (COMMAND (STRING cat) (STRING each)))) (for (FOR_INIT (+ 5 3)) (FOR_CON= D (+ 6 2)) (FOR_MOD (+ 3 1)) (LIST (COMMAND (STRING echo) (STRING yay))))= (select each (COMMAND_SUB (| (COMMAND (STRING ls)) (COMMAND (STRING grep= ) (STRING output)))) (LIST (COMMAND (STRI NG echo) (STRING asdf) (STRING 2) (REDIR > (STRING / dev / null))))) (if= (LIST (COMMAND (STRING echo) (STRING yay2))) (LIST (COMMAND (STRING echo= ) (STRING yay)))) (until (LIST (COMPOUND_COND (KEYWORD_TEST (a (STRING th= is / is . afile))))) (LIST (COMMAND (STRING touch) (STRING this / is . af= ile)))) (while (LIST (COMPOUND_COND (BUILTIN_TEST (n (STRING foobar))))) = (LIST (COMMAND (STRING echo) (STRING (DOUBLE_QUOTED_STRING file found))= ))) (if (LIST (COMPOUND_COND (BUILTIN_TEST (eq 5 6)))) (LIST (COMMAND (ST= RING echo) (STRING (DOUBLE_QUOTED_STRING something ' s wrong))))) (COMM= AND (STRING echo) (STRING this) (STRING command) (STRING has) (STRING mul= tiple) (STRING arguments)) (COMMAND (STRING wc) (PROCESS_SUBSTITUTION < (= LIST (COMMAND (STRING cat) (STRING / usr / share / dict / linux . words))= ))) (|| (&& (&& (&& (COMMAND (STRING cd) (STRING build)) (COMMAND (STRING= . / configure))) (COMMAND (STRING make))) (COMMAND (STRING make_install)= )) (COMMAND (STRING echo) (STRING fail))) (COMMAND (STRING cd) (STRING / usr / bin)) (| (COMMAND (STRING ls) (STR= ING - al)) (COMMAND (STRING grep) (STRING more))) (=3D asdf (STRING param= eters)) (COMMAND (STRING (VAR_REF (USE_DEFAULT_WHEN_UNSET_OR_NULL asdf (S= TRING foo))))) (COMMAND (STRING (VAR_REF (OFFSET asdf 8)))) (COMMAND (STR= ING (VAR_REF (! asdf *)))) (COMMAND (STRING (VAR_REF (! asdf @)))) (COMMA= ND (STRING (VAR_REF (# foo)))) (COMMAND (STRING (VAR_REF (REPLACE_FIRST r= eplaice (STRING with) (STRING pattern))))) (COMMAND (STRING (VAR_REF (# a= sdf (STRING bar))))) (COMMAND (STRING (VAR_REF (## asdf (STRING bar))))) = (COMMAND (STRING (VAR_REF (% asdf (STRING bar))))) (COMMAND (STRING (VAR_= REF (% asdf (STRING bar))))) (COMMAND (STRING (VAR_REF 1)) (VAR_REF @) (V= AR_REF *)) (COMMAND (STRING (VAR_REF ?))) (COMMAND (STRING (VAR_REF (REPL= ACE_ALL PV (STRING .) (STRING _))))) (COMMAND (STRING (VAR_REF (REPLACE_A= T_START PV (STRING foo) (STRING bar))))) (COMMAND (STRING (VAR_REF (REPLA= CE_AT_END PV (STRING foo) (STRING bar)))))=20 (=3D MY_PN (STRING (VAR_REF (REPLACE_FIRST PN (STRING asterisk -))))) (|= (COMMAND (STRING cat) (STRING asdf)) (COMMAND (STRING grep) (STRING thre= e) (STRING 2) (REDIR >& 1) (REDIR > (STRING / dev / null)))) (COMMAND (ST= RING echo) (STRING asdf) (REDIR >> (STRING APPEND))) (COMMAND (STRING ech= o) (STRING cat) (<<< (STRING word)))) +(LIST (function lots_o_echo (CURRSHELL (LIST (COMMAND (STRING echo) (STR= ING (DOUBLE_QUOTED_STRING The number of tests that have faile= d : (VAR_REF failedtests)))) (COMMAND (STRING echo) (STRING (SINGLE_QUO= TED_STRING $ failedtests))) (COMMAND (STRING echo) (STRING (VAR_REF faile= dtests)))))) (function do_some_arith (CURRSHELL (LIST (COMPOUND_ARITH (* = 5 4)) (COMPOUND_ARITH (** 5 4)) (COMPOUND_ARITH (+ (VAR_REF (VAR_REF fail= edtests)) (/ 5 4))) (COMPOUND_ARITH (+ (VAR_REF (VAR_REF z)) (MINUS_SIGN = 3)))))) (function arrays (SUBSHELL (LIST (=3D asdf (ARRAY (STRING a) (STR= ING b) (STRING c) (STRING d))) (COMMAND (STRING echo) (STRING (VAR_REF (a= sdf 3)))) (=3D foo (ARRAY (STRING (COMMAND_SUB (COMMAND (STRING echo) (ST= RING 6)))) (STRING b) (STRING c) (STRING d))) (=3D (arr (VAR_REF foo)) (S= TRING 3)) (=3D bar (ARRAY (STRING a) (STRING b) (=3D 5 (STRING c))))))) (= COMMAND (STRING echo) (STRING (BRACE_EXP (STRING a) (STRING b)))) (COMMAN= D (STRING echo) (STRING (BRACE_EXP (.. a d)))) (COM MAND (STRING echo) (STRING (BRACE_EXP (BRACE_EXP (STRING a) (STRING b)) = (STRING c) (STRING d)))) (COMMAND (STRING echo) (STRING a (BRACE_EXP (STR= ING b) (STRING c)))) (COMMAND (STRING (COMMAND_SUB (COMMAND (STRING echo)= (STRING foobar))))) (| (COMMAND (STRING ls)) (COMMAND (STRING grep) (STR= ING gunit) (REDIR >> (STRING filelist)))) (case (COMMAND_SUB (COMMAND (ST= RING echo) (STRING asdf))) (CASE_PATTERN (STRING gz) (LIST (COMMAND (STRI= NG echo) (STRING yay)))) (CASE_PATTERN (STRING bzip) (LIST (COMMAND (STRI= NG echo) (STRING three)))) (CASE_PATTERN (STRING *) (LIST (COMMAND (STRIN= G echo) (STRING woo))))) (for each (STRING (COMMAND_SUB (| (COMMAND (STRI= NG ls)) (COMMAND (STRING grep) (STRING log))))) (LIST (COMMAND (STRING ec= ho) (STRING (VAR_REF each))) (COMMAND (STRING cat) (STRING each)))) (for = (FOR_INIT (+ 5 3)) (FOR_COND (+ 6 2)) (FOR_MOD (+ 3 1)) (LIST (COMMAND (S= TRING echo) (STRING yay)))) (select each (COMMAND_SUB (| (COMMAND (STRING= ls)) (COMMAND (STRING grep) (STRING outp ut)))) (LIST (COMMAND (STRING echo) (STRING asdf) (STRING 2) (REDIR > (S= TRING / dev / null))))) (if (LIST (COMMAND (STRING echo) (STRING yay2))) = (LIST (COMMAND (STRING echo) (STRING yay)))) (until (LIST (COMPOUND_COND = (KEYWORD_TEST (a (STRING this / is . afile))))) (LIST (COMMAND (STRING to= uch) (STRING this / is . afile)))) (while (LIST (COMPOUND_COND (BUILTIN_T= EST (n (STRING foobar))))) (LIST (COMMAND (STRING echo) (STRING (DOUBLE_Q= UOTED_STRING file found))))) (if (LIST (COMPOUND_COND (BUILTIN_TEST (eq= 5 6)))) (LIST (COMMAND (STRING echo) (STRING (DOUBLE_QUOTED_STRING somet= hing ' s wrong))))) (COMMAND (STRING echo) (STRING this) (STRING comman= d) (STRING has) (STRING multiple) (STRING arguments)) (COMMAND (STRING wc= ) (PROCESS_SUBSTITUTION < (LIST (COMMAND (STRING cat) (STRING / usr / sha= re / dict / linux . words))))) (|| (&& (&& (&& (COMMAND (STRING cd) (STRI= NG build)) (COMMAND (STRING . / configure))) (COMMAND (STRING make))) (CO= MMAND (STRING make_install))) (COMMAND (S TRING echo) (STRING fail))) (COMMAND (STRING cd) (STRING / usr / bin)) (= | (COMMAND (STRING ls) (STRING - al)) (COMMAND (STRING grep) (STRING more= ))) (=3D asdf (STRING parameters)) (COMMAND (STRING (VAR_REF (USE_DEFAULT= _WHEN_UNSET_OR_NULL asdf (STRING foo))))) (COMMAND (STRING (VAR_REF (OFFS= ET asdf 8)))) (COMMAND (STRING (VAR_REF (! asdf *)))) (COMMAND (STRING (V= AR_REF (! asdf @)))) (COMMAND (STRING (VAR_REF (# foo)))) (COMMAND (STRIN= G (VAR_REF (REPLACE_FIRST replaice (STRING with) (STRING pattern))))) (CO= MMAND (STRING (VAR_REF (# asdf (STRING bar))))) (COMMAND (STRING (VAR_REF= (## asdf (STRING bar))))) (COMMAND (STRING (VAR_REF (% asdf (STRING bar)= )))) (COMMAND (STRING (VAR_REF (% asdf (STRING bar))))) (COMMAND (STRING = (VAR_REF 1)) (STRING (VAR_REF @)) (STRING (VAR_REF *))) (COMMAND (STRING = (VAR_REF ?))) (COMMAND (STRING (VAR_REF (REPLACE_ALL PV (STRING .) (STRIN= G _))))) (COMMAND (STRING (VAR_REF (REPLACE_AT_START PV (STRING foo) (STR= ING bar))))) (COMMAND (STRING (VAR_REF (REP LACE_AT_END PV (STRING foo) (STRING bar))))) (=3D MY_PN (STRING (VAR_REF= (REPLACE_FIRST PN (STRING asterisk -))))) (| (COMMAND (STRING cat) (STRI= NG asdf)) (COMMAND (STRING grep) (STRING three) (STRING 2) (REDIR >& 1) (= REDIR > (STRING / dev / null)))) (COMMAND (STRING echo) (STRING asdf) (RE= DIR >> (STRING APPEND))) (COMMAND (STRING echo) (STRING cat) (<<< (STRING= word)))) diff --git a/bashast/gunit/brace.gunit b/bashast/gunit/brace.gunit index 937e594..2030d58 100644 --- a/bashast/gunit/brace.gunit +++ b/bashast/gunit/brace.gunit @@ -24,7 +24,7 @@ brace_expansion: "{a..d}" -> (BRACE_EXP (.. a d)) "{{a,b},c,d}" -> (BRACE_EXP (BRACE_EXP (STRING a) (STRING b)) (STRING c)= (STRING d)) =20 -arg: +fname: "a{b,c}" -> (STRING a (BRACE_EXP (STRING b) (STRING c))) "{c..d}f" -> (STRING (BRACE_EXP (.. c d)) f) "a{a,b}b{c,d}" -> (STRING a (BRACE_EXP (STRING a) (STRING b)) b (BRACE_E= XP (STRING c) (STRING d))) diff --git a/bashast/gunit/compound.gunit b/bashast/gunit/compound.gunit index 9bf658f..587d869 100644 --- a/bashast/gunit/compound.gunit +++ b/bashast/gunit/compound.gunit @@ -80,8 +80,8 @@ for_expr: echo \"file found\" done" -> (for each (STRING (COMMAND_SUB (| (COMMAND (STRING ls)) (COMMAN= D (STRING grep) (STRING log))))) (LIST (COMMAND (STRING echo) (STRING (DO= UBLE_QUOTED_STRING file found))))) "for each in `ls |grep log`; do echo \"file found\"; done" -> (for each = (STRING (COMMAND_SUB (| (COMMAND (STRING ls)) (COMMAND (STRING grep) (STR= ING log))))) (LIST (COMMAND (STRING echo) (STRING (DOUBLE_QUOTED_STRING f= ile found))))) -"for i in 'foo' 'bar'; do echo $i; done" -> (for i (STRING (SINGLE_QUOTE= D_STRING foo)) (STRING (SINGLE_QUOTED_STRING bar)) (LIST (COMMAND (STRING= echo) (VAR_REF i)))) -"for i in foo$var bar; do echo $i; done" -> (for i (STRING foo (VAR_REF = var)) (STRING bar) (LIST (COMMAND (STRING echo) (VAR_REF i)))) +"for i in 'foo' 'bar'; do echo $i; done" -> (for i (STRING (SINGLE_QUOTE= D_STRING foo)) (STRING (SINGLE_QUOTED_STRING bar)) (LIST (COMMAND (STRING= echo) (STRING (VAR_REF i))))) +"for i in foo$var bar; do echo $i; done" -> (for i (STRING foo (VAR_REF = var)) (STRING bar) (LIST (COMMAND (STRING echo) (STRING (VAR_REF i))))) "for each in `ls |grep log`; do echo file done" FAIL =20 "for ((5+3;5+3;5+3)); do echo yay; done" -> (for (FOR_INIT (+ 5 3)) (FOR= _COND (+ 5 3)) (FOR_MOD (+ 5 3)) (LIST (COMMAND (STRING echo) (STRING yay= )))) diff --git a/bashast/gunit/expansions.gunit b/bashast/gunit/expansions.gu= nit index 1ddd97b..4ea5924 100644 --- a/bashast/gunit/expansions.gunit +++ b/bashast/gunit/expansions.gunit @@ -25,5 +25,5 @@ list: "[[ while=3Dwhile ]] && echo true" -> (LIST (&& (COMPOUND_COND (KEYWORD_= TEST (=3D (STRING while) (STRING while)))) (COMMAND (STRING echo) (STRING= true)))) "for each in `ls |grep output`; do echo $each -done" -> (LIST (for each (STRING (COMMAND_SUB (| (COMMAND (STRING ls)) (= COMMAND (STRING grep) (STRING output))))) (LIST (COMMAND (STRING echo) (V= AR_REF each))))) +done" -> (LIST (for each (STRING (COMMAND_SUB (| (COMMAND (STRING ls)) (= COMMAND (STRING grep) (STRING output))))) (LIST (COMMAND (STRING echo) (S= TRING (VAR_REF each)))))) "wc <(cat /usr/share/dict/linux.words)" -> (LIST (COMMAND (STRING wc) (P= ROCESS_SUBSTITUTION < (LIST (COMMAND (STRING cat) (STRING / usr / share /= dict / linux . words)))))) diff --git a/bashast/gunit/simp_command.gunit b/bashast/gunit/simp_comman= d.gunit index e877cab..b2dc74a 100644 --- a/bashast/gunit/simp_command.gunit +++ b/bashast/gunit/simp_command.gunit @@ -28,3 +28,4 @@ simple_command: "[[while" -> (COMMAND (STRING [ [ while)) "./foob=C3=A4r" -> (COMMAND (STRING . / foob =C3=A4 r)) "cat ~/Documents/todo.txt" -> (COMMAND (STRING cat) (STRING ~ / Document= s / todo . txt)) +"dodir ${foo}/${bar}" -> (COMMAND (STRING dodir) (STRING (VAR_REF foo) /= (VAR_REF bar)))