From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from pigeon.gentoo.org ([208.92.234.80] helo=lists.gentoo.org) by finch.gentoo.org with esmtp (Exim 4.60) (envelope-from ) id 1SXxEX-00059o-A9 for garchives@archives.gentoo.org; Fri, 25 May 2012 16:19:21 +0000 Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 4BA32E079D; Fri, 25 May 2012 16:18:54 +0000 (UTC) Received: from smtp.gentoo.org (smtp.gentoo.org [140.211.166.183]) by pigeon.gentoo.org (Postfix) with ESMTP id 0CAEAE079D for ; Fri, 25 May 2012 16:18:53 +0000 (UTC) Received: from hornbill.gentoo.org (hornbill.gentoo.org [94.100.119.163]) (using TLSv1 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 572FC1B4005 for ; Fri, 25 May 2012 16:18:53 +0000 (UTC) Received: from localhost.localdomain (localhost [127.0.0.1]) by hornbill.gentoo.org (Postfix) with ESMTP id 199B8E542A for ; Fri, 25 May 2012 16:18:52 +0000 (UTC) From: "Mike Frysinger" To: gentoo-commits@lists.gentoo.org Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Mike Frysinger" Message-ID: <1337962812.a1578c654f26cab07309bc9cbddd3c95c0c205b5.vapier@gentoo> Subject: [gentoo-commits] proj/portage:master commit in: pym/repoman/ X-VCS-Repository: proj/portage X-VCS-Files: pym/repoman/checks.py X-VCS-Directories: pym/repoman/ X-VCS-Committer: vapier X-VCS-Committer-Name: Mike Frysinger X-VCS-Revision: a1578c654f26cab07309bc9cbddd3c95c0c205b5 X-VCS-Branch: master Date: Fri, 25 May 2012 16:18:52 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: quoted-printable X-Archives-Salt: e328bf6b-dd33-44e1-adf0-2b93a7c8f7f5 X-Archives-Hash: 50527b3487eb11d180bec2e2b059bf49 commit: a1578c654f26cab07309bc9cbddd3c95c0c205b5 Author: Mike Frysinger gentoo org> AuthorDate: Thu May 24 04:05:30 2012 +0000 Commit: Mike Frysinger gentoo org> CommitDate: Fri May 25 16:20:12 2012 +0000 URL: http://git.overlays.gentoo.org/gitweb/?p=3Dproj/portage.git;a= =3Dcommit;h=3Da1578c65 repoman: unroll escaped lines so we can check the entirety of it Sometimes people wrap long lines in their ebuilds to make it easier to read, but this causes us issues when doing line-by-line checking. So automatically unroll those lines before passing the full content down to our checkers. Signed-off-by: Mike Frysinger gentoo.org> --- pym/repoman/checks.py | 65 ++++++++++++++++++++++++++++++++++++++++---= ------ 1 files changed, 53 insertions(+), 12 deletions(-) diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py index 77df603..a413968 100644 --- a/pym/repoman/checks.py +++ b/pym/repoman/checks.py @@ -5,6 +5,7 @@ """This module contains functions used in Repoman to ascertain the quali= ty and correctness of an ebuild.""" =20 +import codecs import re import time import repoman.errors as errors @@ -692,8 +693,11 @@ _here_doc_re =3D re.compile(r'.*\s<<[-]?(\w+)$') _ignore_comment_re =3D re.compile(r'^\s*#') =20 def run_checks(contents, pkg): + unicode_escape_codec =3D codecs.lookup('unicode_escape') + unicode_escape =3D lambda x: unicode_escape_codec.decode(x)[0] checks =3D _constant_checks here_doc_delim =3D None + multiline =3D None =20 for lc in checks: lc.new(pkg) @@ -707,19 +711,56 @@ def run_checks(contents, pkg): here_doc =3D _here_doc_re.match(line) if here_doc is not None: here_doc_delim =3D re.compile(r'^\s*%s$' % here_doc.group(1)) + if here_doc_delim is not None: + continue + + # Unroll multiline escaped strings so that we can check things: + # inherit foo bar \ + # moo \ + # cow + # This will merge these lines like so: + # inherit foo bar moo cow + try: + # A normal line will end in the two bytes: <\> <\n>. So decoding + # that will result in python thinking the <\n> is being escaped + # and eat the single <\> which makes it hard for us to detect. + # Instead, strip the newline (which we know all lines have), and + # append a <0>. Then when python escapes it, if the line ended + # in a <\>, we'll end up with a <\0> marker to key off of. This + # shouldn't be a problem with any valid ebuild ... + line_escaped =3D unicode_escape(line.rstrip('\n') + '0') + except SystemExit: + raise + except: + # Who knows what kind of crazy crap an ebuild will have + # in it -- don't allow it to kill us. + line_escaped =3D line + if multiline: + # Chop off the \ and \n bytes from the previous line. + multiline =3D multiline[:-2] + line + if not line_escaped.endswith('\0'): + line =3D multiline + num =3D multinum + multiline =3D None + else: + continue + else: + if line_escaped.endswith('\0'): + multinum =3D num + multiline =3D line + continue =20 - if here_doc_delim is None: - # We're not in a here-document. - is_comment =3D _ignore_comment_re.match(line) is not None - for lc in checks: - if is_comment and lc.ignore_comment: - continue - if lc.check_eapi(pkg.metadata['EAPI']): - ignore =3D lc.ignore_line - if not ignore or not ignore.match(line): - e =3D lc.check(num, line) - if e: - yield lc.repoman_check_name, e % (num + 1) + # Finally we have a full line to parse. + is_comment =3D _ignore_comment_re.match(line) is not None + for lc in checks: + if is_comment and lc.ignore_comment: + continue + if lc.check_eapi(pkg.metadata['EAPI']): + ignore =3D lc.ignore_line + if not ignore or not ignore.match(line): + e =3D lc.check(num, line) + if e: + yield lc.repoman_check_name, e % (num + 1) =20 for lc in checks: i =3D lc.end()