public inbox for gentoo-commits@lists.gentoo.org
 help / color / mirror / Atom feed
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-08-02 15:14 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-08-02 15:14 UTC (permalink / raw
  To: gentoo-commits

commit:     1ef8dc98806610e180bab0cf7d69e697d9285d69
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Thu Aug  2 14:58:36 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Thu Aug  2 14:58:36 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=1ef8dc98

EbuildJobChannel: remove race condition comment

resolver->start <-> wait_for_resolver should no longer
be a race condition due to changes in the resolver code

---
 roverlay/depres/channels.py |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/roverlay/depres/channels.py b/roverlay/depres/channels.py
index f15a6d0..626d7a0 100644
--- a/roverlay/depres/channels.py
+++ b/roverlay/depres/channels.py
@@ -199,8 +199,6 @@ class EbuildJobChannel ( _EbuildJobChannelBase ):
 			# tell the resolver to start
 			self._depres_master.start()
 
-			# ^, race condition, running resolver vs waiting queue (FIXME)
-
 			# wait for one result at least
 			satisfiable = handle_queue_item ( self._depres_queue.get() )
 


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2014-08-23 19:03 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2014-08-23 19:03 UTC (permalink / raw
  To: gentoo-commits

commit:     e27f0b7bfc4555d1ac9720e2545d01c584d318d3
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Sat Aug 23 18:57:52 2014 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Sat Aug 23 18:57:52 2014 +0000
URL:        http://sources.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=e27f0b7b

roverlay/depres/depenv: skip vmod dict lookup

When checking/normalizing vmod_str, set vmod directly if possible.

---
 roverlay/depres/depenv.py | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index 7d1bf5e..c0ea6e1 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -218,14 +218,19 @@ class DepEnv ( object ):
             # fix versions like ".9" (-> "0.9")
             if version [0] == '.': version = '0' + version
 
-            vmod = m.group ( 'vmod' )
+            vmod_str = m.group ( 'vmod' )
 
-            if not vmod:
+            if not vmod_str:
                # version required, but no modifier: set vmod to '>='
-               vmod = '>='
-            elif vmod == '==':
+               vmod     = self.VMOD_GE
+               vmod_str = '>='
+               #vmod_weak = True # for slot-resolve // TODO-MAYBE
+            elif vmod_str == '==':
                # "normalize"
-               vmod = '='
+               vmod     = self.VMOD_EQ
+               vmod_str = '='
+            else:
+               vmod = self.VMOD.get ( vmod_str, self.VMOD_UNDEF )
 
             version_strlist = version.split ( '.' )
             version_iparts  = list()
@@ -243,13 +248,13 @@ class DepEnv ( object ):
             result.append ( dict (
                name             = m.group ( 'name' ),
                name_low         = m.group ( 'name' ).lower(),
-               version_modifier = vmod,
+               version_modifier = vmod_str,
                version          = version,
                version_strlist  = version_strlist,
                version_tuple    = roverlay.versiontuple.IntVersionTuple (
                   version_iparts
                ),
-               vmod             = self.VMOD.get ( vmod, self.VMOD_UNDEF ),
+               vmod             = vmod,
             ) )
 
             if not self.try_all_regexes: break


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2014-06-05 22:09 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2014-06-05 22:09 UTC (permalink / raw
  To: gentoo-commits

commit:     ffbde283146338c406e67089b3bc813bfb21d3ab
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed May  7 01:35:21 2014 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed May  7 01:35:21 2014 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=ffbde283

roverlay/depres/deptype: cleanup

---
 roverlay/depres/deptype.py | 34 ++++++++++++++++++++++------------
 1 file changed, 22 insertions(+), 12 deletions(-)

diff --git a/roverlay/depres/deptype.py b/roverlay/depres/deptype.py
index 91a3f3f..66f03b7 100644
--- a/roverlay/depres/deptype.py
+++ b/roverlay/depres/deptype.py
@@ -18,22 +18,32 @@ try_other = 1
 mandatory = 2
 external  = 4
 internal  = 8
-#internal does not imply selfdep
-# internal := dependency on package
-# selfdep  := created overlay has dependency
 selfdep   = 16
 
-_MAX = 31
+#internal does not imply selfdep
+#  external,internal control whether a rule can resolve dependency strings
+#  with a deptype of internal/external
+#  (or, whether a dep str expects to be resolved as R or system package)
+#
+#  internal := dependency on a (R) package "internal" to
+#   the R package ecosystem - not necessarily hosted by the generated overlay
+#  external := dep on a system package
+#
+#  selfdep  := dependency (the ebuild) is hosted by the created overlay,
+#              which allows selfdep validation etc.
+#
+# => any combination of {external,internal,selfdep} is legal
+
 
-#VIRTUAL = try_other | mandatory | selfdep
+_MAX = 31
 
-NONE = 0
-ALL  = external | internal | mandatory
-RESOLVE_ALL = external | internal
+NONE          = 0
+RESOLVE_ALL   = external  | internal
+ALL           = mandatory | RESOLVE_ALL
+MANDATORY_TRY = mandatory | try_other
+VIRTUAL       = selfdep   | MANDATORY_TRY
 
 # "system first"
-SYS = mandatory | ( external | try_other )
+SYS = external | MANDATORY_TRY
 # "package first"
-PKG = mandatory | ( internal | try_other )
-
-MANDATORY_TRY = try_other | mandatory
+PKG = internal | MANDATORY_TRY


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-09-03  8:35 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-09-03  8:35 UTC (permalink / raw
  To: gentoo-commits

commit:     f1b422139b303c41fe479a0a345ff867207c6089
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Tue Sep  3 08:34:04 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Tue Sep  3 08:34:04 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=f1b42213

DynamicDependencyRulePool: change reload() behavior

---
 roverlay/depres/deprule.py | 15 ++++++++++-----
 1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/roverlay/depres/deprule.py b/roverlay/depres/deprule.py
index 66a43f2..305995c 100644
--- a/roverlay/depres/deprule.py
+++ b/roverlay/depres/deprule.py
@@ -262,13 +262,18 @@ class DependencyRulePool ( DependencyRulePoolBase ):
 
 class DynamicDependencyRulePool ( DependencyRulePoolBase ):
 
-   @roverlay.util.objects.abstractmethod
-   def reload ( self ):
-      pass
-   # --- end of reload (...) ---
-
    def accepts_other ( self, dep_env ):
       return False
    # --- end of accepts_other (...) ---
 
+   @roverlay.util.objects.abstractmethod
+   def reload_rules ( self ):
+      pass
+   # --- end of reload_rules (...) ---
+
+   def reload ( self ):
+      self.reload_rules()
+      self.sort()
+   # --- end of reload (...) ---
+
 # --- end of DynamicDependencyRulePool ---


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-09-02 13:20 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-09-02 13:20 UTC (permalink / raw
  To: gentoo-commits

commit:     d55a09ed11aa8401263ec359013b8902382cfb16
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Sep  2 13:11:55 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Sep  2 13:11:55 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=d55a09ed

roverlay/depres/listeners: remove unused import

---
 roverlay/depres/listeners.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/roverlay/depres/listeners.py b/roverlay/depres/listeners.py
index 7058bfe..e7f0e39 100644
--- a/roverlay/depres/listeners.py
+++ b/roverlay/depres/listeners.py
@@ -15,7 +15,6 @@ import threading
 import os
 
 from roverlay.depres               import events
-from roverlay.depres.depenv        import DepEnv
 from roverlay.depres.communication import DependencyResolverListener
 
 


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-08-28  9:38 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-08-28  9:38 UTC (permalink / raw
  To: gentoo-commits

commit:     13c5623a12b5f85e68272a719e981462650111c1
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed Aug 28 09:35:51 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed Aug 28 09:35:51 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=13c5623a

roverlay/depres/channels: add_dependencies_filtered()

Like add_dependencies(), but accepts two additional args, a common and a
specific dep string blacklist. A dep str won't be added if it appears in any of
these blacklists.

---
 roverlay/depres/channels.py | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/roverlay/depres/channels.py b/roverlay/depres/channels.py
index d3ba725..ed51df4 100644
--- a/roverlay/depres/channels.py
+++ b/roverlay/depres/channels.py
@@ -136,6 +136,33 @@ class _EbuildJobChannelBase ( DependencyResolverChannel ):
          self.add_dependency ( dep_str=dep_str, deptype_mask=deptype_mask )
    # --- end of add_dependencies (...) ---
 
+   def add_dependencies_filtered (
+      self, dep_list, deptype_mask, common_blacklist, specific_blacklist
+   ):
+      if common_blacklist:
+         if specific_blacklist:
+            for dep_str in dep_list:
+               if (
+                  dep_str not in common_blacklist and
+                  dep_str not in specific_blacklist
+               ):
+                  self.add_dependency ( dep_str, deptype_mask )
+
+         else:
+            for dep_str in dep_list:
+               if dep_str not in common_blacklist:
+                  self.add_dependency ( dep_str, deptype_mask )
+
+      elif specific_blacklist:
+         for dep_str in dep_list:
+            if dep_str not in specific_blacklist:
+               self.add_dependency ( dep_str, deptype_mask )
+      else:
+         for dep_str in dep_list:
+            self.add_dependency ( dep_str, deptype_mask )
+   # --- end of add_dependencies_filtered (...) ---
+
+
    def collect_dependencies ( self ):
       """Returns a list that contains all resolved deps,
       including ignored deps that resolve to None.


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-08-27 15:39 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-08-27 15:39 UTC (permalink / raw
  To: gentoo-commits

commit:     8c2873506aa2e1131a6df00d147afb245d20ea3f
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Tue Aug 27 15:30:35 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Tue Aug 27 15:30:35 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=8c287350

fix deptype.PKG, deptype.SYS

according to the usage guide, these dependency types should be "package
_first_" / "system _first_", not "package _only_" / ....

---
 roverlay/depres/deptype.py | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/roverlay/depres/deptype.py b/roverlay/depres/deptype.py
index f430316..91a3f3f 100644
--- a/roverlay/depres/deptype.py
+++ b/roverlay/depres/deptype.py
@@ -31,7 +31,9 @@ NONE = 0
 ALL  = external | internal | mandatory
 RESOLVE_ALL = external | internal
 
-SYS = external | mandatory
-PKG = internal | mandatory
+# "system first"
+SYS = mandatory | ( external | try_other )
+# "package first"
+PKG = mandatory | ( internal | try_other )
 
 MANDATORY_TRY = try_other | mandatory


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-08-27 15:39 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-08-27 15:39 UTC (permalink / raw
  To: gentoo-commits

commit:     27b6e6ef78b781e9b645738bea3db5b0d42b28dc
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Tue Aug 27 15:33:15 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Tue Aug 27 15:33:15 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=27b6e6ef

depresolver: log deptype

---
 roverlay/depres/depresolver.py | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index 3d96d86..eeed396 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -182,17 +182,21 @@ class DependencyResolver ( object ):
 
       returns: None (implicit)
       """
+      # COULDFIX: needs some cleanup
+      #            for example, it's already known whether event==(UN)RESOLVED
       event_type = events.DEPRES_EVENTS [event]
       if self.logmask & event_type:
          # log this event
          if event_type == events.DEPRES_EVENTS ['RESOLVED']:
             self.logger_resolved.info (
-               "{!r} as {!r}".format (
+               "0x{:x}, {!r} as {!r}".format (
+                  dep_env.deptype_mask,
                   dep_env.dep_str, dep_env.resolved_by.dep
                )
             )
          elif event_type == events.DEPRES_EVENTS ['UNRESOLVABLE']:
-            self.logger_unresolvable.info ( "{!r}".format ( dep_env.dep_str ) )
+            self.logger_unresolvable.info (
+               "0x{:x}, {!r}".format ( dep_env.deptype_mask, dep_env.dep_str ) )
          else:
             # "generic" event, expects that kw msg is set
             self.logger.debug ( "event {}: {}".format ( event, msg ) )


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-08-27 15:39 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-08-27 15:39 UTC (permalink / raw
  To: gentoo-commits

commit:     a97a5b84ea7f9f1a215e04af44e65092c2f3ee6d
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Tue Aug 27 15:32:48 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Tue Aug 27 15:32:48 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=a97a5b84

depres unresolvable listener: log deptype

---
 roverlay/depres/listeners.py | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/roverlay/depres/listeners.py b/roverlay/depres/listeners.py
index 0bfb96c..7058bfe 100644
--- a/roverlay/depres/listeners.py
+++ b/roverlay/depres/listeners.py
@@ -18,6 +18,20 @@ from roverlay.depres               import events
 from roverlay.depres.depenv        import DepEnv
 from roverlay.depres.communication import DependencyResolverListener
 
+
+def get_resolved_str ( dep_env ):
+   return "{dep_str!r} as {dep!r}".format (
+      dep_str=dep_env.dep_str, dep=dep_env.resolved_by.dep
+   )
+# --- end of get_resolved_str (...) ---
+
+def get_unresolved_str ( dep_env ):
+   return "0x{dep_type:x}, {dep_str}".format (
+      dep_type=dep_env.deptype_mask, dep_str=dep_env.dep_str
+   )
+# --- end of get_unresolved_str (...) ---
+
+
 class FileListener ( DependencyResolverListener ):
    """A dependency resolution listener that writes events to a file."""
 
@@ -111,10 +125,7 @@ class ResolvedFileListener ( FileListener ):
    # --- end of __init__ (...) ---
 
    def notify ( self, event_type, dep_env=None, pkg_env=None, **extra ):
-      self._event ( event_type,
-         "{dep_str!r} as {dep!r}".format (
-            dep_str=dep_env.dep_str, dep=dep_env.resolved_by.dep
-      ) )
+      self._event ( event_type, get_resolved_str ( dep_env ) )
    # --- end of notify (...) ---
 
 
@@ -127,7 +138,7 @@ class UnresolvableFileListener ( FileListener ):
    # --- end of __init__ (...) ---
 
    def notify ( self, event_type, dep_env=None, pkg_env=None, **extra ):
-      self._event ( event_type, dep_env.dep_str )
+      self._event ( event_type, get_unresolved_str ( dep_env ) )
    # --- end of notify (...) ---
 
 
@@ -141,5 +152,5 @@ class UnresolvableSetFileListener ( SetFileListener ):
    # --- end of __init__ (...) ---
 
    def notify ( self, event_type, dep_env=None, pkg_env=None, **extra ):
-      self._event ( event_type, dep_env.dep_str )
+      self._event ( event_type, get_unresolved_str ( dep_env ) )
    # --- end of notify (...) ---


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-08-23 13:52 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-08-23 13:52 UTC (permalink / raw
  To: gentoo-commits

commit:     958ea80e3999379df6ed269a7ef9d0d202626741
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Fri Aug 23 13:39:35 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Fri Aug 23 13:39:35 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=958ea80e

roverlay/depres/depresult: ConstantDepResult

dep result class that cannot participate in selfdep validation.

---
 roverlay/depres/depresult.py | 81 ++++++++++++++++++++++++++++----------------
 1 file changed, 51 insertions(+), 30 deletions(-)

diff --git a/roverlay/depres/depresult.py b/roverlay/depres/depresult.py
index d084ea0..9a320b6 100644
--- a/roverlay/depres/depresult.py
+++ b/roverlay/depres/depresult.py
@@ -17,37 +17,22 @@ DEBUG = False
 
 EMPTY_STR = ""
 
-class _DepResult ( object ):
-   """dependency resolution result data container"""
-
-   def __init__ ( self,
-      dep, score, matching_rule, dep_env=None, fuzzy=None
-   ):
-      """Initializes a dependency resolution result object.
-
-      arguments:
-      * dep           -- resolving dependency (string or None)
-      * score         -- score (int)
-      * matching_rule -- (reference to) the rule that resolved dep
-      * dep_env       -- dependency environment (optional)
-      * fuzzy         -- fuzzy dep (sub-)environment (optional)
-      """
-      super ( _DepResult, self ).__init__()
-      self.dep        = dep
-      self.score      = score
-      #assert hasattr ( matching_rule, 'is_selfdep' )
-      self.is_selfdep = matching_rule.is_selfdep if matching_rule else 0
+class ConstantDepResult ( object ):
+   # COULDFIX: class name not really accurate,
+   # because all dep results are "constant" once they have been created
 
-      if self.is_selfdep:
-         self.fuzzy = fuzzy
-         self.resolving_package = matching_rule.resolving_package
-   # --- end of DepResult ---
+   def __init__ ( self, dep, score, is_selfdep=0 ):
+      super ( ConstantDepResult, self ).__init__()
+      self.dep = dep
+      self.score = score
+      self.is_selfdep = is_selfdep
+   # --- end of __init__ (...) ---
 
    def __eq__ ( self, other ):
       """Compares this dep result with another result or a string."""
       if isinstance ( other, str ):
          return str ( self ) == other
-      elif isinstance ( other, DepResult ):
+      elif isinstance ( other, ConstantDepResult ):
          return (
             self.score          == other.score
             and self.is_selfdep == other.is_selfdep
@@ -57,16 +42,16 @@ class _DepResult ( object ):
          return NotImplemented
    # --- end of __eq__ (...) ---
 
-   def __hash__ ( self ):
-      return id ( self )
-   # --- end of __hash__ (...) ---
-
    def __bool__ ( self ):
       """Returns True if this dep result has a valid score (>0)."""
       return self.score > 0
       #and self.dep is not False
    # --- end of __bool__ (...) ---
 
+   def __hash__ ( self ):
+      return hash ( self.dep )
+   # --- end of __hash__ (...) ---
+
    def __repr__ ( self ):
       return "<{} object {!r} at 0x{:x}>".format (
          self.__class__.__name__, self.dep, id ( self )
@@ -77,6 +62,42 @@ class _DepResult ( object ):
       return self.dep if self.dep is not None else EMPTY_STR
    # --- end of __str__ (...) ---
 
+   def is_valid ( self ):
+      return True
+   # --- end of is_valid (...) ---
+
+# --- end of ConstantDepResult ---
+
+
+class _DepResult ( ConstantDepResult ):
+   """dependency resolution result data container"""
+
+   def __init__ ( self,
+      dep, score, matching_rule, dep_env=None, fuzzy=None
+   ):
+      """Initializes a dependency resolution result object.
+
+      arguments:
+      * dep           -- resolving dependency (string or None)
+      * score         -- score (int)
+      * matching_rule -- (reference to) the rule that resolved dep
+      * dep_env       -- dependency environment (optional)
+      * fuzzy         -- fuzzy dep (sub-)environment (optional)
+      """
+      #assert hasattr ( matching_rule, 'is_selfdep' )
+      super ( _DepResult, self ).__init__(
+         dep, score, ( matching_rule.is_selfdep if matching_rule else 0 )
+      )
+
+      if self.is_selfdep:
+         self.fuzzy = fuzzy
+         self.resolving_package = matching_rule.resolving_package
+   # --- end of __init__ (...) ---
+
+   def __hash__ ( self ):
+      return id ( self )
+   # --- end of __hash__ (...) ---
+
    def prepare_selfdep_reduction ( self ):
       """Prepares this dep result for selfdep validation by creating all
       necessary variables.
@@ -258,4 +279,4 @@ class _DebuggedDepResult ( _DepResult ):
 DepResult = _DebuggedDepResult if DEBUG else _DepResult
 
 # static object for unresolvable dependencies
-DEP_NOT_RESOLVED = _DepResult ( dep=None, score=-2, matching_rule=None )
+DEP_NOT_RESOLVED = ConstantDepResult ( None, -2 )


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-08-20 21:46 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-08-20 21:46 UTC (permalink / raw
  To: gentoo-commits

commit:     be93230665f3672c7e29cbeb247d1a42fff030fb
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Tue Aug 20 16:25:35 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Tue Aug 20 16:25:35 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=be932306

dependency resolver: get_threadcount()

---
 roverlay/depres/depresolver.py | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index 22a0e4d..3d96d86 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -95,6 +95,10 @@ class DependencyResolver ( object ):
          self.all_channel_ids = set()
    # --- end of __init__ (...) ---
 
+   def get_threadcount ( self ):
+      return self._jobs
+   # --- end of get_threadcount (...) ---
+
    def _sort ( self ):
       """Sorts the rule pools of this resolver."""
       for pool in self.static_rule_pools: pool.sort()


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-08-20 21:46 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-08-20 21:46 UTC (permalink / raw
  To: gentoo-commits

commit:     0b0d88215de41754ba1525d656af10868cd5a460
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Tue Aug 20 16:26:05 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Tue Aug 20 16:30:49 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=0b0d8821

roverlay/depres: non-greedy depres channel

The non-greedy channel processes all deps, whether successful or not. The
default behavior (EbuildJobChannel) is to stop if any mandatory dep cannot be
resolved. This is desirable when running overlay creation (it's faster),
but can hide unresolvable dep strings, which is not useful when writing dep
rules. Single-threaded depres shouldn't be affected, though, since the ebuild
channel has to wait until the resolver is done.

So, the typical use case for non-greedy depres channels is when dealing with
partial results (which overlay creation doesn't).

---
 roverlay/depres/channels.py | 181 ++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 168 insertions(+), 13 deletions(-)

diff --git a/roverlay/depres/channels.py b/roverlay/depres/channels.py
index 73459e5..d3ba725 100644
--- a/roverlay/depres/channels.py
+++ b/roverlay/depres/channels.py
@@ -160,19 +160,20 @@ class EbuildJobChannel ( _EbuildJobChannelBase ):
 
 
    def satisfy_request ( self,
-      close_if_unresolvable=True, preserve_order=False
+      close_if_unresolvable=True, preserve_order=False, want_tuple=True
    ):
       """Tells to the dependency resolver to run.
-      It blocks until this channel is done, which means that either all
+      Blocks until this channel is done, which means that either all
       deps are resolved or a mandatory one is unresolvable.
 
       arguments:
       * close_if_unresolvable -- close the channel if one dep is unresolvable
                                  this seems reasonable and defaults to True
       * preserve_order        -- if set and True:
-                                 return resolved deps as tuple, not as
-                                 frozenset
+                                 return resolved deps as list or tuple,
+                                 not as frozenset
                                  Note that this doesn't filter out duplicates!
+      * want_tuple            -- sets the return type for preserve_order
 
       Returns a 2-tuple ( <resolved dependencies>, <unresolvable dep strings> )
       if all mandatory dependencies could be resolved, else None.
@@ -180,9 +181,9 @@ class EbuildJobChannel ( _EbuildJobChannelBase ):
       could not be resolved.
       """
       dep_collected     = list()
-      dep_unresolveable = list()
+      dep_unresolvable = list()
       resolved          = dep_collected.append
-      unresolvable      = dep_unresolveable.append
+      unresolvable      = dep_unresolvable.append
 
       def handle_queue_item ( dep_env ):
          self._depdone += 1
@@ -233,18 +234,172 @@ class EbuildJobChannel ( _EbuildJobChannelBase ):
          # DEPEND/RDEPEND/.. later, seewave requires sci-libs/fftw
          # in both DEPEND and RDEPEND for example
          self._collected_deps    = frozenset ( dep_collected )
-         self._unresolvable_deps = frozenset ( dep_unresolveable ) \
-            if len ( dep_unresolveable ) > 0 else None
+         self._unresolvable_deps = frozenset ( dep_unresolvable ) \
+            if len ( dep_unresolvable ) > 0 else None
 
          if preserve_order:
-            return (
-               tuple ( dep_collected ),
-               tuple ( dep_unresolveable ) \
-                  if len ( dep_unresolveable ) > 0 else None
-            )
+            if want_tuple:
+               return (
+                  tuple ( dep_collected ),
+                  tuple ( dep_unresolvable ) if dep_unresolvable else None
+               )
+            else:
+               return ( dep_collected, dep_unresolvable or None )
          else:
             return ( self._collected_deps, self._unresolvable_deps )
       else:
          if close_if_unresolvable: self.close()
          return None
    # --- end of satisfy_request (...) ---
+
+# --- end of EbuildJobChannel ---
+
+class NonGreedyDepresChannel ( _EbuildJobChannelBase ):
+
+   def handle_request ( self,
+      preserve_order, allow_close=True, want_tuple=True
+   ):
+      """Tells to the dependency resolver to run.
+      Blocks until this channel is done, which means that all deps have been
+      processed, whether successful or not.
+
+      arguments:
+
+      * preserve_order -- if True: return resolved deps as list or tuple,
+                                   not as frozenset
+                          Note that this doesn't filter out duplicates!
+      * allow_close    -- whether to allow automatic close() on error
+                          Defaults to True.
+      * want_tuple     -- sets the return type for preserve_order
+
+      Returns a 3-tuple
+      ( <satisfiable>, <resolved dependencies>, <unresolvable dep strings> )
+
+      Also stores resolved/unresolvable deps in self._collected_deps,
+      self._unresolvable_deps.
+
+      Automatically closes this channel if on-error mode and returns a
+      3-tuple (None,None,None) unless allow_close is set to False.
+      """
+      dep_collected     = list()
+      dep_unresolvable = list()
+      resolved          = dep_collected.append
+      unresolvable      = dep_unresolvable.append
+
+      def handle_queue_item ( dep_env ):
+         self._depdone += 1
+
+         if dep_env is None:
+            # queue unblocked -> on_error mode, return False
+            #ret = False
+            ret = None
+         elif dep_env.is_resolved():
+            # successfully resolved
+            resolved ( dep_env.get_resolved() )
+            ret = True
+         else:
+            # dep_env not resolved,
+            #  resolve it as "not resolved" (placeholder if preserve_order)
+            #  and add it to the list of unresolvable deps
+            # return false if mandatory bit is set
+            if preserve_order:
+               resolved ( roverlay.depres.depresult.DEP_NOT_RESOLVED )
+            unresolvable ( dep_env.dep_str )
+            ret = bool ( deptype.mandatory & ~dep_env.deptype_mask )
+
+         self._depres_queue.task_done()
+         return ret
+      # --- end of handle_queue_item (...) ---
+
+      satisfiable = True
+      process_dep_result = True
+
+      # loop until
+      #  (a) satisfiable is None (= on_error mode) or
+      #  (b) all deps processed or
+      #  (c) error queue not empty
+      while (
+         ( satisfiable is not None ) and
+         self._depdone < self._depcount and self.err_queue.empty
+      ):
+         # tell the resolver to start
+         self._depres_master.start()
+
+         # wait for one result at least
+         process_dep_result = handle_queue_item ( self._depres_queue.get() )
+         if process_dep_result is None:
+            satisfiable = None
+         elif process_dep_result is False:
+            satisfiable = False
+
+         # and process all available results
+         while ( satisfiable is not None ) and not self._depres_queue.empty():
+            process_dep_result = handle_queue_item (
+               self._depres_queue.get_nowait()
+            )
+            if process_dep_result is None:
+               satisfiable = None
+            elif process_dep_result is False:
+               satisfiable = False
+      # --- end while
+
+      if allow_close and (
+         not self.err_queue.empty or satisfiable is None
+      ):
+         self.close()
+         return ( None, None, None )
+      elif preserve_order:
+         if want_tuple:
+            # COULDFIX: tests/depres expects tuples for comparision
+            self._collected_deps    = tuple ( dep_collected )
+            self._unresolvable_deps = tuple ( dep_unresolvable )
+         else:
+            self._collected_deps    = dep_collected
+            self._unresolvable_deps = dep_unresolvable
+      else:
+         self._collected_deps    = frozenset ( dep_collected )
+         self._unresolvable_deps = frozenset ( dep_unresolvable )
+
+      return ( satisfiable, self._collected_deps, self._unresolvable_deps )
+   # --- end of handle_request (...) ---
+
+   def satisfy_request ( self,
+      close_if_unresolvable=True, preserve_order=False, want_tuple=True
+   ):
+      """Tells to the dependency resolver to run.
+      Blocks until this channel is done, which means that all deps have been
+      processed, whether successful or not.
+
+      arguments:
+      * close_if_unresolvable -- close the channel if one dep is unresolvable
+                                 this seems reasonable and defaults to True
+      * preserve_order        -- if set and True:
+                                 return resolved deps as tuple,
+                                 not as frozenset
+                                 Note that this doesn't filter out duplicates!
+      * want_tuple            -- sets the return type for preserve_order
+
+      Returns a 2-tuple
+         ( <resolved dependencies>, <unresolvable dep strings>|None )
+      if all mandatory deps could be resolved, else None.
+
+      Calls handle_request() for doing the actual work. This is a compat
+      function that provides a common interface for ebuild creation jobs,
+      consider using handle_request() directly.
+      """
+      satisfiable, dep_resolved, dep_unresolable = self.handle_request (
+         preserve_order=preserve_order, allow_close=close_if_unresolvable,
+         want_tuple=want_tuple
+      )
+
+      if satisfiable and self.err_queue.empty:
+         return ( self._collected_deps, self._unresolvable_deps or None )
+      elif satisfiable is None or not close_if_unresolvable:
+         # already closed or don't close
+         return None
+      else:
+         self.close()
+         return None
+   # --- end of satisfy_request (...) ---
+
+# --- end of NonGreedyDepresChannel ---


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:gsoc13/next commit in: roverlay/depres/
@ 2013-07-18 19:25 André Erdmann
  2013-07-23  7:51 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
  0 siblings, 1 reply; 40+ messages in thread
From: André Erdmann @ 2013-07-18 19:25 UTC (permalink / raw
  To: gentoo-commits

commit:     e499fb1751df277f49df3abfbbd3a873762f8918
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Thu Jul 18 07:29:58 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Thu Jul 18 07:29:58 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=e499fb17

roverlay/depres/depresult: remove __getitem__()

---
 roverlay/depres/depresult.py | 15 ---------------
 1 file changed, 15 deletions(-)

diff --git a/roverlay/depres/depresult.py b/roverlay/depres/depresult.py
index 2ad2488..d084ea0 100644
--- a/roverlay/depres/depresult.py
+++ b/roverlay/depres/depresult.py
@@ -4,8 +4,6 @@
 # Distributed under the terms of the GNU General Public License;
 # either version 2 of the License, or (at your option) any later version.
 
-from __future__ import print_function
-
 __all__ = [ 'DepResult', 'DEP_NOT_RESOLVED', ]
 
 import logging
@@ -79,19 +77,6 @@ class _DepResult ( object ):
       return self.dep if self.dep is not None else EMPTY_STR
    # --- end of __str__ (...) ---
 
-   def __getitem__ ( self, key ):
-      # for backwards compatibility, indexing is supported
-      print ( "FIXME: __getitem__ is deprecated" )
-      if key == 0:
-         return self.score
-      elif key == 1:
-         return self.dep
-      elif isinstance ( key, int ):
-         raise IndexError ( key )
-      else:
-         raise KeyError ( key )
-   # --- end of __getitem__ (...) ---
-
    def prepare_selfdep_reduction ( self ):
       """Prepares this dep result for selfdep validation by creating all
       necessary variables.


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:gsoc13/next commit in: roverlay/depres/
@ 2013-07-17 18:10 André Erdmann
  2013-07-17 18:10 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
  0 siblings, 1 reply; 40+ messages in thread
From: André Erdmann @ 2013-07-17 18:10 UTC (permalink / raw
  To: gentoo-commits

commit:     3cae70d2b3df58af8957e2c49089eb9d835486f2
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed Jul 17 18:06:37 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed Jul 17 18:06:37 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=3cae70d2

add debug capabilities to depresult

---
 roverlay/depres/depresult.py | 112 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 108 insertions(+), 4 deletions(-)

diff --git a/roverlay/depres/depresult.py b/roverlay/depres/depresult.py
index 2049719..2ad2488 100644
--- a/roverlay/depres/depresult.py
+++ b/roverlay/depres/depresult.py
@@ -4,13 +4,22 @@
 # Distributed under the terms of the GNU General Public License;
 # either version 2 of the License, or (at your option) any later version.
 
+from __future__ import print_function
+
 __all__ = [ 'DepResult', 'DEP_NOT_RESOLVED', ]
 
+import logging
+
 import roverlay.depres.depenv
 
+# two dep result classes are available
+#  they're identical, but the "debugged-" one produces a lot of output
+#  and calculates some operations twice
+DEBUG = False
+
 EMPTY_STR = ""
 
-class DepResult ( object ):
+class _DepResult ( object ):
    """dependency resolution result data container"""
 
    def __init__ ( self,
@@ -25,6 +34,7 @@ class DepResult ( object ):
       * dep_env       -- dependency environment (optional)
       * fuzzy         -- fuzzy dep (sub-)environment (optional)
       """
+      super ( _DepResult, self ).__init__()
       self.dep        = dep
       self.score      = score
       #assert hasattr ( matching_rule, 'is_selfdep' )
@@ -98,6 +108,7 @@ class DepResult ( object ):
 
          # DEBUG bind
          self.version = version
+         self.vmod    = vmod
 
          self.version_compare = version.get_package_comparator ( vmod )
 
@@ -155,7 +166,7 @@ class DepResult ( object ):
       """'reduce' operation for selfdep validation.
 
       Eliminates candidates that are no longer valid and returns the number
-      of removed candiates (0 if nothing removed).
+      of removed candidates (0 if nothing removed).
       """
       candidates = list (
          p for p in self.candidates if p.has_valid_selfdeps()
@@ -166,7 +177,100 @@ class DepResult ( object ):
       return num_removed
    # --- end of do_reduce (...) ---
 
-# --- end of DepResult ---
+# --- end of _DepResult ---
+
+
+class _DebuggedDepResult ( _DepResult ):
+   LOGGER = logging.getLogger ( 'depresult' )
+
+   def __init__ ( self, *args, **kwargs ):
+      super ( _DebuggedDepResult, self ).__init__ ( *args, **kwargs )
+      if self.is_selfdep:
+         self.logger = self.__class__.LOGGER.getChild ( self.dep )
+
+   def deps_satisfiable ( self ):
+      self.logger.debug (
+         "deps satisfiable? {}, <{}>".format (
+            ( "Yes" if self.candidates else "No" ),
+            ', '.join (
+               (
+                  "{}::{}-{}".format (
+                     (
+                        p['origin'].name if p.has ( 'origin' ) else "<undef>"
+                     ),
+                     p['name'], p['ebuild_verstr']
+                  ) for p in self.candidates
+               )
+            )
+         )
+      )
+      return super ( DebuggedDepResult, self ).deps_satisfiable()
+   # --- end of deps_satisfiable (...) ---
+
+   def link_if_version_matches ( self, p ):
+      ret = super ( DebuggedDepResult, self ).link_if_version_matches ( p )
+
+      pf = "{PN}-{PVR}".format (
+         PN     = p ['name'],
+         PVR    = p ['ebuild_verstr'],
+      )
+
+      info_append = ' p\'valid={valid!r}'.format (
+         valid  = bool ( p.is_valid() ),
+      )
+
+      if self.fuzzy:
+         info_append += (
+            'this\'fuzzy={fuzzy!r}, vmatch={vmatch!r}, '
+            'this\'version={myver!r}, p\'version={pver}, '
+            'vcomp={vcomp!r} vmod={vmod:d}'.format (
+               fuzzy  = bool ( self.fuzzy ),
+               vmatch = self.version_compare ( p ),
+               myver  = self.version,
+               pver   = p ['version'],
+               vcomp  = self.version_compare,
+               vmod   = self.vmod,
+            )
+         )
+      else:
+         info_append += ', this\'fuzzy=False'
+
+
+      if ret:
+         self.logger.debug (
+            "Added {PF} to candidate list.".format ( PF=pf )
+            + info_append
+         )
+      else:
+         # recalculate for logging
+         self.logger.debug (
+            'Rejected {PF} as candidate.'.format ( PF=pf )
+            + info_append
+         )
+
+      return ret
+   # --- end of link_if_version_matches (...) ---
+
+   def do_reduce ( self ):
+      """'reduce' operation for selfdep validation.
+
+      Eliminates candidates that are no longer valid and returns the number
+      of removed candidates (0 if nothing removed).
+      """
+      self.logger.debug ( "Checking candidates:" )
+      for p in self.candidates:
+         "{}-{}: {}".format (
+            p['name'], p['ebuild_verstr'], p.has_valid_selfdeps()
+         )
+      ret = super ( DebuggedDepResult, self ).do_reduce()
+      self.logger.debug ( "Dropped {:d} candidates.".format ( ret ) )
+      return ret
+   # --- end of do_reduce (...) ---
+
+# --- end of _DebuggedDepResult ---
+
+# DepResult <=  _DepResult | _DebuggedDepResult
+DepResult = _DebuggedDepResult if DEBUG else _DepResult
 
 # static object for unresolvable dependencies
-DEP_NOT_RESOLVED = DepResult ( dep=None, score=-2, matching_rule=None )
+DEP_NOT_RESOLVED = _DepResult ( dep=None, score=-2, matching_rule=None )


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-07-16 16:36 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-07-16 16:36 UTC (permalink / raw
  To: gentoo-commits

commit:     21890525bfd7419ad60a0b370cb14de468da2f36
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Jul 15 22:30:17 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Jul 15 22:30:17 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=21890525

roverlay/depres/depenv, fuzzy: ignore certain words

---
 roverlay/depres/depenv.py | 26 ++++++++++++++++----------
 1 file changed, 16 insertions(+), 10 deletions(-)

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index 7455360..c8fea7e 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -54,7 +54,7 @@ class DepEnv ( object ):
 
    # excluding A-Z since dep_str_low will be used to find a match
    # _NAME ::= word{<whitespace><word>}
-   _NAME = '(?P<name>[a-z0-9_\-/.+-]+(\s+[a-z0-9_\-/.+-]+)*)'
+   _NAME = '(?P<name>[a-z0-9_\-/.:+-]+(\s+[a-z0-9_\-/.+-]+)*)'
 
    # _VER              ::= [[<version_separator>]*<digit>[<digit>]*]*
    # digit             ::= {0..9}
@@ -65,6 +65,9 @@ class DepEnv ( object ):
    # { <, >, ==, <=, >=, =, != }
    _VERMOD = '(?P<vmod>[<>]|[=<>!]?[=])'
 
+   _NAME_PREFIX = '(?P<name_prefix>for building from source[:])'
+   _NAME_SUFFIX = '(?P<name_suffix>lib|library)'
+
    # integer representation of version modifiers
    ## duplicate of versiontuple.py
    VMOD_NONE  = roverlay.versiontuple.VMOD_NONE
@@ -91,22 +94,26 @@ class DepEnv ( object ):
    VERSION_REGEX = frozenset (
       re.compile ( r ) for r in ((
          # 'R >= 2.15', 'R >=2.15' etc. (but not 'R>=2.15'!)
-         '^{name}\s+{vermod}?\s*{ver}\s*$'.format (
-            name=_NAME, vermod=_VERMOD, ver=_VER
+         '^{prefix}?\s*{name}\s+{vermod}?\s*{ver}\s*{suffix}?\s*$'.format (
+            name=_NAME, vermod=_VERMOD, ver=_VER,
+            prefix=_NAME_PREFIX, suffix=_NAME_SUFFIX,
          ),
 
          # 'R (>= 2.15)', 'R(>=2.15)' etc.
-         '^{name}\s*\(\s*{vermod}?\s*{ver}\s*\)$'.format (
-            name=_NAME, vermod=_VERMOD, ver=_VER
+         '^{prefix}?\s*{name}\s*\(\s*{vermod}?\s*{ver}\s*\)\s*{suffix}?$'.format (
+            name=_NAME, vermod=_VERMOD, ver=_VER,
+            prefix=_NAME_PREFIX, suffix=_NAME_SUFFIX,
          ),
          # 'R [>= 2.15]', 'R[>=2.15]' etc.
-         '^{name}\s*\[\s*{vermod}?\s*{ver}\s*\]$'.format (
-            name=_NAME, vermod=_VERMOD, ver=_VER
+         '^{prefix}?\s*{name}\s*\[\s*{vermod}?\s*{ver}\s*\]\s*{suffix}?$'.format (
+            name=_NAME, vermod=_VERMOD, ver=_VER,
+            prefix=_NAME_PREFIX, suffix=_NAME_SUFFIX,
          ),
 
          # 'R {>= 2.15}', 'R{>=2.15}' etc.
-         '^{name}\s*\{{\s*{vermod}?\s*{ver}\s*\}}$'.format (
-            name=_NAME, vermod=_VERMOD, ver=_VER
+         '^{prefix}?\s*{name}\s*\{{\s*{vermod}?\s*{ver}\s*\}}\s*{suffix}?$'.format (
+            name=_NAME, vermod=_VERMOD, ver=_VER,
+            prefix=_NAME_PREFIX, suffix=_NAME_SUFFIX,
          ),
       ))
    )
@@ -207,7 +214,6 @@ class DepEnv ( object ):
                   version_iparts.append ( int ( v2 ) if v2 else 0 )
 
 
-
             result.append ( dict (
                name             = m.group ( 'name' ),
                name_low         = m.group ( 'name' ).lower(),


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:gsoc13/next commit in: roverlay/depres/
@ 2013-07-15 22:31 André Erdmann
  2013-07-16 16:36 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
  0 siblings, 1 reply; 40+ messages in thread
From: André Erdmann @ 2013-07-15 22:31 UTC (permalink / raw
  To: gentoo-commits

commit:     bf85b8297c1581384d9331d750e725a3ff8a650c
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Jul 15 22:29:26 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Jul 15 22:29:26 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=bf85b829

roverlay/depres/listeners: sort unresolvable

---
 roverlay/depres/listeners.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/roverlay/depres/listeners.py b/roverlay/depres/listeners.py
index 3d5ceaf..91d4a0b 100644
--- a/roverlay/depres/listeners.py
+++ b/roverlay/depres/listeners.py
@@ -83,7 +83,9 @@ class SetFileListener ( DependencyResolverListener ):
          fh = open ( self._file, 'w' )
 
          if sort_entries:
-            fh.write ( '\n'.join ( sorted ( self._buffer ) ) )
+            fh.write ( '\n'.join (
+               sorted ( self._buffer, key=lambda x: x.lower() ) )
+            )
          else:
             fh.write ( '\n'.join ( self._buffer ) )
 


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:gsoc13/next commit in: roverlay/depres/
@ 2013-07-11 16:44 André Erdmann
  2013-07-11 16:44 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
  0 siblings, 1 reply; 40+ messages in thread
From: André Erdmann @ 2013-07-11 16:44 UTC (permalink / raw
  To: gentoo-commits

commit:     f7829a4b4809ea2dc42c5b28ceb1cc40a70ac1f1
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Thu Jul 11 16:42:05 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Thu Jul 11 16:42:05 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=f7829a4b

roverlay/depres/depenv: split ANDed dependency str

Dependency strings containing an '<whitespace>and<whitespace>' will now be
splitted, and each part is resolved individually.

Note that this "breaks" some of the existing dependency rules.

---
 roverlay/depres/depenv.py | 19 +++++++++++++------
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index 277290c..7455360 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -114,6 +114,8 @@ class DepEnv ( object ):
    FIXVERSION_REGEX = re.compile ( '[_\-]' )
    URI_PURGE        = re.compile ( '\s*from\s*(http|ftp|https)://[^\s]+' )
    WHITESPACE       = re.compile ( '\s+' )
+   #AND_SPLIT        = re.compile ( '\s+and\s+|\s+&&\s+', flags=re.IGNORECASE )
+   AND_SPLIT        = re.compile ( '\s+and\s+', flags=re.IGNORECASE )
 
    # try all version regexes if True, else break after first match
    TRY_ALL_REGEXES  = False
@@ -124,22 +126,27 @@ class DepEnv ( object ):
 
    @classmethod
    def from_str ( cls, dep_str, deptype_mask ):
-      """(Pre-)parses a dependency string and creates DepEnv objects for it."""
+      """Generator that (pre-)parses a dependency string and creates
+      DepEnv objects for it.
+
+      arguments:
+      * dep_str      --
+      * deptype_mask --
+      """
       # split dep_str into logically ANDed dependency strings,
       # unquote them, remove "from <uri>.." entries and replace all
       # whitespace by a single ' ' char
-      return (
-         cls (
+      for substring in cls.AND_SPLIT.split ( dep_str ):
+         yield cls (
             dep_str = (
                cls.WHITESPACE.sub ( ' ',
                   cls.URI_PURGE.sub ( '',
-                     strutil.unquote ( dep_str )
+                     strutil.unquote ( substring )
                   )
                ).strip()
             ),
             deptype_mask = deptype_mask,
-         ),
-      )
+         )
    # --- end of from_str (...) ---
 
    def __init__ ( self, dep_str, deptype_mask ):


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-07-11  8:49 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-07-11  8:49 UTC (permalink / raw
  To: gentoo-commits

commit:     fb5c3772408a021f02a654bd3bfa4a220e486b50
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Thu Jul 11 08:43:48 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Thu Jul 11 08:43:48 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=fb5c3772

roverlay/depres: create multiple DepEnv instances

Create multiple DepEnv instances for a dependency string.
This can be used to split strings like '<dep1> and <dep2>'.

---
 roverlay/depres/channels.py |  8 +++++---
 roverlay/depres/depenv.py   | 32 ++++++++++++++++++++------------
 2 files changed, 25 insertions(+), 15 deletions(-)

diff --git a/roverlay/depres/channels.py b/roverlay/depres/channels.py
index b1f39bd..60875f0 100644
--- a/roverlay/depres/channels.py
+++ b/roverlay/depres/channels.py
@@ -112,9 +112,11 @@ class _EbuildJobChannelBase ( DependencyResolverChannel ):
             "This channel is 'done', it doesn't accept new dependencies."
          )
       else:
-         dep_env = DepEnv ( dep_str=dep_str, deptype_mask=deptype_mask )
-         self._depcount += 1
-         self._depres_master.enqueue ( dep_env, self.ident )
+         for dep_env in DepEnv.from_str (
+            dep_str=dep_str, deptype_mask=deptype_mask
+         ):
+            self._depcount += 1
+            self._depres_master.enqueue ( dep_env, self.ident )
 
    # --- end of add_dependency (...) ---
 

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index 972dc79..277290c 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -122,17 +122,25 @@ class DepEnv ( object ):
    STATUS_RESOLVED     = 2
    STATUS_UNRESOLVABLE = 4
 
-   def _depstr_fix ( self, dep_str ):
-      """Removes cruft from a dep string."""
-      # unquote dep_str, remove "from <uri>.." entries and replace all
+   @classmethod
+   def from_str ( cls, dep_str, deptype_mask ):
+      """(Pre-)parses a dependency string and creates DepEnv objects for it."""
+      # split dep_str into logically ANDed dependency strings,
+      # unquote them, remove "from <uri>.." entries and replace all
       # whitespace by a single ' ' char
-      cls = self.__class__
-      return cls.WHITESPACE.sub ( ' ',
-         cls.URI_PURGE.sub ( '',
-            strutil.unquote ( dep_str )
-         )
-      ).strip()
-   # --- end of _depstr_fix (...) ---
+      return (
+         cls (
+            dep_str = (
+               cls.WHITESPACE.sub ( ' ',
+                  cls.URI_PURGE.sub ( '',
+                     strutil.unquote ( dep_str )
+                  )
+               ).strip()
+            ),
+            deptype_mask = deptype_mask,
+         ),
+      )
+   # --- end of from_str (...) ---
 
    def __init__ ( self, dep_str, deptype_mask ):
       """Initializes a dependency environment that represents the dependency
@@ -147,8 +155,8 @@ class DepEnv ( object ):
       self.status       = DepEnv.STATUS_UNDONE
       self.resolved_by  = None
 
-      self.dep_str      = self._depstr_fix ( dep_str )
-      self.dep_str_low  = self.dep_str.lower()
+      self.dep_str      = dep_str
+      self.dep_str_low  = dep_str.lower()
 
       self.try_all_regexes = self.__class__.TRY_ALL_REGEXES
 


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2013-06-13 16:34 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2013-06-13 16:34 UTC (permalink / raw
  To: gentoo-commits

commit:     575a672de2d76fdfb128444a88fe717aa5aff99d
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Thu Jun 13 16:27:23 2013 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Thu Jun 13 16:27:23 2013 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=575a672d

correct comment typo

---
 roverlay/depres/deptype.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/roverlay/depres/deptype.py b/roverlay/depres/deptype.py
index 226d871..269f666 100644
--- a/roverlay/depres/deptype.py
+++ b/roverlay/depres/deptype.py
@@ -11,7 +11,7 @@
 # * allows to add other types
 
 # <deptype> ::= 2**k | k in {0,1,2,...}
-# try_call indicates that the dep can be checked world-wide (in non-accepting
+# try_other indicates that the dep can be checked world-wide (in non-accepting
 # rule pools) after unsuccessful resolution
 try_other = 1
 mandatory = 2


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-07-30 15:53 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-07-30 15:53 UTC (permalink / raw
  To: gentoo-commits

commit:     1be81df9840f60ba4e9a354e191df7295f4b2c12
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Jul 30 15:49:36 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Jul 30 15:53:31 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=1be81df9

depenv: regex changes

---
 roverlay/depres/depenv.py |   27 +++++++++++++++++++--------
 1 files changed, 19 insertions(+), 8 deletions(-)

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index e70c6d3..f9bef8c 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -7,14 +7,12 @@ from roverlay import strutil
 class DepEnv ( object ):
 
 	# excluding A-Z since dep_str_low will be used to find a match
-	_NAME = '(?P<name>[a-z0-9_\-/.+-]+)'
+	# _NAME ::= word{<whitespace><word>}
+	_NAME = '(?P<name>[a-z0-9_\-/.+-]+(\s+[a-z0-9_\-/.+-]+)*)'
 	_VER  = '(?P<ver>[0-9._\-]+)'
-	# { <, >, ==, <=, >=, =, != } (TODO !=)
+	# { <, >, ==, <=, >=, =, != }
 	_VERMOD = '(?P<vmod>[<>]|[=<>!]?[=])'
 
-	# FIXME: "boost library (>1.0)" not resolved as >=dev-libs/boost-1.0,
-	# regex \s
-
 	V_REGEX_STR = frozenset ( (
 		# 'R >= 2.15', 'R >=2.15' etc. (but not 'R>=2.15'!)
 		'^{name}\s+{vermod}?\s*{ver}\s*$'.format (
@@ -40,12 +38,25 @@ class DepEnv ( object ):
 		re.compile ( regex ) for regex in V_REGEX_STR
 	)
 	FIXVERSION_REGEX = re.compile ( '[_\-]' )
+
+	URI_PURGE = re.compile ( '\s*from\s*(http|ftp|https)://[^\s]+' )
+	WHITESPACE = re.compile ( '\s+' )
+
 	TRY_ALL_REGEXES  = False
 
 	STATUS_UNDONE       = 1
 	STATUS_RESOLVED     = 2
 	STATUS_UNRESOLVABLE = 4
 
+	def _depstr_fix ( self, dep_str ):
+		cls = self.__class__
+		return cls.WHITESPACE.sub ( ' ',
+			cls.URI_PURGE.sub ( '',
+				strutil.unquote ( dep_str )
+			)
+		).strip()
+	# --- end of _depstr_fix (...) ---
+
 	def __init__ ( self, dep_str, deptype_mask ):
 		"""Initializes a dependency environment that represents the dependency
 		resolution of one entry in the description data of an R package.
@@ -56,16 +67,16 @@ class DepEnv ( object ):
 		* dep_str -- dependency string at it appears in the description data.
 		"""
 		self.deptype_mask = deptype_mask
-		self.dep_str      = strutil.unquote ( dep_str )
-		self.dep_str_low  = self.dep_str.lower()
 		self.status       = DepEnv.STATUS_UNDONE
 		self.resolved_by  = None
 
+		self.dep_str      = self._depstr_fix ( dep_str )
+		self.dep_str_low  = self.dep_str.lower()
+
 		self.try_all_regexes = self.__class__.TRY_ALL_REGEXES
 
 		self._depsplit()
 
-
 		# TODO: analyze dep_str:
 		#   extract dep name, dep version, useless comments,...
 


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:overlay_wip commit in: roverlay/depres/
@ 2012-07-30  8:52 André Erdmann
  2012-07-30  8:52 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
  0 siblings, 1 reply; 40+ messages in thread
From: André Erdmann @ 2012-07-30  8:52 UTC (permalink / raw
  To: gentoo-commits

commit:     6a2487574fcf8b40f42b9e3046976b3ca0658195
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Jul 30 08:44:06 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Jul 30 08:44:06 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=6a248757

unquote dep strings, accept == version modifier

---
 roverlay/depres/depenv.py |   19 +++++++++++++------
 1 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index 69e9d57..e70c6d3 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -2,14 +2,18 @@
 # Copyright 2006-2012 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 import re
+from roverlay import strutil
 
 class DepEnv ( object ):
 
 	# excluding A-Z since dep_str_low will be used to find a match
-	_NAME = '(?P<name>[a-z0-9_\-/]+)'
+	_NAME = '(?P<name>[a-z0-9_\-/.+-]+)'
 	_VER  = '(?P<ver>[0-9._\-]+)'
-	# { <, >, <=, >=, =, != } (TODO !=)
-	_VERMOD = '(?P<vmod>[<>]|[<>!]?[=])'
+	# { <, >, ==, <=, >=, =, != } (TODO !=)
+	_VERMOD = '(?P<vmod>[<>]|[=<>!]?[=])'
+
+	# FIXME: "boost library (>1.0)" not resolved as >=dev-libs/boost-1.0,
+	# regex \s
 
 	V_REGEX_STR = frozenset ( (
 		# 'R >= 2.15', 'R >=2.15' etc. (but not 'R>=2.15'!)
@@ -52,8 +56,8 @@ class DepEnv ( object ):
 		* dep_str -- dependency string at it appears in the description data.
 		"""
 		self.deptype_mask = deptype_mask
-		self.dep_str      = dep_str
-		self.dep_str_low  = dep_str.lower()
+		self.dep_str      = strutil.unquote ( dep_str )
+		self.dep_str_low  = self.dep_str.lower()
 		self.status       = DepEnv.STATUS_UNDONE
 		self.resolved_by  = None
 
@@ -79,9 +83,12 @@ class DepEnv ( object ):
 				# fix versions like ".9" (-> "0.9")
 				if version [0] == '.': version = '0' + version
 
+				vmod = m.group ( 'vmod' )
+				if vmod == '==' : vmod = '='
+
 				result.append ( dict (
 					name             = m.group ( 'name' ),
-					version_modifier = m.group ( 'vmod' ),
+					version_modifier = vmod,
 					version          = version
 				) )
 


^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:overlay_wip commit in: roverlay/depres/
@ 2012-07-19 16:53 André Erdmann
  2012-07-30  8:52 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
  0 siblings, 1 reply; 40+ messages in thread
From: André Erdmann @ 2012-07-19 16:53 UTC (permalink / raw
  To: gentoo-commits

commit:     b934d0728451c9c6127edcbfc40d5d28730c64a4
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Thu Jul 19 16:24:48 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Thu Jul 19 16:24:48 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=b934d072

handle versions correctly that start with a dot

---
 roverlay/depres/depenv.py |   10 +++++++---
 1 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index 7cef1c9..69e9d57 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -73,12 +73,16 @@ class DepEnv ( object ):
 			m = r.match ( self.dep_str_low )
 			if m is not None:
 
+				version = self.__class__.FIXVERSION_REGEX.sub (
+					'.', m.group ( 'ver' )
+				)
+				# fix versions like ".9" (-> "0.9")
+				if version [0] == '.': version = '0' + version
+
 				result.append ( dict (
 					name             = m.group ( 'name' ),
 					version_modifier = m.group ( 'vmod' ),
-					version          = self.__class__.FIXVERSION_REGEX.sub (
-												'.', m.group ( 'ver' )
-											)
+					version          = version
 				) )
 
 				if not self.try_all_regexes: break



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-07-16 16:15 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-07-16 16:15 UTC (permalink / raw
  To: gentoo-commits

commit:     313ea3d9c2ab34d4bb72423e0bf9a99a0d094b9f
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Jul 16 09:14:13 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Jul 16 09:16:45 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=313ea3d9

depresolver: separate run method for threads #2

Re-added threaded resolving which runs async now
(as "daemon" - start once and ask for depres at any time).

	geändert:   roverlay/depres/depresolver.py

---
 roverlay/depres/depresolver.py |  182 +++++++++++++++++++--------------------
 1 files changed, 89 insertions(+), 93 deletions(-)

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index 5f85ad1..40d696e 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -39,14 +39,17 @@ class DependencyResolver ( object ):
 			'RESOLVED', 'UNRESOLVABLE'
 		)
 
-		self._jobs = config.get ( "DEPRES.jobcount", 0 )
+		self._jobs = config.get ( "DEPRES.jobcount", 1 )
 
 		# used to lock the run methods,
 		self._runlock = threading.Lock()
 
-		if self._jobs > 0:
+
+
+		if self._jobs > 1:
 			# the dep res main thread
 			self._mainthread = None
+			self._thread_close = False
 
 
 		self.err_queue = err_queue
@@ -270,7 +273,7 @@ class DependencyResolver ( object ):
 	# --- end of _queue_previously_failed (...) ---
 
 	def start ( self ):
-		if self._jobs == 0:
+		if self._jobs < 2:
 			if not self._depqueue.empty():
 				self._run_resolver()
 			if not self.err_queue.really_empty():
@@ -357,7 +360,6 @@ class DependencyResolver ( object ):
 						break
 			# --
 
-
 		# -- done with resolving
 
 		if is_resolved != 2:
@@ -412,9 +414,89 @@ class DependencyResolver ( object ):
 	# --- end of _run_resolver (...) ---
 
 	def _thread_run_resolver ( self ):
-		raise Exception ( "method stub" )
+		"""master thread"""
+		try:
+			self.logger.debug (
+				"Running in concurrent mode with {} worker threads.".format (
+					self._jobs
+				)
+			)
+			send_queues = tuple (
+				queue.Queue ( maxsize=1 ) for k in range ( self._jobs )
+			)
+			rec_queues  = tuple (
+				queue.Queue ( maxsize=1 ) for k in range ( self._jobs )
+			)
+			threads = tuple (
+				threading.Thread (
+					target=self._thread_resolve,
+					# this thread's send queue is the worker thread's receive queue
+					# and vice versa
+					kwargs={ 'recq' : send_queues [n], 'sendq' : rec_queues [n] }
+				) for n in range ( self._jobs )
+			)
+
+			try:
+				for t in threads: t.start()
+
+				# *loop forever*
+				# wait for the resolver threads to process the dep queue,
+				# mark remaining deps as unresolvable and
+				# tell the threads to continue
+				while self.err_queue.really_empty() and not self._thread_close:
+					for q in rec_queues:
+						if q.get() != 0:
+							self._thread_close = True
+							break
+					else:
+						self._process_unresolvable_queue()
+						# tell the threads to continue
+						for q in send_queues: q.put_nowait ( 0 )
+
+			except ( Exception, KeyboardInterrupt ) as e:
+				self.err_queue.push ( context=id ( self ), error=e )
+
+			self._thread_close = True
+
+			# on-error code (self.err_queue not empty or close requested)
+			try:
+				for q in send_queues: q.put_nowait ( 2 )
+			except:
+				pass
+
+			for t in threads: t.join()
+
+		finally:
+			self._runlock.release()
 	# --- end of _thread_run_resolver (...) ---
 
+	def _thread_resolve ( self, sendq=0, recq=0 ):
+		"""worker thread"""
+		try:
+			while not self._thread_close and self.err_queue.empty:
+				try:
+					# process remaining deps
+					while not self._thread_close and self.err_queue.empty:
+						self._process_dep ( self._depqueue.get_nowait() )
+				except queue.Empty:
+					pass
+
+				# dep queue has been processed,
+				# let the master thread process all unresolvable deps
+				# only 0 means continue, anything else stops this thread
+				sendq.put_nowait ( 0 )
+				if recq.get() != 0: break
+		except ( Exception, KeyboardInterrupt ) as e:
+			self.err_queue.push ( id ( self ), e )
+
+		# this is on-error code (err_queue is not empty or close requested)
+		self._thread_close = True
+		try:
+			sendq.put_nowait ( 2 )
+		except queue.Full:
+			pass
+	# --- end of _thread_resolve (...) ---
+
 	def enqueue ( self, dep_env, channel_id ):
 		"""Adds a DepEnv to the queue of deps to resolve.
 
@@ -425,11 +507,11 @@ class DependencyResolver ( object ):
 		returns: None (implicit)
 		"""
 		self._depqueue.put ( ( channel_id, dep_env ) )
-
 	# --- end of enqueue (...) ---
 
 	def close ( self ):
-		if self._jobs > 0:
+		if self._jobs > 1:
+			self._thread_close = True
 			if self._mainthread:
 				self._mainthread.join()
 		for lis in self.listeners: lis.close()
@@ -439,89 +521,3 @@ class DependencyResolver ( object ):
 				"{} channels were in use.".format ( len ( self.all_channel_ids ) )
 			)
 	# --- end of close (...) ---
-
-
-	def _thread_run_main ( self ):
-		"""Tells the resolver to run."""
-		raise Exception ( "to be removed" )
-		jobcount = self.__class__.NUMTHREADS
-
-		try:
-			if jobcount < 1:
-				( self.logger.warning if jobcount < 0 else self.logger.debug ) (
-					"Running in sequential mode."
-				)
-				self._thread_run_resolve()
-			else:
-
-				# wait for old threads
-				if not self._threads is None:
-					self.logger.warning ( "Waiting for old threads..." )
-					for t in self._threads: t.join()
-
-				self.logger.debug (
-					"Running in concurrent mode with %i jobs." % jobcount
-				)
-
-				# create threads,
-				self._threads = tuple (
-					threading.Thread ( target=self._thread_run_resolve )
-					for n in range (jobcount)
-				)
-				# run them
-				for t in self._threads: t.start()
-				# and wait until done
-				for t in self._threads: t.join()
-
-				# finally remove them
-				del self._threads
-				self._threads = None
-
-			# iterate over _depqueue_failed and report unresolved
-			## todo can thread this
-			self._process_unresolvable_queue()
-
-			if not self.err_queue.really_empty():
-				self.err_queue.unblock_queues()
-
-		except ( Exception, KeyboardInterrupt ) as e:
-			if jobcount > 0:
-				self.err_queue.push ( id ( self ), e )
-				return
-			else:
-				raise e
-
-		finally:
-			# release the lock
-			self._runlock.release()
-
-	# --- end of _thread_run_main (...) ---
-
-	def _thread_run_resolve ( self ):
-		"""Resolves dependencies (thread target).
-
-		returns: None (implicit)
-		"""
-		raise Exception ( "to be removed" )
-		try:
-			while self.err_queue.empty and not self._depqueue.empty():
-				try:
-					to_resolve = self._depqueue.get_nowait()
-					self._process_dep ( queue_item=to_resolve )
-					self._depqueue.task_done()
-				except queue.Empty:
-					# this thread is done when the queue is empty, so this is
-					# no error, but just the result of the race condition between
-					# queue.empty() and queue.get(False)
-					return
-			# --- end while
-
-		except ( Exception, KeyboardInterrupt ) as e:
-			if self.__class__.NUMTHREADS > 0:
-				self.err_queue.push ( id ( self ), e )
-				return
-			else:
-				raise e
-
-
-	# --- end of _thread_run_resolve (...) ---



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:depres_wip commit in: roverlay/depres/
@ 2012-07-13 10:44 André Erdmann
  2012-07-16 16:15 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
  0 siblings, 1 reply; 40+ messages in thread
From: André Erdmann @ 2012-07-13 10:44 UTC (permalink / raw
  To: gentoo-commits

commit:     bedc1a5d0029f17c255422310bd5ef98aa7d3fb8
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Fri Jul 13 10:39:41 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Fri Jul 13 10:39:41 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=bedc1a5d

depresolver: separate run method for threads

the run-method code for the depresolver is being split up into
a threaded and not-threaded variant, to increase readability,
stability and performance.

* threaded resolving temporarily removed

---
 roverlay/depres/depresolver.py |  355 ++++++++++++++++++++--------------------
 1 files changed, 179 insertions(+), 176 deletions(-)

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index bbe58b4..5f85ad1 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -16,10 +16,8 @@ from roverlay        import config
 from roverlay.depres import communication, deptype, events
 #from roverlay.depres import simpledeprule
 
-#from roverlay.depres.depenv import DepEnv (implicit)
 
-
-# if false: do not using the "negative" result caching which stores
+# if false: do not use the "negative" result caching which stores
 # unresolvable deps in a set for should-be faster lookups
 USING_DEPRES_CACHE = True
 
@@ -29,9 +27,6 @@ SAFE_CHANNEL_IDS = True
 class DependencyResolver ( object ):
 	"""Main object for dependency resolution."""
 
-
-	NUMTHREADS = config.get ( "DEPRES.jobcount", 0 )
-
 	def __init__ ( self, err_queue ):
 		"""Initializes a DependencyResolver."""
 
@@ -44,16 +39,17 @@ class DependencyResolver ( object ):
 			'RESOLVED', 'UNRESOLVABLE'
 		)
 
-		# this lock tells whether a dep res 'master' thread is running (locked)
-		self.runlock     = threading.Lock()
-		self.startlock   = threading.Lock()
-		# the dep res main thread
-		self._mainthread = None
-		# the dep res worker threads
-		self._threads    = None
+		self._jobs = config.get ( "DEPRES.jobcount", 0 )
 
-		self.err_queue   = err_queue
+		# used to lock the run methods,
+		self._runlock = threading.Lock()
 
+		if self._jobs > 0:
+			# the dep res main thread
+			self._mainthread = None
+
+
+		self.err_queue = err_queue
 
 		# the list of registered listener modules
 		self.listeners = list ()
@@ -139,13 +135,13 @@ class DependencyResolver ( object ):
 			# log this event
 			if event_type == events.DEPRES_EVENTS ['RESOLVED']:
 				self.logger_resolved.info (
-					"'%s' as '%s'" % ( dep_env.dep_str, dep_env.resolved_by )
+					"{!r} as {!r}".format ( dep_env.dep_str, dep_env.resolved_by )
 				)
 			elif event_type == events.DEPRES_EVENTS ['UNRESOLVABLE']:
-				self.logger_unresolvable.info ( "'%s'" % dep_env.dep_str )
+				self.logger_unresolvable.info ( "{!r}".format ( dep_env.dep_str ) )
 			else:
 				# "generic" event, expects that kw msg is set
-				self.logger.debug ( "event %s : %s" % ( event, msg ) )
+				self.logger.debug ( "event {}: {}".format ( event, msg ) )
 		# --- if
 
 		if self.listenermask & event_type:
@@ -274,48 +270,180 @@ class DependencyResolver ( object ):
 	# --- end of _queue_previously_failed (...) ---
 
 	def start ( self ):
-		# -- verify whether resolver has to be started
-		if self._depqueue.empty():
-			# nothing to resolve
-			return
+		if self._jobs == 0:
+			if not self._depqueue.empty():
+				self._run_resolver()
+			if not self.err_queue.really_empty():
+				self.err_queue.unblock_queues()
+		else:
+			# new resolver threads run async and
+			# can be started with an empty depqueue
+			if self._runlock.acquire ( False ):
+				# else resolver is running
+
+				self._mainthread = threading.Thread (
+					target=self._thread_run_resolver
+				)
+				self._mainthread.start()
+				# _thread_run_resolver has to release the lock when done
+	# --- end of start (...) ---
+
+	def _process_unresolvable_queue ( self ):
+		# iterate over _depqueue_failed and report unresolved
+		while not self._depqueue_failed.empty() and self.err_queue.empty:
+			try:
+				channel_id, dep_env = self._depqueue_failed.get_nowait()
+			except queue.Empty:
+				# race cond empty() <-> get_nowait()
+				return
+
+			dep_env.set_unresolvable()
+			self._report_event ( 'UNRESOLVABLE', dep_env )
+
+			try:
+				if channel_id in self._depqueue_done:
+					self._depqueue_done [channel_id].put_nowait ( dep_env )
+			except KeyError:
+				# channel has been closed before calling put, ignore this
+				pass
+	# --- end of _process_unresolvable_queue (...) ---
+
+	def _process_dep ( self, queue_item ):
+		channel_id, dep_env = queue_item
+
+		# drop dep if channel closed
+		if not channel_id in self._depqueue_done: return
+
+		self.logger.debug (
+			"Trying to resolve {!r}.".format ( dep_env.dep_str )
+		)
 
-		if not self.startlock.acquire ( False ):
-			# another channel/.. is starting the resolver
-			return
-		elif self._depqueue.empty():
-			self.startlock.release()
-			return
+		resolved = None
+		# resolved can be None, so use a tri-state int for checking
+		#  0 -> unresolved, but resolvable
+		#  1 -> unresolved and (currently, new rules may change this)
+		#        not resolvable
+		#  2 -> resolved
+		is_resolved = 0
 
-		# -- verify...
+		if USING_DEPRES_CACHE and dep_env.dep_str_low in self._dep_unresolvable:
+			# cannot resolve
+			is_resolved = 1
 
-		# acquire the run lock (that locks _run_main)
+		else:
+			# search for a match in the rule pools that accept the dep type
+			for rulepool in (
+				p for p in self.static_rule_pools \
+					if p.deptype_mask & dep_env.deptype_mask
+			):
+				result = rulepool.matches ( dep_env )
+				if result [0] > 0:
+					resolved    = result [1]
+					is_resolved = 2
+					break
+
+			if is_resolved == 0 and dep_env.deptype_mask & deptype.try_other:
+				## TRY_OTHER bit is set
+				# search for a match in the rule pools
+				#  that (normally) don't accept the dep type
+				for rulepool in (
+					p for p in self.static_rule_pools \
+						if p.deptype_mask & ~dep_env.deptype_mask
+				):
+					result = rulepool.matches ( dep_env )
+					if result [0] > 0:
+						resolved    = result [1]
+						is_resolved = 2
+						break
+			# --
+
+
+		# -- done with resolving
+
+		if is_resolved != 2:
+			# could not resolve dep_env
+			self._depqueue_failed.put ( queue_item )
+			if USING_DEPRES_CACHE:
+				# does not work when adding new rules is possible
+				self._dep_unresolvable.add ( dep_env.dep_str_low )
+		else:
+			# successfully resolved
+			dep_env.set_resolved ( resolved, append=False )
+			self._report_event ( 'RESOLVED', dep_env )
+			try:
+				self._depqueue_done [channel_id].put ( dep_env )
+			except KeyError:
+				# channel gone while resolving
+				pass
+
+			"""
+			## only useful if new rules can be created
+			# new rule found, requeue all previously
+			#  failed dependency searches
+			if have_new_rule:
+				self._queue_previously_failed
+				if USING_DEPRES_CACHE:
+					self._dep_unresolvable.clear() #?
+			"""
+	# --- end of _process_dep (...) ---
+
+	def _run_resolver ( self ):
+		# single-threaded variant of run
+		#  still checking err_queue 'cause other modules
+		#  could be run with threads
+		if self._depqueue.empty(): return
 		try:
-			self.runlock.acquire()
+			self._runlock.acquire()
+			while not self._depqueue.empty() and self.err_queue.empty:
+				to_resolve = self._depqueue.get_nowait()
+				self._process_dep ( queue_item=to_resolve )
+				self._depqueue.task_done()
+
+			self._process_unresolvable_queue()
+		except ( Exception, KeyboardInterrupt ) as e:
+			# single-threaded exception catcher:
+			# * push exception to inform other threads (if any)
+			# * unblock queues (automatically when calling push)
+			# * reraise
+			self.err_queue.push ( id ( self ), e )
+			raise e
 		finally:
-			self.startlock.release()
+			self._runlock.release()
+	# --- end of _run_resolver (...) ---
 
-		if self._depqueue.empty():
-			self.runlock.release()
-			return
+	def _thread_run_resolver ( self ):
+		raise Exception ( "method stub" )
+	# --- end of _thread_run_resolver (...) ---
 
-		if DependencyResolver.NUMTHREADS > 0:
-			# no need to wait for the old thread
-			# FIXME: could remove the following block
-			if self._mainthread is not None:
-				self._mainthread.join()
-				del self._mainthread
+	def enqueue ( self, dep_env, channel_id ):
+		"""Adds a DepEnv to the queue of deps to resolve.
 
-			self._mainthread = threading.Thread ( target=self._thread_run_main )
-			self._mainthread.start()
+		arguments:
+		* dep_env -- to add
+		* channel_id -- identifier of the channel associated with the dep_env
 
-		else:
-			self._thread_run_main()
+		returns: None (implicit)
+		"""
+		self._depqueue.put ( ( channel_id, dep_env ) )
+
+	# --- end of enqueue (...) ---
+
+	def close ( self ):
+		if self._jobs > 0:
+			if self._mainthread:
+				self._mainthread.join()
+		for lis in self.listeners: lis.close()
+		del self.listeners
+		if SAFE_CHANNEL_IDS:
+			self.logger.debug (
+				"{} channels were in use.".format ( len ( self.all_channel_ids ) )
+			)
+	# --- end of close (...) ---
 
-		# self.runlock is released when _thread_run_main is done
-	# --- end of start (...) ---
 
 	def _thread_run_main ( self ):
 		"""Tells the resolver to run."""
+		raise Exception ( "to be removed" )
 		jobcount = self.__class__.NUMTHREADS
 
 		try:
@@ -351,26 +479,7 @@ class DependencyResolver ( object ):
 
 			# iterate over _depqueue_failed and report unresolved
 			## todo can thread this
-			while not self._depqueue_failed.empty() and self.err_queue.empty:
-				try:
-					channel_id, dep_env = self._depqueue_failed.get_nowait()
-
-				except queue.Empty:
-					# race cond empty() <-> get_nowait()
-					return
-
-
-				dep_env.set_unresolvable()
-
-				self._report_event ( 'UNRESOLVABLE', dep_env )
-
-
-				try:
-					if channel_id in self._depqueue_done:
-						self._depqueue_done [channel_id].put_nowait ( dep_env )
-				except KeyError:
-					# channel has been closed before calling put, ignore this
-					pass
+			self._process_unresolvable_queue()
 
 			if not self.err_queue.really_empty():
 				self.err_queue.unblock_queues()
@@ -384,7 +493,7 @@ class DependencyResolver ( object ):
 
 		finally:
 			# release the lock
-			self.runlock.release()
+			self._runlock.release()
 
 	# --- end of _thread_run_main (...) ---
 
@@ -393,100 +502,18 @@ class DependencyResolver ( object ):
 
 		returns: None (implicit)
 		"""
+		raise Exception ( "to be removed" )
 		try:
 			while self.err_queue.empty and not self._depqueue.empty():
 				try:
 					to_resolve = self._depqueue.get_nowait()
+					self._process_dep ( queue_item=to_resolve )
+					self._depqueue.task_done()
 				except queue.Empty:
 					# this thread is done when the queue is empty, so this is
 					# no error, but just the result of the race condition between
 					# queue.empty() and queue.get(False)
 					return
-
-				channel_id, dep_env = to_resolve
-
-				if channel_id in self._depqueue_done:
-					# else channel has been closed, drop dep
-
-					self.logger.debug (
-						"Trying to resolve '%s'." % dep_env.dep_str
-					)
-
-					#have_new_rule = False
-
-					resolved = None
-					# resolved can be None, so use a tri-state int for checking
-					#  0 -> unresolved, but resolvable
-					#  1 -> unresolved and (currently, new rules may change this)
-					#        not resolvable
-					#  2 -> resolved
-					is_resolved = 0
-
-					# TODO:
-					#  (threading: could search the pools in parallel)
-
-					if USING_DEPRES_CACHE:
-						if dep_env.dep_str_low in self._dep_unresolvable:
-							# cannot resolve
-							is_resolved = 1
-
-					if is_resolved == 0:
-						# search for a match in the rule pools that accept
-						# the dep type
-						for rulepool in ( p for p in self.static_rule_pools \
-							if p.deptype_mask & dep_env.deptype_mask
-						):
-							result = rulepool.matches ( dep_env )
-							if not result is None and result [0] > 0:
-								resolved    = result [1]
-								is_resolved = 2
-								break
-
-					if is_resolved == 0 and \
-						dep_env.deptype_mask & deptype.try_other \
-					:
-						# search for a match in the rule pools that (normally)
-						# don't accept the dep type
-						for rulepool in ( p for p in self.static_rule_pools \
-							if p.deptype_mask & ~dep_env.deptype_mask
-						):
-							result = rulepool.matches ( dep_env )
-							if not result is None and result [0] > 0:
-								resolved    = result [1]
-								is_resolved = 2
-								break
-
-
-
-
-
-					if is_resolved == 2:
-						dep_env.set_resolved ( resolved, append=False )
-						self._report_event ( 'RESOLVED', dep_env )
-						try:
-							self._depqueue_done [channel_id].put ( dep_env )
-						except KeyError:
-							# channel gone while resolving
-							pass
-					else:
-						self._depqueue_failed.put ( to_resolve )
-
-						if USING_DEPRES_CACHE:
-							# does not work when adding new rules is possible
-							self._dep_unresolvable.add ( dep_env.dep_str_low )
-
-					"""
-					## only useful if new rules can be created
-					# new rule found, requeue all previously
-					#  failed dependency searches
-					if have_new_rule:
-						self._queue_previously_failed
-						if USING_DEPRES_CACHE:
-							self._dep_unresolvable.clear() #?
-					"""
-				# --- end if channel_id in self._depqueue_done
-
-				self._depqueue.task_done()
 			# --- end while
 
 		except ( Exception, KeyboardInterrupt ) as e:
@@ -498,27 +525,3 @@ class DependencyResolver ( object ):
 
 
 	# --- end of _thread_run_resolve (...) ---
-
-	def enqueue ( self, dep_env, channel_id ):
-		"""Adds a DepEnv to the queue of deps to resolve.
-
-		arguments:
-		* dep_env -- to add
-		* channel_id -- identifier of the channel associated with the dep_env
-
-		returns: None (implicit)
-		"""
-		self._depqueue.put ( ( channel_id, dep_env ) )
-
-	# --- end of enqueue (...) ---
-
-	def close ( self ):
-		if isinstance ( self._mainthread, threading.Thread ):
-			self._mainthread.join()
-		for lis in self.listeners: lis.close()
-		del self.listeners
-		if SAFE_CHANNEL_IDS:
-			self.logger.debug (
-				"%i channels were in use." % len ( self.all_channel_ids )
-			)
-	# --- end of close (...) ---



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:depres_wip commit in: roverlay/depres/
@ 2012-07-13 10:44 André Erdmann
  2012-07-16 16:15 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
  0 siblings, 1 reply; 40+ messages in thread
From: André Erdmann @ 2012-07-13 10:44 UTC (permalink / raw
  To: gentoo-commits

commit:     4e33a9f0fec2416376e234ace12a43a5816c837a
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Fri Jul 13 10:37:56 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Fri Jul 13 10:37:56 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=4e33a9f0

return negative score if not resolved

---
 roverlay/depres/deprule.py |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/roverlay/depres/deprule.py b/roverlay/depres/deprule.py
index 32c73e1..2b5754f 100644
--- a/roverlay/depres/deprule.py
+++ b/roverlay/depres/deprule.py
@@ -111,5 +111,5 @@ class DependencyRulePool ( object ):
 						return result
 
 
-		return None
+		return ( -1, None )
 	# --- end of matches (...) ---



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-07-02 16:52 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-07-02 16:52 UTC (permalink / raw
  To: gentoo-commits

commit:     5092211ac965d6f24d1a665c7aa7ce0ee292a4f1
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Jul  2 16:47:24 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Jul  2 16:47:24 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=5092211a

depres channels: lock id generator usage

	modified:   roverlay/depres/communication.py

---
 roverlay/depres/communication.py |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/roverlay/depres/communication.py b/roverlay/depres/communication.py
index 14d6193..daeb79e 100644
--- a/roverlay/depres/communication.py
+++ b/roverlay/depres/communication.py
@@ -6,13 +6,11 @@ import threading
 import sys
 
 def channel_counter ():
-	lock    = threading.Lock()
 	last_id = long ( -1 ) if sys.version_info < ( 3, ) else int ( -1 )
 
 	while True:
-		with lock:
-			last_id += 1
-			yield last_id
+		last_id += 1
+		yield last_id
 
 
 class DependencyResolverListener ( object ):
@@ -59,6 +57,7 @@ class DependencyResolverListener ( object ):
 
 class DependencyResolverChannel ( object ):
 
+	id_gen_lock  = threading.Lock()
 	id_generator = channel_counter()
 
 	def __init__ ( self, main_resolver ):
@@ -73,7 +72,8 @@ class DependencyResolverChannel ( object ):
 		#super ( DependencyResolverChannel, self ) . __init__ ()
 		# channel identifiers must be unique even when the channel has been
 		# deleted (id does not guarantee that)
-		self.ident          = next ( DependencyResolverChannel.id_generator )
+		with DependencyResolverChannel.id_gen_lock:
+			self.ident = next ( DependencyResolverChannel.id_generator )
 		self._depres_master = main_resolver
 	# --- end of __init__ (...) ---
 



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-07-02 16:52 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-07-02 16:52 UTC (permalink / raw
  To: gentoo-commits

commit:     169dbd47cc0795101f25304e709f7f0045d7f364
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Jul  2 16:45:40 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Jul  2 16:45:40 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=169dbd47

fix export_rule for simple dep rules

	modified:   roverlay/depres/abstractsimpledeprule.py
	modified:   roverlay/depres/simpledeprule.py

---
 roverlay/depres/abstractsimpledeprule.py |   60 +++++++++++++----------------
 roverlay/depres/simpledeprule.py         |   26 ++++++-------
 2 files changed, 39 insertions(+), 47 deletions(-)

diff --git a/roverlay/depres/abstractsimpledeprule.py b/roverlay/depres/abstractsimpledeprule.py
index b03ce63..811f88e 100644
--- a/roverlay/depres/abstractsimpledeprule.py
+++ b/roverlay/depres/abstractsimpledeprule.py
@@ -1,5 +1,6 @@
 import logging
 
+from roverlay import config
 from roverlay.depres import deprule
 
 TMP_LOGGER = logging.getLogger ('simpledeps')
@@ -29,8 +30,6 @@ class SimpleRule ( deprule.DependencyRule ):
 		if not dep_str is None:
 			self.dep_alias.append ( dep_str )
 
-		self.logger.debug ( "new rule (%s) for %s" % ( self.__class__.__name__, self.resolving_package ) )
-
 	# --- end of __init__ (...) ---
 
 	def done_reading ( self ):
@@ -80,45 +79,40 @@ class SimpleRule ( deprule.DependencyRule ):
 		return None
 	# --- end of matches (...) ---
 
-	def export_rule ( self, resolving_to=None ):
-		"""Returns this rule as a list of text lines that can be written into
-		a file.
-		An empty list will be returned if dep_alias has zero length.
-
-		arguments:
-		* resolving_to -- portage package that the exported rule should
-		                  resolve to, defaults to self.resolving_package or
-		                  an ignore keyword such as '!'.
+	def export_rule ( self ):
+		"""Generates text lines for this rule that can later be read using
+		the SimpleDependencyRuleReader.
 		"""
+		# todo hardcoded rule format here
+		if self.resolving_package is None:
+			resolving = ''
+		else:
+			resolving = self.resolving_package
 
-		alias_count = len ( self.dep_alias )
+			cat = config.get ( 'OVERLAY.category', None )
 
-		retlist = []
+			if cat is not None:
+				resolving = resolving.replace ( cat + '/', '')
 
-		if alias_count:
-			if resolving_to is None:
-				if hasattr ( self, 'resolving_package'):
-					resolving_package = self.resolving_package
-				else:
-					resolving_package = '!'
-			else:
-				resolving_package = resolving_to
+		if hasattr ( self.__class__, 'RULE_PREFIX' ):
+			resolving = self.__class__.RULE_PREFIX + resolving
 
-			# todo hardcoded rule format here
-			if alias_count > 1:
+		if len ( self.dep_alias ) == 0:
+			pass
 
-				retlist = [ resolving_package + ' {\n' ] + \
-					[ "\t%s\n" % alias for alias in self.dep_alias ] + \
-					[ '}\n' ]
-			else:
-				retlist = [
-					"%s :: %s\n" % ( resolving_package, self.dep_alias [0] )
-				]
+		elif len ( self.dep_alias ) == 1:
+			yield "%s :: %s\n" % ( resolving, iter ( self.dep_alias ).next() )
+
+		else:
+			yield resolving + ' {'
+			for alias in self.dep_alias:
+				yield "\t" + alias
+			yield '}'
+
+	def __str__ ( self ):
+		return '\n'.join ( self.export_rule() )
 
-		# -- if
 
-		return retlist
-	# --- end of export_rule (...) ---
 
 class FuzzySimpleRule ( SimpleRule ):
 

diff --git a/roverlay/depres/simpledeprule.py b/roverlay/depres/simpledeprule.py
index ce4ba10..04a47a6 100644
--- a/roverlay/depres/simpledeprule.py
+++ b/roverlay/depres/simpledeprule.py
@@ -13,6 +13,8 @@ TMP_LOGGER = logging.getLogger ('simpledeps')
 
 class SimpleIgnoreDependencyRule ( SimpleRule ):
 
+	RULE_PREFIX = '!'
+
 	def __init__ ( self, dep_str=None, priority=50, resolving_package=None ):
 		super ( SimpleIgnoreDependencyRule, self ) . __init__ (
 			dep_str=dep_str,
@@ -43,6 +45,8 @@ class SimpleDependencyRule ( SimpleRule ):
 
 class SimpleFuzzyIgnoreDependencyRule ( FuzzySimpleRule ):
 
+	RULE_PREFIX = '%'
+
 	def __init__ ( self, dep_str=None, priority=51, resolving_package=None ):
 		super ( SimpleFuzzyIgnoreDependencyRule, self ) . __init__ (
 			dep_str=dep_str,
@@ -52,6 +56,9 @@ class SimpleFuzzyIgnoreDependencyRule ( FuzzySimpleRule ):
 		)
 
 class SimpleFuzzyDependencyRule ( FuzzySimpleRule ):
+
+	RULE_PREFIX = '~'
+
 	def __init__ ( self, resolving_package, dep_str=None, priority=71 ):
 		super ( SimpleFuzzyDependencyRule, self ) . __init__ (
 			dep_str=dep_str,
@@ -117,11 +124,8 @@ class SimpleDependencyRulePool ( deprule.DependencyRulePool ):
 		raises: IOError (fh)
 		"""
 		for rule in self.rules:
-			to_write = fh.export_rule()
-			if isinstance ( to_write, str ):
-				fh.write ( to_write )
-			else:
-				fh.writelines ( to_write )
+			fh.write ( str ( rule ) )
+			fh.write ( '\n' )
 
 	# --- end of export_rules (...) ---
 
@@ -134,9 +138,9 @@ class SimpleDependencyRuleReader ( object ):
 	comment_chars      = "#;"
 
 	# todo: const/config?
-	package_ignore = '!'
-	fuzzy          = '~'
-	fuzzy_ignore   = '%'
+	package_ignore = SimpleIgnoreDependencyRule.RULE_PREFIX
+	fuzzy          = SimpleFuzzyDependencyRule.RULE_PREFIX
+	fuzzy_ignore   = SimpleFuzzyIgnoreDependencyRule.RULE_PREFIX
 
 	BREAK_PARSING  = frozenset (( '#! NOPARSE', '#! BREAK' ))
 
@@ -146,12 +150,6 @@ class SimpleDependencyRuleReader ( object ):
 		pass
 	# --- end of __init__  (...) ---
 
-
-	def _make_rule ( self, rule_str ):
-		CLS = self.__class__
-
-
-
 	def read_file ( self, filepath ):
 		"""Reads a file that contains simple dependency rules
 		(SimpleIgnoreDependencyRules/SimpleDependencyRules).



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-29 22:48 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-29 22:48 UTC (permalink / raw
  To: gentoo-commits

commit:     0fb00e79cfb4e686c8b68d8645b177531e5a7668
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Fri Jun 29 22:34:34 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Fri Jun 29 22:34:34 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=0fb00e79

DepEnv: collect more infos about the dep_str

* using regex for extracting dep name, version, ..

	modified:   roverlay/depres/depenv.py

---
 roverlay/depres/depenv.py |   57 +++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 57 insertions(+), 0 deletions(-)

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index 0cb81e8..eba03a8 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -1,6 +1,38 @@
 # R overlay --
 # Copyright 2006-2012 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
+import re
+
+
+# excluding A-Z since dep_str_low will be used to find a match
+_NAME = '(?P<name>[a-z0-9_\-/]+)'
+_VER  = '(?P<ver>[0-9._\-]+)'
+# { <, >, <=, >=, =, != } (TODO !=)
+_VERMOD = '(?P<vmod>[<>]|[<>!]?[=])'
+
+# this lists ... <fixme>
+V_REGEX_STR = frozenset ( (
+	# 'R >= 2.15', 'R >=2.15' etc. (but not 'R>=2.15'!)
+	'^%s\s+%s?\s*%s\s*$' % ( _NAME, _VERMOD, _VER ),
+
+	# TODO: merge these regexes: ([{ )]}]
+
+	# 'R (>= 2.15)', 'R(>=2.15)' etc.
+	'^%s\s*\(%s?\s*%s\s*\)$' % ( _NAME, _VERMOD, _VER ),
+
+	# 'R [>= 2.15]', 'R[>=2.15]' etc.
+	'^%s\s*\[%s?\s*%s\s*\]$' % ( _NAME, _VERMOD, _VER ),
+
+
+	# 'R {>= 2.15}', 'R{>=2.15}' etc.
+	'^%s\s*\{%s?\s*%s\s*\}$' % ( _NAME, _VERMOD, _VER ),
+) )
+
+VERSION_REGEX = frozenset ( re.compile ( regex ) for regex in V_REGEX_STR )
+
+FIXVERSION_REGEX = re.compile ( '[_\-]' )
+
+TRY_ALL_REGEXES = False
 
 
 class DepEnv ( object ):
@@ -22,11 +54,36 @@ class DepEnv ( object ):
 		self.status      = DepEnv.STATUS_UNDONE
 		self.resolved_by = None
 
+		self.try_all_regexes = TRY_ALL_REGEXES
+
+		self._depsplit()
+
+
 		# TODO: analyze dep_str:
 		#   extract dep name, dep version, useless comments,...
 
 	# --- end of __init__ (...) ---
 
+	def _depsplit ( self ):
+		result = list()
+		for r in VERSION_REGEX:
+			m = r.match ( self.dep_str_low )
+			if m is not None:
+
+				result.append ( dict (
+					name             = m.group ( 'name' ),
+					version_modifier = m.group ( 'vmod' ),
+					version          = FIXVERSION_REGEX.sub (
+												'.', m.group ( 'ver' )
+											)
+				) )
+
+				if not self.try_all_regexes: break
+
+		if result:
+			self.fuzzy = tuple ( result )
+	# --- end of _depsplit (...) ---
+
 	def set_resolved ( self, resolved_by, append=False ):
 		"""Marks this DepEnv as resolved with resolved_by as corresponding
 		portage package.



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-29 22:48 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-29 22:48 UTC (permalink / raw
  To: gentoo-commits

commit:     f4fe2f55298624cd522b554aa57b505591a4138a
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Fri Jun 29 22:36:33 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Fri Jun 29 22:36:33 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=f4fe2f55

dependency resolution: 'fuzzy' rules

These extended simple rules match various dependency strings
with a single rule, e.g. a fuzzy "R" rule matches
"R (>= 2.10)" as ">=dev-lang/R-2.10" and "R <2.15" as "<dev-lang/R-2.15".

Also added easier syntax for R package dependencies (sound -> sci-R/sound)
in the dep rule file(s), which can now be written as 'sound' instead of
a category-hardcoded 'sci-R/sound :: sound' statement.

	new file:   roverlay/depres/abstractsimpledeprule.py
	modified:   roverlay/depres/deprule.py
	modified:   roverlay/depres/simpledeprule.py

---
 roverlay/depres/abstractsimpledeprule.py |  204 ++++++++++++++++++++++++
 roverlay/depres/deprule.py               |   11 +-
 roverlay/depres/simpledeprule.py         |  254 ++++++++++++++----------------
 3 files changed, 325 insertions(+), 144 deletions(-)

diff --git a/roverlay/depres/abstractsimpledeprule.py b/roverlay/depres/abstractsimpledeprule.py
new file mode 100644
index 0000000..b03ce63
--- /dev/null
+++ b/roverlay/depres/abstractsimpledeprule.py
@@ -0,0 +1,204 @@
+import logging
+
+from roverlay.depres import deprule
+
+TMP_LOGGER = logging.getLogger ('simpledeps')
+
+class SimpleRule ( deprule.DependencyRule ):
+	"""A dependency rule that represents an ignored package in portage."""
+
+	def __init__ ( self,
+		dep_str=None, priority=50, resolving_package=None,
+		logger_name='simple_rule'
+	):
+		"""Initializes a SimpleIgnoreDependencyRule.
+
+		arguments:
+		* dep_str -- a dependency string that this rule is able to resolve
+		* priority -- priority of this rule
+		"""
+		super ( SimpleRule, self ) . __init__ ( priority )
+		self.dep_alias = list()
+
+		self.logger = TMP_LOGGER.getChild ( logger_name )
+
+		self.resolving_package = resolving_package
+
+		self.prepare_lowercase_alias = True
+
+		if not dep_str is None:
+			self.dep_alias.append ( dep_str )
+
+		self.logger.debug ( "new rule (%s) for %s" % ( self.__class__.__name__, self.resolving_package ) )
+
+	# --- end of __init__ (...) ---
+
+	def done_reading ( self ):
+		self.dep_alias = frozenset ( self.dep_alias )
+		if self.prepare_lowercase_alias:
+			self.dep_alias_low = frozenset ( x.lower() for x in self.dep_alias )
+
+	def add_resolved ( self, dep_str ):
+		"""Adds an dependency string that should be matched by this rule.
+
+		arguments:
+		* dep_str --
+		"""
+		self.dep_alias.append ( dep_str )
+	# --- end of add_resolved (...) ---
+
+	def _find ( self, dep_str, lowercase ):
+		if lowercase:
+			if hasattr ( self, 'dep_alias_low' ):
+				if dep_str in self.dep_alias_low:
+					return True
+			else:
+				if dep_str in ( alias.lower() for alias in self.dep_alias ):
+					return True
+
+		return dep_str in self.dep_alias
+	# --- end of _find (...) ---
+
+	def matches ( self, dep_env, lowercase=True ):
+		"""Returns True if this rule matches the given DepEnv, else False.
+
+		arguments:
+		* dep_env --
+		* lowercase -- if True: be case-insensitive when iterating over all
+		               stored dep_strings
+		"""
+
+		if self._find (
+			dep_env.dep_str_low if lowercase else dep_env.dep_str, lowercase
+		):
+			self.logger.debug (
+				"matches %s with score %i and priority %i."
+					% ( dep_env.dep_str, self.max_score, self.priority )
+			)
+			return ( self.max_score, self.resolving_package )
+
+		return None
+	# --- end of matches (...) ---
+
+	def export_rule ( self, resolving_to=None ):
+		"""Returns this rule as a list of text lines that can be written into
+		a file.
+		An empty list will be returned if dep_alias has zero length.
+
+		arguments:
+		* resolving_to -- portage package that the exported rule should
+		                  resolve to, defaults to self.resolving_package or
+		                  an ignore keyword such as '!'.
+		"""
+
+		alias_count = len ( self.dep_alias )
+
+		retlist = []
+
+		if alias_count:
+			if resolving_to is None:
+				if hasattr ( self, 'resolving_package'):
+					resolving_package = self.resolving_package
+				else:
+					resolving_package = '!'
+			else:
+				resolving_package = resolving_to
+
+			# todo hardcoded rule format here
+			if alias_count > 1:
+
+				retlist = [ resolving_package + ' {\n' ] + \
+					[ "\t%s\n" % alias for alias in self.dep_alias ] + \
+					[ '}\n' ]
+			else:
+				retlist = [
+					"%s :: %s\n" % ( resolving_package, self.dep_alias [0] )
+				]
+
+		# -- if
+
+		return retlist
+	# --- end of export_rule (...) ---
+
+class FuzzySimpleRule ( SimpleRule ):
+
+	def __init__ ( self, *args, **kw ):
+		super ( FuzzySimpleRule, self ) . __init__ ( *args, **kw )
+		self.prepare_lowercase_alias = True
+
+		# 0 : version with modifier, 1 : version w/o mod, 2 : name only, 3 : std
+		self.fuzzy_score = ( 1250, 1000, 750, 500 )
+		self.max_score   = max ( self.fuzzy_score )
+
+	def matches ( self, dep_env ):
+		if self._find ( dep_env.dep_str_low, lowercase=True ):
+			# non-fuzzy match
+			self.logger.debug (
+				"matches %s with score %i and priority %i."
+					% ( dep_env.dep_str, self.max_score, self.priority )
+			)
+			return ( self.fuzzy_score[3], self.resolving_package )
+
+		elif hasattr ( dep_env, 'fuzzy' ):
+			for fuzzy in dep_env.fuzzy:
+				if 'name' in fuzzy:
+					if self._find ( fuzzy ['name'], lowercase=True ):
+						# fuzzy match found
+
+						if self.resolving_package is None:
+							# ignore rule
+							res   = None
+							score = self.fuzzy_score [2]
+
+
+						elif 'version' in fuzzy:
+
+							ver_pkg = '-'.join ( (
+								self.resolving_package, fuzzy ['version']
+							) )
+
+							if 'version_modifier' in fuzzy:
+								vmod = fuzzy ['version_modifier']
+								if '!' in vmod:
+									# package matches, but specific version is forbidden
+									# ( !<package>-<specific verion> <package> )
+									res = '( !=%s %s )' % (
+										ver_pkg,
+										self.resolving_package
+									)
+
+								else:
+									# std vmod: >=, <=, =, <, >
+									res = vmod + ver_pkg
+
+								score = self.fuzzy_score[0]
+
+							else:
+								# version required, but no modifier: defaults to '>='
+
+								res   = '>=' + ver_pkg
+								score = self.fuzzy_score[1]
+
+						else:
+							# substring match
+							#  currently not possible (see DepEnv's regexes)
+							score = fuzzy[2]
+							res   = self.resolving_package
+						# --- if resolving... elif version ... else
+
+
+						self.logger.debug (
+							"fuzzy-match: %s resolved as '%s' with score=%i."
+								% ( dep_env.dep_str, res, score )
+						)
+						return ( score, res )
+					# --- if find (=match found)
+				# --- if name in
+			# --- for fuzzy
+		# --- elif hasattr
+
+		return None
+	# --- end of matches (...) ---
+
+
+

diff --git a/roverlay/depres/deprule.py b/roverlay/depres/deprule.py
index b02213c..72e706d 100644
--- a/roverlay/depres/deprule.py
+++ b/roverlay/depres/deprule.py
@@ -16,8 +16,9 @@ class DependencyRule ( object ):
 	# --- end of __init__ (...) ---
 
 	def matches ( self, dep_env ):
-		"""Returns an int > 0 if this rule matches the given DepEnv."""
-		return 0
+		"""Returns a tuple ( score ::= int > 0, matching dep ::= str )
+		if this rule matches the given DepEnv, else None"""
+		return None
 	# --- end of matches (...) ---
 
 # --- end of DependencyRule ---
@@ -97,12 +98,12 @@ class DependencyRulePool ( object ):
 				order.reverse()
 
 			for index in order:
-				score = self.rules [index].matches ( dep_env )
-				if score:
+				result = self.rules [index].matches ( dep_env )
+				if result is not None and result [0] > 0:
 					if skipped < skip_matches:
 						skipped += 1
 					else:
-						return ( score, self.rules [index].get_dep () )
+						return result
 
 
 		return None

diff --git a/roverlay/depres/simpledeprule.py b/roverlay/depres/simpledeprule.py
index ee40045..ce4ba10 100644
--- a/roverlay/depres/simpledeprule.py
+++ b/roverlay/depres/simpledeprule.py
@@ -5,122 +5,23 @@
 import re
 import logging
 
+from roverlay import config
 from roverlay.depres import deprule
+from roverlay.depres.abstractsimpledeprule import SimpleRule, FuzzySimpleRule
 
 TMP_LOGGER = logging.getLogger ('simpledeps')
 
-class SimpleIgnoreDependencyRule ( deprule.DependencyRule ):
-	"""A dependency rule that represents an ignored package in portage."""
-
-	def __init__ ( self, dep_str=None, priority=50 ):
-		"""Initializes a SimpleIgnoreDependencyRule.
-
-		arguments:
-		* dep_str -- a dependency string that this rule is able to resolve
-		* priority -- priority of this rule
-		"""
-		super ( SimpleIgnoreDependencyRule, self ) . __init__ ( priority )
-		self.dep_alias = set ()
-
-		self.logger = TMP_LOGGER.getChild ( 'IGNORE_DEPS' )
-
-		if not dep_str is None:
-			self.dep_alias.add ( dep_str )
-
-	# --- end of __init__ (...) ---
-
-	def add_resolved ( self, dep_str ):
-		"""Adds an dependency string that should be matched by this rule.
-
-		arguments:
-		* dep_str --
-		"""
-		self.dep_alias.add ( dep_str )
-	# --- end of add_resolved (...) ---
-
-	def matches ( self, dep_env, lowercase=True ):
-		"""Returns True if this rule matches the given DepEnv, else False.
-
-		arguments:
-		* dep_env --
-		* lowercase -- if True: be case-insensitive when iterating over all
-		               stored dep_strings
-		"""
-
-		def logmatch ( score=self.max_score ):
-			"""Wrapper function that logs a successful match and
-			returns its score.
-
-			arguments:
-			* score -- score of this match, defaults to self.max_score
-			"""
-
-			self.logger.debug ( "matches %s with score %i and priority %i." %
-				( dep_env.dep_str, score, self.priority )
-			)
-			return score
-		# --- end of logmatch (...) ---
-
-		if lowercase:
-			#lower_dep_str = dep_env.dep_str.lower()
-			for alias in self.dep_alias:
-				if alias.lower() == dep_env.dep_str_low:
-					return logmatch ()
-		elif dep_env.dep_str in self.dep_alias:
-			return logmatch ()
-
-		return 0
-	# --- end of matches (...) ---
-
-	def get_dep ( self ):
-		"""Returns the textual portage package representation of this rule,
-		which is None 'cause this is an ignored dependency.
-		"""
-		return None
-	# --- end of get_dep (...) ---
-
-	def export_rule ( self, resolving_to=None ):
-		"""Returns this rule as a list of text lines that can be written into
-		a file.
-		An empty list will be returned if dep_alias has zero length.
-
-		arguments:
-		* resolving_to -- portage package that the exported rule should
-		                  resolve to, defaults to self.resolving_package or
-		                  an ignore keyword such as '!'.
-		"""
-
-		alias_count = len ( self.dep_alias )
-
-		retlist = []
-
-		if alias_count:
-			if resolving_to is None:
-				if hasattr ( self, 'resolving_package'):
-					resolving_package = self.resolving_package
-				else:
-					resolving_package = '!'
-			else:
-				resolving_package = resolving_to
-
-			# todo hardcoded rule format here
-			if alias_count > 1:
-
-				retlist = [ resolving_package + ' {\n' ] + \
-					[ "\t%s\n" % alias for alias in self.dep_alias ] + \
-					[ '}\n' ]
-			else:
-				retlist = [
-					"%s :: %s\n" % ( resolving_package, self.dep_alias [0] )
-				]
-
-		# -- if
-
-		return retlist
-	# --- end of export_rule (...) ---
+class SimpleIgnoreDependencyRule ( SimpleRule ):
 
+	def __init__ ( self, dep_str=None, priority=50, resolving_package=None ):
+		super ( SimpleIgnoreDependencyRule, self ) . __init__ (
+			dep_str=dep_str,
+			priority=priority,
+			resolving_package=None,
+			logger_name = 'IGNORE_DEPS',
+		)
 
-class SimpleDependencyRule ( SimpleIgnoreDependencyRule ):
+class SimpleDependencyRule ( SimpleRule ):
 
 	def __init__ ( self, resolving_package, dep_str=None, priority=70 ):
 		"""Initializes a SimpleDependencyRule. This is
@@ -132,21 +33,32 @@ class SimpleDependencyRule ( SimpleIgnoreDependencyRule ):
 		* priority --
 		"""
 		super ( SimpleDependencyRule, self ) . __init__ (
-			dep_str=dep_str, priority=priority
+			dep_str=dep_str,
+			priority=priority,
+			resolving_package=resolving_package,
+			logger_name=resolving_package
 		)
 
-		self.resolving_package = resolving_package
+	# --- end of __init__ (...) ---
 
-		self.logger = TMP_LOGGER.getChild ( resolving_package )
+class SimpleFuzzyIgnoreDependencyRule ( FuzzySimpleRule ):
 
-	# --- end of __init__ (...) ---
+	def __init__ ( self, dep_str=None, priority=51, resolving_package=None ):
+		super ( SimpleFuzzyIgnoreDependencyRule, self ) . __init__ (
+			dep_str=dep_str,
+			priority=priority,
+			resolving_package=None,
+			logger_name = 'FUZZY.IGNORE_DEPS',
+		)
 
-	def get_dep ( self ):
-		"""Returns the textual portage package representation of this rule,
-		e.g. 'dev-lang/R'.
-		"""
-		return self.resolving_package
-	# --- end of get_dep (...) ---
+class SimpleFuzzyDependencyRule ( FuzzySimpleRule ):
+	def __init__ ( self, resolving_package, dep_str=None, priority=71 ):
+		super ( SimpleFuzzyDependencyRule, self ) . __init__ (
+			dep_str=dep_str,
+			priority=priority,
+			resolving_package=resolving_package,
+			logger_name = 'FUZZY.' + resolving_package,
+		)
 
 
 class SimpleDependencyRulePool ( deprule.DependencyRulePool ):
@@ -175,7 +87,7 @@ class SimpleDependencyRulePool ( deprule.DependencyRulePool ):
 		arguments:
 		* rule --
 		"""
-		if isinstance ( rule, SimpleIgnoreDependencyRule ):
+		if isinstance ( rule, SimpleRule ):
 			self.rules.append ( rule )
 		else:
 			raise Exception ( "bad usage (simple dependency rule expected)." )
@@ -220,8 +132,13 @@ class SimpleDependencyRuleReader ( object ):
 	multiline_start    = '{'
 	multiline_stop     = '}'
 	comment_chars      = "#;"
+
 	# todo: const/config?
-	package_ignore     = [ '!' ]
+	package_ignore = '!'
+	fuzzy          = '~'
+	fuzzy_ignore   = '%'
+
+	BREAK_PARSING  = frozenset (( '#! NOPARSE', '#! BREAK' ))
 
 
 	def __init__ ( self ):
@@ -229,6 +146,12 @@ class SimpleDependencyRuleReader ( object ):
 		pass
 	# --- end of __init__  (...) ---
 
+
+	def _make_rule ( self, rule_str ):
+		CLS = self.__class__
+
+
+
 	def read_file ( self, filepath ):
 		"""Reads a file that contains simple dependency rules
 		(SimpleIgnoreDependencyRules/SimpleDependencyRules).
@@ -244,6 +167,8 @@ class SimpleDependencyRuleReader ( object ):
 			logging.debug ( "Reading simple dependency rule file %s." % filepath )
 			fh = open ( filepath, 'r' )
 
+			CLS = self.__class__
+
 			# the list of read rules
 			rules = list ()
 
@@ -258,47 +183,95 @@ class SimpleDependencyRuleReader ( object ):
 					# empty
 					pass
 
+				elif line in CLS.BREAK_PARSING:
+					# stop reading here
+					break
+
 				elif not next_rule is None:
 					# in a multiline rule
 
-					if line [0] == SimpleDependencyRuleReader.multiline_stop:
+					if line [0] == CLS.multiline_stop:
 						# end of a multiline rule,
 						#  add rule to rules and set next_rule to None
+						next_rule.done_reading()
 						rules.append ( next_rule )
 						next_rule = None
 					else:
 						# new resolved str
 						next_rule.add_resolved ( line )
 
-				elif line [0] in SimpleDependencyRuleReader.comment_chars:
+				elif line [0] in CLS.comment_chars:
 					# comment
 					#  it is intented that multi line rules cannot contain comments
 					pass
 
-				elif line [-1] == SimpleDependencyRuleReader.multiline_start:
+				elif line [-1] == CLS.multiline_start:
 					# start of a multiline rule
 					portpkg = line [:-1].rstrip()
-					if portpkg in SimpleDependencyRuleReader.package_ignore:
+
+					if portpkg == CLS.fuzzy_ignore:
+						next_rule = SimpleFuzzyIgnoreDependencyRule ( None )
+					elif portpkg == CLS.fuzzy:
+						next_rule = SimpleFuzzyDependencyRule ( portpkg[1:], None )
+					elif portpkg == CLS.package_ignore:
 						next_rule = SimpleIgnoreDependencyRule ( None, 60 )
 					else:
 						next_rule = SimpleDependencyRule ( portpkg, None, 70 )
 
 				else:
-					# single line rule?
-					rule_str = \
-						SimpleDependencyRuleReader.one_line_separator.split (line, 1)
+					# single line rule, either selfdep,
+					#  e.g. '~zoo' -> fuzzy sci-R/zoo :: zoo
+					#  or normal rule 'dev-lang/R :: R'
+					# selfdeps are always single line statements (!)
+					rule_str = CLS.one_line_separator.split (line, 1)
+
+					new_rule   = None
+					rule_class = None
+					resolving  = None
+
+					first_char = rule_str [0][0] if len ( rule_str [0] ) else ''
+
+					if first_char == CLS.fuzzy:
+						rule_class = SimpleFuzzyDependencyRule
+						resolving  = rule_str [0] [1:]
+
+					elif rule_str [0] == CLS.fuzzy_ignore:
+						rule_class = SimpleFuzzyIgnoreDependencyRule
+						resolving  = None
+
+					elif rule_str [0] == CLS.package_ignore:
+						rule_class = SimpleIgnoreDependencyRule
+
+					else:
+						rule_class = SimpleDependencyRule
+						resolving  = rule_str [0]
 
 					if len ( rule_str ) == 2:
-						# is a single line rule
-
-						if rule_str [0] in SimpleDependencyRuleReader.package_ignore:
-							rules.append (
-								SimpleIgnoreDependencyRule ( rule_str [1], 40 )
-							)
-						else:
-							rules.append (
-								SimpleDependencyRule ( rule_str [0], rule_str [1], 50 )
-							)
+						# normal rule
+						new_rule = rule_class (
+							resolving_package=resolving,
+							dep_str=rule_str [1]
+						)
+
+					elif resolving is not None:
+						# selfdep
+						dep_str = resolving
+						resolving = '/'.join ( (
+							config.get_or_fail ( 'OVERLAY.category' ),
+							resolving
+						) )
+						new_rule = rule_class (
+							resolving_package=resolving,
+							dep_str=dep_str
+						)
+
+					# else: error
+
+
+					if new_rule:
+						new_rule.done_reading()
+						rules.append ( new_rule )
+
 					else:
 						logging.error (
 							"In %s, line %i : cannot use this line." %
@@ -308,6 +281,9 @@ class SimpleDependencyRuleReader ( object ):
 
 			if fh: fh.close ()
 
+			if next_rule is not None:
+				logging.warning ( "Multi line rule does not end at EOF - ignored" )
+
 			logging.info (
 				"%s: read %i dependency rules (in %i lines)" %
 					( filepath, len ( rules ), lineno )



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-28 13:29 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-28 13:29 UTC (permalink / raw
  To: gentoo-commits

commit:     7d3d0d7ffc8b2c24198da55dfc4812b90c48e4f9
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Thu Jun 28 13:25:35 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Thu Jun 28 13:25:35 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=7d3d0d7f

depresolver: ensure that channel ids are unique

	modified:   roverlay/depres/depresolver.py

---
 roverlay/depres/depresolver.py |   53 +++++++++++++++++++++++++++++++--------
 1 files changed, 42 insertions(+), 11 deletions(-)

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index 3b0e481..26040c0 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -22,18 +22,20 @@ from roverlay.depres import simpledeprule, communication, events
 # unresolvable deps in a set for should-be faster lookups
 USING_DEPRES_CACHE = True
 
+# if True: verify that channels are unique for a resolver instance
+SAFE_CHANNEL_IDS = True
+
 class DependencyResolver ( object ):
 	"""Main object for dependency resolution."""
 
-	LOGGER = logging.getLogger ( "DependencyResolver" )
 
-	NUMTHREADS = config.get ( "DEPRES.jobcount", 2 )
+	NUMTHREADS = config.get ( "DEPRES.jobcount", 0 )
 
 	def __init__ ( self ):
 		"""Initializes a DependencyResolver."""
 
 		# these loggers are temporary helpers
-		self.logger              = DependencyResolver.LOGGER
+		self.logger              = logging.getLogger ( self.__class__.__name__ )
 		self.logger_unresolvable = self.logger.getChild ( "UNRESOLVABLE" )
 		self.logger_resolved     = self.logger.getChild ( "RESOLVED" )
 
@@ -54,7 +56,6 @@ class DependencyResolver ( object ):
 		self.listeners = list ()
 
 		# fifo queue for dep resolution
-		#  (threads: could use queue.Queue instead of collections.deque)
 		self._depqueue = queue.Queue()
 
 		# the queue of failed dep resolutions
@@ -76,6 +77,13 @@ class DependencyResolver ( object ):
 
 		# list of rule pools that have been created from reading files
 		self.static_rule_pools = list ()
+
+
+		if SAFE_CHANNEL_IDS:
+			# this lock is used in register_channel
+			self._chanlock       = threading.Lock()
+			# this stores all channel ids ever registered to this resolver
+			self.all_channel_ids = set()
 	# --- end of __init__ (...) ---
 
 	def _sort ( self ):
@@ -184,15 +192,34 @@ class DependencyResolver ( object ):
 
 		returns: channel
 		 """
-		if channel in self._depqueue_done:
-			raise Exception ( "channel is already registered." )
+		if SAFE_CHANNEL_IDS:
+			try:
+				self._chanlock.acquire()
 
-		# register channel and allocate a queue in depqueue_done
-		self._depqueue_done [channel.ident] = queue.Queue()
+				if channel.ident in self.all_channel_ids:
+					raise Exception ( "channel id reused!" )
+				else:
+					self.all_channel_ids.add ( channel.ident )
 
-		channel.set_resolver (
-			self, channel_queue=self._depqueue_done [channel.ident]
-		)
+				# register channel and allocate a queue in depqueue_done
+				self._depqueue_done [channel.ident] = queue.Queue()
+
+				channel.set_resolver (
+					self, channel_queue=self._depqueue_done [channel.ident]
+				)
+
+			finally:
+				self._chanlock.release()
+		else:
+			if channel.ident in self._depqueue_done:
+				raise Exception ( "channel is already registered." )
+
+			# register channel and allocate a queue in depqueue_done
+			self._depqueue_done [channel.ident] = queue.Queue()
+
+			channel.set_resolver (
+				self, channel_queue=self._depqueue_done [channel.ident]
+			)
 
 		return channel
 	# --- end of register_channel (...) ---
@@ -408,4 +435,8 @@ class DependencyResolver ( object ):
 			self._mainthread.join()
 		for lis in self.listeners: lis.close()
 		del self.listeners
+		if SAFE_CHANNEL_IDS:
+			self.logger.debug (
+				"%i channels were in use." % len ( self.all_channel_ids )
+			)
 	# --- end of close (...) ---



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-28 13:29 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-28 13:29 UTC (permalink / raw
  To: gentoo-commits

commit:     8d858c0d38e9425bc38136ed2e3249b7db18c71e
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Thu Jun 28 13:24:34 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Thu Jun 28 13:24:34 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=8d858c0d

depres channels: using int/long as id

	modified:   roverlay/depres/communication.py

---
 roverlay/depres/communication.py |   17 +++++++++++++++--
 1 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/roverlay/depres/communication.py b/roverlay/depres/communication.py
index 943e2ca..14d6193 100644
--- a/roverlay/depres/communication.py
+++ b/roverlay/depres/communication.py
@@ -2,7 +2,18 @@
 # Copyright 2006-2012 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
-import uuid
+import threading
+import sys
+
+def channel_counter ():
+	lock    = threading.Lock()
+	last_id = long ( -1 ) if sys.version_info < ( 3, ) else int ( -1 )
+
+	while True:
+		with lock:
+			last_id += 1
+			yield last_id
+
 
 class DependencyResolverListener ( object ):
 
@@ -48,6 +59,8 @@ class DependencyResolverListener ( object ):
 
 class DependencyResolverChannel ( object ):
 
+	id_generator = channel_counter()
+
 	def __init__ ( self, main_resolver ):
 		"""Initializes a DependencyResolverChannel which can be used to
 		communicate with the dep resolver.
@@ -60,7 +73,7 @@ class DependencyResolverChannel ( object ):
 		#super ( DependencyResolverChannel, self ) . __init__ ()
 		# channel identifiers must be unique even when the channel has been
 		# deleted (id does not guarantee that)
-		self.ident          = uuid.uuid4()
+		self.ident          = next ( DependencyResolverChannel.id_generator )
 		self._depres_master = main_resolver
 	# --- end of __init__ (...) ---
 



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-27 14:46 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-27 14:46 UTC (permalink / raw
  To: gentoo-commits

commit:     9832cfd9e91f47e05e8112b22097b419aad66c10
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed Jun 27 14:41:27 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed Jun 27 14:41:27 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=9832cfd9

SetFileListener in depres

* listen to events and stores the resulting string in a set
  that is written to a file on close

	modified:   roverlay/depres/listeners.py

---
 roverlay/depres/listeners.py |   55 +++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 54 insertions(+), 1 deletions(-)

diff --git a/roverlay/depres/listeners.py b/roverlay/depres/listeners.py
index 0c123c0..0815036 100644
--- a/roverlay/depres/listeners.py
+++ b/roverlay/depres/listeners.py
@@ -26,7 +26,7 @@ class FileListener ( DependencyResolverListener ):
 		self._file = _file
 
 		if self._file is None:
-			raise Exception ("...")
+			raise Exception ( "no file assigned" )
 	# --- end of __init__ (...) ---
 
 	def _event ( self, event_type, to_write ):
@@ -43,6 +43,45 @@ class FileListener ( DependencyResolverListener ):
 	# --- end of close (...) ---
 
 
+class SetFileListener ( DependencyResolverListener ):
+	def __init__ ( self, _file, listen_mask ):
+		super ( SetFileListener, self ) . __init__ ()
+
+		self._buffer = set()
+
+		self.mask  = listen_mask
+		self._file = _file
+
+		if self._file is None:
+			raise Exception ( "no file assigned" )
+	# --- end of __init__ (...) ---
+
+	def _event ( self, event_type, to_add ):
+		if self.mask & event_type:
+			self._buffer.add ( to_add )
+	# --- end of _event (...) ---
+
+	def write ( self, sort_entries=True ):
+		try:
+			fh = open ( self._file, 'w' )
+
+			if sort_entries:
+				fh.write ( '\n'.join ( sorted ( self._buffer ) ) )
+			else:
+				fh.write ( '\n'.join ( self._buffer ) )
+
+			fh.write ( '\n' )
+
+			fh.close()
+		finally:
+			if 'fh' in locals(): fh.close()
+	# --- end of write (...) ---
+
+	def close ( self ):
+		self.write()
+	# --- end of close (...) ---
+
+
 class ResolvedFileListener ( FileListener ):
 	"""A FileListener that listens to 'dependency resolved' events."""
 
@@ -71,3 +110,17 @@ class UnresolvableFileListener ( FileListener ):
 		self._event ( event_type, dep_env.dep_str )
 	# --- end of notify (...) ---
 
+
+class UnresolvableSetFileListener ( SetFileListener ):
+	"""A SetFileListener that listens to 'dependency unresolvable' events."""
+
+	def __init__ ( self, _file ):
+		super ( UnresolvableSetFileListener, self ) . __init__ (
+			_file, events.DEPRES_EVENTS ['UNRESOLVABLE']
+		)
+	# --- end of __init__ (...) ---
+
+	def notify ( self, event_type, dep_env=None, pkg_env=None, **extra ):
+		#self._event ( event_type, dep_env.dep_str_low )
+		self._event ( event_type, dep_env.dep_str )
+	# --- end of notify (...) ---



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-20 19:03 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-20 19:03 UTC (permalink / raw
  To: gentoo-commits

commit:     8a115d6934f9a67eb4191f4ac7e0a7728ac4fad1
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed Jun 20 10:34:04 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed Jun 20 10:34:04 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=8a115d69

fix satisfiable check in depres, EbuildJobChannel

	modified:   depres/channels.py

---
 roverlay/depres/channels.py |   17 +++++++----------
 1 files changed, 7 insertions(+), 10 deletions(-)

diff --git a/roverlay/depres/channels.py b/roverlay/depres/channels.py
index 570d032..765d241 100644
--- a/roverlay/depres/channels.py
+++ b/roverlay/depres/channels.py
@@ -159,16 +159,12 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 		else None.
 		"""
 
-
-
-
-
 		# using a set allows easy difference() operations between
 		# DEPEND/RDEPEND/.. later, seewave requires sci-libs/fftw
 		# in both DEPEND and RDEPEND for example
 		dep_collected = set()
+		satisfiable   = True
 
-		satisfiable = True
 
 		def handle_queue_item ( dep_env ):
 			self._depdone += 1
@@ -176,11 +172,12 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 				### and dep_env in self.dep_env_list
 				# successfully resolved
 				dep_collected.add ( dep_env.get_result() [1] )
+				self._depres_queue.task_done()
+				return True
 			else:
 				# failed
-				satisfiable = False
-
-			self._depres_queue.task_done()
+				self._depres_queue.task_done()
+				return False
 		# --- end of handle_queue_item (...) ---
 
 		while self._depdone < len ( self.dep_env_list ) and satisfiable:
@@ -188,10 +185,10 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 			self._depres_master.start()
 
 			# wait for one result at least
-			handle_queue_item ( self._depres_queue.get() )
+			satisfiable = handle_queue_item ( self._depres_queue.get() )
 
 			while not self._depres_queue.empty() and satisfiable:
-				handle_queue_item ( self._depres_queue.get_nowait() )
+				satisfiable = handle_queue_item ( self._depres_queue.get_nowait() )
 		# --- end while
 
 		if satisfiable:



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-13 16:34 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-13 16:34 UTC (permalink / raw
  To: gentoo-commits

commit:     6fe656141b6dcf2795e0aeeb94c1aa119acd6a77
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed Jun 13 08:04:28 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed Jun 13 08:04:28 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=6fe65614

fix indent
	modified:   roverlay/depres/depresolver.py

---
 roverlay/depres/depresolver.py |    6 +++---
 1 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index 2cf9c54..cb70c14 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -369,9 +369,9 @@ class DependencyResolver ( object ):
 				else:
 					self._depqueue_failed.put ( to_resolve )
 
-				if USING_DEPRES_CACHE:
-					# does not work when adding new rules is possible
-					self._dep_unresolvable.add ( dep_env.dep_str_low )
+					if USING_DEPRES_CACHE:
+						# does not work when adding new rules is possible
+						self._dep_unresolvable.add ( dep_env.dep_str_low )
 
 				"""
 				## only useful if new rules can be created



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-13  7:45 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-13  7:45 UTC (permalink / raw
  To: gentoo-commits

commit:     fdbdc009d664b09278104a1207f81f5dfd220f19
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed Jun 13 07:44:57 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed Jun 13 07:44:57 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=fdbdc009

dep resolver: make the result cache configurable

	modified:   roverlay/depres/depresolver.py

---
 roverlay/depres/depresolver.py |   26 ++++++++++++++++++--------
 1 files changed, 18 insertions(+), 8 deletions(-)

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index 299d3c6..2cf9c54 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -17,6 +17,11 @@ from roverlay.depres import simpledeprule, communication, events
 
 #from roverlay.depres.depenv import DepEnv (implicit)
 
+
+# if false: do not using the "negative" result caching which stores
+# unresolvable deps in a set for should-be faster lookups
+USING_DEPRES_CACHE = True
+
 class DependencyResolver ( object ):
 	"""Main object for dependency resolution."""
 
@@ -60,7 +65,8 @@ class DependencyResolver ( object ):
 		# the 'negative' result cache, stores unresolvable deps
 		# has to be (selectively?) cleared when
 		# new dep rule found / new rulepool etc.
-		self._dep_unresolvable = set ()
+		if USING_DEPRES_CACHE:
+			self._dep_unresolvable = set ()
 
 		# map: channel identifier -> queue of done deps (resolved/unresolvable)
 		# this serves two purposes:
@@ -88,7 +94,8 @@ class DependencyResolver ( object ):
 		* pool_type -- ignored.
 		"""
 		self.static_rule_pools.append ( rulepool )
-		self._dep_unresolvable.clear()
+		if USING_DEPRES_CACHE:
+			self._dep_unresolvable.clear()
 		self._sort()
 	# --- end of add_rulepool (...) ---
 
@@ -339,9 +346,10 @@ class DependencyResolver ( object ):
 				# TODO:
 				#  (threading: could search the pools in parallel)
 
-#				if dep_env.dep_str_low in self._dep_unresolvable:
-#					# cannot resolve
-#					is_resolved = 1
+				if USING_DEPRES_CACHE:
+					if dep_env.dep_str_low in self._dep_unresolvable:
+						# cannot resolve
+						is_resolved = 1
 
 				if is_resolved == 0:
 					# search for a match in the rule pools
@@ -361,15 +369,17 @@ class DependencyResolver ( object ):
 				else:
 					self._depqueue_failed.put ( to_resolve )
 
-#					# does not work when adding new rules is possible
-#					self._dep_unresolvable.add ( dep_env.dep_str_low )
+				if USING_DEPRES_CACHE:
+					# does not work when adding new rules is possible
+					self._dep_unresolvable.add ( dep_env.dep_str_low )
 
 				"""
 				## only useful if new rules can be created
 				# new rule found, requeue all previously failed dependency searches
 				if have_new_rule:
 					self._queue_previously_failed
-					self._dep_unresolvable.clear() #?
+					if USING_DEPRES_CACHE:
+						self._dep_unresolvable.clear() #?
 				"""
 			# --- end if channel_id in self._depqueue_done
 



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-13  7:45 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-13  7:45 UTC (permalink / raw
  To: gentoo-commits

commit:     f414aac1da097573250886740ead2868145a4a65
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed Jun 13 07:44:06 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed Jun 13 07:44:06 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=f414aac1

remove debug print statement
	modified:   roverlay/depres/channels.py

---
 roverlay/depres/channels.py |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/roverlay/depres/channels.py b/roverlay/depres/channels.py
index 34cf0ba..570d032 100644
--- a/roverlay/depres/channels.py
+++ b/roverlay/depres/channels.py
@@ -187,8 +187,6 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 			# tell the resolver to start
 			self._depres_master.start()
 
-			self.logger.critical ( "WAITING..." )
-
 			# wait for one result at least
 			handle_queue_item ( self._depres_queue.get() )
 



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-12 17:17 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-12 17:17 UTC (permalink / raw
  To: gentoo-commits

commit:     adb14edc11a41eea30f1897e807c0cedfb80aa99
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Tue Jun 12 17:07:23 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Tue Jun 12 17:07:23 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=adb14edc

dependency resolution

* added a 'negative' result cache that stores unresolvable deps
* improved threading: channels now use a queue to wait for results from the
  resolver instead of inefficient looping until all deps are done
* remove unused code

	modified:   roverlay/depres/channels.py
	modified:   roverlay/depres/communication.py
	modified:   roverlay/depres/depenv.py
	modified:   roverlay/depres/depresolver.py
	modified:   roverlay/depres/simpledeprule.py

---
 roverlay/depres/channels.py      |  184 +++++++++++++++++---------------
 roverlay/depres/communication.py |    6 +
 roverlay/depres/depenv.py        |    1 +
 roverlay/depres/depresolver.py   |  221 +++++++++++++++++++++-----------------
 roverlay/depres/simpledeprule.py |   40 ++++---
 5 files changed, 248 insertions(+), 204 deletions(-)

diff --git a/roverlay/depres/channels.py b/roverlay/depres/channels.py
index 960b60a..34cf0ba 100644
--- a/roverlay/depres/channels.py
+++ b/roverlay/depres/channels.py
@@ -16,55 +16,61 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 	poll until done and collect the dependencies afterwards.
 
 	Note that this channel has a strict control flow:
-	add deps -> while not done(): "wait" -> if satisfy_request(): collect/lookup
+	  add deps, then satisfy_request(): collect/lookup
 	"""
 
-	def __init__ ( self, main_resolver=None, name=None ):
+	def __init__ ( self, name=None, logger=None ):
 		"""EbuildJobChannel
 
 		arguments:
-		* main_resolver -- reference to the dep resolver,
-		                    optional - dep resolver is automatically assigned
-		                    when calling depresolver.register_channel ( $this ).
 		* name -- name of this channel, optional
+		* logger --
 		"""
-		super ( EbuildJobChannel, self ) . __init__ ( main_resolver )
-		self.dep_env_list    = list ()
-		self._done           = False
+		super ( EbuildJobChannel, self ) . __init__ ( main_resolver=None )
+
+		# this is the number of resolved deps so far, should only be modified
+		# in the join()-method
+		self._depdone = 0
+
+		# set of portage packages (resolved deps)
+		#  this is None unless all deps have been successfully resolved
 		self._collected_deps = None
 
+		# used to communicate with the resolver
+		self._depres_queue   = None
+
+		# the list of dependency lookups assigned to this channel
+		## todo: could remove this list
+		self.dep_env_list = list ()
+
+		_logger = logger if hasattr ( logger, 'log' ) else COMLINK
 		if name:
-			self.name = name
-			self.logger = COMLINK.getChild ( 'EbuildJobChannel.' + name )
+			self.name   = name
+			self.logger = _logger.getChild ( 'channel.' + name )
 		else:
-			self.logger = COMLINK.getChild ( 'EbuildJobChannel' )
+			self.logger = _logger.getChild ( 'channel' )
 	# --- end of __init__ (...) ---
 
-	def get_name ( self ):
-		"""Returns the name of this channel or None if unset."""
-		return self.name if hasattr ( self, 'name' ) else None
-	# --- end of get_name (...) ---
+	def close ( self ):
+		if self._depdone >= 0:
+			# else already closed
+
+			super ( EbuildJobChannel, self ) . close()
+			del self.dep_env_list, self._collected_deps, self._depres_queue
+			del self.logger
+
+			self._depdone = -1
+			self._collected_deps = None
+	# --- end of close (...) ---
 
-	def depcount ( self ):
-		"""Returns the number of dependency lookups assigned to this channel."""
-		if isinstance ( self.dep_env_list, list ):
-			return len ( self.dep_env_list )
+	def set_resolver ( self, resolver, channel_queue, **extra ):
+		""" comment todo """
+		if self._depres_master is None:
+			self._depres_master = resolver
+			self._depres_queue  = channel_queue
 		else:
-			return -1
-	# --- end of depcount (...) ---
-
-	def done ( self ):
-		"""Returns True if the dep resolver has resolved all deps of this
-		channel.
-		Note that there is 'no' way back to adding dependencies when/after this
-		method returns True.
-		"""
-		if not self._done:
-			self._done = self._depres_master.done (
-				self.ident, len ( self.dep_env_list )
-			)
-		return self._done
-	# --- end of done (...) ---
+			raise Exception ( "channel already bound to a resolver." )
+	# --- end of set_resolver (...) ---
 
 	def add_dependency ( self, dep_str ):
 		"""Adds a dependency string that should be looked up.
@@ -76,9 +82,9 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 
 		raises: Exception if this channel is done
 
-		returns: None
+		returns: None (implicit)
 		"""
-		if self._done:
+		if self._depdone:
 			raise Exception (
 				"This channel is 'done', it doesn't accept new dependencies."
 			)
@@ -87,7 +93,6 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 			self.dep_env_list.append ( dep_env )
 			self._depres_master.enqueue ( dep_env, self.ident )
 
-		return None
 	# --- end of add_dependency (...) ---
 
 	def add_dependencies ( self, dep_list ):
@@ -104,39 +109,6 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 		for dep_str in dep_list: self.add_dependency ( dep_str )
 	# --- end of add_dependencies (...) ---
 
-	def satisfy_request ( self, delete_list_if_unresolvable=True ):
-		"""Returns True if all dependencies can be satisfied, else False.
-		It also prepares the list of resolved dependencies.
-
-		arguments:
-		* delete_list_if_unresolvable -- deletes the DepEnv list before
-		                                 returning False, else has no effect
-		"""
-
-		if self._done:
-			if self.dep_env_list is None:
-				return False
-
-			# using a set allows easy difference() operations between
-			# DEPEND/RDEPEND/.. later, seewave requires sci-libs/fftw
-			# in both DEPEND and RDEPEND for example
-			dep_collected = set()
-
-			for dep_env in self.dep_env_list:
-				if dep_env.is_resolved ():
-					dep_collected.add ( dep_env.get_result() [1] )
-				else:
-					if delete_list_if_unresolvable:
-						del self.dep_env_list
-						self.dep_env_list = None
-					return False
-
-			self._collected_deps = dep_collected
-			return True
-
-		return False
-	# --- end of satisfy_request (...) ---
-
 	def lookup ( self, dep_str ):
 		"""Looks up a specific dep str. Use the (faster) collect_dependencies()
 		method for getting all dependencies if order doesn't matter.
@@ -148,6 +120,7 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 
 		raises: Exception if dep_str not in the dep env list.
 		"""
+		# it's no requirement that this channel is done when calling this method
 		for de in self.dep_env_list:
 			if de.dep_str == dep_str:
 				return de.get_result() [1]
@@ -165,7 +138,7 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 
 		raises: Exception
 		"""
-		if self._done:
+		if not self._collected_deps is None:
 			self.logger.debug (
 				"returning collected deps: %s." % self._collected_deps
 			)
@@ -173,25 +146,62 @@ class EbuildJobChannel ( DependencyResolverChannel ):
 		raise Exception ( "cannot do that" )
 	# --- end of collect_dependencies (...) ---
 
-	def trigger_run ( self ):
-		"""Tells the dependency resolver to run.
+	def satisfy_request ( self, close_if_unresolvable=True ):
+		"""Tells to the dependency resolver to run.
+		It blocks until this channel is done, which means that either all
+		deps are resolved or one is unresolvable.
 
-		returns: None (implicit)
-		"""
-		self._depres_master.start()
-	# --- end of trigger_run (...) ---
+		arguments:
+		* close_if_unresolvable -- close the channel if one dep is unresolvable
+		                           this seems reasonable and defaults to True
 
-	def join ( self ):
-		"""Tells to the dependency resolver to run until this channel is done
-		(= all deps are done (either resolved or unresolvable)). Blocks.
+		Returns the list of resolved dependencies if all could be resolved,
+		else None.
 		"""
 
-		self._done = self._depres_master.done (
-			self.ident, len ( self.dep_env_list )
-		)
-		while not self._done:
+
+
+
+
+		# using a set allows easy difference() operations between
+		# DEPEND/RDEPEND/.. later, seewave requires sci-libs/fftw
+		# in both DEPEND and RDEPEND for example
+		dep_collected = set()
+
+		satisfiable = True
+
+		def handle_queue_item ( dep_env ):
+			self._depdone += 1
+			if dep_env.is_resolved():
+				### and dep_env in self.dep_env_list
+				# successfully resolved
+				dep_collected.add ( dep_env.get_result() [1] )
+			else:
+				# failed
+				satisfiable = False
+
+			self._depres_queue.task_done()
+		# --- end of handle_queue_item (...) ---
+
+		while self._depdone < len ( self.dep_env_list ) and satisfiable:
+			# tell the resolver to start
 			self._depres_master.start()
-			self._done = self._depres_master.done (
-				self.ident, len ( self.dep_env_list )
-			)
+
+			self.logger.critical ( "WAITING..." )
+
+			# wait for one result at least
+			handle_queue_item ( self._depres_queue.get() )
+
+			while not self._depres_queue.empty() and satisfiable:
+				handle_queue_item ( self._depres_queue.get_nowait() )
+		# --- end while
+
+		if satisfiable:
+			self._collected_deps = dep_collected
+			return self._collected_deps
+		else:
+			if close_if_unresolvable: self.close()
+			return None
+
+
 	# --- end of join (...) ---

diff --git a/roverlay/depres/communication.py b/roverlay/depres/communication.py
index add075c..33e1e8f 100644
--- a/roverlay/depres/communication.py
+++ b/roverlay/depres/communication.py
@@ -64,9 +64,15 @@ class DependencyResolverChannel ( object ):
 		self._depres_master = main_resolver
 	# --- end of __init__ (...) ---
 
+	def set_resolver ( self, resolver, channel_queue=None, **extra ):
+		"""comment todo."""
+		self._depres_master = resolver
+	# --- end of set_resolver (...) ---
+
 	def close ( self ):
 		"""Closes this channel."""
 		self._depres_master.channel_closed ( self.ident )
+		del self._depres_master
 	# --- end of close (...) ---
 
 	def enabled ( self ):

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
index ad414d5..889af9c 100644
--- a/roverlay/depres/depenv.py
+++ b/roverlay/depres/depenv.py
@@ -18,6 +18,7 @@ class DepEnv ( object ):
 		"""
 		self.ident       = id ( self )
 		self.dep_str     = dep_str
+		self.dep_str_low = dep_str.lower()
 		self.status      = DepEnv.STATUS_UNDONE
 		self.resolved_by = None
 

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index a7f9dfb..299d3c6 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -2,8 +2,6 @@
 # Copyright 2006-2012 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
-# todo depres result cache
-
 import logging
 import threading
 
@@ -14,39 +12,17 @@ except ImportError:
 	import Queue as queue
 
 
-from roverlay               import config
-from roverlay.depres        import simpledeprule, communication, events
-from roverlay.depres.worker import DepResWorker
+from roverlay        import config
+from roverlay.depres import simpledeprule, communication, events
 
 #from roverlay.depres.depenv import DepEnv (implicit)
 
-class PseudoAtomicCounter ( object ):
-	def __init__ ( self, number=0 ):
-		self.nlock   = threading.Lock()
-		self._number = number
-	# --- end of __init__ (...) ---
-
-	def increment_and_get ( self, step=1 ):
-		with self.nlock:
-			self._number += step
-			ret           = self._number
-		return ret
-	# --- end of increment_and_get (...) ---
-
-	def get ( self ): return self._number
-
-	def __ge__ ( self, other_int ): return self._number >= other_int
-	def __gt__ ( self, other_int ): return self._number >  other_int
-	def __le__ ( self, other_int ): return self._number <= other_int
-	def __lt__ ( self, other_int ): return self._number <  other_int
-
-
 class DependencyResolver ( object ):
 	"""Main object for dependency resolution."""
 
 	LOGGER = logging.getLogger ( "DependencyResolver" )
 
-	NUMTHREADS = config.get ( "DEPRES.jobcount", 0 )
+	NUMTHREADS = config.get ( "DEPRES.jobcount", 2 )
 
 	def __init__ ( self ):
 		"""Initializes a DependencyResolver."""
@@ -61,8 +37,13 @@ class DependencyResolver ( object ):
 			'RESOLVED', 'UNRESOLVABLE'
 		)
 
-		self.runlock  = threading.Lock()
-		self._threads = None
+		# this lock tells whether a dep res 'master' thread is running (locked)
+		self.runlock     = threading.Lock()
+		# the dep res main thread
+		self._mainthread = None
+		# the dep res worker threads
+		self._threads    = None
+
 
 		# the list of registered listener modules
 		self.listeners = list ()
@@ -76,12 +57,16 @@ class DependencyResolver ( object ):
 		#  or marked as unresolvable
 		self._depqueue_failed = queue.Queue()
 
-		# map: channel identifier -> number of done deps (resolved/unresolvable)
+		# the 'negative' result cache, stores unresolvable deps
+		# has to be (selectively?) cleared when
+		# new dep rule found / new rulepool etc.
+		self._dep_unresolvable = set ()
+
+		# map: channel identifier -> queue of done deps (resolved/unresolvable)
 		# this serves two purposes:
-		# (a) obviously: the number of resolved deps which is useful for channels
-		# (b) the keys of this dict is the list of known channels
-		#
-		self._depdone = dict ()
+		# (a) channels can do a blocking call on this queue
+		# (b) the keys of this dict are the list of known channels
+		self._depqueue_done = dict ()
 
 		# list of rule pools that have been created from reading files
 		self.static_rule_pools = list ()
@@ -103,6 +88,7 @@ class DependencyResolver ( object ):
 		* pool_type -- ignored.
 		"""
 		self.static_rule_pools.append ( rulepool )
+		self._dep_unresolvable.clear()
 		self._sort()
 	# --- end of add_rulepool (...) ---
 
@@ -191,37 +177,39 @@ class DependencyResolver ( object ):
 
 		returns: channel
 		 """
-		if channel in self._depdone:
+		if channel in self._depqueue_done:
 			raise Exception ( "channel is already registered." )
 
-		if channel._depres_master is None:
-			channel._depres_master = self
+		# register channel and allocate a queue in depqueue_done
+		self._depqueue_done [channel.ident] = queue.Queue()
 
-		# register channel and allocate a result counter in depdone
-		self._depdone [channel.ident] = PseudoAtomicCounter (0)
+		channel.set_resolver (
+			self, channel_queue=self._depqueue_done [channel.ident]
+		)
 
 		return channel
 	# --- end of register_channel (...) ---
 
 	def channel_closed ( self, channel_id ):
-		# TODO
+		"""Tells the dependency resolver that a channel has been closed.
+		It will then unregister the channel; this operation does not fail if the
+		channel is not registered with this resolver.
+
+		arguments:
+		* channel_id -- identifier of the closed channel
+
+		returns: None (implicit)
+		"""
 
 		# not removing channel_id's DepEnvs from the queues
 		# 'cause this costs time
-		del self._depdone [channel_id]
-	# --- end of channel_closed (...) ---
+		try:
+			del self._depqueue_done [channel_id]
+		except KeyError as expected:
+			# ok
+			pass
 
-	def get_worker ( self, max_dep_resolve=0 ):
-		"""Returns a dep resolver worker (thread).
-		-- Threading is not implemented, this method is just a reminder.
-
-		arguments:
-		* max_dep_resolve -- if > 0 : worker stops after resolving # deps
-		                     if   0 : worker stops when queue is empty
-		                     else   : worker does not stop unless told to do so
-		"""
-		raise Exception ( "DependencyResolver.get_worker(...) is TODO!" )
-	# --- end of get_worker (...) ---
+	# --- end of channel_closed (...) ---
 
 	def _queue_previously_failed ( self ):
 		"""Inserts all previously failed dependency lookups into the queue
@@ -232,34 +220,50 @@ class DependencyResolver ( object ):
 		while not self._depqueue_failed.empty():
 			# it has to be guaranteed that no items are removed from
 			# _depqueue_failed while calling this method,
-			# else Queue.Empty will be raised
+			# else queue.Empty will be raised
 			self._depqueue.put ( self._depqueue_failed.get_nowait() )
 	# --- end of _queue_previously_failed (...) ---
 
-
 	def start ( self ):
-		"""Tells the resolver to run."""
 		if not self.runlock.acquire ( False ):
 			# already running
 			return True
 		# --
 
+		if DependencyResolver.NUMTHREADS > 0:
+			# no need to wait for the old thread
+			self._mainthread = threading.Thread ( target=self._thread_run_main )
+			self._mainthread.start()
+		else:
+			self._thread_run_main()
+
+		# self.runlock is released when _thread_run_main is done
+	# --- end of start (...) ---
+
+	def _thread_run_main ( self ):
+		"""Tells the resolver to run."""
+
 		jobcount = DependencyResolver.NUMTHREADS
 
 		if jobcount < 1:
-			if jobcount < 0:
-				self.logger.warning ( "Running in sequential mode." )
-			else:
-				self.logger.debug ( "Running in sequential mode." )
-			self.thread_run ()
+			( self.logger.warning if jobcount < 0 else self.logger.debug ) (
+				"Running in sequential mode."
+			)
+			self._thread_run_resolve()
 		else:
+
+			# wait for old threads
+			if not self._threads is None:
+				self.logger.warning ( "Waiting for old threads..." )
+				for t in self._threads: t.join()
+
 			self.logger.warning (
 				"Running in concurrent mode with %i jobs." % jobcount
 			)
 
 			# create threads,
 			self._threads = [
-				threading.Thread ( target=self.thread_run )
+				threading.Thread ( target=self._thread_run_resolve )
 				for n in range (jobcount)
 			]
 			# run them
@@ -273,18 +277,33 @@ class DependencyResolver ( object ):
 
 
 		# iterate over _depqueue_failed and report unresolved
-		while not ( self._depqueue_failed.empty() ):
+		## todo can thread this
+		while not self._depqueue_failed.empty():
+			try:
+				channel_id, dep_env = self._depqueue_failed.get_nowait()
 
-			channel_id, dep_env = self._depqueue_failed.get_nowait()
-			dep_env.set_unresolvable()
-			self._depdone [channel_id].increment_and_get()
+				dep_env.set_unresolvable()
 
-			self._report_event ( 'UNRESOLVABLE', dep_env )
+				self._report_event ( 'UNRESOLVABLE', dep_env )
+
+				if channel_id in self._depqueue_done:
+					## todo/fixme/whatever: this 'if' can filter out channels that have
+					##  been added again
+					self._depqueue_done [channel_id].put ( dep_env )
+
+			except queue.Empty:
+				# race cond empty() <-> get_nowait()
+				break
+			except KeyError:
+				# channel has been closed before calling put, ignore this err
+				pass
 
+		# release the lock
 		self.runlock.release()
-	# --- end of start (...) ---
 
-	def thread_run ( self ):
+	# --- end of _thread_run_main (...) ---
+
+	def _thread_run_resolve ( self ):
 		"""Resolves dependencies (thread target).
 
 		returns: None (implicit)
@@ -293,7 +312,7 @@ class DependencyResolver ( object ):
 		while not self._depqueue.empty():
 
 			try:
-				to_resolve    = self._depqueue.get_nowait()
+				to_resolve = self._depqueue.get_nowait()
 			except queue.Empty:
 				# this thread is done when the queue is empty, so this is
 				# no error, but just the result of the race condition between
@@ -302,49 +321,64 @@ class DependencyResolver ( object ):
 
 			channel_id, dep_env = to_resolve
 
-			if channel_id in self._depdone:
+			if channel_id in self._depqueue_done:
 				# else channel has been closed, drop dep
 
 				self.logger.debug ( "Trying to resolve '%s'." % dep_env.dep_str )
 
 				#have_new_rule = False
 
-				# resolved can be None, so use a bool for checking
 				resolved    = None
-				is_resolved = False
+				# resolved can be None, so use a tri-state int for checking
+				#  0 -> unresolved, but resolvable
+				#  1 -> unresolved and (currently, new rules may change this)
+				#        not resolvable
+				#  2 -> resolved
+				is_resolved = 0
 
-				# search for a match in the rule pools
-				for rulepool in self.static_rule_pools:
-					result = rulepool.matches ( dep_env )
-					if not result is None and result [0] > 0:
-						resolved    = result [1]
-						is_resolved = True
-						break
+				# TODO:
+				#  (threading: could search the pools in parallel)
 
+#				if dep_env.dep_str_low in self._dep_unresolvable:
+#					# cannot resolve
+#					is_resolved = 1
 
+				if is_resolved == 0:
+					# search for a match in the rule pools
+					for rulepool in self.static_rule_pools:
+						result = rulepool.matches ( dep_env )
+						if not result is None and result [0] > 0:
+							resolved    = result [1]
+							is_resolved = 2
+							break
 
-				if is_resolved:
-					dep_env.set_resolved ( resolved, append=False )
-					self._depdone [channel_id].increment_and_get()
 
+
+				if is_resolved == 2:
+					dep_env.set_resolved ( resolved, append=False )
 					self._report_event ( 'RESOLVED', dep_env )
+					self._depqueue_done [channel_id].put ( dep_env )
 				else:
 					self._depqueue_failed.put ( to_resolve )
 
+#					# does not work when adding new rules is possible
+#					self._dep_unresolvable.add ( dep_env.dep_str_low )
+
 				"""
 				## only useful if new rules can be created
 				# new rule found, requeue all previously failed dependency searches
 				if have_new_rule:
 					self._queue_previously_failed
+					self._dep_unresolvable.clear() #?
 				"""
-			# --- end if channel_id in self._depdone
+			# --- end if channel_id in self._depqueue_done
 
 			self._depqueue.task_done()
 		# --- end while
 
 
 
-	# --- end of thread_run (...) ---
+	# --- end of _thread_run_resolve (...) ---
 
 	def enqueue ( self, dep_env, channel_id ):
 		"""Adds a DepEnv to the queue of deps to resolve.
@@ -359,22 +393,9 @@ class DependencyResolver ( object ):
 
 	# --- end of enqueue (...) ---
 
-	def done ( self, channel_id, numdeps ):
-		"""Returns True if channel_id exists in depdone and at least numdeps
-		dependencies have been resolved for that channel.
-
-		arguments:
-		* channel_id --
-		* numdeps --
-		"""
-
-		if channel_id in self._depdone:
-			return self._depdone [channel_id] >= numdeps
-		else:
-			return False
-	# --- end of done (...) ---
-
 	def close ( self ):
+		if isinstance ( self._mainthread, threading.Thread ):
+			self._mainthread.join()
 		for lis in self.listeners: lis.close()
 		del self.listeners
 	# --- end of close (...) ---

diff --git a/roverlay/depres/simpledeprule.py b/roverlay/depres/simpledeprule.py
index 3bad42b..55732c4 100644
--- a/roverlay/depres/simpledeprule.py
+++ b/roverlay/depres/simpledeprule.py
@@ -62,9 +62,9 @@ class SimpleIgnoreDependencyRule ( deprule.DependencyRule ):
 		# --- end of logmatch (...) ---
 
 		if lowercase:
-			lower_dep_str = dep_env.dep_str.lower()
+			#lower_dep_str = dep_env.dep_str.lower()
 			for alias in self.dep_alias:
-				if alias.lower() == lower_dep_str:
+				if alias.lower() == dep_env.dep_str_low:
 					return logmatch ()
 		elif dep_env.dep_str in self.dep_alias:
 			return logmatch ()
@@ -85,9 +85,9 @@ class SimpleIgnoreDependencyRule ( deprule.DependencyRule ):
 		An empty list will be returned if dep_alias has zero length.
 
 		arguments:
-		* resolving_to -- portage package that the exported rule should resolve to,
-		                  defaults to self.resolving_package or an ignore
-		                  keyword such as '!'.
+		* resolving_to -- portage package that the exported rule should
+		                  resolve to, defaults to self.resolving_package or
+		                  an ignore keyword such as '!'.
 		"""
 
 		alias_count = len ( self.dep_alias )
@@ -123,15 +123,17 @@ class SimpleIgnoreDependencyRule ( deprule.DependencyRule ):
 class SimpleDependencyRule ( SimpleIgnoreDependencyRule ):
 
 	def __init__ ( self, resolving_package, dep_str=None, priority=70 ):
-		"""Initializes a SimpleDependencyRule.
-		This is a SimpleIgnoreDependencyRule extended by a portage package string.
+		"""Initializes a SimpleDependencyRule. This is
+		a SimpleIgnoreDependencyRule extended by a portage package string.
 
 		arguments:
 		* resolving package --
 		* dep_str --
 		* priority --
 		"""
-		super ( SimpleDependencyRule, self ) . __init__ ( dep_str=dep_str, priority=priority )
+		super ( SimpleDependencyRule, self ) . __init__ (
+			dep_str=dep_str, priority=priority
+		)
 
 		self.resolving_package = resolving_package
 
@@ -140,8 +142,8 @@ class SimpleDependencyRule ( SimpleIgnoreDependencyRule ):
 	# --- end of __init__ (...) ---
 
 	def get_dep ( self ):
-		"""Returns the textual portage package representation of this rule, e.g.
-		'dev-lang/R'.
+		"""Returns the textual portage package representation of this rule,
+		e.g. 'dev-lang/R'.
 		"""
 		return self.resolving_package
 	# --- end of get_dep (...) ---
@@ -151,12 +153,13 @@ class SimpleDependencyRulePool ( deprule.DependencyRulePool ):
 
 	def __init__ ( self, name, priority=70, filepath=None ):
 		"""Initializes a SimpleDependencyRulePool, which is a DependencyRulePool
-		specialized in simple dependency rules; it offers loading rules from files.
+		specialized in simple dependency rules;
+		it offers loading rules from files.
 
 		arguments:
-		* name -- string identifier for this pool
+		* name     -- string identifier for this pool
 		* priority -- of this pool
-		* filepath -- if set and not None: load a rule file directly into this pool
+		* filepath -- if set and not None: load a rule file directly
 		"""
 		super ( SimpleDependencyRulePool, self ) . __init__ ( name, priority )
 
@@ -282,13 +285,16 @@ class SimpleDependencyRuleReader ( object ):
 
 				else:
 					# single line rule?
-					rule_str = SimpleDependencyRuleReader.one_line_separator.split ( line, 1 )
+					rule_str = \
+						SimpleDependencyRuleReader.one_line_separator.split (line, 1)
 
 					if len ( rule_str ) == 2:
 						# is a single line rule
 
 						if rule_str [0] in SimpleDependencyRuleReader.package_ignore:
-							rules.append ( SimpleIgnoreDependencyRule ( rule_str [1], 40 ) )
+							rules.append (
+								SimpleIgnoreDependencyRule ( rule_str [1], 40 )
+							)
 						else:
 							rules.append (
 								SimpleDependencyRule ( rule_str [0], rule_str [1], 50 )
@@ -296,7 +302,7 @@ class SimpleDependencyRuleReader ( object ):
 					else:
 						logging.error (
 							"In %s, line %i : cannot use this line." %
-							( filepath, lineno )
+								( filepath, lineno )
 						)
 				# ---
 
@@ -304,7 +310,7 @@ class SimpleDependencyRuleReader ( object ):
 
 			logging.info (
 				"%s: read %i dependency rules (in %i lines)" %
-				( filepath, len ( rules ), lineno )
+					( filepath, len ( rules ), lineno )
 			)
 
 			return rules



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-06 19:52 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-06 19:52 UTC (permalink / raw
  To: gentoo-commits

commit:     1202ca5c63080cf066a6997709a23088996322ef
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Wed Jun  6 19:44:50 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Wed Jun  6 19:44:50 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=1202ca5c

dependency resolution

* can now resolve dependencies using dictionary-like 'SimpleDependencyRules'

	new file:   roverlay/depres/communication.py
	new file:   roverlay/depres/depenv.py
	modified:   roverlay/depres/depresolver.py
	modified:   roverlay/depres/deprule.py
	deleted:    roverlay/depres/rulereader.py
	new file:   roverlay/depres/simpledeprule.py

---
 roverlay/depres/communication.py |  115 +++++++++++++++++++++++++
 roverlay/depres/depenv.py        |   47 +++++++++++
 roverlay/depres/depresolver.py   |  161 +++++++++++++++++++++++++++++-------
 roverlay/depres/deprule.py       |   90 +++++----------------
 roverlay/depres/rulereader.py    |   88 --------------------
 roverlay/depres/simpledeprule.py |  170 ++++++++++++++++++++++++++++++++++++++
 6 files changed, 482 insertions(+), 189 deletions(-)

diff --git a/roverlay/depres/communication.py b/roverlay/depres/communication.py
new file mode 100644
index 0000000..65050f1
--- /dev/null
+++ b/roverlay/depres/communication.py
@@ -0,0 +1,115 @@
+# R overlay --
+# Copyright 2006-2012 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+import logging
+
+from roverlay.depres.depenv import DepEnv
+
+class DependencyResolverListener:
+
+	def __init__ ( self ):
+		self.ident = id ( self )
+
+	def notify ( event_type, pkg_env=None, **extra ):
+		"""Notify this listener about an event."""
+		pass
+
+
+class DependencyResolverChannel ( DependencyResolverListener ):
+
+	def __init__ ( self, main_resolver, *args ):
+		super ( DependencyResolverChannel, self ) . __init__ ()
+		self._depres_master = main_resolver
+
+	def close ( self ):
+		"""Closes this channel."""
+		pass
+
+	def enabled ( self ):
+		return True
+
+
+class EbuildJobChannel ( DependencyResolverChannel ):
+	# this channel has a strict control flow:
+	#  add deps -> while not done(): "wait" -> if satisfy_request(): collect/lookup
+
+	def __init__ ( self, main_resolver=None, name=None ):
+		super ( EbuildJobChannel, self ) . __init__ ( main_resolver )
+		self.dep_env_list    = list ()
+		self._done           = False
+		self._collected_deps = None
+
+		if not name is None:
+			self.name = name
+
+	def get_name ( self ):
+		return self.name if hasattr ( self, 'name' ) else None
+
+
+
+	def depcount ( self ):
+		if isinstance ( self.dep_env_list, list ):
+			return len ( self.dep_env_list )
+		else:
+			return -1
+
+
+	def done ( self ):
+		if not self._done:
+			self._done = self._depres_master.done (
+				self.ident, len ( self.dep_env_list )
+			)
+		return self._done
+
+	def add_dependency ( self, dep_str ):
+		if self._done:
+			raise Exception ( "This channel is 'done', it doesn't accept new dependencies." )
+		else:
+			dep_env = DepEnv ( dep_str )
+			self.dep_env_list.append ( dep_env )
+			self._depres_master.enqueue ( dep_env, self.ident )
+
+		return None
+
+	def add_dependencies ( self, dep_list ):
+		for dep_str in dep_list: self.add_dependency ( dep_str )
+		return None
+
+	def satisfy_request ( self, delete_list_if_unresolvable=True ):
+		if self._done:
+			if self.dep_env_list is None:
+				return False
+
+			dep_collected = list()
+
+			for dep_env in self.dep_env_list:
+				if dep_env.is_resolved ():
+					dep_collected.append ( dep_env.get_result() [1] )
+				else:
+					if delete_list_if_unresolvable:
+						del self.dep_env_list
+						self.dep_env_list = None
+					return False
+
+			self._collected_deps = dep_collected
+			return True
+
+		return False
+
+	def lookup ( self, dep_str ):
+		"""Looks up a specific dep str. Use collect_dependencies() for getting
+		all dependencies if order doesn't matter.
+		"""
+		raise Exception ( "TODO" )
+		return None
+
+	def collect_dependencies ( self ):
+		"""Returns a tuple ( dep str, resolved by )."""
+		if self._done:
+			logging.critical ( str ( self._collected_deps ) )
+			return self._collected_deps
+		raise Exception ( "cannot do that" )
+
+	def trigger_run ( self ):
+		self._depres_master.run()

diff --git a/roverlay/depres/depenv.py b/roverlay/depres/depenv.py
new file mode 100644
index 0000000..b6c945c
--- /dev/null
+++ b/roverlay/depres/depenv.py
@@ -0,0 +1,47 @@
+# R overlay --
+# Copyright 2006-2012 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+
+class DepEnv:
+
+	STATUS_UNDONE       = 1
+	STATUS_RESOLVED     = 2
+	STATUS_UNRESOLVABLE = 4
+
+	def __init__ ( self, dep_str ):
+		self.ident       = id ( self )
+		self.dep_str     = dep_str
+		self.status      = DepEnv.STATUS_UNDONE
+		self.resolved_by = None
+
+		# TODO: analyze dep_str: extract dep name, dep version, useless comments,...
+
+
+	def set_resolved ( self, resolved_by, append=False ):
+		if self.resolved_by is None:
+			self.resolved_by = resolved_by
+		elif append:
+			raise Exception ( "appending is not supported..." ) #TODO
+			#self.resolved_by.append ( resolved_by )
+		else:
+			raise Exception ( "dependency is already resolved and append is disabled." )
+
+		self.status |= DepEnv.STATUS_RESOLVED
+
+	def set_unresolvable ( self, force=False ):
+		if force or not self.status & DepEnv.STATUS_RESOLVED:
+			self.resolved_by = None
+			self.status |= DepEnv.STATUS_UNRESOLVABLE
+		else:
+			raise Exception ("dependency is already marked as resolved." )
+
+	def zap ( self ):
+		self.status      = DepEnv.STATUS_UNDONE
+		self.resolved_by = None
+
+	def is_resolved ( self ):
+		return bool ( self.status & DepEnv.STATUS_RESOLVED )
+
+	def get_result ( self ):
+		return ( self.dep_str, self.resolved_by )

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index ebb88cd..c8b4729 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -2,14 +2,53 @@
 # Copyright 2006-2012 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
+import logging
+
+from collections import deque
+
+#from roverlay.depres import deprule
+from roverlay.depres import simpledeprule, communication
+#from roverlay.depres.depenv import DepEnv
+
+
 
 class DependencyResolver:
 
-	def __init__ ( self ):
-		self.logger    = None
-		self.channels  = list ()
-		self.listeners = list ()
+	LOGGER = logging.getLogger ( "DependencyResolver" )
 
+	def __init__ ( self ):
+		# these loggers are temporary helpers
+		self.logger              = DependencyResolver.LOGGER
+		self.logger_unresolvable = self.logger.getChild ( "UNRESOLVABLE" )
+		self.logger_resolved     = self.logger.getChild ( "RESOLVED" )
+		self.channels            = dict ()
+		self.listeners           = dict ()
+
+		self._depqueue           = deque ()
+		self._depqueue_failed    = deque ()
+		self._depdone            = dict ()
+
+		self.static_rule_pools = list ()
+
+	def log_unresolvable ( self, dep_env ):
+		msg = "'" + dep_env.dep_str + "'"
+		if not dep_env.dep_str:
+			self.logger_unresolvable.warning ( msg + " empty? fix that!" )
+		else:
+			self.logger_unresolvable.info ( msg )
+
+	def log_resolved ( self, dep_env ):
+		msg = "'" + dep_env.dep_str + "' as '" + str ( dep_env.resolved_by ) + "'"
+		self.logger_resolved.info ( msg )
+
+	def sort ( self ):
+		for pool in self.static_rule_pools: pool.sort()
+		self.static_rule_pools.sort ( key = lambda pool : ( pool.priority, pool.rule_weight ) )
+
+	def add_rulepool ( self, rulepool, pool_type=None ):
+		self.static_rule_pools.append ( rulepool )
+		self.sort()
+		return None
 
 	def _report_event ( self, event, pkg_env=None, **extra ):
 		"""Reports an event to the log, channels and listeners."""
@@ -26,7 +65,7 @@ class DependencyResolver:
 		"""
 		pass
 
-
+	# def add_listener ( self, listener ): FIXME
 	def add_listener ( self ):
 		"""Adds a listener, which listens to events such as
 		"dependency is unresolvable". Possible use cases include redirecting
@@ -35,11 +74,13 @@ class DependencyResolver:
 		arguments:
 		* listener_type
 		"""
+		# broken due to module "outsourcing"
 		new_listener = DependencyResolverListener()
 		# register the new listener
 		self.listeners [new_listener.ident] = new_listener
 		return new_listener
 
+	# FIXME get_channel is not useful when using various channel types
 	def get_channel ( self, readonly=False ):
 		"""Returns a communication channel to the DependencyResolver.
 		This channel can be used to _talk_, e.g. queue dependencies for resolution
@@ -48,50 +89,108 @@ class DependencyResolver:
 		arguments:
 		* readonly -- whether this channel has write access or not
 		"""
-		new channel = DependencyResolverChannel ( self )
+		# broken due to module "outsourcing"
+		new_channel = DependencyResolverChannel ( self )
 		self.channels [new_channel.ident] = new_channel
 		return new_channel
 
+	def register_channel ( self, channel ):
+		if channel in self.channels:
+			raise Exception ( "channel is already registered." )
 
+		if channel._depres_master is None:
+			channel._depres_master = self
 
-class DependencyResolverListener:
+		self.channels [channel.ident] = channel
 
-	def __init__ ( self ):
-		self.ident = id ( self )
+		return channel
 
-	def notify ( event_type, pkg_env=None, **extra ):
-		"""Notify this listener about an event."""
-		pass
 
+	def get_worker ( self, max_dep_resolve=0 ):
+		"""Returns a dep resolver worker (thread).
+		-- Threading is not implemented, this method is just a reminder.
 
-class DependencyResolverChannel ( DependencyResolverListener ):
+		arguments:
+		* max_dep_resolve -- if > 0 : worker stops after resolving max_dep_resolve deps
+		                     if   0 : worker stops when queue is empty
+		                     else   : worker does not stop unless told to do so
+		"""
+		raise Exception ( "DependencyResolver.get_worker(...) is TODO!" )
 
-	def __init__ ( self, main_resolver, *args ):
-		super ( DependencyResolverChannel, self ) . __init__ ()
-		self._depres_master = main_resolver
+	def _queue_previously_failed( self ):
+		queue_again = self._depqueue_failed
+		self._depqueue_failed = deque()
+		self._depqueue.extend ( queue_again )
 
-	def close ( self ):
-		"""Closes this channel."""
-		pass
 
+	def run ( self ):
+		"""Resolves dependencies."""
+		# TODO: this method has to be replaced when using threads
 
-class EbuildJobChannel ( DependencyResolverChannel ):
-	def __init__ ( self, main_resolver, *args ):
-		super ( EbuildJobChannel, self ) . __init__ ( main_resolver )
 
+		while len ( self._depqueue ):
 
-	def done ( self ):
-		pass
+			to_resolve = self._depqueue.popleft()
+			channel_id, dep_env = to_resolve
 
-	def add_dependency ( self, dep_str ):
-		pass
+			self.logger.debug ( "Trying to resolve '" + dep_env.dep_str + "'." )
 
-	def add_dependencies ( self, dep_list ):
-		pass
+			if not channel_id in self.channels:
+				# channel has been closed but did not request a cleanup
+				raise Exception ( "dirty queue!" )
 
-	def satisfy_request ( self ):
-		pass
+			#have_new_rule = False
+			resolved      = None
+
+			for rulepool in self.static_rule_pools:
+				result = rulepool.matches ( dep_env )
+				if not result is None and result [0] > 0:
+					resolved = result [1]
+					break
+
+
+
+			if resolved:
+				dep_env.set_resolved ( resolved, append=False )
+				self.log_resolved ( dep_env )
+				self._depdone [channel_id] +=1
+			else:
+				self._depqueue_failed.append ( to_resolve )
+
+			"""
+			## only useful if new rules can be created
+			# new rule found, queue all previously failed dependency searches again
+			if have_new_rule:
+				self._queue_previously_failed
+			"""
+		# --- end while
+
+		# iterate over _depqueue_failed and report unresolved
+
+		while len ( self._depqueue_failed ):
+
+			channel_id, dep_env = self._depqueue_failed.popleft()
+			dep_env.set_unresolvable()
+			self._depdone [channel_id] += 1
+
+			# TODO: temporary log
+			self.log_unresolvable ( dep_env )
 
-	def lookup ( self, dep_str ):
 		return None
 
+
+	def enqueue ( self, dep_env, channel_id ):
+		self._depqueue.append ( ( channel_id, dep_env ) )
+		if not channel_id in self._depdone:
+			# allocate a result counter in depdone
+			self._depdone [channel_id] = 0
+
+
+	def done ( self, channel_id, numdeps ):
+
+		if channel_id in self._depdone:
+			return bool ( self._depdone [channel_id] >= numdeps )
+		else:
+			return False
+
+

diff --git a/roverlay/depres/deprule.py b/roverlay/depres/deprule.py
index b5d1116..83cff66 100644
--- a/roverlay/depres/deprule.py
+++ b/roverlay/depres/deprule.py
@@ -6,60 +6,26 @@ class DependencyRule:
 	def __init__ ( self ):
 		self.max_score = 1000
 
-	def matches ( dep_str ):
+	def matches ( self, dep_env ):
 		return 0
 
-class SimpleDependencyRule ( DependencyRule ):
-
-	def __init__ ( self, resolving_package, dep_str=None, priority=100 ):
-		super ( SimpleDependencyRule, self ) . __init__ ( )
-		self.dep_alias = set ()
-		if dep_str:
-			self.dep_alias.add ( dep_str )
-
-		self.resolving_package = resolving_package
-
-		self.priority = priority
-	# --- end of __init__ (...) ---
-
-	def add_resolved ( self, dep_str ):
-		self.dep_alias.add ( dep_str )
-	# --- end of add_resolved (...) ---
-
-	def matches ( self, dep_str, lowercase=True ):
-		if lowercase:
-			lower_dep_str = dep_str
-			for alias in self.dep_alias:
-				if alias.lower() == lower_dep_str:
-					return self.max_score
-		elif dep_str in self.dep_alias:
-			return self.max_score
-
-		return 0
-	# --- end of matches (...) ---
-
-	def get_dep ( self ):
-		return self.resolving_package
-
-	def export_rule ( self ):
-		pass
-
 class DependencyRulePool:
 
-	def __init__ ( self, name ):
-		self.rules    = list ()
-		self.name     = name
-		self.priority = 0
+	def __init__ ( self, name, priority ):
+		self.rules       = list ()
+		self.name        = name
+		self.priority    = priority
+		# the "rule weight" is the sum of the rules' priorities - it's used to
+		#  compare/sort dependency pools with the same priority (lesser weight is better)
+		self.rule_weight = 0
 	# --- end of __init__ (...) ---
 
 	def sort ( self ):
 		self.rules.sort ( key=lambda rule : rule.priority )
 
-		priority_sum = 0
-		for r in self.rules:
-			priority_sum += r.priority
-
-		self.priority = int ( priority_sum / len ( self.rules ) ) if len ( self.rules ) else 0
+		rule_priority_sum = 0
+		for r in self.rules: rule_priority_sum += r.priority
+		self.rule_weight = rule_priority_sum
 
 		return None
 	# --- end of _sort_rules (...) ---
@@ -73,13 +39,15 @@ class DependencyRulePool:
 		return None
 	# --- end of add (...) ---
 
-	def matches ( self, dep_str, skip_matches=0 ):
+	def matches ( self, dep_env, skip_matches=0 ):
 		"""Tries to find a match in this dependency rule pool.
 		The first match is immediatly returned unless skip_matches is != 0, in
 		which case the first (>0) / last (<0) skip_matches matches are skipped.
+		Returns a tuple ( score, portage dependency ),
+		e.g. ( 1000, 'sys-apps/which' ), if match found, else None.
 
 		arguments:
-		* dep_str -- dependency to look up
+		* dep_env -- dependency to look up
 		* skip_matches --
 		"""
 
@@ -89,41 +57,23 @@ class DependencyRulePool:
 
 		else:
 			skipped = 0
-			order = range ( len ( self.rules ) )
+			# python3 requires list ( range ... )
+			order = list ( range ( len ( self.rules ) ) )
 
 			if skip_matches < 1:
 				skip_matches *= -1
 				order.reverse()
 
 			for index in order:
-				score = self.rules [index].matches ( dep_str )
+				score = self.rules [index].matches ( dep_env )
 				if score:
 					if skipped < skip_matches:
 						skipped += 1
 					else:
-						return score, self.rules [index].get_dep ()
-
+						return ( score, self.rules [index].get_dep () )
 
-		return 0, None
 
+		return None
 
-class SimpleDependencyRulePool ( DependencyRulePool )
 
-	def __init__ ( self, name ):
-		super ( SimpleDependencyRulePool, self ) . __init__ ( name )
-	# --- end of __init__ (...) ---
-
-	def add ( self, rile ):
-		if isinstance ( rule, SimpleDependencyRule ):
-			self.rules.append ( rule )
-		else:
-			raise Exception ( "bad usage (simple dependency rule expected)." )
-	# --- end of add (...) ---
 
-	def export_rules ( self, fh ):
-		for rule in self.rules:
-			to_write = fh.export_rule()
-			if isinstance ( to_write, str ):
-				fh.write ( to_write )
-			else:
-				fh.writelines ( to_write )

diff --git a/roverlay/depres/rulereader.py b/roverlay/depres/rulereader.py
deleted file mode 100644
index 6998cb2..0000000
--- a/roverlay/depres/rulereader.py
+++ /dev/null
@@ -1,88 +0,0 @@
-
-import re
-import logging
-
-from roverlay.depres.deprule import SimpleDependencyRule
-
-class SimpleDependencyRuleReader:
-
-	one_line_separator = re.compile ( '\s+::\s+' )
-	multiline_start    = '{'
-	multiline_stop     = '}'
-	comment_chars      = list ( '#;' )
-
-
-	def __init__ ( self ):
-		pass
-	# --- end of __init__  (...) ---
-
-	def read_file ( self, filepath ):
-		"""Reads a file that contains SimpleDescriptionRules.
-
-		arguments:
-		* filepath -- file to read
-		"""
-
-		lineno = 0
-
-		try:
-			logging.debug ( "Reading simple dependency rule file " + filepath + "." )
-			fh = open ( filepath, 'r' )
-
-			next_rule = None
-			rules = list ()f
-
-			for line in fh.readlines():
-				lineno += 1
-				line = line.strip()
-
-				if not line:
-					pass
-
-				elif not next_rule is None:
-					# in a multiline rule
-					if line [0] == SimpleDependencyRuleReader.multiline_stop:
-						# end of a multiline rule
-						rules.append ( next_rule )
-						next_rule = None
-					else:
-						# new resolved str
-						next_rule.add_resolved ( line )
-
-				elif line [0] in SimpleDependencyRuleReader.comment_chars:
-					# comment
-					pass
-
-				elif line [-1] == SimpleDependencyRuleReader.multiline_start:
-					# start of multiline rule
-					next_rule = SimpleDependencyRule ( line [:-1].rstrip(), None, 100 )
-
-				else:
-					# one line rule?
-					rule_str = SimpleDependencyRuleReader.one_line_separator.split ( line, 1 )
-
-					if len ( rule_str ) == 2:
-						rules.append (
-							SimpleDependencyRule ( rule_str [0], rule_str [1], 90 )
-						)
-					else:
-						logging.error (
-							"In " + filepath + ", line " + str ( lineno ) + ": cannot use this line."
-						)
-				# ---
-
-			if fh: fh.close ()
-
-			logging.info ( filepath + ": read " + str ( len ( rules ) ) + " dependency rules." )
-
-			return rules
-
-		except IOError as ioerr:
-			if lineno:
-				logging.error ( "Failed to read file " + filepath + " after " + str ( lineno ) " lines."
-			else:
-				logging.error ( "Could not read file " + filepath + "." )
-			raise
-
-		# --- end of read_file (...) ---
-

diff --git a/roverlay/depres/simpledeprule.py b/roverlay/depres/simpledeprule.py
new file mode 100644
index 0000000..0f916ec
--- /dev/null
+++ b/roverlay/depres/simpledeprule.py
@@ -0,0 +1,170 @@
+
+import re
+import logging
+
+
+
+from roverlay.depres import deprule
+
+class SimpleDependencyRule ( deprule.DependencyRule ):
+
+	TMP_LOGGER = logging.getLogger ( 'SimpleDependencyRule' )
+
+	def __init__ ( self, resolving_package, dep_str=None, priority=100 ):
+		super ( SimpleDependencyRule, self ) . __init__ ( )
+		self.dep_alias = set ()
+
+		self.logger = SimpleDependencyRule.TMP_LOGGER.getChild ( resolving_package )
+		self.logger.debug ( "new rule" )
+
+		if dep_str:
+			self.logger.debug ( "resolves '" + dep_str + "' now." )
+			self.dep_alias.add ( dep_str )
+
+		self.resolving_package = resolving_package
+
+		self.priority = priority
+
+
+	# --- end of __init__ (...) ---
+
+	def add_resolved ( self, dep_str ):
+		self.dep_alias.add ( dep_str )
+	# --- end of add_resolved (...) ---
+
+	def matches ( self, dep_env, lowercase=True ):
+
+		if lowercase:
+			lower_dep_str = dep_env.dep_str.lower()
+			for alias in self.dep_alias:
+				if alias.lower() == lower_dep_str:
+					self.logger.debug ( "matches '" + lower_dep_str + "'" )
+					return self.max_score
+		elif dep_env.dep_str in self.dep_alias:
+			self.logger.debug ( "matches '" + dep_env.dep_str + "'" )
+			return self.max_score
+
+		return 0
+	# --- end of matches (...) ---
+
+	def get_dep ( self ):
+		return self.resolving_package
+
+	def export_rule ( self ):
+		pass
+
+class SimpleDependencyRulePool ( deprule.DependencyRulePool ):
+
+	def __init__ ( self, name, priority=70, filepath=None ):
+		super ( SimpleDependencyRulePool, self ) . __init__ ( name, priority )
+
+		if not filepath is None:
+			self.load_rule_file ( filepath )
+	# --- end of __init__ (...) ---
+
+	def add ( self, rule ):
+		if isinstance ( rule, SimpleDependencyRule ):
+			self.rules.append ( rule )
+		else:
+			raise Exception ( "bad usage (simple dependency rule expected)." )
+	# --- end of add (...) ---
+
+	def load_rule_file ( self, filepath ):
+		reader = SimpleDependencyRuleReader()
+
+		new_rules = reader.read_file ( filepath )
+		for rule in new_rules:
+			self.add ( rule )
+
+	def export_rules ( self, fh ):
+		for rule in self.rules:
+			to_write = fh.export_rule()
+			if isinstance ( to_write, str ):
+				fh.write ( to_write )
+			else:
+				fh.writelines ( to_write )
+
+
+
+class SimpleDependencyRuleReader:
+
+	one_line_separator = re.compile ( '\s+::\s+' )
+	multiline_start    = '{'
+	multiline_stop     = '}'
+	comment_chars      = list ( '#;' )
+
+
+	def __init__ ( self ):
+		pass
+	# --- end of __init__  (...) ---
+
+	def read_file ( self, filepath ):
+		"""Reads a file that contains SimpleDescriptionRules.
+
+		arguments:
+		* filepath -- file to read
+		"""
+
+		lineno = 0
+
+		try:
+			logging.debug ( "Reading simple dependency rule file " + filepath + "." )
+			fh = open ( filepath, 'r' )
+
+			next_rule = None
+			rules = list ()
+
+			for line in fh.readlines():
+				lineno += 1
+				line = line.strip()
+
+				if not line:
+					pass
+
+				elif not next_rule is None:
+					# in a multiline rule
+					if line [0] == SimpleDependencyRuleReader.multiline_stop:
+						# end of a multiline rule
+						rules.append ( next_rule )
+						next_rule = None
+					else:
+						# new resolved str
+						next_rule.add_resolved ( line )
+
+				elif line [0] in SimpleDependencyRuleReader.comment_chars:
+					# comment
+					pass
+
+				elif line [-1] == SimpleDependencyRuleReader.multiline_start:
+					# start of multiline rule
+					next_rule = SimpleDependencyRule ( line [:-1].rstrip(), None, 100 )
+
+				else:
+					# one line rule?
+					rule_str = SimpleDependencyRuleReader.one_line_separator.split ( line, 1 )
+
+					if len ( rule_str ) == 2:
+						rules.append (
+							SimpleDependencyRule ( rule_str [0], rule_str [1], 90 )
+						)
+					else:
+						logging.error (
+							"In " + filepath + ", line " + str ( lineno ) + ": cannot use this line."
+						)
+				# ---
+
+			if fh: fh.close ()
+
+			logging.info ( filepath + ": read " + str ( len ( rules ) ) + " dependency rules." )
+
+			return rules
+
+		except IOError as ioerr:
+			if lineno:
+				logging.error ( "Failed to read file " + filepath + " after " + str ( lineno ) + " lines." )
+			else:
+				logging.error ( "Could not read file " + filepath + "." )
+			raise
+
+		# --- end of read_file (...) ---
+



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-05 17:30 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-05 17:30 UTC (permalink / raw
  To: gentoo-commits

commit:     f65931ef830da61d738dab1dcc9f5588cc260243
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Tue Jun  5 17:20:39 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Tue Jun  5 17:20:39 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=f65931ef

dependency resolution

* added simple rule reader
* started with the dependency resolver

---
 roverlay/depres/depresolver.py |   78 +++++++++++++++++++++++++++---
 roverlay/depres/deprule.py     |  103 +++++++++++++++++++++++++++++++++-------
 roverlay/depres/rulereader.py  |   88 ++++++++++++++++++++++++++++++++++
 3 files changed, 245 insertions(+), 24 deletions(-)

diff --git a/roverlay/depres/__init__.py b/roverlay/depres/__init__.py
new file mode 100644
index 0000000..e69de29

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
index 7d7c5e6..ebb88cd 100644
--- a/roverlay/depres/depresolver.py
+++ b/roverlay/depres/depresolver.py
@@ -2,22 +2,55 @@
 # Copyright 2006-2012 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
+
 class DependencyResolver:
 
 	def __init__ ( self ):
-		self.channels  = dict()
-		self.listeners = dict()
+		self.logger    = None
+		self.channels  = list ()
+		self.listeners = list ()
+
+
+	def _report_event ( self, event, pkg_env=None, **extra ):
+		"""Reports an event to the log, channels and listeners."""
+		pass
+
+
+	def set_logmask ( self, logmask ):
+		"""Sets the logmask for this DependencyResolver which can be used to
+		filter events that would normally go into the log file.
+		Useful if a listener module reports such events in an extra file.
+
+		arguments:
+		* logmask -- new logmask
+		"""
+		pass
 
 
-	def get_listener ():
+	def add_listener ( self ):
+		"""Adds a listener, which listens to events such as
+		"dependency is unresolvable". Possible use cases include redirecting
+		such events into a file for further parsing.
+
+		arguments:
+		* listener_type
+		"""
 		new_listener = DependencyResolverListener()
 		# register the new listener
 		self.listeners [new_listener.ident] = new_listener
 		return new_listener
 
-	def get_channel ( readonly=False ):
-		# ... TODO
-		pass
+	def get_channel ( self, readonly=False ):
+		"""Returns a communication channel to the DependencyResolver.
+		This channel can be used to _talk_, e.g. queue dependencies for resolution
+		and collect the results later.
+
+		arguments:
+		* readonly -- whether this channel has write access or not
+		"""
+		new channel = DependencyResolverChannel ( self )
+		self.channels [new_channel.ident] = new_channel
+		return new_channel
 
 
 
@@ -26,8 +59,39 @@ class DependencyResolverListener:
 	def __init__ ( self ):
 		self.ident = id ( self )
 
+	def notify ( event_type, pkg_env=None, **extra ):
+		"""Notify this listener about an event."""
+		pass
+
+
 class DependencyResolverChannel ( DependencyResolverListener ):
 
 	def __init__ ( self, main_resolver, *args ):
-		super ( DependencyResolverChannel, self ) . __init__ ( args )
+		super ( DependencyResolverChannel, self ) . __init__ ()
 		self._depres_master = main_resolver
+
+	def close ( self ):
+		"""Closes this channel."""
+		pass
+
+
+class EbuildJobChannel ( DependencyResolverChannel ):
+	def __init__ ( self, main_resolver, *args ):
+		super ( EbuildJobChannel, self ) . __init__ ( main_resolver )
+
+
+	def done ( self ):
+		pass
+
+	def add_dependency ( self, dep_str ):
+		pass
+
+	def add_dependencies ( self, dep_list ):
+		pass
+
+	def satisfy_request ( self ):
+		pass
+
+	def lookup ( self, dep_str ):
+		return None
+

diff --git a/roverlay/depres/deprule.py b/roverlay/depres/deprule.py
index 3a82cbe..b5d1116 100644
--- a/roverlay/depres/deprule.py
+++ b/roverlay/depres/deprule.py
@@ -4,20 +4,20 @@
 
 class DependencyRule:
 	def __init__ ( self ):
-		pass
+		self.max_score = 1000
 
-	def matches ( pkg_env ):
-		return False
+	def matches ( dep_str ):
+		return 0
 
 class SimpleDependencyRule ( DependencyRule ):
 
 	def __init__ ( self, resolving_package, dep_str=None, priority=100 ):
-		super ( SimpleDependencyRule, self ) . __init__ ( self )
+		super ( SimpleDependencyRule, self ) . __init__ ( )
 		self.dep_alias = set ()
 		if dep_str:
 			self.dep_alias.add ( dep_str )
 
-		self.package = resolving_package
+		self.resolving_package = resolving_package
 
 		self.priority = priority
 	# --- end of __init__ (...) ---
@@ -26,35 +26,104 @@ class SimpleDependencyRule ( DependencyRule ):
 		self.dep_alias.add ( dep_str )
 	# --- end of add_resolved (...) ---
 
-	def matches ( self, pkg_env, lowercase=True ):
+	def matches ( self, dep_str, lowercase=True ):
 		if lowercase:
-			lower_dep_str = pkg_env.dep_str
+			lower_dep_str = dep_str
 			for alias in self.dep_alias:
 				if alias.lower() == lower_dep_str:
-					return True
-		elif pkg_env.dep_str in self.dep_alias:
-			return True
+					return self.max_score
+		elif dep_str in self.dep_alias:
+			return self.max_score
 
-		return False
+		return 0
 	# --- end of matches (...) ---
 
+	def get_dep ( self ):
+		return self.resolving_package
+
+	def export_rule ( self ):
+		pass
+
 class DependencyRulePool:
 
-	def __init__ ( self ):
-		self.rules = list ()
-		self._priofunc = lambda x : x.priority
+	def __init__ ( self, name ):
+		self.rules    = list ()
+		self.name     = name
+		self.priority = 0
 	# --- end of __init__ (...) ---
 
-	def _sort_rules ( self ):
-		self.rules.sort ( key=self._priofunc )
+	def sort ( self ):
+		self.rules.sort ( key=lambda rule : rule.priority )
+
+		priority_sum = 0
+		for r in self.rules:
+			priority_sum += r.priority
+
+		self.priority = int ( priority_sum / len ( self.rules ) ) if len ( self.rules ) else 0
+
 		return None
 	# --- end of _sort_rules (...) ---
 
 	def add ( self, rule ):
 		if issubclass ( rule, DependencyRule ):
-			self.rules.add ( rule )
+			self.rules.append ( rule )
 		else:
 			raise Exception ( "bad usage (dependency rule expected)." )
 
 		return None
 	# --- end of add (...) ---
+
+	def matches ( self, dep_str, skip_matches=0 ):
+		"""Tries to find a match in this dependency rule pool.
+		The first match is immediatly returned unless skip_matches is != 0, in
+		which case the first (>0) / last (<0) skip_matches matches are skipped.
+
+		arguments:
+		* dep_str -- dependency to look up
+		* skip_matches --
+		"""
+
+		if abs ( skip_matches ) >= len ( self.rules ):
+			# all matches ignored; cannot expect a result in this case - abort now
+			pass
+
+		else:
+			skipped = 0
+			order = range ( len ( self.rules ) )
+
+			if skip_matches < 1:
+				skip_matches *= -1
+				order.reverse()
+
+			for index in order:
+				score = self.rules [index].matches ( dep_str )
+				if score:
+					if skipped < skip_matches:
+						skipped += 1
+					else:
+						return score, self.rules [index].get_dep ()
+
+
+		return 0, None
+
+
+class SimpleDependencyRulePool ( DependencyRulePool )
+
+	def __init__ ( self, name ):
+		super ( SimpleDependencyRulePool, self ) . __init__ ( name )
+	# --- end of __init__ (...) ---
+
+	def add ( self, rile ):
+		if isinstance ( rule, SimpleDependencyRule ):
+			self.rules.append ( rule )
+		else:
+			raise Exception ( "bad usage (simple dependency rule expected)." )
+	# --- end of add (...) ---
+
+	def export_rules ( self, fh ):
+		for rule in self.rules:
+			to_write = fh.export_rule()
+			if isinstance ( to_write, str ):
+				fh.write ( to_write )
+			else:
+				fh.writelines ( to_write )

diff --git a/roverlay/depres/rulereader.py b/roverlay/depres/rulereader.py
new file mode 100644
index 0000000..6998cb2
--- /dev/null
+++ b/roverlay/depres/rulereader.py
@@ -0,0 +1,88 @@
+
+import re
+import logging
+
+from roverlay.depres.deprule import SimpleDependencyRule
+
+class SimpleDependencyRuleReader:
+
+	one_line_separator = re.compile ( '\s+::\s+' )
+	multiline_start    = '{'
+	multiline_stop     = '}'
+	comment_chars      = list ( '#;' )
+
+
+	def __init__ ( self ):
+		pass
+	# --- end of __init__  (...) ---
+
+	def read_file ( self, filepath ):
+		"""Reads a file that contains SimpleDescriptionRules.
+
+		arguments:
+		* filepath -- file to read
+		"""
+
+		lineno = 0
+
+		try:
+			logging.debug ( "Reading simple dependency rule file " + filepath + "." )
+			fh = open ( filepath, 'r' )
+
+			next_rule = None
+			rules = list ()f
+
+			for line in fh.readlines():
+				lineno += 1
+				line = line.strip()
+
+				if not line:
+					pass
+
+				elif not next_rule is None:
+					# in a multiline rule
+					if line [0] == SimpleDependencyRuleReader.multiline_stop:
+						# end of a multiline rule
+						rules.append ( next_rule )
+						next_rule = None
+					else:
+						# new resolved str
+						next_rule.add_resolved ( line )
+
+				elif line [0] in SimpleDependencyRuleReader.comment_chars:
+					# comment
+					pass
+
+				elif line [-1] == SimpleDependencyRuleReader.multiline_start:
+					# start of multiline rule
+					next_rule = SimpleDependencyRule ( line [:-1].rstrip(), None, 100 )
+
+				else:
+					# one line rule?
+					rule_str = SimpleDependencyRuleReader.one_line_separator.split ( line, 1 )
+
+					if len ( rule_str ) == 2:
+						rules.append (
+							SimpleDependencyRule ( rule_str [0], rule_str [1], 90 )
+						)
+					else:
+						logging.error (
+							"In " + filepath + ", line " + str ( lineno ) + ": cannot use this line."
+						)
+				# ---
+
+			if fh: fh.close ()
+
+			logging.info ( filepath + ": read " + str ( len ( rules ) ) + " dependency rules." )
+
+			return rules
+
+		except IOError as ioerr:
+			if lineno:
+				logging.error ( "Failed to read file " + filepath + " after " + str ( lineno ) " lines."
+			else:
+				logging.error ( "Could not read file " + filepath + "." )
+			raise
+
+		# --- end of read_file (...) ---
+



^ permalink raw reply related	[flat|nested] 40+ messages in thread
* [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/
@ 2012-06-04 19:07 André Erdmann
  0 siblings, 0 replies; 40+ messages in thread
From: André Erdmann @ 2012-06-04 19:07 UTC (permalink / raw
  To: gentoo-commits

commit:     b878c706058d51594c4d47bf41a637c43b5125b3
Author:     André Erdmann <dywi <AT> mailerd <DOT> de>
AuthorDate: Mon Jun  4 19:04:31 2012 +0000
Commit:     André Erdmann <dywi <AT> mailerd <DOT> de>
CommitDate: Mon Jun  4 19:04:31 2012 +0000
URL:        http://git.overlays.gentoo.org/gitweb/?p=proj/R_overlay.git;a=commit;h=b878c706

initial work for dependency resolution

---
 roverlay/depres/depresolver.py |   33 ++++++++++++++++++++++
 roverlay/depres/deprule.py     |   60 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 93 insertions(+), 0 deletions(-)

diff --git a/roverlay/depres/depresolver.py b/roverlay/depres/depresolver.py
new file mode 100644
index 0000000..7d7c5e6
--- /dev/null
+++ b/roverlay/depres/depresolver.py
@@ -0,0 +1,33 @@
+# R overlay --
+# Copyright 2006-2012 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+class DependencyResolver:
+
+	def __init__ ( self ):
+		self.channels  = dict()
+		self.listeners = dict()
+
+
+	def get_listener ():
+		new_listener = DependencyResolverListener()
+		# register the new listener
+		self.listeners [new_listener.ident] = new_listener
+		return new_listener
+
+	def get_channel ( readonly=False ):
+		# ... TODO
+		pass
+
+
+
+class DependencyResolverListener:
+
+	def __init__ ( self ):
+		self.ident = id ( self )
+
+class DependencyResolverChannel ( DependencyResolverListener ):
+
+	def __init__ ( self, main_resolver, *args ):
+		super ( DependencyResolverChannel, self ) . __init__ ( args )
+		self._depres_master = main_resolver

diff --git a/roverlay/depres/deprule.py b/roverlay/depres/deprule.py
new file mode 100644
index 0000000..3a82cbe
--- /dev/null
+++ b/roverlay/depres/deprule.py
@@ -0,0 +1,60 @@
+# R overlay --
+# Copyright 2006-2012 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+class DependencyRule:
+	def __init__ ( self ):
+		pass
+
+	def matches ( pkg_env ):
+		return False
+
+class SimpleDependencyRule ( DependencyRule ):
+
+	def __init__ ( self, resolving_package, dep_str=None, priority=100 ):
+		super ( SimpleDependencyRule, self ) . __init__ ( self )
+		self.dep_alias = set ()
+		if dep_str:
+			self.dep_alias.add ( dep_str )
+
+		self.package = resolving_package
+
+		self.priority = priority
+	# --- end of __init__ (...) ---
+
+	def add_resolved ( self, dep_str ):
+		self.dep_alias.add ( dep_str )
+	# --- end of add_resolved (...) ---
+
+	def matches ( self, pkg_env, lowercase=True ):
+		if lowercase:
+			lower_dep_str = pkg_env.dep_str
+			for alias in self.dep_alias:
+				if alias.lower() == lower_dep_str:
+					return True
+		elif pkg_env.dep_str in self.dep_alias:
+			return True
+
+		return False
+	# --- end of matches (...) ---
+
+class DependencyRulePool:
+
+	def __init__ ( self ):
+		self.rules = list ()
+		self._priofunc = lambda x : x.priority
+	# --- end of __init__ (...) ---
+
+	def _sort_rules ( self ):
+		self.rules.sort ( key=self._priofunc )
+		return None
+	# --- end of _sort_rules (...) ---
+
+	def add ( self, rule ):
+		if issubclass ( rule, DependencyRule ):
+			self.rules.add ( rule )
+		else:
+			raise Exception ( "bad usage (dependency rule expected)." )
+
+		return None
+	# --- end of add (...) ---



^ permalink raw reply related	[flat|nested] 40+ messages in thread
end of thread, other threads:[~2014-08-23 19:03 UTC | newest]

Thread overview: 40+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-08-02 15:14 [gentoo-commits] proj/R_overlay:master commit in: roverlay/depres/ André Erdmann
  -- strict thread matches above, loose matches on Subject: below --
2014-08-23 19:03 André Erdmann
2014-06-05 22:09 André Erdmann
2013-09-03  8:35 André Erdmann
2013-09-02 13:20 André Erdmann
2013-08-28  9:38 André Erdmann
2013-08-27 15:39 André Erdmann
2013-08-27 15:39 André Erdmann
2013-08-27 15:39 André Erdmann
2013-08-23 13:52 André Erdmann
2013-08-20 21:46 André Erdmann
2013-08-20 21:46 André Erdmann
2013-07-18 19:25 [gentoo-commits] proj/R_overlay:gsoc13/next " André Erdmann
2013-07-23  7:51 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
2013-07-17 18:10 [gentoo-commits] proj/R_overlay:gsoc13/next " André Erdmann
2013-07-17 18:10 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
2013-07-16 16:36 André Erdmann
2013-07-15 22:31 [gentoo-commits] proj/R_overlay:gsoc13/next " André Erdmann
2013-07-16 16:36 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
2013-07-11 16:44 [gentoo-commits] proj/R_overlay:gsoc13/next " André Erdmann
2013-07-11 16:44 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
2013-07-11  8:49 André Erdmann
2013-06-13 16:34 André Erdmann
2012-07-30 15:53 André Erdmann
2012-07-30  8:52 [gentoo-commits] proj/R_overlay:overlay_wip " André Erdmann
2012-07-30  8:52 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
2012-07-19 16:53 [gentoo-commits] proj/R_overlay:overlay_wip " André Erdmann
2012-07-30  8:52 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
2012-07-16 16:15 André Erdmann
2012-07-13 10:44 [gentoo-commits] proj/R_overlay:depres_wip " André Erdmann
2012-07-16 16:15 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
2012-07-13 10:44 [gentoo-commits] proj/R_overlay:depres_wip " André Erdmann
2012-07-16 16:15 ` [gentoo-commits] proj/R_overlay:master " André Erdmann
2012-07-02 16:52 André Erdmann
2012-07-02 16:52 André Erdmann
2012-06-29 22:48 André Erdmann
2012-06-29 22:48 André Erdmann
2012-06-28 13:29 André Erdmann
2012-06-28 13:29 André Erdmann
2012-06-27 14:46 André Erdmann
2012-06-20 19:03 André Erdmann
2012-06-13 16:34 André Erdmann
2012-06-13  7:45 André Erdmann
2012-06-13  7:45 André Erdmann
2012-06-12 17:17 André Erdmann
2012-06-06 19:52 André Erdmann
2012-06-05 17:30 André Erdmann
2012-06-04 19:07 André Erdmann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox