Polish up Cabrillo output. (#4)

In detail: The output of attributes is order-independent according to
the spec.  But sanity demands that the same software outputs the same
stuff in a consistent order.  This was not the case previously, as
plain Python dict had been used and had been iterated over, resulting
in ever-changing output produced from run to run.  But consistency
has been achieved with this change.

The Cabrillo spec demands: "All QSO lines must appear in chronological
order."  Now the Cabrillo timestamp granularity is only one minute,
and QSO rates above 60 per hour are not at all uncommon.  So (strictly
speaking), chronological order cannot be achieved by sorting QSOs via
timestamp.  It is somewhat unclear whether anybody would actually care
to receive a file with QSO lines not in chronological order, but only
sorted by Cabrillo timestamps.

We take a conservative stance and never re-order QSO data.  We assume
that incoming data already is ordered chronologically.

This required a move of functionality: The distinction between valid
QSO data and X-QSO data used to be done by using different lists.
Now, one single list is used, and the QSO itself knows whether it
is valid (QSO:) or not (X-QSO:).

We also check, on input, whether incoming data is indeed ordered
chronologically, as the Cabrillo spec demands and as far as can be
checked, given the coarse timestamps.  On disorder found, an exception
is raised.  If you do not need ordering, this check can be switched
off.  If you do switch it off, Cabrillo output generation is switched
off as well, as there is a high risk the output might not be valid
Cabrillo.

README.md

@@ -3,18 +3,52 @@ cabrillo [![Build Status](https://travis-ci.com/thxo/cabrillo.svg?branch=master)
 A Python library to parse Cabrillo-format amateur radio contest logs. 
 
 # Getting Started
+
 ## Basic Parsing
+
 ```python
 >>> from cabrillo.parser import parse_log_file
 >>> cab = parse_log_file('tests/CQWPX.log')
 >>> cab.callsign
 'AA1ZZZ'
 >>> cab.qso
 [<cabrillo.qso.QSO object at 0x10cb09f28>, <cabrillo.qso.QSO object at 0x10cbc8860>]
->>> cab.write_text()
-'START-OF-LOG: 3.0\nCALLSIGN: AA1ZZZ\nCONTEST: CQ-WPX-CW\n[...snip...]END-OF-LOG:'
+>>> cab.text()
+'START-OF-LOG: 3.0\nCREATED-BY: WriteLog V10.72C\nCALLSIGN: AA1ZZZ\n[...snip...]END-OF-LOG:\n'
+```
+
+You can also write to a file:
+
+```python
+with open('out.cbr', 'w') as o:
+    cab.write(o)
+```
+
+The same works for text-file-like objects.
+
+Finally, if you desire to parse Cabrillo data already present as a Python string,
+you can do so with, e.g.,
+
+```python
+from cabrillo.parser import parse_log_text
+
+cabrillo_text = """START-OF-LOG: 3.0
+END-OF-LOG:
+"""
+
+cab = parse_log_text(cabrillo_text)
 ```
+
+## Ignoring malorder
+
+Cabrillo logs must be time-sorted. If you want to read files that are
+not so sorted, but other than that are Cabrillo files, you can do so by
+adding a keyword argument `ignore_order=False` to either `parse_log_file`
+or `parse_log_text`. If you do that, the resulting Cabrillo object
+will refuse to generate (potentially non-)Cabrillo output.
+
 ## Matching Two QSOs in Contest Scoring
+
 ```python
 >>> # We start off with a pair with complementary data.
 >>> from cabrillo import QSO
@@ -33,9 +67,10 @@ False
 ```
 
 # Attributes
+
 Use these attributes to access and construct individual objects.
 
-```
+```python
 class Cabrillo(builtins.object)
  |  Cabrillo(check_categories=True, **d)
  |  
@@ -53,46 +88,48 @@ class Cabrillo(builtins.object)
  |        category_station: One of CATEGORY-STATION.
  |        category_time: One of CATEGORY-TIME.
  |        category_transmitter: One of CATEGORY-TRANSMITTER. Optional for
- |        multi-op.
+ |          multi-op.
  |        category_overlay: One of CATEGORY-OVERLAY.
  |        certificate: If certificate by post. Boolean.
  |        claimed_score: Claimed score in int.
  |        club: Club represented.
  |        created_by: Software responsible for creating this log file.
- |        Optional and defaults to "cabrillo (Python)".
+ |          Optional, defaults to "cabrillo (Python)".
  |        email: Email address of the submitter.
  |        location: State/section/ID depending on contest.
- |        name: Name.
- |        address: Mailing address in list, each entry is each line.
+ |        name: Log submitter's name.
+ |        address: Mailing address, as a list, one entry per line.
  |        address_city: Optional granular address info.
  |        address_state_province: Optional granular address info.
  |        address_postalcode: Optional granular address info.
  |        address_country: Optional granular address info.
- |        operators: List containing each operator's callsign of the station.
+ |        operators: List of operators' callsigns.
  |        offtime: List containing two datetime objects denoting start and
- |        end of off-time.
- |        soapbox: List containing each line of soapbox text at their own entry.
- |        qso: QSO data containing QSO objects.
- |        x_qso: Ignored QSO data containing QSO objects.
- |        x_anything: A dict of ignored/unknown attributes.
+ |          end of off-time.
+ |        soapbox: List of lines of soapbox text.
+ |        qso: List of all QSO objects, including ignored QSOs.
+ |        valid_qso: List of all valid QSOs (excluding X-QSO) (read-only).
+ |        x_qso: List of QSO objects for ignored QSOs (X-QSO only) (read-only).
+ |        x_anything: An ordered mapping of ignored/unknown attributes of the Cabrillo file.
 ```
- 
- ```
+
+```python
  class QSO(builtins.object)
- |  QSO(freq, mo, date, de_call, dx_call, de_exch=None, dx_exch=None, t=None)
+ |  QSO(freq, mo, date, de_call, dx_call, de_exch=[], dx_exch=[], t=None, valid=True)
  |  
  |  Representation of a single QSO.
  |  
  |  Attributes:
- |      freq: Frequency in str representation.
- |      mo: Two letter of QSO. See MODES.
- |      date: UTC time in datetime.datetime object.
+ |      freq: Frequency in kHz in str representation.
+ |      mo: Transmission mode of QSO.
+ |      date: UTC time as datetime.datetime object.
  |      de_call: Sent callsign.
- |      de_exch: Sent exchange incl. RST. List of each component.
+ |      de_exch: Sent exchange. List, first item is RST, second tends to be context exchange.
  |      dx_call: Received callsign.
- |      dx_exch: Received exchange incl. RST. List of each component.
+ |      dx_exch: Received exchange. List, first item is RST, second tends to be context exchange.
  |      t: Transmitter ID for multi-transmitter categories in int. 0/1.
- ```
+ |      valid: True for QSO that counts, False for an X-QSO.
+```
 
 ## Contributors
 
@@ -103,7 +140,7 @@ They assume Python 3.3 or later.
 
 For Posix plattforms (which includes Mac and Linux):
 
-```
+```sh
 git clone https://github.com/thxo/cabrillo.git
 cd cabrillo
 python3 -m venv python-venv

cabrillo/cabrillo.py

@@ -1,10 +1,13 @@
 """Contains code pertaining to parsing and holding individual Cabrillo data.
 """
 # pylint: disable=E1101, E0203
+
+import collections
+import io
+
 from cabrillo import data
 from cabrillo.errors import InvalidLogException
 
-
 class Cabrillo:
     """Representation of a Cabrillo log file.
 
@@ -20,13 +23,13 @@ class Cabrillo:
           category_station: One of CATEGORY-STATION.
           category_time: One of CATEGORY-TIME.
           category_transmitter: One of CATEGORY-TRANSMITTER. Optional for
-          multi-op.
+            multi-op.
           category_overlay: One of CATEGORY-OVERLAY.
           certificate: If certificate by post. Boolean.
-          claimed_score: Claimed score in int.
+          claimed_score: Claimed score as int.
           club: Club represented.
           created_by: Software responsible for creating this log file.
-          Optional and defaults to "cabrillo (Python)".
+            Optional, defaults to "cabrillo (Python)".
           email: Email address of the submitter.
           location: State/section/ID depending on contest.
           name: Name.
@@ -37,15 +40,15 @@ class Cabrillo:
           address_country: Optional granular address info.
           operators: List containing each operator's callsign of the station.
           offtime: List containing two datetime objects denoting start and
-          end of off-time.
-          soapbox: List containing each line of soapbox text at their own entry.
-          qso: QSO data containing QSO objects.
-          x_qso: Ignored QSO data containing QSO objects.
-          x_anything: A dict of ignored/unknown attributes.
-
+            end of off-time.
+          soapbox: List of lines of soapbox text.
+          qso: List of all QSOs, including ignored QSOs.
+          valid_qso: List of all valid QSOs (excluding ignored X-QSO) (read-only).
+          x_qso: List of all invalid QSOs (X-QSO only) (read-only).
+          x_anything: An ordered mapping of ignored/unknown attributes.
     """
 
-    def __init__(self, check_categories=True, **d):
+    def __init__(self, check_categories=True, ignore_order=False, **d):
         """Construct a Cabrillo object.
 
         Use named arguments only.
@@ -58,34 +61,50 @@ def __init__(self, check_categories=True, **d):
         Raises:
             InvalidLogException
         """
-        d.setdefault('version', '3.0')
         d.setdefault('created_by', 'cabrillo (Python)')
-        for key in data.KEYWORD_MAP:
-            setattr(self, key, d.setdefault(key, None))
+        for key in data.OUTPUT_KEYWORD_MAP:
+            setattr(self, key, d.get(key, None))
 
-        self.x_anything = d.setdefault('x_anything', dict())
+        self.x_anything = d.get('x_anything', collections.OrderedDict())
 
-        if self.version != '3.0':
+        version = d.get('version', '3.0')
+        if version != '3.0':
             raise InvalidLogException("Only Cabrillo v3 supported, "
-                                      "got {}".format(self.version))
+                                      "got {}".format(version))
+        else:
+            self.version = version
 
-        if not self.qso:
-            self.qso = list()
+        self.qso = []
+        for qso in d.get('qso', []):
+            self.append_qso(qso, ignore_order)
 
-        if not self.x_qso:
-            self.x_qso = list()
+        self.ignore_order = ignore_order
 
         if check_categories:
             for attribute, candidates in data.VALID_CATEGORIES_MAP.items():
                 value = getattr(self, attribute, None)
                 if value and value not in candidates:
                     raise InvalidLogException(
                         'Got {} for {} but expecting one of {}.'.format(
-                            value,
-                            attribute, candidates))
+                            value, attribute, candidates))
+
+    valid_qso = property(fget=lambda self: [qso for qso in self.qso if qso.valid])
+    x_qso = property(fget=lambda self: [qso for qso in self.qso if not qso.valid])
+
+    def append_qso(self, qso, ignore_order):
+        """Add one QSO to the end of this log."""
+        if 0 < len(self.qso) and qso.date < self.qso[-1].date and not ignore_order:
+            # The Cabrillo spec says QSOs need to be ordered by time.
+            # The timestamps from Cabrillo's point of view
+            # give time only to minute precision, and
+            # QSO rates above 60 / hour are by no means uncommon.
+            # So we refrain from ordering QSOs by timestamps ourselves.
+            raise InvalidLogException("QSOs need to be ordered time-wise.")
+
+        self.qso.append(qso)
 
-    def write_text(self):
-        """write_text generates a Cabrillo log text.
+    def text(self):
+        """Generate the Cabrillo log text.
 
         Arguments:
             None.
@@ -95,46 +114,63 @@ def write_text(self):
               names are automatically replaced to `-` upon text output.
 
         Raises:
-            InvalidLogException when target Cabrillo version is not 3.0.
+            InvalidLogException when target Cabrillo version is not 3.0
+            or ignore_ordered mode is active.
+        """
+        with io.StringIO() as out:
+            self.write(out)
+            return out.getvalue()
+
+    def write(self, file):
+        """writes a Cabrillo log text to the text-file-like object file.
+
+        Arguments:
+            file: Anything that has a write() - method accepting a string.
+                  Cabrillo log file text is written here. `_` in attribute
+                  names are automatically replaced by `-`.
+
+        Raises:
+            InvalidLogException when target Cabrillo version is not 3.0
+            or ignore_ordered mode is active.
         """
         if self.version != '3.0':
             raise InvalidLogException("Only Cabrillo v3 supported.")
 
-        lines = list()
-        lines.append('START-OF-LOG: {}'.format(self.version))
+        if self.ignore_order:
+            raise InvalidLogException("Refuse produce output in ignore_ordered mode as Cabrillo logs need to be ordered time-wise.")
+
+        print('START-OF-LOG: {}'.format(self.version), file=file)
 
         # Output known attributes.
-        for attribute, keyword in data.KEYWORD_MAP.items():
+        for attribute, keyword in data.OUTPUT_KEYWORD_MAP.items():
             value = getattr(self, attribute, None)
             if value is not None:
                 if attribute == 'certificate':
                     # Convert boolean to YES/NO.
-                    if value:
-                        lines.append('{}: YES'.format(keyword))
-                    else:
-                        lines.append('{}: NO'.format(keyword))
-                elif attribute in ['address', 'soapbox', 'qso', 'x_qso']:
+                    print('{}: {}'.format(keyword, 'YES' if value else 'NO'), file=file)
+                elif attribute in ['address', 'soapbox']:
                     # Process multi-line attributes.
-                    output_lines = ['{}: {}'.format(keyword, str(x)) for x in
-                                    value]
-                    lines += output_lines
+                    for x in value:
+                        print('{}: {}'.format(keyword, x), file=file)
                 elif attribute == 'operators':
                     # Process attributes delimited by space.
-                    lines.append('{}: {}'.format(keyword, ' '.join(value)))
+                    print('{}: {}'.format(keyword, ' '.join(value)), file=file)
                 elif attribute == 'offtime':
                     # Process offtime dates.
-                    lines.append('{}: {}'.format(keyword, ' '.join(
-                        [x.strftime("%Y-%m-%d %H%M") for x in value])))
+                    print('{}: {}'.format(keyword, ' '.join(
+                        [x.strftime("%Y-%m-%d %H%M") for x in value])), file=file)
                 elif value and attribute != 'version':
-                    lines.append('{}: {}'.format(keyword, value))
+                    print('{}: {}'.format(keyword, value), file=file)
 
         # Output ignored attributes.
         for attribute, keyword in self.x_anything.items():
-            lines.append('{}: {}'.format(attribute.replace('_', '-'), keyword))
+            print('{}: {}'.format(attribute.replace('_', '-'), keyword), file=file)
 
-        lines.append('END-OF-LOG:')
+        # Output QSOs:
+        for qso in self.qso:
+            print(qso, file=file)
 
-        return '\n'.join(lines)
+        print('END-OF-LOG:', file=file)
 
     def __str__(self):
         return '<Cabrillo for {}>'.format(self.callsign)