Challenge Outline & Goals

  • OPNFV uses PDF to specific POD Descriptor File
  • Airship does not use PDF; instead it uses a “treasure map” manifest file that’s a combination of PDF + SDF (Scenario Descriptor File)
  • Discussion around creating a PDF to Airship Manifest 1.0 converter
  • Discussion around expanding to scope to potentially replacing PDF with the new format

Resources

Attendees

  1. Mark Shostak
  2. Daniel Balsiger
  3. Al Morton

Notes, etc.

  • TBD

Treasuremap Files

When looking shortly at treasuremap I see at least those files affected, i.e. containing information which is in OPNFV PDF (it is not a complete list):

site/${your_site}/profiles/hardware/${your_bm_node_type}.yaml
site/${your_site}/baremetal/nodes.yaml
site/${your_site}/networks/common-addresses.yaml
site/${your_site}/networks/physical/networks.yaml
site/${your_site}/profiles/host/compute.yaml
site/${your_site}/secrets/passphrases/ipmi_admin_password.yaml

Team, Here is a link to the actual manifest files used to deploy various pods in the Intel lab: https://github.com/opnfv/airship/tree/master/site

This is the OPNFV deployment guide for Airship, and is focused on deploying in the pods in the Intel lab: https://wiki.opnfv.org/display/AIR/Airship+Installer+Deployment+Guide

Here are the generic treasuremaps, released every few months (1.6 is the latest at the time of writing): https://github.com/airshipit/treasuremap/releases

Machine generated PDF from UNH:

sample_pdf_from_UNH.yaml

  • No labels

21 Comments

  1. James Gu

    Is there a time and zoom link for remote attendees?

    1. Mark Shostak

      Hi James, Unfortunately we're holding these F2F meetings in one of the smaller break-out rooms, and there is no zoom channel that I'm aware of. Jim Baker Jim, can you confirm pls? Thanks, -Mark

  2. Daniel Balsiger

    Could someone paste a link to the PDF file we are about to use as a basis ?

    1. Paweł Wieczorek

      Hi Daniel, I believe relevant PDFs might be these: https://git.opnfv.org/pharos/tree/config/pdf (taken from slide #4: https://wiki.opnfv.org/download/attachments/6816289/OPNFV_PDF_v1.0_28_Feb_2018.pptx linked at: https://wiki.opnfv.org/display/INF/POD+Descriptor)

      I would appreciate if links above could be confirmed by OPNFV/Airship experts

      1. Daniel Balsiger

        Thank you Pawel, as far as I can judge, those are the ones shown in Mark's talk.

      2. Aric Gardner

        Yes I would also like to see an actual PDF as spit out by LAAS


        It appears that airship runs on intel-pod17 and we only have a PDF for intel-pod18

        https://github.com/opnfv/pharos/blob/master/labs/intel/pod18.yaml


        1. Mark Shostak

          Aric, Were you following the URL from the wiki (above)? https://github.com/opnfv/airship/tree/master/site

          I see manifests for all 4 pods. Can anyone else see the other 3 pods at that link?

          -M


  3. Mark Shostak

    LibYAML

    Following up on some ad-hoc conversations by the carb station, the name of the python library for serializing/de-serializing YAML is called LibYAML.

    ( https://github.com/yaml/pyyaml.org/blob/master/wiki/LibYAML.md )

  4. Aric Gardner

    I have used this before https://pypi.org/project/ruamel.yaml/ 

    It can modify/ammend etc existing yaml files, which may come in handy 

  5. Aric Gardner

    Maybe ipmi user/pass is a good place to start. 

    So we have https://github.com/opnfv/airship/blob/master/site/intel-pod18/secrets/passphrases/ipmi_admin_password.yaml on the airship side

    and  https://github.com/opnfv/pharos/blob/master/labs/intel/pod18.yaml#L42 on the PDF side 

  6. Aric Gardner

    okay this work, it reads the "passord" from the PDF and then writes it to the ipmi_admin_password.yaml


      import sys
  import ruamel.yaml
  import yaml
  ryaml = ruamel.yaml.YAML()
  ryaml.preserve_quotes = True
  ryaml.indent(mapping=4, sequence=6, offset=4)
  ryaml.explicit_start = True

  pdf = "pod18.yaml"
  ipmi_file = "ipmi_admin_password.yaml"

  with open(pdf) as f:
      pdf_data = yaml.safe_load(f)

  #get password from pdf
  password = pdf_data['jumphost']['remote_params']['pass']


  with open(ipmi_file, 'r+') as f:
      ipmi_data = ryaml.load(f)
      #The password in the ipmi_admin_password.yaml file is called "data" a bit confusing... 
      ipmi_data['data'] = password

      with open(ipmi_file, 'w') as f:
          ryaml.dump(ipmi_data, f)

    1. Daniel Balsiger

      Good to read, I tried something similar, you were faster (smile)

      1. Mark Shostak

        Nice Aric! Can you show us the output?

        1. Daniel Balsiger

          When using 'testpasswd' in pod18.yaml and running Aric's code, the ipmi_admin_password.yaml looks like this:

          ---
schema: deckhand/Passphrase/v1
metadata:
    schema: metadata/Document/v1
    name: ipmi_admin_password
    layeringDefinition:
        abstract: false
        layer: site
    labels:
        name: ipmi-admin-password-site
    storagePolicy: cleartext
data: testpasswd

          I consider that as successful.

  7. Mark Shostak

    All, I've attached a machine generated PDF from LaaS, that Lincoln helped us out with.

    sample_pdf_from_UNH.yaml

    1. Daniel Balsiger

      Aric's code successfully modifies the ipmi_admin_password.yaml when using Lincolns sample PDF.

      1. Mark Shostak

        That's awesome! The next question is, what are our options to scale this? The options that come to my mind are:

        • Extend Aric's script // this would require hardcoding everything (sad)
        • Use something table-driven
        • Hack up something based on jcopy from Georg's email
        • Other ideas?

        What do people think?

        1. Aric Gardner

          Let's make yet another yaml file to do the mapping! XD

          1. Mark Shostak

            I figured you'd have already done it! (smile)

            1. Mark Shostak

              Team, Daniel and I had a quick chat and have a proposal for scaling Aric's demo script. Basically, we want to format the code so the actual translation portion is isolated, clearly identified and can be easily replicated. Then, anyone on the team who has a few minutes can add a translation for another datum, and with a combined effort we can brute-force it. It's not as flexible as meta-data driven solution, but hey, it's a hack. That's the whole point in a hackathon!

              Regarding a source "repository", our thinking is to start a quick and dirty email thread, strictly to hold the source. It's short and only a single file, so that should suffice. If someone wants to put it on github, that's great, but our thinking is to save the time and use the brute-force approach and knock out as much as we can.

              Thoughts? Aric, what do you think?

  8. Mark Shostak

    Reminder - Can everyone who attended on Monday, enter their name under "Attendees" in the upper-right of the wiki page. Thanks!