AdminSysGNULinux

Outils pour utilisateurs

Outils du site

Piste : • documentation • notes_json
tech:notes_json

Ceci est une ancienne révision du document !

Table des matières

, ,

Notes JSON et YAML

jc JSON Convert JSONifies the output of many CLI tools and file-types
jq Command-line JSON processor
json_pp JSON Pretty Printer
JMESPath JMESPath is a query language for JSON (utilisé par json_query d'Ansible
jp The jp command is a command line interface to JMESPath

Voir :

  • JSONPath et go_template
  • JSON RFC 6902 patch

Voir Ansible :

Voir aussi : 

sudo apt-get install jc
dig www.google.com | jc --dig -p
[
  {
    "id": 17598,
    "opcode": "QUERY",
    "status": "NOERROR",
    "flags": [
      "qr",
      "rd",
      "ra"
    ],
    "query_num": 1,
    "answer_num": 1,
    "authority_num": 0,
    "additional_num": 1,
    "opt_pseudosection": {
      "edns": {
        "version": 0,
        "flags": [],
        "udp": 4096
      }
    },
    "question": {
      "name": "www.google.com.",
      "class": "IN",
      "type": "A"
    },
    "answer": [
      {
        "name": "www.google.com.",
        "class": "IN",
        "type": "A",
        "ttl": 38,
        "data": "142.250.178.132"
      }
    ],
    "query_time": 8,
    "server": "208.67.222.123#53(208.67.222.123) (UDP)",
    "when": "Sun Oct 16 21:54:04 CEST 2022",
    "rcvd": 59,
    "when_epoch": 1665950044,
    "when_epoch_utc": null
  }
]
cat fic.json |python3 -m json.tool |native2ascii -encoding UTF-8 -reverse

Extension Firefox http://jsonview.com/ 

sudo apt-get install yajl-tools

yajl-tools

  • json_reformat
  • json_verify
mongoimport --host localhost --db database --collection collection <fichier.json --jsonArray

Convert JSON to YAML 

yq -P sample.json

Ou encore 

python -c 'import sys, yaml, json; yaml.safe_dump(json.load(sys.stdin), sys.stdout, default_flow_style=False)' < /opt/stack/keystone/etc/policy.v3cloudsample.json > /etc/keystone/policy.yaml
pip install jsbeautifier
js-beautify file.js

Dico vers JSON ? 

cat plop.t |tr \' \" |sed -e 's/None/""/g' |jq .

VIM - Formater le JSON 

:%!python -m json.tool

Requête (Query)

Avec curl 

curl -k https://aap-controller.local/api/v2/users/27/roles/ -X POST -u user:password --data-raw '{"id":534}'

Avec jq voir : https://hyperpolyglot.org/json

Top level sections 

cat single-instance.json |jq 'keys[]'

Exemple wildcard 

 
$ curl -s -u "${AAP_USER}:${AAP_PASS}" "https://aap.acme.local/api/v2/job_templates/" | jq '.results[0].id'
1680

$ curl -s -u "${AAP_USER}:${AAP_PASS}" "https://aap.acme.local/api/v2/job_templates/" | jq '.results[].id' | tail -3
1572
1569
2343

jq retours sans guillemet (quote) : 

jq -r
docker inspect portainer/portainer-ce |jq '.[0].RepoTags'
ansible -i testsrv, -m setup all |sed -e 's/^.*=>//' |jq -r '.ansible_facts.ansible_mounts[] |.mount'

Filtres

Conversion string integer avec tonumber

Exemple 

jq '.blockdevices[].size | tonumber'

Autres

Ansible - Utilisation de boolean en extra vars via JSON

ansible-playbook playbook.yml -i inventory.ini -e '{ "remove_vhost": false }'

YAML

yq portable command-line YAML processor
yb YAML Parser in bash

Voir :

Contrôle syntaxe :

Ce que jq est à json, yq l'est pour le yaml.

yq permet également de convertir des fichiers JSON en YAML 

cat .kube/config |egrep -v "\-data"  |yq .
# pip install --user yq
sudo apt-get install yq
openstack image show IMAGE1 -c properties -f yaml |yq '.properties.direct_url'

Filtrer sur les clefs (keys) 

cat meta/runtime.yml | yq -r '.action_groups | keys'
# Ou
cat meta/runtime.yml | yq -r '.action_groups | keys[]'

Mofifier un fichier YAML

avec yq :

En Python et autres :

Avec Ansible : https://github.com/kwoodson/ansible-role-yedit

Exemple non-specific tag 

configuration: !include config.d/*.yaml

Reférences 

vars:
  webapp:
      version: &my_version 1.0
      custom_name:
          - "ToDo_App"
          - *my_version

Exemple Python

import yaml
 
with open("filter.yml", "r") as yamlfile:
    #filters = yaml.load(yamlfile, Loader=yaml.FullLoader)
 
    filters = yaml.safe_load(yamlfile)

Échapper certains caractères

Passer un fichier YAML à une API REST qui fonctionne en JSON

cat inv.yaml |sed -e 's/$/\\n/' -e 's/"/\\"/g' |tr -d '\n'

Exemple YAML vers JSON

inv.yaml 

---
server1:
  hosts:
    server1.inf.acme.lan:
  vars:
    SET_LINGERING_LIST:
      - SET_LINGERING_USER: testplop
        SET_LINGERING_ENABLED: "false"
      - SET_LINGERING_USER: testplop
        SET_LINGERING_ENABLED: "true"

type_host:
  hosts:
    server1.inf.acme.lan: {}
  vars:
    type: host
#! /bin/bash
set -euo pipefail
 
YAML="$(cat inv.yaml |sed -e 's/$/\\n/' -e 's/"/\\"/g' |tr -d '\n')"
 
curl -v -k -u user1:'P@ssw0rd' -H 'Content-Type: application/json' -X POST https://awx.acme.fr/api/v2/job_templates/81/launch/ -d '{
"extra_vars": {
"foo1": "bar1",
"foo2": "bar2",
"inventory_content": "'"${YAML}"'"
 
}
}'

Ou avec un heredoc 

YAML="$(cat <<'EOF' | sed -e 's/$/\\n/' -e 's/"/\\"/g' |tr -d '\n'
---
server1:
  hosts:
    server1.inf.acme.lan:
  vars:
    SET_LINGERING_LIST:
      - SET_LINGERING_USER: testplop
        SET_LINGERING_ENABLED: "false"
      - SET_LINGERING_USER: testplop
        SET_LINGERING_ENABLED: "true"
 
type_host:
  hosts:
    server1.inf.acme.lan: {}
  vars:
    type: host
EOF
)"

En convertissant le YAML en JSON

En passant un JSON dans un JSON en string 

#JS="$(ansible-inventory -i inv.yaml --list)"
JS="$(cat inv.js)"
 
JS="$(echo $JS |sed 's/"/\\"/g')"
URL=''
 
curl -v -k -u user1:'P@ssw0rd' -H 'Content-Type: application/json' -X POST https://awx.acme.fr/api/v2/job_templates/81/launch/ -d '{
"extra_vars": {
"foo1": "bar1",
"foo2": "bar2",
"inventory_content": "'"${JS}"'"
 
}
}'

Exemple de yaml

CRON_HOST_TARGETS:
  all:
    - CRON_FILENAME: crontab_app1
      CRON_USER: app1
      CRONS:
        - DESCRIPTION: "topmem.sh exécuté toutes les heures"
	        SCHED: "0 * * * *"
	        CMDLINE: "~/scripts/topmem.sh >> ~app1/logs/topmem.out 2>&1"

  sc01:
    - CRON_FILENAME: crontab_app1_sc01
      CRON_USER: app1
      CRONS:
        - DESCRIPTION: "archivage de certains fichiers applicatifs"
	        SCHED: "06 03 * * *"
	        CMDLINE: ". ~/scripts/archive_plopsc.sh -l${LINE} >> ~app1/logs/archive_sc_${LINE}.out 2>&1"

        - DESCRIPTION: Performances
	        SCHED: "05 03 * * *"
	        CMDLINE: ". ~/scripts/appexec.sh appenv.sh ${LINE} > /dev/null; ~/scripts/stat_report_run.sh ${LINE} > /dev/null"

Pb

Compile Error - Échapper certains caractères

plop.yml 

rec:plop:
  stage: plop
  variables:
    ENV: "RECETTE"

prd:plop:
  stage: plop
  variables:
    ENV: "PRODUCTION"
$ cat plop.yml |yq '.rec:plop'
jq: error: syntax error, unexpected ':', expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.rec:plop    
jq: 1 compile error

Solution

cat plop.yml |yq '.["rec:plop"]'
tech/notes_json.1764845465.txt.gz · Dernière modification : de Jean-Baptiste
Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki