Table des matières
3 billet(s) pour janvier 2026
| Notes rsh rcp | 2026/01/21 18:08 | Jean-Baptiste |
| Git - Duplication d'un dépôt | 2026/01/19 10:22 | Jean-Baptiste |
| Exemple simple de conf Nagios | 2026/01/14 10:07 | Jean-Baptiste |
Draft Python3
List comprehensions and generator expressions
(short form: “listcomps” and “genexps”)
line_list = [' line 1\n', 'line 2 \n', ' \n', ''] # Generator expression -- returns iterator stripped_iter = (line.strip() for line in line_list) # List comprehension -- returns list stripped_list = [line.strip() for line in line_list] stripped_list = [line.strip() for line in line_list if line != ""]
Autres
def inc(f, id): """ Exemple : inc(lambda x : x+1, 0) """ try : global counter counter = f(counter) except NameError: counter = 0 return counter
http://mgautier.fr/blog/Astuce/changer-lenvironnement-bash-avec-python.html
List unique remove duplicate
def unique(items): found = set([]) keep = [] for item in items: if item not in found: found.add(item) keep.append(item) return keep
How do you split a list into evenly sized chunks in Python?
Source : http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python
def chunks(l, n): n = max(1, n) return [l[i:i + n] for i in range(0, len(l), n)]
Determine if variable is defined in Python
http://stackoverflow.com/questions/1592565/determine-if-variable-is-defined-in-python
try: thevariable except NameError: print "well, it WASN'T defined after all!" else: print "sure, it was defined."
I think it's better to avoid the situation. It's cleaner and clearer to write:
a = None if condition: a = 42
Getting file size in Python
http://stackoverflow.com/questions/6591931/getting-file-size-in-python
def get_Size(file): file.seek(0,2) # move the cursor to the end of the file size = file.tell() return size
Exemple :
with open('plop.bin','rb') as file: size=get_Size(file) with open('plop.bin','rb') as file: #size=get_Size(file) for i in range(size): un, deux = read_hexafile() msg.append(un) msg.append(deux)
/dev/null
fnull = open(os.devnull, 'w')
Message d'erreur
How to print to stderr in Python?
http://stackoverflow.com/questions/5574702/how-to-print-to-stderr-in-python
def warning(*objs): print(*objs, file=sys.stderr)
Conversion hexa
import numpy as np carac=re.sub('^','0x',carac) carac= int(carac, 16) print(carac) file.write(np.byte(carac)) with open('plop.bin', 'bw') as file: file.write(b'\x50\x40\x73') hex(11) # '0xb' a = int('0x100', 16) print(a) #256 print('%x' % a) #100 import binascii binascii.unhexlify('7061756c') # b'paul' "{0:8b}".format(int("a",16)) # ' 1010'
http://stackoverflow.com/questions/16843108/how-to-read-a-hex-file-into-numpy-array
with open(myfile) as f: iv = binascii.unhexlify(f.readline().strip()) key = binascii.unhexlify(f.readline().strip()) count = int(f.readline()) a = np.fromiter((binascii.unhexlify(line.strip()) for line in f), dtype='|S16')
hashlib.sha512('Bonjour'.encode('utf-8')).hexdigest() hex(int.from_bytes('Bonjour'.encode('utf-8'), 'big'))
Créer un Dictionnaire à partir de deux listes (l'une clef, l'autre valeur)
clef = ['a', 'b', 'c'] valeur = [1, 2, 3] dictionnaire=dict(zip(clef, valeur))
Test
Voir :
- nox
- unittest
Exception
http://stackoverflow.com/questions/16138232/is-it-a-good-practice-to-use-try-except-else-in-python
try: s except NameError: pass else: # If no exception occured, do : s.user.logout()
Temps / time
Voir :
- timeit
start = time.time() UnifiedJob.objects.filter(id=1096679).update(status='canceled') end = time.time() print(end - start)
Strings
Debug
Source : https://www.geekarea.fr/wordpress/?p=763
Level 1
f = open('/tmp/debug','a') f.write(variable + '\n') f.close()
Level 2
from pprint import pprint pprint(variable.__class__.__name__, f) pprint(dir(variable), f) pprint(vars(variable), f)
Level 3 (sur une exception)
import traceback f.write(str(traceback.format_exc()))
map reduce filter
Aures
A noter que sur RedHat 8 le chemin vers python est /usr/libexec/platform-python
Python virtualenv pyvenv pyenv pipenv poetry
Voir :
- poetry
pyvenv
Remplacé par python3 -m venv
pyvenv plop cd plop source bin/activate pip install --upgrade pip #deactivate
pyvenv cd env/ cd .. source env/bin/activate pip install -r requirements.txt
python3 -m venv plop source plop/bin/activate
pipenv
Source : http://sametmax.com/pipenv-solution-moderne-pour-remplacer-pip-et-virtualenv/
Préciser la version de Python à utiliser
| python2 | --two |
| python3 | --three |
Install via PIP
python3 -m pip install pip --upgrade --user python3 -m pip install pipenv --user
Install via apt
apt-get install pipenv
Exemple
pipenv run python pipenv shell pipenv install --dev
# Créer le fichier des dépendances pipenv lock # Installer toutes les dépendances pipenv install
Afficher les dépendances
pipenv graph
Chemin du dossier du virtualenv
pipenv --venv
Pb
AttributeError: module 'pip' has no attribute 'get_installed_distributions'
$ pipenv check
Checking PEP 508 requirements…
Passed!
Checking installed package safety…
An error occurred:
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/__main__.py", line 8, in <module>
File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 722, in __call__
File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 697, in main
File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 1066, in invoke
File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 895, in invoke
File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/click/core.py", line 535, in invoke
File "/usr/lib/python3/dist-packages/pipenv/patched/safety.zip/safety/cli.py", line 52, in check
AttributeError: module 'pip' has no attribute 'get_installed_distributions'
Solution
sudo apt-get -y autoremove --purge pipenv python3-virtualenv-clone
$ python3 -m pip install pip --upgrade --user $ python3 -m pip install pipenv --user
Python tests unitaires
Voir aussi:
- unittest (moins pratique que pytest)
Source : http://sametmax.com/un-gros-guide-bien-gras-sur-les-tests-unitaires-en-python-partie-3/
Voir aussi les différents types de test :
- Tests unitaires (au format unittest ou pytest) : units
- Tests d’intégration : integration
- Tests syntaxique : sanity
#pip3 install pytest apt-get install python3-pytest
Voir aussi : Testinfra
test_plop.py
#! /usr/bin/python3 # coding: utf-8 #import pytest from func import plop def test_plop(): assert foo == bar
pytest-3
Autre
~/swift/.unittests
#!/bin/bash TOP_DIR=$(dirname $(realpath "$0")) cd $TOP_DIR/test/unit nosetests --exe --with-coverage --cover-package swift --cover-erase --cover-branches --cover-html --cover-html-dir="$TOP_DIR"/cover $@ rvalue=$? rm -f .coverage cd - exit $rvalue
$ dpkg -S /usr/bin/nosetests python-nose: /usr/bin/nosetests
Note Python pip
Voir aussi :
- poetry
Do not run pip install as root (or with sudo)
Installation dans ~/.local/bin/
easy_install --user pip
Liste des “packages” installés dans la sessions de l'utilisateur :
pip freeze --local
Ou
python -m ensurepip
Voir aussi pipreqs
Installer un package depuis un dépôt git
pip install --user git+https://github.com/benoit-intrw/livestreamer
Mise à jour des “packages” locaux
pip install --local -U livestreamer
Installation d'un “package” dans la session de l'utilisateur (pas besoin de droit root) (pas besoin de prendre le risque d’abîmer sa distro)
pip install --user livestreamer
Tous mettre à jour localement
pip freeze --local | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U
Source http://stackoverflow.com/questions/2720014/upgrading-all-packages-with-pip
Fichier requirements.txt
env1/bin/pip freeze > requirements.txt env2/bin/pip install -r requirements.txt
Voir aussi pipreqs
Exemple de fichier requirements.txt
requirements.txt
numpy>=1.15.0 opencv-python>=3.4.2.17 pandas>=0.23.3 --no-binary pandas Pillow>=5.2.0 tensorflow==1.8.0 tensorflow-gpu==1.8.0
Variable Python - utiliser les dist-packages depuis un environnement virtuel
~/.bashrc
export PYTHONPATH=$PYTHONPATH:/usr/lib/python3/dist-packages/:$HOME/test/lib/python3.5/site-packages/
Utiliser /usr/local
~/.config/pip/pip.conf
[global] target = /usr/local/lib/python2.7/site-packages
Ou
PIP_TARGET=/usr/local/lib/python2.7/site-packages
~/.bashrc
export PYTHONPATH=$PYTHONPATH:/usr/local/lib/python2.7/site-packages
Exemple de PIP en interne
pip.conf
[global] timeout = 300 index-url = https://pypi.acme.fr/pypi/+simple/ cert = /etc/ssl/certs/ca-certificates.crt
pip.conf
mkdir -p ~/.pip/ && \ echo "[global]" >> ~/.pip/pip.conf && \ URLPIP="https://artifactory.packages.install-os.acme.local/api/pypi/ext_pypi/simple" && \ echo "index = $URLPIP " >> ~/.pip/pip.conf && \ echo "index-url = $URLPIP " >> ~/.pip/pip.conf && \ echo "cert = /etc/ssl/certs/ca-certificates.crt " >> ~/.pip/pip.conf
Autres
ssh -R3128:192.168.56.1:3128 user@192.168.205.11 export http_proxy=http://127.0.0.1:3128 export https_proxy=http://127.0.0.1:3128 python3 -m venv test cd test source bin/activate pip install -U pip # pip install --use-feature=2020-resolver plop pip install --no-binary pandas pandas #deactivate
export PIP_BREAK_SYSTEM_PACKAGES=1
constraints
You can achieve this with a constraints file. Just put all your constraints into that file:
constraints.txt
google-api-core==1.16.0
Then you can install via:
python -m pip install -c constraints.txt google-cloud-secret-manager
This will try every version of google-cloud-secret-manager, starting from the most recent version, until it finds a version that is compatible with the given constraints.
pipx
python3 -m pip install --user pipx python3 -m pipx ensurepath
Cluster Pacemaker et Corosync partition GFS2
Brouillon
Voir :
Voir aussi :
/sbin/lvmconf --enable-cluster
ou
/etc/lvm/lvm.conf
locking_type = 3
sed -i -e "s/use_lvmetad \= 1/use_lvmetad \= 0/g" /etc/lvm/lvm.conf systemctl stop lvm2-lvmetad.service dracut -H -f /boot/initramfs-$(uname -r).img $(uname -r)
yum install rgmanager lvm2-cluster gfs2-utils mkfs.gfs2 -p lock_dlm -t my_cluster:FSName -j 2 /dev/sdb
-j 2
2 journaux car deux nœuds
Teste
mount -t gfs2 -o locktable="my_cluster:FSName" /dev/sdb /mnt/gfs/
pcs resource create dlm ocf:pacemaker:controld op monitor interval=30s on-fail=fence clone interleave=true ordered=true pcs resource create clvmd ocf:heartbeat:clvm op monitor interval=30s on-fail=fence clone interleave=true ordered=true pcs constraint order start dlm-clone then clvmd-clone pcs constraint colocation add clvmd-clone with dlm-clone
pcs resource create clusterfs Filesystem device="/dev/vg_data/lv_test1" directory="/mnt/gfs" fstype="gfs2" "options=noatime" \ op monitor interval=10s on-fail=fence clone interleave=true
Pb
Transport endpoint is not connected
mount: mount /dev/sdb on /mnt/gfs failed: Transport endpoint is not connected
Solution
systemctl start dlm
systemctl enable dlm
