東川印記

一本東川,笑看爭龍斗虎;寰茫兦者,度橫佰昧人生。

CentOS 8.1配置Shell脚本通过crontab自动运行Python3应用

2021年2月22日星期一



本地可以用

服务器也能用


1,确定系统版本

[root@ss2020x2 senRsl]# uname -a
Linux ss2020x2 4.18.0-147.5.1.el8_1.x86_64 #1 SMP Wed Feb 5 02:00:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[root@ss2020x2 senRsl]# uname -r
4.18.0-147.5.1.el8_1.x86_64
[root@ss2020x2 senRsl]# lsb_release -a
bash: lsb_release: command not found
[root@ss2020x2 senRsl]# cat /etc/centos-release
CentOS Linux release 8.1.1911 (Core)
[root@ss2020x2 senRsl]# getconf LONG_BIT
64
[root@ss2020x2 senRsl]#

系统版本为CentOS 8.1.1911 64位


2,安装Python3和pip3

先更新一下源。。。。

[root@ss2020x2 senRsl]# yum install -y update
Failed to set locale, defaulting to C.UTF-8
Last metadata expiration check: 0:25:49 ago on Mon Feb 22 04:04:47 2021.
No match for argument: update
Error: Unable to find a match: update
[root@ss2020x2 senRsl]# yum clean all
Failed to set locale, defaulting to C.UTF-8
39 files removed
[root@ss2020x2 senRsl]# yum clean metadata
Failed to set locale, defaulting to C.UTF-8
Cache was expired
0 files removed
[root@ss2020x2 senRsl]# yum install -y update
Failed to set locale, defaulting to C.UTF-8
CentOS-8 - AppStream                                                                                                    4.5 MB/s | 6.3 MB     00:01   
CentOS-8 - Base                                                                                                         3.7 MB/s | 2.3 MB     00:00   
CentOS-8 - Extras                                                                                                        16 kB/s | 8.6 kB     00:00   
CentOS-8 - PowerTools                                                                                                   2.7 MB/s | 2.0 MB     00:00   
Google Compute Engine                                                                                                   3.7 kB/s | 7.4 kB     00:02   
Google Cloud SDK                                                                                                         15 MB/s |  26 MB     00:01   
No match for argument: update
Error: Unable to find a match: update
[root@ss2020x2 senRsl]#
[root@ss2020x2 senRsl]# cd /etc/yum.repos.d/
[root@ss2020x2 yum.repos.d]# ls
CentOS-AppStream.repo  CentOS-Debuginfo.repo  CentOS-HA.repo          CentOS-Sources.repo     CentOS-fasttrack.repo
CentOS-Base.repo       CentOS-Devel.repo      CentOS-Media.repo       CentOS-Vault.repo       google-cloud.repo
CentOS-CR.repo         CentOS-Extras.repo     CentOS-PowerTools.repo  CentOS-centosplus.repo
[root@ss2020x2 yum.repos.d]#

好吧,版本旧点就旧点,不更新源了。。。。

[root@ss2020x2 senRsl]# yum install python3
Failed to set locale, defaulting to C.UTF-8
CentOS-8 - AppStream                                                                                                    5.7 MB/s | 6.3 MB     00:01   
CentOS-8 - Base                                                                                                         3.7 MB/s | 2.3 MB     00:00   
CentOS-8 - Extras                                                                                                        10 kB/s | 8.6 kB     00:00   
CentOS-8 - PowerTools                                                                                                   3.0 MB/s | 2.0 MB     00:00   
Google Compute Engine                                                                                                   3.6 kB/s | 7.4 kB     00:02   
Google Cloud SDK                                                                                                         14 MB/s |  26 MB     00:01   
Package python36-3.6.8-2.module_el8.1.0+245+c39af44f.x86_64 is already installed.
Dependencies resolved.
========================================================================================================================================================
 Package                      Architecture               Version                                                    Repository                     Size
========================================================================================================================================================
Upgrading:
 python36                     x86_64                     3.6.8-2.module_el8.3.0+562+e162826a                        AppStream                      19 k

Transaction Summary
========================================================================================================================================================
Upgrade  1 Package

Total download size: 19 k
Is this ok [y/N]: y
Downloading Packages:
python36-3.6.8-2.module_el8.3.0+562+e162826a.x86_64.rpm                                                                 381 kB/s |  19 kB     00:00   
--------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                    34 kB/s |  19 kB     00:00    
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                1/1
  Upgrading        : python36-3.6.8-2.module_el8.3.0+562+e162826a.x86_64                                                                            1/2
  Running scriptlet: python36-3.6.8-2.module_el8.3.0+562+e162826a.x86_64                                                                            1/2
  Cleanup          : python36-3.6.8-2.module_el8.1.0+245+c39af44f.x86_64                                                                            2/2
  Running scriptlet: python36-3.6.8-2.module_el8.1.0+245+c39af44f.x86_64                                                                            2/2
  Verifying        : python36-3.6.8-2.module_el8.3.0+562+e162826a.x86_64                                                                            1/2
  Verifying        : python36-3.6.8-2.module_el8.1.0+245+c39af44f.x86_64                                                                            2/2

Upgraded:
  python36-3.6.8-2.module_el8.3.0+562+e162826a.x86_64                                                                                                  

Complete!
[root@ss2020x2 senRsl]# python3 --version
Python 3.6.8
[root@ss2020x2 senRsl]#

python 有了。。。。

python3安装完成后自带了pip3

[root@ss2020x2 senRsl]# pip3 --version
pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)
[root@ss2020x2 senRsl]#

省事

3,clone要自动运行的Python脚本

git clone 要改git库的权限,还是 scp传本地更省事

[root@ss2020x2 senRsl]# mkdir blog

SENRSL:blog senrsl$ scp -r blogger dong_M@ss.dcjz.ml:/home/dong_M/
insertShort.py                                                                                                        100% 5261    28.3KB/s   00:00   
blogger.dat                                                                                                           100% 1151     6.2KB/s   00:00   
getOnce.py                                                                                                            100% 1060     5.8KB/s   00:00   
.DS_Store                                                                                                             100% 6148    33.9KB/s   00:00   
getOnce.cpython-39.pyc                                                                                                100%  941     5.3KB/s   00:00   
insert_once.log                                                                                                       100% 2783    15.6KB/s   00:00   
sanchuan.iml                                                                                                          100%  338     1.9KB/s   00:00   
client_secrets.json                                                                                                   100%  504     2.9KB/s   00:00   
.DS_Store                                                                                                             100% 6148    32.8KB/s   00:00   
blogger.py                                                                                                            100% 8859    48.2KB/s   00:00   
blogspot163.iml                                                                                                       100%  338     1.8KB/s   00:00   
dom.cpython-39.pyc                                                                                                    100%  481     2.7KB/s   00:00   
test.py                                                                                                               100%   75     0.4KB/s   00:00   
client_secrets.json                                                                                                   100%  496     2.7KB/s   00:00   
dom.py                                                                                                                100% 3196    17.5KB/s   00:00   
.DS_Store                                                                                                             100% 8196    43.8KB/s   00:00   
blogger.iml                                                                                                           100%  338     1.9KB/s   00:00   
test.json                                                                                                             100%    0     0.0KB/s   00:00   
.gitignore                                                                                                            100%   25     0.1KB/s   00:00   
senrsl.iml                                                                                                            100%  338     1.9KB/s   00:00   
.DS_Store                                                                                                             100% 6148    33.3KB/s   00:00   
list.py                                                                                                               100% 2809    15.1KB/s   00:00   
client_secrets.json                                                                                                   100%  496     2.7KB/s   00:00   
startAuto.sh                                                                                                          100%  286     1.6KB/s   00:00   
Project_Default.xml                                                                                                   100%  410     2.3KB/s   00:00   
dbnavigator.xml                                                                                                       100%   23KB 124.0KB/s   00:00   
vcs.xml                                                                                                               100%  183     0.9KB/s   00:00   
.gitignore                                                                                                            100%  176     0.9KB/s   00:00   
workspace.xml                                                                                                         100%   10KB  55.3KB/s   00:00   
modules.xml                                                                                                           100%  609     3.3KB/s   00:00   
misc.xml                                                                                                              100%  251     1.4KB/s   00:00   
SENRSL:blog senrsl$

[root@ss2020x2 senRsl]# cp -r ../dong_M/blogger blog/
[root@ss2020x2 senRsl]# ls blog/
blogger
[root@ss2020x2 senRsl]# ls -l blog/blogger/
total 8
-rw-r--r-- 1 root root 338 Feb 22 04:54 blogger.iml
drwxr-xr-x 3 root root 139 Feb 22 04:54 blogspot163
drwxr-xr-x 3 root root 171 Feb 22 04:54 sanchuan
drwxr-xr-x 2 root root  83 Feb 22 04:54 senrsl
-rwxr--r-- 1 root root 286 Feb 22 04:54 startAuto.sh
-rw-r--r-- 1 root root   0 Feb 22 04:54 test.json

[root@ss2020x2 senRsl]# rm -rf ../dong_M/blogger
[root@ss2020x2 senRsl]#

脚本有了

吃饭2021年02月22日12:58:21


4,配置Python脚本所需环境

安装venv

[dong_M@ss2020x2 blog]$ sudo su
[root@ss2020x2 blog]# pip3 install virtualenv

这个安了命令找不到,就当没安过。。。。

直接安 google api client

[root@ss2020x2 blog]# pip3 install google-api-python-client
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting google-api-python-client

...

Installing collected packages: setuptools, pyasn1, pyasn1-modules, rsa, six, cachetools, google-auth, protobuf, pyparsing, packaging, certifi, chardet, urllib3, requests, googleapis-common-protos, pytz, google-api-core, httplib2, google-auth-httplib2, uritemplate, google-api-python-client
Successfully installed cachetools-4.2.1 certifi-2020.12.5 chardet-4.0.0 google-api-core-1.26.0 google-api-python-client-1.12.8 google-auth-1.27.0 google-auth-httplib2-0.0.4 googleapis-common-protos-1.52.0 httplib2-0.19.0 packaging-20.9 protobuf-3.15.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyparsing-2.4.7 pytz-2021.1 requests-2.25.1 rsa-4.7.1 setuptools-53.0.0 six-1.15.0 uritemplate-3.0.1 urllib3-1.26.3
[root@ss2020x2 blog]#

再安google-api-client的登录依赖库。。。。

[root@ss2020x2 blog]# pip3 install --upgrade oauth2client
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
。。。
Installing collected packages: oauth2client
Successfully installed oauth2client-4.1.3
[root@ss2020x2 blog]#

环境应该可以了,运行脚本验证下。。。。

没安虚拟环境,就不需要切虚拟环境了,直接运行。。。。

[root@ss2020x2 blog]# cd blogger/sanchuan/
[root@ss2020x2 sanchuan]# python insertShort.py --logging_level=DEBUG --noauth_local_webserver
bash: python: command not found
[root@ss2020x2 sanchuan]# python3 insertShort.py --logging_level=DEBUG --noauth_local_webserver
This user's display name is: 三川
The blog named 'TEST BY SENRSL' is at: http://testbysenrsl.blogspot.com/,171507978679241883
  我一生都在等你 (http://testbysenrsl.blogspot.com/2021/02/blog-post_22.html)
  This is the Beijing municipal public security bureau official weibo, here you can learn about the B (http://testbysenrsl.blogspot.com/2021/02/this-is-beijing-municipal-public_22.html)
  Contribute to QQ email @ QQFocus on quantitative research investment, the stock market investment (http://testbysenrsl.blogspot.com/2021/02/contribute-to-qq-email-qqfocus-on.html)
  Elder sister a list of the daily topic. Interactive topic every nightJust want to be a beloved cl (http://testbysenrsl.blogspot.com/2021/02/elder-sister-list-of-daily-topic_22.html)
  Every day of the rest of your life with love and happiness to defineThere is a smile is called un (http://testbysenrsl.blogspot.com/2021/02/every-day-of-rest-of-your-life-with.html)
  A plain, sincere, love lifeA plain, sincere, love lifeA plain, sincere, love lifeA plain, sin (http://testbysenrsl.blogspot.com/2021/02/a-plain-sincere-love-lifea-plain_22.html)
  Pearl pill of finches, Jane does not play. < br  > kingdom of expensive, not cooked fish. < br  > w (http://testbysenrsl.blogspot.com/2021/02/pearl-pill-of-finches-jane-does-not.html)
  Tencent video (v.q q) located in Chinas leading integrated video portal, content is given priority  (http://testbysenrsl.blogspot.com/2021/02/tencent-video-vq-q-located-in-chinas.html)
  A plain, sincere, love lifeState the family whats going on in the world, everything is concerned. (http://testbysenrsl.blogspot.com/2021/02/a-plain-sincere-love-lifestate-family.html)
  The Peoples Daily corporate microblog. Participation, communication, record times.Contribute to Q (http://testbysenrsl.blogspot.com/2021/02/the-peoples-daily-corporate-microblog_21.html)
总共帖子 10 条
返回 200
地窖,
 地址 http://testbysenrsl.blogspot.com/2021/02/blog-post_46.html ,作者 三川 分类 ['短篇集']
[root@ss2020x2 sanchuan]#

看起来很轻松。。。。

然后只要稍微改改脚本就行了。。。。

[root@ss2020x2 senRsl]# ./blog/blogger/startAuto.sh
start auto...
。。。
思想控制的荒唐史,
 地址 http://testbysenrsl.blogspot.com/2021/02/blog-post_16.html ,作者 三川 分类 ['短篇集']
[root@ss2020x2 senRsl]#

脚本

[root@ss2020x2 senRsl]# cat /home/senRsl/blog/blogger/startAuto.sh
#!/usr/bin/env sh

# SENRSL:blog senrsl$ ./blogger/startAuto.sh

echo "start auto..."

# export http_proxy=http://127.0.0.1:1087;export https_proxy=http://127.0.0.1:1087;
# source venv/bin/activate
cd /home/senRsl/blog/blogger/sanchuan/
python3 insertShort.py --logging_level=DEBUG --noauth_local_webserver
[root@ss2020x2 senRsl]#

环境就好了。。。。


5,配置crontab自动运行命令

幸好crontab是系统默认安装上的。。。。

[root@ss2020x2 senRsl]# systemctl status crond
● crond.service - Command Scheduler
   Loaded: loaded (/usr/lib/systemd/system/crond.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2021-02-11 13:12:27 UTC; 1 weeks 3 days ago
 Main PID: 1413 (crond)
    Tasks: 1 (limit: 4771)
   Memory: 3.7M
   CGroup: /system.slice/crond.service
           └─1413 /usr/sbin/crond -n

Feb 22 03:01:01 ss2020x2 CROND[7751]: (root) CMD (run-parts /etc/cron.hourly)
Feb 22 03:01:01 ss2020x2 anacron[7760]: Anacron started on 2021-02-22
Feb 22 03:01:01 ss2020x2 anacron[7760]: Will run job `cron.daily' in 48 min.
Feb 22 03:01:01 ss2020x2 anacron[7760]: Jobs will be executed sequentially
Feb 22 03:49:01 ss2020x2 anacron[7760]: Job `cron.daily' started
Feb 22 03:49:01 ss2020x2 anacron[7760]: Job `cron.daily' terminated
Feb 22 03:49:01 ss2020x2 anacron[7760]: Normal exit (1 job run)
Feb 22 04:01:01 ss2020x2 CROND[7879]: (root) CMD (run-parts /etc/cron.hourly)
Feb 22 05:01:01 ss2020x2 CROND[9211]: (root) CMD (run-parts /etc/cron.hourly)
Feb 22 06:01:01 ss2020x2 CROND[9342]: (root) CMD (run-parts /etc/cron.hourly)
[root@ss2020x2 senRsl]# crontab -l
no crontab for root
[root@ss2020x2 senRsl]#

先编辑个简单点的吧。。。。

[root@ss2020x2 senRsl]# crontab -e
no crontab for root - using an empty one
crontab: installing new crontab
[root@ss2020x2 senRsl]# crontab -l
22 */8 */2 * * /home/senRsl/blog/blogger/startAuto.sh

[root@ss2020x2 dong_M]# systemctl restart crond.service

[root@ss2020x2 senRsl]#

每隔两天,每隔8小时的22分执行脚本。。。。

crontab规则为  分时日月周 命令

给shell里加上 推送通知脚本,就可以监测运行情况了。。。。

然后写了个notice.py,遇到一个脚本在上级目录一直无法运行问题。。。。

最终,找到一个能用的。。。。

# 导入上级目录中文件,屮。。。。
# sys.path.append("../")
# from .. import notice
# from ... import blogger
# from ..notice import toDingTalk
# import path, sys
# folder = path.path(__file__).abspath()
# sys.path.append(folder.parent.parent)
sys.path.append("..")
sys.path.extend([os.path.join(root, name) for root, dirs, _ in os.walk("../") for name in dirs])
from notice import toDingTalk

浪费小半天时间。。。。


--
senRsl
2021年02月22日12:01:22

没有评论 :

发表评论