new doc

Author: digoal zhou

201606/20160626_01.md

@@ -47,7 +47,7 @@ https://yq.aliyun.com/articles/38377
 逻辑更复杂一些的场景，需要将数据取到应用端，在应用端处理，这会涉及到move data，也会较大程度的放大网络RT。move data的模式正在逐渐成为影响用户体验、效率，浪费成本的罪魁祸首。    
 
 ![2](https://oss-cn-hangzhou.aliyuncs.com/yqfiles/62dfdae72172a450199e8101efc7c28f8fba0e16.png)  
-如果能把数据库打造成为同事具备数据存储、管理与处理能力为一体的产品。在数据库硬件资源充足的情况下，把一些数据库能处理的逻辑交给数据库处理，将极大的降低延迟，在高并发低延迟的应用场景非常有效。    
+如果能把数据库打造成为同时具备数据存储、管理与处理能力为一体的产品。在数据库硬件资源充足的情况下，把一些数据库能处理的逻辑交给数据库处理，将极大的降低延迟，在高并发低延迟的应用场景非常有效。    
 
 这考验的就是数据库的扩展能力。    
 <br />  

201607/20160728_01.md

@@ -565,7 +565,7 @@ UBER文章说的 查询会与恢复堵塞，说的是物理备库，但必须纠
 
 * Application of a vacuum cleanup record from WAL conflicts with standby transactions whose snapshots can still "see" any of the rows to be removed.  
 
-  主库回收dead tuple的REDO，同事备库当前的query snapshot需要看到这些记录。     
+  主库回收dead tuple的REDO，同时备库当前的query snapshot需要看到这些记录。     
 
   这种情况可以通过参数控制，恢复优先，或查询优先。 可以配置时间窗口。     
 

201608/20160815_03.md

@@ -39,7 +39,7 @@ PostgreSQL 的基于流复制的物理备库是基于redo的物理块复制备
 
 * Application of a vacuum cleanup record from WAL conflicts with standby transactions whose snapshots can still "see" any of the rows to be removed.  
 
-  主库回收dead tuple的REDO，同事备库当前的query snapshot需要看到这些记录。     
+  主库回收dead tuple的REDO，同时备库当前的query snapshot需要看到这些记录。     
 
   这种情况可以通过参数控制，恢复优先，或查询优先。 可以配置时间窗口。     
 

201610/20161012_01.md

@@ -14,7 +14,7 @@ PostgreSQL , DaaS , 模板 , schema , database , apply delay , standby
 ## 背景    
 市面上有一些提供DaaS服务的厂商，例如heroKu，可能有上百万的数据库服务；  
 
-又比如提供PaaS平台的服务商，数据库也会有很多，同事这些数据库可能也是模板化的，这些厂商并不一定是为每个客户都新建一个数据库集群来满足数据库的需求。  
+又比如提供PaaS平台的服务商，数据库也会有很多，同时这些数据库可能也是模板化的，这些厂商并不一定是为每个客户都新建一个数据库集群来满足数据库的需求。  
 
 很有可能是使用数据库或者schema来隔离不同用户的。  
 

201702/20170215_02.md

@@ -25,7 +25,7 @@ https://blog.tankywoo.com/2015/01/29/sysctl-load-order.html
 
 昨天恰好又出现了, 于是今天准备写一个监控net.ipv4.ip_forward值的脚本, 才写到一半, 跳闸了, 网关机器启动后, 发现无法转发, nat表都是正常的, 继而发现ip_forward也被改为0了.  
 
-改回来后, 过了一会, 又跳闸, 启动后确认, 基本确定了是重启会造成这个问题. 因为之前网关机器的服务异常时, 同事都是重启了机器.  
+改回来后, 过了一会, 又跳闸, 启动后确认, 基本确定了是重启会造成这个问题. 因为之前网关机器的服务异常时, 同时都是重启了机器.  
 
 继续排查, Gentoo的默认/etc/sysctl.conf配置(主要部分):  
 

201911/20191112_01.md

@@ -14,7 +14,7 @@ PostgreSQL , pg_basebackup , 阿里云 , standby
 ## 背景        
 自建异地rds pg 11从库，不管是同机房，还是异地，只要网络通就可以。利用PG的流复制。  
 
-以异地pg 从库 on ecs为例。首先要确保ecs的存储，内存与rds pg规格相当。同事建议  
+以异地pg 从库 on ecs为例。首先要确保ecs的存储，内存与rds pg规格相当。    
 
 ## 例子  
 1、centos 7.x x64  

202003/20200310_01.md

@@ -230,6 +230,52 @@ rn               | 1
 
 也许未来内核会支持这样的接口, 可以找到精确的堵塞wal replay的query.  
 
+目前PG提供的查询blocking的系统函数只能用于查询重量级锁冲突:   
+```
+-[ RECORD 1 ]-------+-----------------------------------------------------------------------------------------------
+Schema              | pg_catalog
+Name                | pg_blocking_pids
+Result data type    | integer[]
+Argument data types | integer
+Type                | func
+Volatility          | volatile
+Parallel            | safe
+Owner               | postgres
+Security            | invoker
+Access privileges   | 
+Language            | internal
+Source code         | pg_blocking_pids
+Description         | get array of PIDs of sessions blocking specified backend PID from acquiring a heavyweight lock
+-[ RECORD 2 ]-------+-----------------------------------------------------------------------------------------------
+Schema              | pg_catalog
+Name                | pg_isolation_test_session_is_blocked
+Result data type    | boolean
+Argument data types | integer, integer[]
+Type                | func
+Volatility          | volatile
+Parallel            | safe
+Owner               | postgres
+Security            | invoker
+Access privileges   | 
+Language            | internal
+Source code         | pg_isolation_test_session_is_blocked
+Description         | isolationtester support function
+-[ RECORD 3 ]-------+-----------------------------------------------------------------------------------------------
+Schema              | pg_catalog
+Name                | pg_safe_snapshot_blocking_pids
+Result data type    | integer[]
+Argument data types | integer
+Type                | func
+Volatility          | volatile
+Parallel            | safe
+Owner               | postgres
+Security            | invoker
+Access privileges   | 
+Language            | internal
+Source code         | pg_safe_snapshot_blocking_pids
+Description         | get array of PIDs of sessions blocking specified backend PID from acquiring a safe snapshot
+```
+  
 ## 堵塞了多少wal没有被replay  
 ```  
 db1=# select pg_is_wal_replay_paused(),  

202005/20200518_01.md

@@ -148,7 +148,7 @@ psql postgresql://localhost:1921/postgres?options="-c statement_timeout%3D5s -c
 - 当从库有big(long) query时, 主库IO暴增, 垃圾回收进程空转, 显示为表有垃圾, autovacuum发起扫描表垃圾, 但是回收不掉(因为从库依赖这些垃圾版本), 导致大量无用IO.  
 - 当从库有big(long) query时, 主库表膨胀, 因为某些时刻更多的垃圾无法被及时回收, 导致膨胀.    
 
-5、如果是standby本身资源问题导致delay, 那么建议查看standby节点的网络贷款、cpu、io能力是否存在瓶颈. 该加资源就加资源.    
+5、如果是standby本身资源问题导致delay, 那么建议查看standby节点的网络带宽、cpu、io能力是否存在瓶颈. 该加资源就加资源.    
 
 
 ## 参考  

202011/20201117_02.md

@@ -14,7 +14,7 @@ PostgreSQL , 冲突 , replay , standby , query
 ## 背景      
 https://www.cybertec-postgresql.com/en/streaming-replication-conflicts-in-postgresql/  
 
-几种冲突的解释说得比较清楚, 最后Avoiding buffer pin conflicts可以尝试在主节点开启vacuum_defer_cleanup_age.  
+几种冲突的解释说得比较清楚.   
 
 一切避免冲突的手段都可能导致主节点的垃圾回收做无用功, 费劲IO和CPU却不回收垃圾.   
 
@@ -41,14 +41,14 @@ This is the most frequent replication conflict.
 Snapshot conflicts can occur if VACUUM processes a table and removes dead tuples. This removal is replayed on the standby. Now a query on the standby may have started before VACUUM on the primary (it has an older snapshot), so it can still see the tuples that should be removed. This constitutes a snapshot conflict.  
 
 ### Lock replication conflicts  
-The queries on a standby server take an ACCESS EXCLUSIVE lock on the tables they are reading. So any ACCESS EXCLUSIVE lock on the primary (which conflicts with ACCESS SHARE) must be replayed on the standby to keep incompatible operations on the table from happening. PostgreSQL takes such a lock for operations that conflict with SELECT, for example DROP TABLE, TRUNCATE and many ALTER TABLE statements. If the standby should replay such a lock on a table that a query uses, we have a lock conflict.  
+The queries on a standby server take an ACCESS SHARE lock on the tables they are reading. So any ACCESS EXCLUSIVE lock on the primary (which conflicts with ACCESS SHARE) must be replayed on the standby to keep incompatible operations on the table from happening. PostgreSQL takes such a lock for operations that conflict with SELECT, for example DROP TABLE, TRUNCATE and many ALTER TABLE statements. If the standby should replay such a lock on a table that a query uses, we have a lock conflict.  
 
 ### Buffer pin replication conflicts  
-One way to reduce the need for VACUUM is to use HOT updates. Then any query on the primary that accesses a page with dead heap-only tuples and can get an exclusive lock on it will prune the HOT chains. PostgreSQL always holds such page locks for a short time, so there is no conflict with processing on the primary. There are other causes for page locks, but this is perhaps the most frequent one.  
+One way to reduce the need for VACUUM is to use HOT updates. Then any query on the primary that accesses a page with dead heap-only tuples and can get an exclusive lock on it will prune the HOT chains.(查询时修改page, 缩短hot chain. 加轻量级锁) PostgreSQL always holds such page locks for a short time, so there is no conflict with processing on the primary. There are other causes for page locks, but this is perhaps the most frequent one.  
 
-When the standby server should replay such an exclusive page lock and a query is using the page (“has the page pinned” in PostgreSQL jargon), you get a buffer pin replication conflict. Pages can be pinned for a while, for example during a sequential scan of a table on the outer side of a nested loop join.  
+When the standby server should replay such an exclusive page lock and a query is using the page (“has the page pinned” in PostgreSQL jargon), you get a buffer pin replication conflict. Pages can be pinned for a while, for example during a sequential scan of a table on the outer side of a nested loop join. (当从库上有nest loop join, 并且外表是全表扫描, 而且刚好这个外表有prune HOT chains的wal replay时, 这个replay可能长时间等待).    
 
-HOT chain pruning can of course also lead to snapshot replication conflicts.  
+HOT chain pruning can of course also lead to snapshot replication conflicts. (HOT chain pruning也会导致snapshot 冲突) 
 
 ### Rare kinds of replication conflicts  
 The following types of conflict are rare and will not bother you:  

202112/20211216_01.md

@@ -1,4 +1,4 @@
-## Linux ftrace  
+## Linux ftrace - 内核性能分析 - 火焰图  
                                     
 ### 作者                                    
 digoal                                    
@@ -7,7 +7,7 @@ digoal
 2021-12-16                              
 
 ### 标签                                 
-PostgreSQL , Linux , trace , ftrace , strace , ptrace , read , write , IO                      
+PostgreSQL , Linux , 火焰图 , trace , ftrace , strace , ptrace , read , write , IO                      
 
 ----