Merge branch 'public-single-node' into pmm-6401-read-prometheus-data-files

This commit is contained in:
Aliaksandr Valialkin 2020-10-06 16:12:25 +03:00
commit 317ddb84b9
42 changed files with 3489 additions and 553 deletions

View file

@ -16,7 +16,7 @@ jobs:
- name: Setup Go
uses: actions/setup-go@master
with:
go-version: 1.14
go-version: 1.15
id: go
- name: Dependencies
env:

View file

@ -1,6 +1,25 @@
# tip
# [v1.43.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.43.0)
* FEATURE: reduce CPU usage for repeated queries over sliding time window when no new time series are added to the database.
Typical use cases: repeated evaluation of alerting rules in [vmalert](https://victoriametrics.github.io/vmalert.html) or dashboard auto-refresh in Grafana.
* FEATURE: vmagent: add OpenStack service discovery aka [openstack_sd_config](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#openstack_sd_config).
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/728 .
* FEATURE: vmalert: make `-maxIdleConnections` configurable for datasource HTTP client. This option can be used for minimizing connection churn.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/795 .
* FEATURE: add `-influx.maxLineSize` command-line flag for configuring the maximum size for a single Influx line during parsing.
See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/807
* BUGFIX: properly handle `inf` values during [background merge of LSM parts](https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282).
Previously `Inf` values could result in `NaN` values for adjancent samples in time series. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/805 .
* BUGFIX: fill gaps on graphs for `range_*` and `running_*` functions. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/806 .
* BUGFIX: make a copy of label with new name during relabeling with `action: labelmap` in the same way as Prometheus does.
Previously the original label name has been replaced. See https://github.com/VictoriaMetrics/VictoriaMetrics/issues/812 .
* BUGFIX: support parsing floating-point timestamp like Graphite Carbon does. Such timestmaps are truncated to seconds.
# [v1.42.0](https://github.com/VictoriaMetrics/VictoriaMetrics/releases/tag/v1.42.0)
* FEATURE: use all the available CPU cores when accepting data via a single TCP connection

View file

@ -2,6 +2,7 @@ package main
import (
"flag"
"fmt"
"net/http"
"os"
"time"
@ -66,6 +67,10 @@ func main() {
}
func requestHandler(w http.ResponseWriter, r *http.Request) bool {
if r.RequestURI == "/" {
fmt.Fprintf(w, "Single-node VictoriaMetrics. See docs at https://victoriametrics.github.io/")
return true
}
if vminsert.RequestHandler(w, r) {
return true
}

View file

@ -138,6 +138,10 @@ func main() {
}
func requestHandler(w http.ResponseWriter, r *http.Request) bool {
if r.RequestURI == "/" {
fmt.Fprintf(w, "vmagent - see docs at https://victoriametrics.github.io/vmagent.html")
return true
}
path := strings.Replace(r.URL.Path, "//", "/", -1)
switch path {
case "/api/v1/write":

View file

@ -26,6 +26,7 @@
* [Billy: how VictoriaMetrics deals with more than 500 billion rows](https://medium.com/@valyala/billy-how-victoriametrics-deals-with-more-than-500-billion-rows-e82ff8f725da)
* [How to migrate data from Prometheus to VictoriaMetrics](https://medium.com/@romanhavronenko/victoriametrics-how-to-migrate-data-from-prometheus-d44a6728f043)
* [Filtering and modifying time series during import to VictoriaMetrics](https://medium.com/@romanhavronenko/victoriametrics-how-to-migrate-data-from-prometheus-filtering-and-modifying-time-series-6d40cea4bf21)
* [Anomaly Detection in VictoriaMetrics](https://medium.com/@VictoriaMetrics/anomaly-detection-in-victoriametrics-9528538786a7)
## Third-party articles and slides

View file

@ -73,13 +73,14 @@ See [Monitoring K8S with VictoriaMetrics](https://docs.google.com/presentation/d
Numbers:
* The number of active time series per VictoriaMetrics instance is 20M.
* The number of active time series per VictoriaMetrics instance is 40M.
* The total number of time series per VictoriaMetrics instance is 400M+.
* Ingestion rate per VictoriaMetrics instance is 800K data points per second.
* Ingestion rate per VictoriaMetrics instance is 1M data points per second.
* The total number of datapoints per VictoriaMetrics instance is 8 trillions.
* The average time series churn rate is ~3M per day.
* The average query rate is ~1K per minute (mostly alert queries).
* Query duration: median is ~70ms, 99th percentile is ~2sec.
* Retention: 6 months.
* The average query rate is ~100 per second (mostly alert queries).
* Query duration: median is ~70ms, 99th percentile is ~1.5sec.
* Retention: 3 months.
> Alternatives that weve played with before choosing VictoriaMetrics are: federated Prometheus, Cortex, IronDB and Thanos.
> Points that were critical to us when we were choosing a central tsdb, in order of importance:

13
go.mod
View file

@ -1,7 +1,7 @@
module github.com/VictoriaMetrics/VictoriaMetrics
require (
cloud.google.com/go v0.67.0 // indirect
cloud.google.com/go v0.68.0 // indirect
cloud.google.com/go/storage v1.12.0
github.com/VictoriaMetrics/fastcache v1.5.7
@ -10,11 +10,11 @@ require (
github.com/VictoriaMetrics/fasthttp v1.0.7
github.com/VictoriaMetrics/metrics v1.12.3
github.com/VictoriaMetrics/metricsql v0.6.0
github.com/aws/aws-sdk-go v1.34.34
github.com/aws/aws-sdk-go v1.35.3
github.com/cespare/xxhash/v2 v2.1.1
github.com/go-kit/kit v0.10.0
github.com/golang/snappy v0.0.2
github.com/klauspost/compress v1.11.0
github.com/klauspost/compress v1.11.1
github.com/prometheus/common v0.14.0 // indirect
github.com/prometheus/procfs v0.2.0 // indirect
github.com/prometheus/prometheus v1.8.2-0.20200911110723-e83ef207b6c2
@ -25,10 +25,13 @@ require (
github.com/valyala/histogram v1.1.2
github.com/valyala/quicktemplate v1.6.3
go.uber.org/atomic v1.7.0 // indirect
golang.org/x/net v0.0.0-20201002202402-0a1ea396d57c // indirect
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43
golang.org/x/sys v0.0.0-20200929083018-4d22bbb62b3c
golang.org/x/tools v0.0.0-20200929223013-bf155c11ec6f // indirect
golang.org/x/sync v0.0.0-20200930132711-30421366ff76 // indirect
golang.org/x/sys v0.0.0-20201005172224-997123666555
golang.org/x/tools v0.0.0-20201005185003-576e169c3de7 // indirect
google.golang.org/api v0.32.0
google.golang.org/genproto v0.0.0-20201006033701-bcad7cf615f2 // indirect
gopkg.in/yaml.v2 v2.3.0
)

33
go.sum
View file

@ -15,10 +15,9 @@ cloud.google.com/go v0.56.0/go.mod h1:jr7tqZxxKOVYizybht9+26Z/gUq7tiRzu+ACVAMbKV
cloud.google.com/go v0.57.0/go.mod h1:oXiQ6Rzq3RAkkY7N6t3TcE6jE+CIBBbA36lwQ1JyzZs=
cloud.google.com/go v0.62.0/go.mod h1:jmCYTdRCQuc1PHIIJ/maLInMho30T/Y0M4hTdTShOYc=
cloud.google.com/go v0.65.0/go.mod h1:O5N8zS7uWy9vkA9vayVHs65eM1ubvY4h553ofrNHObY=
cloud.google.com/go v0.66.0 h1:DZeAkuQGQqnm9Xv36SbMJEU8aFBz4wL04UpMWPWwjzg=
cloud.google.com/go v0.66.0/go.mod h1:dgqGAjKCDxyhGTtC9dAREQGUJpkceNm1yt590Qno0Ko=
cloud.google.com/go v0.67.0 h1:YIkzmqUfVGiGPpT98L8sVvUIkDno6UlrDxw4NR6z5ak=
cloud.google.com/go v0.67.0/go.mod h1:YNan/mUhNZFrYUor0vqrsQ0Ffl7Xtm/ACOy/vsTS858=
cloud.google.com/go v0.68.0 h1:AnVkaPGAuWaIY/8a75HlNzZNrHDee6YL4rWkwS+CeyE=
cloud.google.com/go v0.68.0/go.mod h1:91NO4SCDjUfe1zeC0f4/dpckkUNpuNEyqm4X2KLrzNQ=
cloud.google.com/go/bigquery v1.0.1/go.mod h1:i/xbL2UlR5RvWAURpBYZTtm/cXjCha9lbfbpx4poX+o=
cloud.google.com/go/bigquery v1.3.0/go.mod h1:PjpwJnslEMmckchkHFfq+HTD2DmtT67aNFKH1/VBDHE=
cloud.google.com/go/bigquery v1.4.0/go.mod h1:S8dzgnTigyfTmLBfrtrhyYhwRxG72rYxvftPBK2Dvzc=
@ -115,8 +114,8 @@ github.com/asaskevich/govalidator v0.0.0-20200108200545-475eaeb16496/go.mod h1:o
github.com/aws/aws-lambda-go v1.13.3/go.mod h1:4UKl9IzQMoD+QF79YdCuzCwp8VbmG4VAQwij/eHl5CU=
github.com/aws/aws-sdk-go v1.27.0/go.mod h1:KmX6BPdI08NWTb3/sm4ZGu5ShLoqVDhKgpiN924inxo=
github.com/aws/aws-sdk-go v1.34.9/go.mod h1:5zCpMtNQVjRREroY7sYe8lOMRSxkhG6MZveU8YkpAk0=
github.com/aws/aws-sdk-go v1.34.34 h1:5dC0ZU0xy25+UavGNEkQ/5MOQwxXDA2YXtjCL1HfYKI=
github.com/aws/aws-sdk-go v1.34.34/go.mod h1:H7NKnBqNVzoTJpGfLrQkkD+ytBA93eiDYi/+8rV9s48=
github.com/aws/aws-sdk-go v1.35.3 h1:r0puXncSaAfRt7Btml2swUo74Kao+vKhO3VLjwDjK54=
github.com/aws/aws-sdk-go v1.35.3/go.mod h1:H7NKnBqNVzoTJpGfLrQkkD+ytBA93eiDYi/+8rV9s48=
github.com/aws/aws-sdk-go-v2 v0.18.0/go.mod h1:JWVYvqSMppoMJC0x5wdwiImzgXTI9FuZwxzkQq9wy+g=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
@ -459,8 +458,9 @@ github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+o
github.com/klauspost/compress v1.4.0/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.9.5/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0guNDohfE1A=
github.com/klauspost/compress v1.10.7/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.11.0 h1:wJbzvpYMVGG9iTI9VxpnNZfd4DzMPoCWze3GgSqz8yg=
github.com/klauspost/compress v1.11.0/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/compress v1.11.1 h1:bPb7nMRdOZYDrpPMTA3EInUQrdgoBinqUuSwlGdKDdE=
github.com/klauspost/compress v1.11.1/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs=
github.com/klauspost/cpuid v0.0.0-20170728055534-ae7887de9fa5/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek=
github.com/klauspost/crc32 v0.0.0-20161016154125-cb6bfca970f6/go.mod h1:+ZoRqAPRLkC4NPOvfYeR5KNOrY6TD+/sAC3HXPZgDYg=
github.com/klauspost/pgzip v1.0.2-0.20170402124221-0bf5dcad4ada/go.mod h1:Ch1tH69qFZu15pkjo5kYi6mth2Zzwzt50oCQKQE9RUs=
@ -835,6 +835,8 @@ golang.org/x/net v0.0.0-20200822124328-c89045814202/go.mod h1:/O7V0waA8r7cgGh81R
golang.org/x/net v0.0.0-20200904194848-62affa334b73/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20200927032502-5d4f70055728 h1:5wtQIAulKU5AbLQOkjxl32UufnIOqgBX72pS0AV14H0=
golang.org/x/net v0.0.0-20200927032502-5d4f70055728/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/net v0.0.0-20201002202402-0a1ea396d57c h1:dk0ukUIHmGHqASjP0iue2261isepFCC6XRCSd1nHgDw=
golang.org/x/net v0.0.0-20201002202402-0a1ea396d57c/go.mod h1:iQL9McJNjoIa5mjH6nYTCTZXUN6RP+XW3eib7Ya3XcI=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/oauth2 v0.0.0-20190604053449-0f29369cfe45/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
@ -852,6 +854,8 @@ golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sync v0.0.0-20200317015054-43a5402ce75a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208 h1:qwRHBd0NqMbJxfbotnDhm2ByMI1Shq4Y6oRJo21SGJA=
golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20200930132711-30421366ff76 h1:JnxiSYT3Nm0BT2a8CyvYyM6cnrWpidecD1UuSYbhKm0=
golang.org/x/sync v0.0.0-20200930132711-30421366ff76/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20170830134202-bb24a47a89ea/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180823144017-11551d06cbcc/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
@ -912,8 +916,9 @@ golang.org/x/sys v0.0.0-20200803210538-64077c9b5642/go.mod h1:h1NjWce9XRLGQEsW7w
golang.org/x/sys v0.0.0-20200821140526-fda516888d29/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200828194041-157a740278f4/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200905004654-be1d3432aa8f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200929083018-4d22bbb62b3c h1:/h0vtH0PyU0xAoZJVcRw1k0Ng+U0JAy3QDiFmppIlIE=
golang.org/x/sys v0.0.0-20200929083018-4d22bbb62b3c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201005172224-997123666555 h1:fihtqzYxy4E31W1yUlyRGveTZT1JIP0bmKaDZ2ceKAw=
golang.org/x/sys v0.0.0-20201005172224-997123666555/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.0.0-20160726164857-2910a502d2bf/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
@ -994,9 +999,9 @@ golang.org/x/tools v0.0.0-20200828161849-5deb26317202/go.mod h1:njjCfa9FT2d7l9Bc
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20200915173823-2db8f0ff891c/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20200918232735-d647fc253266/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20200929161345-d7fc70abf50f/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20200929223013-bf155c11ec6f h1:7+Nz9MyPqt2qMCTvNiRy1G0zYfkB7UCa+ayT6uVvbyI=
golang.org/x/tools v0.0.0-20200929223013-bf155c11ec6f/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20201002184944-ecd9fd270d5d/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20201005185003-576e169c3de7 h1:YTAUHYgZh/ZOA35/OrjTDmFFKb6ddkBL1Zgtl9r8Di8=
golang.org/x/tools v0.0.0-20201005185003-576e169c3de7/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
@ -1074,8 +1079,10 @@ google.golang.org/genproto v0.0.0-20200831141814-d751682dd103/go.mod h1:FWY/as6D
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200914193844-75d14daec038/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200921151605-7abf4a1a14d5/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200929141702-51c3e5b607fe h1:6SgESkjJknFUnsfQ2yxQbmTAi37BxhwS/riq+VdLo9c=
google.golang.org/genproto v0.0.0-20200929141702-51c3e5b607fe/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201002142447-3860012362da h1:DTQYk4u7nICKkkVZsBv0/0po0ChISxAJ5CTAfUhO0PQ=
google.golang.org/genproto v0.0.0-20201002142447-3860012362da/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201006033701-bcad7cf615f2 h1:rBG1miiV00OG3NTbgxg42kPWohjVNn1sGhJyt4xyPLQ=
google.golang.org/genproto v0.0.0-20201006033701-bcad7cf615f2/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.0/go.mod h1:chYK+tFQF0nDUGJgXMSgLCQk3phJEuONr2DCgLDdAQM=

View file

@ -54,7 +54,7 @@ func (cfg *apiConfig) getFreshAPICredentials() (*apiCredentials, error) {
return nil, fmt.Errorf("cannot refresh OpenStack api token: %w", err)
}
cfg.creds = newCreds
logger.Infof("successfully refreshed OpenStack api token; next expiration: %s", newCreds.expiration)
logger.Infof("successfully refreshed OpenStack api token; expiration: %.3f seconds", time.Until(newCreds.expiration).Seconds())
return newCreds, nil
}

View file

@ -98,12 +98,12 @@ func (r *Row) unmarshal(s string, tagsPool []Tag) ([]Tag, error) {
if err != nil {
return tagsPool, fmt.Errorf("cannot unmarshal value from %q: %w", tail[:n], err)
}
ts, err := fastfloat.ParseInt64(tail[n+1:])
ts, err := fastfloat.Parse(tail[n+1:])
if err != nil {
return tagsPool, fmt.Errorf("cannot unmarshal timestamp from %q: %w", tail[n+1:], err)
}
r.Value = v
r.Timestamp = ts
r.Timestamp = int64(ts)
return tagsPool, nil
}

View file

@ -113,6 +113,16 @@ func TestRowsUnmarshalSuccess(t *testing.T) {
}},
})
// Floating-point timestamp
// See https://github.com/graphite-project/carbon/blob/b0ba62a62d40a37950fed47a8f6ae6d0f02e6af5/lib/carbon/protocols.py#L197
f("aaa 1123 4294.943", &Rows{
Rows: []Row{{
Metric: "aaa",
Value: 1123,
Timestamp: 4294,
}},
})
// Tags
f("foo;bar=baz 1 2", &Rows{
Rows: []Row{{

View file

@ -1,5 +1,12 @@
# Changes
## [0.68.0](https://www.github.com/googleapis/google-cloud-go/compare/v0.67.0...v0.68.0) (2020-10-02)
### Features
* **all:** auto-regenerate gapics , refs [#2952](https://www.github.com/googleapis/google-cloud-go/issues/2952) [#2944](https://www.github.com/googleapis/google-cloud-go/issues/2944) [#2935](https://www.github.com/googleapis/google-cloud-go/issues/2935)
## [0.67.0](https://www.github.com/googleapis/google-cloud-go/compare/v0.66.0...v0.67.0) (2020-09-29)

4
vendor/cloud.google.com/go/go.mod generated vendored
View file

@ -16,8 +16,8 @@ require (
golang.org/x/net v0.0.0-20200927032502-5d4f70055728
golang.org/x/oauth2 v0.0.0-20200902213428-5d25da1a8d43
golang.org/x/text v0.3.3
golang.org/x/tools v0.0.0-20200929161345-d7fc70abf50f
golang.org/x/tools v0.0.0-20201002184944-ecd9fd270d5d
google.golang.org/api v0.32.0
google.golang.org/genproto v0.0.0-20200929141702-51c3e5b607fe
google.golang.org/genproto v0.0.0-20201002142447-3860012362da
google.golang.org/grpc v1.32.0
)

8
vendor/cloud.google.com/go/go.sum generated vendored
View file

@ -377,8 +377,8 @@ golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d h1:szSOL78iTCl0LF1AMjhSWJj
golang.org/x/tools v0.0.0-20200804011535-6c149bb5ef0d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200825202427-b303f430e36d/go.mod h1:njjCfa9FT2d7l9Bc6FUM5FLjQPp3cFF28FI3qnDFljA=
golang.org/x/tools v0.0.0-20200904185747-39188db58858/go.mod h1:Cj7w3i3Rnn0Xh82ur9kSqwfTHTeVxaDqrfMjpcNT6bE=
golang.org/x/tools v0.0.0-20200929161345-d7fc70abf50f h1:18s2P7JILnVhIF2+ZtGJQ9czV5bvTsb13/UGtNPDbjA=
golang.org/x/tools v0.0.0-20200929161345-d7fc70abf50f/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/tools v0.0.0-20201002184944-ecd9fd270d5d h1:vWQvJ/Z0Lu+9/8oQ/pAYXNzbc7CMnBl+tULGVHOy3oE=
golang.org/x/tools v0.0.0-20201002184944-ecd9fd270d5d/go.mod h1:z6u4i615ZeAfBE4XtMziQW1fSVJXACjjbWkB/mvPzlU=
golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
@ -460,8 +460,8 @@ google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c h1:Lq4llNryJoaVFRm
google.golang.org/genproto v0.0.0-20200804131852-c06518451d9c/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200825200019-8632dd797987/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200904004341-0bd0a958aa1d/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20200929141702-51c3e5b607fe h1:6SgESkjJknFUnsfQ2yxQbmTAi37BxhwS/riq+VdLo9c=
google.golang.org/genproto v0.0.0-20200929141702-51c3e5b607fe/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/genproto v0.0.0-20201002142447-3860012362da h1:DTQYk4u7nICKkkVZsBv0/0po0ChISxAJ5CTAfUhO0PQ=
google.golang.org/genproto v0.0.0-20201002142447-3860012362da/go.mod h1:FWY/as6DDZQgahTzZj3fqbO1CbirC29ZNUFHwi0/+no=
google.golang.org/grpc v1.19.0 h1:cfg4PD8YEdSFnm7qLV4++93WcmhH2nIUhMjhdCvl3j8=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.20.1 h1:Hz2g2wirWK7H0qIIhGIqRGTuMwTE8HEKFnDZZ7lm9NU=

View file

@ -26,7 +26,7 @@ import (
// Repo is the current version of the client libraries in this
// repo. It should be a date in YYYYMMDD format.
const Repo = "20200925"
const Repo = "20201002"
// Go returns the Go runtime version. The returned string
// has no whitespace.

230
vendor/cloud.google.com/go/testing.md generated vendored Normal file
View file

@ -0,0 +1,230 @@
# Testing Code that depends on Go Client Libraries
The Go client libraries generated as a part of `cloud.google.com/go` all take
the approach of returning concrete types instead of interfaces. That way, new
fields and methods can be added to the libraries without breaking users. This
document will go over some patterns that can be used to test code that depends
on the Go client libraries.
## Testing gRPC services using fakes
The clients found in `cloud.google.com/go` are gRPC based, with a couple of
notable exceptions being the [`storage`](https://pkg.go.dev/cloud.google.com/go/storage)
and [`bigquery`](https://pkg.go.dev/cloud.google.com/go/bigquery) clients.
Interactions with gRPC services can be faked by serving up your own in-memory
server within your test. One benefit of using this approach is that you dont
need to define an interface in your runtime code; you can keep using
concrete struct types. You instead define a fake server in your test code. For
example, take a look at the following function:
```go
import (
"context"
"fmt"
"log"
"os"
translate "cloud.google.com/go/translate/apiv3"
"github.com/googleapis/gax-go/v2"
translatepb "google.golang.org/genproto/googleapis/cloud/translate/v3"
)
func TranslateTextWithConcreteClient(client *translate.TranslationClient, text string, targetLang string) (string, error) {
ctx := context.Background()
log.Printf("Translating %q to %q", text, targetLang)
req := &translatepb.TranslateTextRequest{
Parent: fmt.Sprintf("projects/%s/locations/global", os.Getenv("GOOGLE_CLOUD_PROJECT")),
TargetLanguageCode: "en-US",
Contents: []string{text},
}
resp, err := client.TranslateText(ctx, req)
if err != nil {
return "", fmt.Errorf("unable to translate text: %v", err)
}
translations := resp.GetTranslations()
if len(translations) != 1 {
return "", fmt.Errorf("expected only one result, got %d", len(translations))
}
return translations[0].TranslatedText, nil
}
```
Here is an example of what a fake server implementation would look like for
faking the interactions above:
```go
import (
"context"
translatepb "google.golang.org/genproto/googleapis/cloud/translate/v3"
)
type fakeTranslationServer struct {
translatepb.UnimplementedTranslationServiceServer
}
func (f *fakeTranslationServer) TranslateText(ctx context.Context, req *translatepb.TranslateTextRequest) (*translatepb.TranslateTextResponse, error) {
resp := &translatepb.TranslateTextResponse{
Translations: []*translatepb.Translation{
&translatepb.Translation{
TranslatedText: "Hello World",
},
},
}
return resp, nil
}
```
All of the generated protobuf code found in [google.golang.org/genproto](https://pkg.go.dev/google.golang.org/genproto)
contains a similar `package.UnimplmentedFooServer` type that is useful for
creating fakes. By embedding the unimplemented server in the
`fakeTranslationServer`, the fake will “inherit” all of the RPCs the server
exposes. Then, by providing our own `fakeTranslationServer.TranslateText`
method you can “override” the default unimplemented behavior of the one RPC that
you would like to be faked.
The test itself does require a little bit of setup: start up a `net.Listener`,
register the server, and tell the client library to call the server:
```go
import (
"context"
"net"
"testing"
translate "cloud.google.com/go/translate/apiv3"
"google.golang.org/api/option"
translatepb "google.golang.org/genproto/googleapis/cloud/translate/v3"
"google.golang.org/grpc"
)
func TestTranslateTextWithConcreteClient(t *testing.T) {
ctx := context.Background()
// Setup the fake server.
fakeTranslationServer := &fakeTranslationServer{}
l, err := net.Listen("tcp", "localhost:0")
if err != nil {
t.Fatal(err)
}
gsrv := grpc.NewServer()
translatepb.RegisterTranslationServiceServer(gsrv, fakeTranslationServer)
fakeServerAddr := l.Addr().String()
go func() {
if err := gsrv.Serve(l); err != nil {
panic(err)
}
}()
// Create a client.
client, err := translate.NewTranslationClient(ctx,
option.WithEndpoint(fakeServerAddr),
option.WithoutAuthentication(),
option.WithGRPCDialOption(grpc.WithInsecure()),
)
if err != nil {
t.Fatal(err)
}
// Run the test.
text, err := TranslateTextWithConcreteClient(client, "Hola Mundo", "en-US")
if err != nil {
t.Fatal(err)
}
if text != "Hello World" {
t.Fatalf("got %q, want Hello World", text)
}
}
```
## Testing using mocks
When mocking code you need to work with interfaces. Lets create an interface
for the `cloud.google.com/go/translate/apiv3` client used in the
`TranslateTextWithConcreteClient` function mentioned in the previous section.
The `translate.Client` has over a dozen methods but this code only uses one of
them. Here is an interface that satisfies the interactions of the
`translate.Client` in this function.
```go
type TranslationClient interface {
TranslateText(ctx context.Context, req *translatepb.TranslateTextRequest, opts ...gax.CallOption) (*translatepb.TranslateTextResponse, error)
}
```
Now that we have an interface that satisfies the method being used we can
rewrite the function signature to take the interface instead of the concrete
type.
```go
func TranslateTextWithAbstractClient(client TranslationClient, text string, targetLang string) (string, error) {
// ...
}
```
This allows a real `translate.Client` to be passed to the method in production
and for a mock implementation to be passed in during testing. This pattern can
be applied to any Go code, not just `cloud.google.com/go`. This is because
interfaces in Go are implicitly satisfied. Structs in the client libraries can
implicitly implement interfaces defined in your codebase. Lets take a look at
what it might look like to define a lightweight mock for the `TranslationClient`
interface.
```go
import (
"context"
"testing"
"github.com/googleapis/gax-go/v2"
translatepb "google.golang.org/genproto/googleapis/cloud/translate/v3"
)
type mockClient struct{}
func (*mockClient) TranslateText(_ context.Context, req *translatepb.TranslateTextRequest, opts ...gax.CallOption) (*translatepb.TranslateTextResponse, error) {
resp := &translatepb.TranslateTextResponse{
Translations: []*translatepb.Translation{
&translatepb.Translation{
TranslatedText: "Hello World",
},
},
}
return resp, nil
}
func TestTranslateTextWithAbstractClient(t *testing.T) {
client := &mockClient{}
text, err := TranslateTextWithAbstractClient(client, "Hola Mundo", "en-US")
if err != nil {
t.Fatal(err)
}
if text != "Hello World" {
t.Fatalf("got %q, want Hello World", text)
}
}
```
If you prefer to not write your own mocks there are mocking frameworks such as
[golang/mock](https://github.com/golang/mock) which can generate mocks for you
from an interface. As a word of caution though, try to not
[overuse mocks](https://testing.googleblog.com/2013/05/testing-on-toilet-dont-overuse-mocks.html).
## Testing using emulators
Some of the client libraries provided in `cloud.google.com/go` support running
against a service emulator. The concept is similar to that of using fakes,
mentioned above, but the server is managed for you. You just need to start it up
and instruct the client library to talk to the emulator by setting a service
specific emulator environment variable. Current services/environment-variables
are:
- bigtable: `BIGTABLE_EMULATOR_HOST`
- datastore: `DATASTORE_EMULATOR_HOST`
- firestore: `FIRESTORE_EMULATOR_HOST`
- pubsub: `PUBSUB_EMULATOR_HOST`
- spanner: `SPANNER_EMULATOR_HOST`
- storage: `STORAGE_EMULATOR_HOST`
- Although the storage client supports an emulator environment variable there is no official emulator provided by gcloud.
For more information on emulators please refer to the
[gcloud documentation](https://cloud.google.com/sdk/gcloud/reference/beta/emulators).

View file

@ -4041,6 +4041,7 @@ var awsPartition = partition{
"ca-central-1": endpoint{},
"eu-central-1": endpoint{},
"eu-north-1": endpoint{},
"eu-south-1": endpoint{},
"eu-west-1": endpoint{},
"eu-west-2": endpoint{},
"eu-west-3": endpoint{},
@ -5556,6 +5557,7 @@ var awsPartition = partition{
"ca-central-1": endpoint{},
"eu-central-1": endpoint{},
"eu-north-1": endpoint{},
"eu-south-1": endpoint{},
"eu-west-1": endpoint{},
"eu-west-2": endpoint{},
"eu-west-3": endpoint{},
@ -6585,9 +6587,21 @@ var awsPartition = partition{
"eu-central-1": endpoint{},
"eu-west-1": endpoint{},
"eu-west-2": endpoint{},
"sa-east-1": endpoint{},
"us-east-1": endpoint{},
"us-west-2": endpoint{},
"fips-us-east-1": endpoint{
Hostname: "workspaces-fips.us-east-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "us-east-1",
},
},
"fips-us-west-2": endpoint{
Hostname: "workspaces-fips.us-west-2.amazonaws.com",
CredentialScope: credentialScope{
Region: "us-west-2",
},
},
"sa-east-1": endpoint{},
"us-east-1": endpoint{},
"us-west-2": endpoint{},
},
},
"xray": service{
@ -7053,6 +7067,23 @@ var awscnPartition = partition{
"cn-northwest-1": endpoint{},
},
},
"iotevents": service{
Endpoints: endpoints{
"cn-north-1": endpoint{},
},
},
"ioteventsdata": service{
Endpoints: endpoints{
"cn-north-1": endpoint{
Hostname: "data.iotevents.cn-north-1.amazonaws.com.cn",
CredentialScope: credentialScope{
Region: "cn-north-1",
},
},
},
},
"iotsecuredtunneling": service{
Endpoints: endpoints{
@ -7179,6 +7210,13 @@ var awscnPartition = partition{
"cn-northwest-1": endpoint{},
},
},
"resource-groups": service{
Endpoints: endpoints{
"cn-north-1": endpoint{},
"cn-northwest-1": endpoint{},
},
},
"route53": service{
PartitionEndpoint: "aws-cn-global",
IsRegionalized: boxedFalse,
@ -8455,6 +8493,12 @@ var awsusgovPartition = partition{
},
},
Endpoints: endpoints{
"fips-us-gov-west-1": endpoint{
Hostname: "pinpoint.us-gov-west-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "us-gov-west-1",
},
},
"us-gov-west-1": endpoint{},
},
},
@ -8954,6 +8998,25 @@ var awsusgovPartition = partition{
"us-gov-west-1": endpoint{},
},
},
"transfer": service{
Endpoints: endpoints{
"fips-us-gov-east-1": endpoint{
Hostname: "transfer-fips.us-gov-east-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "us-gov-east-1",
},
},
"fips-us-gov-west-1": endpoint{
Hostname: "transfer-fips.us-gov-west-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "us-gov-west-1",
},
},
"us-gov-east-1": endpoint{},
"us-gov-west-1": endpoint{},
},
},
"translate": service{
Defaults: endpoint{
Protocols: []string{"https"},
@ -8988,6 +9051,12 @@ var awsusgovPartition = partition{
"workspaces": service{
Endpoints: endpoints{
"fips-us-gov-west-1": endpoint{
Hostname: "workspaces-fips.us-gov-west-1.amazonaws.com",
CredentialScope: credentialScope{
Region: "us-gov-west-1",
},
},
"us-gov-west-1": endpoint{},
},
},

View file

@ -5,4 +5,4 @@ package aws
const SDKName = "aws-sdk-go"
// SDKVersion is the version of this SDK
const SDKVersion = "1.34.34"
const SDKVersion = "1.35.3"

View file

@ -19,23 +19,28 @@ func (a AccessPointARN) GetARN() arn.ARN {
// ParseAccessPointResource attempts to parse the ARN's resource as an
// AccessPoint resource.
//
// Supported Access point resource format:
// - Access point format: arn:{partition}:s3:{region}:{accountId}:accesspoint/{accesspointName}
// - example: arn.aws.s3.us-west-2.012345678901:accesspoint/myaccesspoint
//
func ParseAccessPointResource(a arn.ARN, resParts []string) (AccessPointARN, error) {
if len(a.Region) == 0 {
return AccessPointARN{}, InvalidARNError{a, "region not set"}
return AccessPointARN{}, InvalidARNError{ARN: a, Reason: "region not set"}
}
if len(a.AccountID) == 0 {
return AccessPointARN{}, InvalidARNError{a, "account-id not set"}
return AccessPointARN{}, InvalidARNError{ARN: a, Reason: "account-id not set"}
}
if len(resParts) == 0 {
return AccessPointARN{}, InvalidARNError{a, "resource-id not set"}
return AccessPointARN{}, InvalidARNError{ARN: a, Reason: "resource-id not set"}
}
if len(resParts) > 1 {
return AccessPointARN{}, InvalidARNError{a, "sub resource not supported"}
return AccessPointARN{}, InvalidARNError{ARN: a, Reason: "sub resource not supported"}
}
resID := resParts[0]
if len(strings.TrimSpace(resID)) == 0 {
return AccessPointARN{}, InvalidARNError{a, "resource-id not set"}
return AccessPointARN{}, InvalidARNError{ARN: a, Reason: "resource-id not set"}
}
return AccessPointARN{

View file

@ -1,6 +1,7 @@
package arn
import (
"fmt"
"strings"
"github.com/aws/aws-sdk-go/aws/arn"
@ -25,13 +26,14 @@ func ParseResource(s string, resParser ResourceParser) (resARN Resource, err err
}
if len(a.Partition) == 0 {
return nil, InvalidARNError{a, "partition not set"}
return nil, InvalidARNError{ARN: a, Reason: "partition not set"}
}
if a.Service != "s3" {
return nil, InvalidARNError{a, "service is not S3"}
if a.Service != "s3" && a.Service != "s3-outposts" {
return nil, InvalidARNError{ARN: a, Reason: "service is not supported"}
}
if len(a.Resource) == 0 {
return nil, InvalidARNError{a, "resource not set"}
return nil, InvalidARNError{ARN: a, Reason: "resource not set"}
}
return resParser(a)
@ -66,6 +68,7 @@ type InvalidARNError struct {
Reason string
}
// Error returns a string denoting the occurred InvalidARNError
func (e InvalidARNError) Error() string {
return "invalid Amazon S3 ARN, " + e.Reason + ", " + e.ARN.String()
return fmt.Sprintf("invalid Amazon %s ARN, %s, %s", e.ARN.Service, e.Reason, e.ARN.String())
}

View file

@ -0,0 +1,126 @@
package arn
import (
"strings"
"github.com/aws/aws-sdk-go/aws/arn"
)
// OutpostARN interface that should be satisfied by outpost ARNs
type OutpostARN interface {
Resource
GetOutpostID() string
}
// ParseOutpostARNResource will parse a provided ARNs resource using the appropriate ARN format
// and return a specific OutpostARN type
//
// Currently supported outpost ARN formats:
// * Outpost AccessPoint ARN format:
// - ARN format: arn:{partition}:s3-outposts:{region}:{accountId}:outpost/{outpostId}/accesspoint/{accesspointName}
// - example: arn:aws:s3-outposts:us-west-2:012345678901:outpost/op-1234567890123456/accesspoint/myaccesspoint
//
// * Outpost Bucket ARN format:
// - ARN format: arn:{partition}:s3-outposts:{region}:{accountId}:outpost/{outpostId}/bucket/{bucketName}
// - example: arn:aws:s3-outposts:us-west-2:012345678901:outpost/op-1234567890123456/bucket/mybucket
//
// Other outpost ARN formats may be supported and added in the future.
//
func ParseOutpostARNResource(a arn.ARN, resParts []string) (OutpostARN, error) {
if len(a.Region) == 0 {
return nil, InvalidARNError{ARN: a, Reason: "region not set"}
}
if len(a.AccountID) == 0 {
return nil, InvalidARNError{ARN: a, Reason: "account-id not set"}
}
// verify if outpost id is present and valid
if len(resParts) == 0 || len(strings.TrimSpace(resParts[0])) == 0 {
return nil, InvalidARNError{ARN: a, Reason: "outpost resource-id not set"}
}
// verify possible resource type exists
if len(resParts) < 3 {
return nil, InvalidARNError{
ARN: a, Reason: "incomplete outpost resource type. Expected bucket or access-point resource to be present",
}
}
// Since we know this is a OutpostARN fetch outpostID
outpostID := strings.TrimSpace(resParts[0])
switch resParts[1] {
case "accesspoint":
accesspointARN, err := ParseAccessPointResource(a, resParts[2:])
if err != nil {
return OutpostAccessPointARN{}, err
}
return OutpostAccessPointARN{
AccessPointARN: accesspointARN,
OutpostID: outpostID,
}, nil
case "bucket":
bucketName, err := parseBucketResource(a, resParts[2:])
if err != nil {
return nil, err
}
return OutpostBucketARN{
ARN: a,
BucketName: bucketName,
OutpostID: outpostID,
}, nil
default:
return nil, InvalidARNError{ARN: a, Reason: "unknown resource set for outpost ARN"}
}
}
// OutpostAccessPointARN represents outpost access point ARN.
type OutpostAccessPointARN struct {
AccessPointARN
OutpostID string
}
// GetOutpostID returns the outpost id of outpost access point arn
func (o OutpostAccessPointARN) GetOutpostID() string {
return o.OutpostID
}
// OutpostBucketARN represents the outpost bucket ARN.
type OutpostBucketARN struct {
arn.ARN
BucketName string
OutpostID string
}
// GetOutpostID returns the outpost id of outpost bucket arn
func (o OutpostBucketARN) GetOutpostID() string {
return o.OutpostID
}
// GetARN retrives the base ARN from outpost bucket ARN resource
func (o OutpostBucketARN) GetARN() arn.ARN {
return o.ARN
}
// parseBucketResource attempts to parse the ARN's bucket resource and retrieve the
// bucket resource id.
//
// parseBucketResource only parses the bucket resource id.
//
func parseBucketResource(a arn.ARN, resParts []string) (bucketName string, err error) {
if len(resParts) == 0 {
return bucketName, InvalidARNError{ARN: a, Reason: "bucket resource-id not set"}
}
if len(resParts) > 1 {
return bucketName, InvalidARNError{ARN: a, Reason: "sub resource not supported"}
}
bucketName = strings.TrimSpace(resParts[0])
if len(bucketName) == 0 {
return bucketName, InvalidARNError{ARN: a, Reason: "bucket resource-id not set"}
}
return bucketName, err
}

View file

@ -0,0 +1,189 @@
package s3shared
import (
"fmt"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/internal/s3shared/arn"
)
const (
invalidARNErrorErrCode = "InvalidARNError"
configurationErrorErrCode = "ConfigurationError"
)
// InvalidARNError denotes the error for Invalid ARN
type InvalidARNError struct {
message string
resource arn.Resource
origErr error
}
// Error returns the InvalidARNError
func (e InvalidARNError) Error() string {
var extra string
if e.resource != nil {
extra = "ARN: " + e.resource.String()
}
return awserr.SprintError(e.Code(), e.Message(), extra, e.origErr)
}
// Code returns the invalid ARN error code
func (e InvalidARNError) Code() string {
return invalidARNErrorErrCode
}
// Message returns the message for Invalid ARN error
func (e InvalidARNError) Message() string {
return e.message
}
// OrigErr is the original error wrapped by Invalid ARN Error
func (e InvalidARNError) OrigErr() error {
return e.origErr
}
// NewInvalidARNError denotes invalid arn error
func NewInvalidARNError(resource arn.Resource, err error) InvalidARNError {
return InvalidARNError{
message: "invalid ARN",
origErr: err,
resource: resource,
}
}
// NewInvalidARNWithCustomEndpointError ARN not supported for custom clients endpoints
func NewInvalidARNWithCustomEndpointError(resource arn.Resource, err error) InvalidARNError {
return InvalidARNError{
message: "resource ARN not supported with custom client endpoints",
origErr: err,
resource: resource,
}
}
// NewInvalidARNWithUnsupportedPartitionError ARN not supported for the target partition
func NewInvalidARNWithUnsupportedPartitionError(resource arn.Resource, err error) InvalidARNError {
return InvalidARNError{
message: "resource ARN not supported for the target ARN partition",
origErr: err,
resource: resource,
}
}
// NewInvalidARNWithFIPSError ARN not supported for FIPS region
func NewInvalidARNWithFIPSError(resource arn.Resource, err error) InvalidARNError {
return InvalidARNError{
message: "resource ARN not supported for FIPS region",
resource: resource,
origErr: err,
}
}
// ConfigurationError is used to denote a client configuration error
type ConfigurationError struct {
message string
resource arn.Resource
clientPartitionID string
clientRegion string
origErr error
}
// Error returns the Configuration error string
func (e ConfigurationError) Error() string {
extra := fmt.Sprintf("ARN: %s, client partition: %s, client region: %s",
e.resource, e.clientPartitionID, e.clientRegion)
return awserr.SprintError(e.Code(), e.Message(), extra, e.origErr)
}
// Code returns configuration error's error-code
func (e ConfigurationError) Code() string {
return configurationErrorErrCode
}
// Message returns the configuration error message
func (e ConfigurationError) Message() string {
return e.message
}
// OrigErr is the original error wrapped by Configuration Error
func (e ConfigurationError) OrigErr() error {
return e.origErr
}
// NewClientPartitionMismatchError stub
func NewClientPartitionMismatchError(resource arn.Resource, clientPartitionID, clientRegion string, err error) ConfigurationError {
return ConfigurationError{
message: "client partition does not match provided ARN partition",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
// NewClientRegionMismatchError denotes cross region access error
func NewClientRegionMismatchError(resource arn.Resource, clientPartitionID, clientRegion string, err error) ConfigurationError {
return ConfigurationError{
message: "client region does not match provided ARN region",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
// NewFailedToResolveEndpointError denotes endpoint resolving error
func NewFailedToResolveEndpointError(resource arn.Resource, clientPartitionID, clientRegion string, err error) ConfigurationError {
return ConfigurationError{
message: "endpoint resolver failed to find an endpoint for the provided ARN region",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
// NewClientConfiguredForFIPSError denotes client config error for unsupported cross region FIPS access
func NewClientConfiguredForFIPSError(resource arn.Resource, clientPartitionID, clientRegion string, err error) ConfigurationError {
return ConfigurationError{
message: "client configured for fips but cross-region resource ARN provided",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
// NewClientConfiguredForAccelerateError denotes client config error for unsupported S3 accelerate
func NewClientConfiguredForAccelerateError(resource arn.Resource, clientPartitionID, clientRegion string, err error) ConfigurationError {
return ConfigurationError{
message: "client configured for S3 Accelerate but is not supported with resource ARN",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
// NewClientConfiguredForCrossRegionFIPSError denotes client config error for unsupported cross region FIPS request
func NewClientConfiguredForCrossRegionFIPSError(resource arn.Resource, clientPartitionID, clientRegion string, err error) ConfigurationError {
return ConfigurationError{
message: "client configured for FIPS with cross-region enabled but is supported with cross-region resource ARN",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
// NewClientConfiguredForDualStackError denotes client config error for unsupported S3 Dual-stack
func NewClientConfiguredForDualStackError(resource arn.Resource, clientPartitionID, clientRegion string, err error) ConfigurationError {
return ConfigurationError{
message: "client configured for S3 Dual-stack but is not supported with resource ARN",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}

View file

@ -0,0 +1,62 @@
package s3shared
import (
"strings"
"github.com/aws/aws-sdk-go/aws"
awsarn "github.com/aws/aws-sdk-go/aws/arn"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/internal/s3shared/arn"
)
// ResourceRequest represents the request and arn resource
type ResourceRequest struct {
Resource arn.Resource
Request *request.Request
}
// ARN returns the resource ARN
func (r ResourceRequest) ARN() awsarn.ARN {
return r.Resource.GetARN()
}
// AllowCrossRegion returns a bool value to denote if S3UseARNRegion flag is set
func (r ResourceRequest) AllowCrossRegion() bool {
return aws.BoolValue(r.Request.Config.S3UseARNRegion)
}
// UseFIPS returns true if request config region is FIPS
func (r ResourceRequest) UseFIPS() bool {
return IsFIPS(aws.StringValue(r.Request.Config.Region))
}
// ResourceConfiguredForFIPS returns true if resource ARNs region is FIPS
func (r ResourceRequest) ResourceConfiguredForFIPS() bool {
return IsFIPS(r.ARN().Region)
}
// IsCrossPartition returns true if client is configured for another partition, than
// the partition that resource ARN region resolves to.
func (r ResourceRequest) IsCrossPartition() bool {
return r.Request.ClientInfo.PartitionID != r.Resource.GetARN().Partition
}
// IsCrossRegion returns true if ARN region is different than client configured region
func (r ResourceRequest) IsCrossRegion() bool {
return IsCrossRegion(r.Request, r.Resource.GetARN().Region)
}
// HasCustomEndpoint returns true if custom client endpoint is provided
func (r ResourceRequest) HasCustomEndpoint() bool {
return len(aws.StringValue(r.Request.Config.Endpoint)) > 0
}
// IsFIPS returns true if region is a fips region
func IsFIPS(clientRegion string) bool {
return strings.HasPrefix(clientRegion, "fips-") || strings.HasSuffix(clientRegion, "-fips")
}
// IsCrossRegion returns true if request signing region is not same as configured region
func IsCrossRegion(req *request.Request, otherRegion string) bool {
return req.ClientInfo.SigningRegion != otherRegion
}

File diff suppressed because it is too large Load diff

View file

@ -3,8 +3,8 @@ package s3
import (
"github.com/aws/aws-sdk-go/aws/client"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/internal/s3err"
"github.com/aws/aws-sdk-go/service/s3/internal/arn"
"github.com/aws/aws-sdk-go/internal/s3shared/arn"
"github.com/aws/aws-sdk-go/internal/s3shared/s3err"
)
func init() {
@ -69,6 +69,8 @@ type copySourceSSECustomerKeyGetter interface {
getCopySourceSSECustomerKey() string
}
// endpointARNGetter is an accessor interface to grab the
// the field corresponding to an endpoint ARN input.
type endpointARNGetter interface {
getEndpointARN() (arn.Resource, error)
hasEndpointARN() bool

View file

@ -6,11 +6,9 @@ import (
"github.com/aws/aws-sdk-go/aws"
awsarn "github.com/aws/aws-sdk-go/aws/arn"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/endpoints"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/private/protocol"
"github.com/aws/aws-sdk-go/service/s3/internal/arn"
"github.com/aws/aws-sdk-go/internal/s3shared"
"github.com/aws/aws-sdk-go/internal/s3shared/arn"
)
// Used by shapes with members decorated as endpoint ARN.
@ -22,12 +20,66 @@ func accessPointResourceParser(a awsarn.ARN) (arn.Resource, error) {
resParts := arn.SplitResource(a.Resource)
switch resParts[0] {
case "accesspoint":
if a.Service != "s3" {
return arn.AccessPointARN{}, arn.InvalidARNError{ARN: a, Reason: "service is not s3"}
}
return arn.ParseAccessPointResource(a, resParts[1:])
case "outpost":
if a.Service != "s3-outposts" {
return arn.OutpostAccessPointARN{}, arn.InvalidARNError{ARN: a, Reason: "service is not s3-outposts"}
}
return parseOutpostAccessPointResource(a, resParts[1:])
default:
return nil, arn.InvalidARNError{ARN: a, Reason: "unknown resource type"}
}
}
// parseOutpostAccessPointResource attempts to parse the ARNs resource as an
// outpost access-point resource.
//
// Supported Outpost AccessPoint ARN format:
// - ARN format: arn:{partition}:s3-outposts:{region}:{accountId}:outpost/{outpostId}/accesspoint/{accesspointName}
// - example: arn:aws:s3-outposts:us-west-2:012345678901:outpost/op-1234567890123456/accesspoint/myaccesspoint
//
func parseOutpostAccessPointResource(a awsarn.ARN, resParts []string) (arn.OutpostAccessPointARN, error) {
// outpost accesspoint arn is only valid if service is s3-outposts
if a.Service != "s3-outposts" {
return arn.OutpostAccessPointARN{}, arn.InvalidARNError{ARN: a, Reason: "service is not s3-outposts"}
}
if len(resParts) == 0 {
return arn.OutpostAccessPointARN{}, arn.InvalidARNError{ARN: a, Reason: "outpost resource-id not set"}
}
if len(resParts) < 3 {
return arn.OutpostAccessPointARN{}, arn.InvalidARNError{
ARN: a, Reason: "access-point resource not set in Outpost ARN",
}
}
resID := strings.TrimSpace(resParts[0])
if len(resID) == 0 {
return arn.OutpostAccessPointARN{}, arn.InvalidARNError{ARN: a, Reason: "outpost resource-id not set"}
}
var outpostAccessPointARN = arn.OutpostAccessPointARN{}
switch resParts[1] {
case "accesspoint":
accessPointARN, err := arn.ParseAccessPointResource(a, resParts[2:])
if err != nil {
return arn.OutpostAccessPointARN{}, err
}
// set access-point arn
outpostAccessPointARN.AccessPointARN = accessPointARN
default:
return arn.OutpostAccessPointARN{}, arn.InvalidARNError{ARN: a, Reason: "access-point resource not set in Outpost ARN"}
}
// set outpost id
outpostAccessPointARN.OutpostID = resID
return outpostAccessPointARN, nil
}
func endpointHandler(req *request.Request) {
endpoint, ok := req.Params.(endpointARNGetter)
if !ok || !endpoint.hasEndpointARN() {
@ -37,29 +89,29 @@ func endpointHandler(req *request.Request) {
resource, err := endpoint.getEndpointARN()
if err != nil {
req.Error = newInvalidARNError(nil, err)
req.Error = s3shared.NewInvalidARNError(nil, err)
return
}
resReq := resourceRequest{
resReq := s3shared.ResourceRequest{
Resource: resource,
Request: req,
}
if resReq.IsCrossPartition() {
req.Error = newClientPartitionMismatchError(resource,
req.Error = s3shared.NewClientPartitionMismatchError(resource,
req.ClientInfo.PartitionID, aws.StringValue(req.Config.Region), nil)
return
}
if !resReq.AllowCrossRegion() && resReq.IsCrossRegion() {
req.Error = newClientRegionMismatchError(resource,
req.Error = s3shared.NewClientRegionMismatchError(resource,
req.ClientInfo.PartitionID, aws.StringValue(req.Config.Region), nil)
return
}
if resReq.HasCustomEndpoint() {
req.Error = newInvalidARNWithCustomEndpointError(resource, nil)
req.Error = s3shared.NewInvalidARNWithCustomEndpointError(resource, nil)
return
}
@ -69,47 +121,22 @@ func endpointHandler(req *request.Request) {
if err != nil {
req.Error = err
}
case arn.OutpostAccessPointARN:
// outposts does not support FIPS regions
if resReq.ResourceConfiguredForFIPS() {
req.Error = s3shared.NewInvalidARNWithFIPSError(resource, nil)
return
}
err = updateRequestOutpostAccessPointEndpoint(req, tv)
if err != nil {
req.Error = err
}
default:
req.Error = newInvalidARNError(resource, nil)
req.Error = s3shared.NewInvalidARNError(resource, nil)
}
}
type resourceRequest struct {
Resource arn.Resource
Request *request.Request
}
func (r resourceRequest) ARN() awsarn.ARN {
return r.Resource.GetARN()
}
func (r resourceRequest) AllowCrossRegion() bool {
return aws.BoolValue(r.Request.Config.S3UseARNRegion)
}
func (r resourceRequest) UseFIPS() bool {
return isFIPS(aws.StringValue(r.Request.Config.Region))
}
func (r resourceRequest) IsCrossPartition() bool {
return r.Request.ClientInfo.PartitionID != r.Resource.GetARN().Partition
}
func (r resourceRequest) IsCrossRegion() bool {
return isCrossRegion(r.Request, r.Resource.GetARN().Region)
}
func (r resourceRequest) HasCustomEndpoint() bool {
return len(aws.StringValue(r.Request.Config.Endpoint)) > 0
}
func isFIPS(clientRegion string) bool {
return strings.HasPrefix(clientRegion, "fips-") || strings.HasSuffix(clientRegion, "-fips")
}
func isCrossRegion(req *request.Request, otherRegion string) bool {
return req.ClientInfo.SigningRegion != otherRegion
}
func updateBucketEndpointFromParams(r *request.Request) {
bucket, ok := bucketNameFromReqParams(r.Params)
if !ok {
@ -124,7 +151,7 @@ func updateBucketEndpointFromParams(r *request.Request) {
func updateRequestAccessPointEndpoint(req *request.Request, accessPoint arn.AccessPointARN) error {
// Accelerate not supported
if aws.BoolValue(req.Config.S3UseAccelerate) {
return newClientConfiguredForAccelerateError(accessPoint,
return s3shared.NewClientConfiguredForAccelerateError(accessPoint,
req.ClientInfo.PartitionID, aws.StringValue(req.Config.Region), nil)
}
@ -132,7 +159,7 @@ func updateRequestAccessPointEndpoint(req *request.Request, accessPoint arn.Acce
// are not supported.
req.Config.DisableEndpointHostPrefix = aws.Bool(false)
if err := accessPointEndpointBuilder(accessPoint).Build(req); err != nil {
if err := accessPointEndpointBuilder(accessPoint).build(req); err != nil {
return err
}
@ -141,93 +168,34 @@ func updateRequestAccessPointEndpoint(req *request.Request, accessPoint arn.Acce
return nil
}
func updateRequestOutpostAccessPointEndpoint(req *request.Request, accessPoint arn.OutpostAccessPointARN) error {
// Accelerate not supported
if aws.BoolValue(req.Config.S3UseAccelerate) {
return s3shared.NewClientConfiguredForAccelerateError(accessPoint,
req.ClientInfo.PartitionID, aws.StringValue(req.Config.Region), nil)
}
// Dualstack not supported
if aws.BoolValue(req.Config.UseDualStack) {
return s3shared.NewClientConfiguredForDualStackError(accessPoint,
req.ClientInfo.PartitionID, aws.StringValue(req.Config.Region), nil)
}
// Ignore the disable host prefix for access points since custom endpoints
// are not supported.
req.Config.DisableEndpointHostPrefix = aws.Bool(false)
if err := outpostAccessPointEndpointBuilder(accessPoint).build(req); err != nil {
return err
}
removeBucketFromPath(req.HTTPRequest.URL)
return nil
}
func removeBucketFromPath(u *url.URL) {
u.Path = strings.Replace(u.Path, "/{Bucket}", "", -1)
if u.Path == "" {
u.Path = "/"
}
}
type accessPointEndpointBuilder arn.AccessPointARN
const (
accessPointPrefixLabel = "accesspoint"
accountIDPrefixLabel = "accountID"
accesPointPrefixTemplate = "{" + accessPointPrefixLabel + "}-{" + accountIDPrefixLabel + "}."
)
func (a accessPointEndpointBuilder) Build(req *request.Request) error {
resolveRegion := arn.AccessPointARN(a).Region
cfgRegion := aws.StringValue(req.Config.Region)
if isFIPS(cfgRegion) {
if aws.BoolValue(req.Config.S3UseARNRegion) && isCrossRegion(req, resolveRegion) {
// FIPS with cross region is not supported, the SDK must fail
// because there is no well defined method for SDK to construct a
// correct FIPS endpoint.
return newClientConfiguredForCrossRegionFIPSError(arn.AccessPointARN(a),
req.ClientInfo.PartitionID, cfgRegion, nil)
}
resolveRegion = cfgRegion
}
endpoint, err := resolveRegionalEndpoint(req, resolveRegion)
if err != nil {
return newFailedToResolveEndpointError(arn.AccessPointARN(a),
req.ClientInfo.PartitionID, cfgRegion, err)
}
if err = updateRequestEndpoint(req, endpoint.URL); err != nil {
return err
}
const serviceEndpointLabel = "s3-accesspoint"
// dualstack provided by endpoint resolver
cfgHost := req.HTTPRequest.URL.Host
if strings.HasPrefix(cfgHost, "s3") {
req.HTTPRequest.URL.Host = serviceEndpointLabel + cfgHost[2:]
}
protocol.HostPrefixBuilder{
Prefix: accesPointPrefixTemplate,
LabelsFn: a.hostPrefixLabelValues,
}.Build(req)
req.ClientInfo.SigningName = endpoint.SigningName
req.ClientInfo.SigningRegion = endpoint.SigningRegion
err = protocol.ValidateEndpointHost(req.Operation.Name, req.HTTPRequest.URL.Host)
if err != nil {
return newInvalidARNError(arn.AccessPointARN(a), err)
}
return nil
}
func (a accessPointEndpointBuilder) hostPrefixLabelValues() map[string]string {
return map[string]string{
accessPointPrefixLabel: arn.AccessPointARN(a).AccessPointName,
accountIDPrefixLabel: arn.AccessPointARN(a).AccountID,
}
}
func resolveRegionalEndpoint(r *request.Request, region string) (endpoints.ResolvedEndpoint, error) {
return r.Config.EndpointResolver.EndpointFor(EndpointsID, region, func(opts *endpoints.Options) {
opts.DisableSSL = aws.BoolValue(r.Config.DisableSSL)
opts.UseDualStack = aws.BoolValue(r.Config.UseDualStack)
opts.S3UsEast1RegionalEndpoint = endpoints.RegionalS3UsEast1Endpoint
})
}
func updateRequestEndpoint(r *request.Request, endpoint string) (err error) {
endpoint = endpoints.AddScheme(endpoint, aws.BoolValue(r.Config.DisableSSL))
r.HTTPRequest.URL, err = url.Parse(endpoint + r.Operation.HTTPPath)
if err != nil {
return awserr.New(request.ErrCodeSerialization,
"failed to parse endpoint URL", err)
}
return nil
}

View file

@ -0,0 +1,177 @@
package s3
import (
"net/url"
"strings"
"github.com/aws/aws-sdk-go/aws"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/aws/endpoints"
"github.com/aws/aws-sdk-go/aws/request"
"github.com/aws/aws-sdk-go/internal/s3shared"
"github.com/aws/aws-sdk-go/internal/s3shared/arn"
"github.com/aws/aws-sdk-go/private/protocol"
)
const (
accessPointPrefixLabel = "accesspoint"
accountIDPrefixLabel = "accountID"
accessPointPrefixTemplate = "{" + accessPointPrefixLabel + "}-{" + accountIDPrefixLabel + "}."
outpostPrefixLabel = "outpost"
outpostAccessPointPrefixTemplate = accessPointPrefixTemplate + "{" + outpostPrefixLabel + "}."
)
// accessPointEndpointBuilder represents the endpoint builder for access point arn
type accessPointEndpointBuilder arn.AccessPointARN
// build builds the endpoint for corresponding access point arn
//
// For building an endpoint from access point arn, format used is:
// - Access point endpoint format : {accesspointName}-{accountId}.s3-accesspoint.{region}.{dnsSuffix}
// - example : myaccesspoint-012345678901.s3-accesspoint.us-west-2.amazonaws.com
//
// Access Point Endpoint requests are signed using "s3" as signing name.
//
func (a accessPointEndpointBuilder) build(req *request.Request) error {
resolveService := arn.AccessPointARN(a).Service
resolveRegion := arn.AccessPointARN(a).Region
cfgRegion := aws.StringValue(req.Config.Region)
if s3shared.IsFIPS(cfgRegion) {
if aws.BoolValue(req.Config.S3UseARNRegion) && s3shared.IsCrossRegion(req, resolveRegion) {
// FIPS with cross region is not supported, the SDK must fail
// because there is no well defined method for SDK to construct a
// correct FIPS endpoint.
return s3shared.NewClientConfiguredForCrossRegionFIPSError(arn.AccessPointARN(a),
req.ClientInfo.PartitionID, cfgRegion, nil)
}
resolveRegion = cfgRegion
}
endpoint, err := resolveRegionalEndpoint(req, resolveRegion, resolveService)
if err != nil {
return s3shared.NewFailedToResolveEndpointError(arn.AccessPointARN(a),
req.ClientInfo.PartitionID, cfgRegion, err)
}
if err = updateRequestEndpoint(req, endpoint.URL); err != nil {
return err
}
const serviceEndpointLabel = "s3-accesspoint"
// dual stack provided by endpoint resolver
cfgHost := req.HTTPRequest.URL.Host
if strings.HasPrefix(cfgHost, "s3") {
req.HTTPRequest.URL.Host = serviceEndpointLabel + cfgHost[2:]
}
protocol.HostPrefixBuilder{
Prefix: accessPointPrefixTemplate,
LabelsFn: a.hostPrefixLabelValues,
}.Build(req)
// signer redirection
redirectSigner(req, endpoint.SigningName, endpoint.SigningRegion)
err = protocol.ValidateEndpointHost(req.Operation.Name, req.HTTPRequest.URL.Host)
if err != nil {
return s3shared.NewInvalidARNError(arn.AccessPointARN(a), err)
}
return nil
}
func (a accessPointEndpointBuilder) hostPrefixLabelValues() map[string]string {
return map[string]string{
accessPointPrefixLabel: arn.AccessPointARN(a).AccessPointName,
accountIDPrefixLabel: arn.AccessPointARN(a).AccountID,
}
}
// outpostAccessPointEndpointBuilder represents the Endpoint builder for outpost access point arn.
type outpostAccessPointEndpointBuilder arn.OutpostAccessPointARN
// build builds an endpoint corresponding to the outpost access point arn.
//
// For building an endpoint from outpost access point arn, format used is:
// - Outpost access point endpoint format : {accesspointName}-{accountId}.{outpostId}.s3-outposts.{region}.{dnsSuffix}
// - example : myaccesspoint-012345678901.op-01234567890123456.s3-outposts.us-west-2.amazonaws.com
//
// Outpost AccessPoint Endpoint request are signed using "s3-outposts" as signing name.
//
func (o outpostAccessPointEndpointBuilder) build(req *request.Request) error {
resolveRegion := o.Region
resolveService := o.Service
endpointsID := resolveService
if resolveService == "s3-outposts" {
endpointsID = "s3"
}
endpoint, err := resolveRegionalEndpoint(req, resolveRegion, endpointsID)
if err != nil {
return s3shared.NewFailedToResolveEndpointError(o,
req.ClientInfo.PartitionID, resolveRegion, err)
}
if err = updateRequestEndpoint(req, endpoint.URL); err != nil {
return err
}
// add url host as s3-outposts
cfgHost := req.HTTPRequest.URL.Host
if strings.HasPrefix(cfgHost, endpointsID) {
req.HTTPRequest.URL.Host = resolveService + cfgHost[len(endpointsID):]
}
protocol.HostPrefixBuilder{
Prefix: outpostAccessPointPrefixTemplate,
LabelsFn: o.hostPrefixLabelValues,
}.Build(req)
// set the signing region, name to resolved names from ARN
redirectSigner(req, resolveService, resolveRegion)
err = protocol.ValidateEndpointHost(req.Operation.Name, req.HTTPRequest.URL.Host)
if err != nil {
return s3shared.NewInvalidARNError(o, err)
}
return nil
}
func (o outpostAccessPointEndpointBuilder) hostPrefixLabelValues() map[string]string {
return map[string]string{
accessPointPrefixLabel: o.AccessPointName,
accountIDPrefixLabel: o.AccountID,
outpostPrefixLabel: o.OutpostID,
}
}
func resolveRegionalEndpoint(r *request.Request, region string, endpointsID string) (endpoints.ResolvedEndpoint, error) {
return r.Config.EndpointResolver.EndpointFor(endpointsID, region, func(opts *endpoints.Options) {
opts.DisableSSL = aws.BoolValue(r.Config.DisableSSL)
opts.UseDualStack = aws.BoolValue(r.Config.UseDualStack)
opts.S3UsEast1RegionalEndpoint = endpoints.RegionalS3UsEast1Endpoint
})
}
func updateRequestEndpoint(r *request.Request, endpoint string) (err error) {
endpoint = endpoints.AddScheme(endpoint, aws.BoolValue(r.Config.DisableSSL))
r.HTTPRequest.URL, err = url.Parse(endpoint + r.Operation.HTTPPath)
if err != nil {
return awserr.New(request.ErrCodeSerialization,
"failed to parse endpoint URL", err)
}
return nil
}
// redirectSigner sets signing name, signing region for a request
func redirectSigner(req *request.Request, signingName string, signingRegion string) {
req.ClientInfo.SigningName = signingName
req.ClientInfo.SigningRegion = signingRegion
}

View file

@ -1,151 +0,0 @@
package s3
import (
"fmt"
"github.com/aws/aws-sdk-go/aws/awserr"
"github.com/aws/aws-sdk-go/service/s3/internal/arn"
)
const (
invalidARNErrorErrCode = "InvalidARNError"
configurationErrorErrCode = "ConfigurationError"
)
type invalidARNError struct {
message string
resource arn.Resource
origErr error
}
func (e invalidARNError) Error() string {
var extra string
if e.resource != nil {
extra = "ARN: " + e.resource.String()
}
return awserr.SprintError(e.Code(), e.Message(), extra, e.origErr)
}
func (e invalidARNError) Code() string {
return invalidARNErrorErrCode
}
func (e invalidARNError) Message() string {
return e.message
}
func (e invalidARNError) OrigErr() error {
return e.origErr
}
func newInvalidARNError(resource arn.Resource, err error) invalidARNError {
return invalidARNError{
message: "invalid ARN",
origErr: err,
resource: resource,
}
}
func newInvalidARNWithCustomEndpointError(resource arn.Resource, err error) invalidARNError {
return invalidARNError{
message: "resource ARN not supported with custom client endpoints",
origErr: err,
resource: resource,
}
}
// ARN not supported for the target partition
func newInvalidARNWithUnsupportedPartitionError(resource arn.Resource, err error) invalidARNError {
return invalidARNError{
message: "resource ARN not supported for the target ARN partition",
origErr: err,
resource: resource,
}
}
type configurationError struct {
message string
resource arn.Resource
clientPartitionID string
clientRegion string
origErr error
}
func (e configurationError) Error() string {
extra := fmt.Sprintf("ARN: %s, client partition: %s, client region: %s",
e.resource, e.clientPartitionID, e.clientRegion)
return awserr.SprintError(e.Code(), e.Message(), extra, e.origErr)
}
func (e configurationError) Code() string {
return configurationErrorErrCode
}
func (e configurationError) Message() string {
return e.message
}
func (e configurationError) OrigErr() error {
return e.origErr
}
func newClientPartitionMismatchError(resource arn.Resource, clientPartitionID, clientRegion string, err error) configurationError {
return configurationError{
message: "client partition does not match provided ARN partition",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
func newClientRegionMismatchError(resource arn.Resource, clientPartitionID, clientRegion string, err error) configurationError {
return configurationError{
message: "client region does not match provided ARN region",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
func newFailedToResolveEndpointError(resource arn.Resource, clientPartitionID, clientRegion string, err error) configurationError {
return configurationError{
message: "endpoint resolver failed to find an endpoint for the provided ARN region",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
func newClientConfiguredForFIPSError(resource arn.Resource, clientPartitionID, clientRegion string, err error) configurationError {
return configurationError{
message: "client configured for fips but cross-region resource ARN provided",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
func newClientConfiguredForAccelerateError(resource arn.Resource, clientPartitionID, clientRegion string, err error) configurationError {
return configurationError{
message: "client configured for S3 Accelerate but is supported with resource ARN",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}
func newClientConfiguredForCrossRegionFIPSError(resource arn.Resource, clientPartitionID, clientRegion string, err error) configurationError {
return configurationError{
message: "client configured for FIPS with cross-region enabled but is supported with cross-region resource ARN",
origErr: err,
resource: resource,
clientPartitionID: clientPartitionID,
clientRegion: clientRegion,
}
}

View file

@ -8,7 +8,7 @@ const (
// "BucketAlreadyExists".
//
// The requested bucket name is not available. The bucket namespace is shared
// by all users of the system. Please select a different name and try again.
// by all users of the system. Select a different name and try again.
ErrCodeBucketAlreadyExists = "BucketAlreadyExists"
// ErrCodeBucketAlreadyOwnedByYou for service response error code

View file

@ -108,6 +108,10 @@ type S3API interface {
DeleteBucketMetricsConfigurationWithContext(aws.Context, *s3.DeleteBucketMetricsConfigurationInput, ...request.Option) (*s3.DeleteBucketMetricsConfigurationOutput, error)
DeleteBucketMetricsConfigurationRequest(*s3.DeleteBucketMetricsConfigurationInput) (*request.Request, *s3.DeleteBucketMetricsConfigurationOutput)
DeleteBucketOwnershipControls(*s3.DeleteBucketOwnershipControlsInput) (*s3.DeleteBucketOwnershipControlsOutput, error)
DeleteBucketOwnershipControlsWithContext(aws.Context, *s3.DeleteBucketOwnershipControlsInput, ...request.Option) (*s3.DeleteBucketOwnershipControlsOutput, error)
DeleteBucketOwnershipControlsRequest(*s3.DeleteBucketOwnershipControlsInput) (*request.Request, *s3.DeleteBucketOwnershipControlsOutput)
DeleteBucketPolicy(*s3.DeleteBucketPolicyInput) (*s3.DeleteBucketPolicyOutput, error)
DeleteBucketPolicyWithContext(aws.Context, *s3.DeleteBucketPolicyInput, ...request.Option) (*s3.DeleteBucketPolicyOutput, error)
DeleteBucketPolicyRequest(*s3.DeleteBucketPolicyInput) (*request.Request, *s3.DeleteBucketPolicyOutput)
@ -192,6 +196,10 @@ type S3API interface {
GetBucketNotificationConfigurationWithContext(aws.Context, *s3.GetBucketNotificationConfigurationRequest, ...request.Option) (*s3.NotificationConfiguration, error)
GetBucketNotificationConfigurationRequest(*s3.GetBucketNotificationConfigurationRequest) (*request.Request, *s3.NotificationConfiguration)
GetBucketOwnershipControls(*s3.GetBucketOwnershipControlsInput) (*s3.GetBucketOwnershipControlsOutput, error)
GetBucketOwnershipControlsWithContext(aws.Context, *s3.GetBucketOwnershipControlsInput, ...request.Option) (*s3.GetBucketOwnershipControlsOutput, error)
GetBucketOwnershipControlsRequest(*s3.GetBucketOwnershipControlsInput) (*request.Request, *s3.GetBucketOwnershipControlsOutput)
GetBucketPolicy(*s3.GetBucketPolicyInput) (*s3.GetBucketPolicyOutput, error)
GetBucketPolicyWithContext(aws.Context, *s3.GetBucketPolicyInput, ...request.Option) (*s3.GetBucketPolicyOutput, error)
GetBucketPolicyRequest(*s3.GetBucketPolicyInput) (*request.Request, *s3.GetBucketPolicyOutput)
@ -359,6 +367,10 @@ type S3API interface {
PutBucketNotificationConfigurationWithContext(aws.Context, *s3.PutBucketNotificationConfigurationInput, ...request.Option) (*s3.PutBucketNotificationConfigurationOutput, error)
PutBucketNotificationConfigurationRequest(*s3.PutBucketNotificationConfigurationInput) (*request.Request, *s3.PutBucketNotificationConfigurationOutput)
PutBucketOwnershipControls(*s3.PutBucketOwnershipControlsInput) (*s3.PutBucketOwnershipControlsOutput, error)
PutBucketOwnershipControlsWithContext(aws.Context, *s3.PutBucketOwnershipControlsInput, ...request.Option) (*s3.PutBucketOwnershipControlsOutput, error)
PutBucketOwnershipControlsRequest(*s3.PutBucketOwnershipControlsInput) (*request.Request, *s3.PutBucketOwnershipControlsOutput)
PutBucketPolicy(*s3.PutBucketPolicyInput) (*s3.PutBucketPolicyOutput, error)
PutBucketPolicyWithContext(aws.Context, *s3.PutBucketPolicyInput, ...request.Option) (*s3.PutBucketPolicyOutput, error)
PutBucketPolicyRequest(*s3.PutBucketPolicyInput) (*request.Request, *s3.PutBucketPolicyOutput)

View file

@ -16,20 +16,30 @@ type UploadInput struct {
// The canned ACL to apply to the object. For more information, see Canned ACL
// (https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#CannedACL).
//
// This action is not supported by Amazon S3 on Outposts.
ACL *string `location:"header" locationName:"x-amz-acl" type:"string" enum:"ObjectCannedACL"`
// The readable body payload to send to S3.
Body io.Reader
// Bucket name to which the PUT operation was initiated.
// The bucket name to which the PUT operation was initiated.
//
// When using this API with an access point, you must direct requests to the
// access point hostname. The access point hostname takes the form AccessPointName-AccountId.s3-accesspoint.Region.amazonaws.com.
// When using this operation using an access point through the AWS SDKs, you
// When using this operation with an access point through the AWS SDKs, you
// provide the access point ARN in place of the bucket name. For more information
// about access point ARNs, see Using Access Points (https://docs.aws.amazon.com/AmazonS3/latest/dev/using-access-points.html)
// in the Amazon Simple Storage Service Developer Guide.
//
// When using this API with Amazon S3 on Outposts, you must direct requests
// to the S3 on Outposts hostname. The S3 on Outposts hostname takes the form
// AccessPointName-AccountId.outpostID.s3-outposts.Region.amazonaws.com. When
// using this operation using S3 on Outposts through the AWS SDKs, you provide
// the Outposts bucket ARN in place of the bucket name. For more information
// about S3 on Outposts ARNs, see Using S3 on Outposts (https://docs.aws.amazon.com/AmazonS3/latest/dev/S3onOutposts.html)
// in the Amazon Simple Storage Service Developer Guide.
//
// Bucket is a required field
Bucket *string `location:"uri" locationName:"Bucket" type:"string" required:"true"`
@ -73,15 +83,23 @@ type UploadInput struct {
Expires *time.Time `location:"header" locationName:"Expires" type:"timestamp"`
// Gives the grantee READ, READ_ACP, and WRITE_ACP permissions on the object.
//
// This action is not supported by Amazon S3 on Outposts.
GrantFullControl *string `location:"header" locationName:"x-amz-grant-full-control" type:"string"`
// Allows grantee to read the object data and its metadata.
//
// This action is not supported by Amazon S3 on Outposts.
GrantRead *string `location:"header" locationName:"x-amz-grant-read" type:"string"`
// Allows grantee to read the object ACL.
//
// This action is not supported by Amazon S3 on Outposts.
GrantReadACP *string `location:"header" locationName:"x-amz-grant-read-acp" type:"string"`
// Allows grantee to write the ACL for the applicable object.
//
// This action is not supported by Amazon S3 on Outposts.
GrantWriteACP *string `location:"header" locationName:"x-amz-grant-write-acp" type:"string"`
// Object key for which the PUT operation was initiated.
@ -146,8 +164,12 @@ type UploadInput struct {
// S3 (for example, AES256, aws:kms).
ServerSideEncryption *string `location:"header" locationName:"x-amz-server-side-encryption" type:"string" enum:"ServerSideEncryption"`
// If you don't specify, S3 Standard is the default storage class. Amazon S3
// supports other storage classes.
// By default, Amazon S3 uses the STANDARD Storage Class to store newly created
// objects. The STANDARD storage class provides high durability and high availability.
// Depending on performance needs, you can specify a different Storage Class.
// Amazon S3 on Outposts only uses the OUTPOSTS Storage Class. For more information,
// see Storage Classes (https://docs.aws.amazon.com/AmazonS3/latest/dev/storage-class-intro.html)
// in the Amazon S3 Service Developer Guide.
StorageClass *string `location:"header" locationName:"x-amz-storage-class" type:"string" enum:"StorageClass"`
// The tag-set for the object. The tag-set must be encoded as URL Query parameters.

View file

@ -30,12 +30,13 @@ type encoderOptions struct {
func (o *encoderOptions) setDefault() {
*o = encoderOptions{
// use less ram: true for now, but may change.
concurrent: runtime.GOMAXPROCS(0),
crc: true,
single: nil,
blockSize: 1 << 16,
windowSize: 8 << 20,
level: SpeedDefault,
concurrent: runtime.GOMAXPROCS(0),
crc: true,
single: nil,
blockSize: 1 << 16,
windowSize: 8 << 20,
level: SpeedDefault,
allLitEntropy: true,
}
}

View file

@ -12,10 +12,8 @@ import "syscall"
// We can't use the gc-syntax .s files for gccgo. On the plus side
// much of the functionality can be written directly in Go.
//extern gccgoRealSyscallNoError
func realSyscallNoError(trap, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r uintptr)
//extern gccgoRealSyscall
func realSyscall(trap, a1, a2, a3, a4, a5, a6, a7, a8, a9 uintptr) (r, errno uintptr)
func SyscallNoError(trap, a1, a2, a3 uintptr) (r1, r2 uintptr) {

View file

@ -21,6 +21,9 @@ struct ret {
uintptr_t err;
};
struct ret gccgoRealSyscall(uintptr_t trap, uintptr_t a1, uintptr_t a2, uintptr_t a3, uintptr_t a4, uintptr_t a5, uintptr_t a6, uintptr_t a7, uintptr_t a8, uintptr_t a9)
__asm__(GOSYM_PREFIX GOPKGPATH ".realSyscall");
struct ret
gccgoRealSyscall(uintptr_t trap, uintptr_t a1, uintptr_t a2, uintptr_t a3, uintptr_t a4, uintptr_t a5, uintptr_t a6, uintptr_t a7, uintptr_t a8, uintptr_t a9)
{
@ -32,6 +35,9 @@ gccgoRealSyscall(uintptr_t trap, uintptr_t a1, uintptr_t a2, uintptr_t a3, uintp
return r;
}
uintptr_t gccgoRealSyscallNoError(uintptr_t trap, uintptr_t a1, uintptr_t a2, uintptr_t a3, uintptr_t a4, uintptr_t a5, uintptr_t a6, uintptr_t a7, uintptr_t a8, uintptr_t a9)
__asm__(GOSYM_PREFIX GOPKGPATH ".realSyscallNoError");
uintptr_t
gccgoRealSyscallNoError(uintptr_t trap, uintptr_t a1, uintptr_t a2, uintptr_t a3, uintptr_t a4, uintptr_t a5, uintptr_t a6, uintptr_t a7, uintptr_t a8, uintptr_t a9)
{

View file

@ -527,7 +527,7 @@ ccflags="$@"
$2 ~ /^RND/ ||
$2 ~ /^KEY_(SPEC|REQKEY_DEFL)_/ ||
$2 ~ /^KEYCTL_/ ||
$2 ~ /^PERF_EVENT_IOC_/ ||
$2 ~ /^PERF_/ ||
$2 ~ /^SECCOMP_MODE_/ ||
$2 ~ /^SPLICE_/ ||
$2 ~ /^SYNC_FILE_RANGE_/ ||
@ -546,7 +546,7 @@ ccflags="$@"
$2 ~ /^XATTR_(CREATE|REPLACE|NO(DEFAULT|FOLLOW|SECURITY)|SHOWCOMPRESSION)/ ||
$2 ~ /^ATTR_(BIT_MAP_COUNT|(CMN|VOL|FILE)_)/ ||
$2 ~ /^FSOPT_/ ||
$2 ~ /^WDIOC_/ ||
$2 ~ /^WDIO[CFS]_/ ||
$2 ~ /^NFN/ ||
$2 ~ /^XDP_/ ||
$2 ~ /^RWF_/ ||

View file

@ -32,6 +32,10 @@ func cmsgAlignOf(salen int) int {
if runtime.GOARCH == "arm" {
salign = 8
}
// NetBSD aarch64 requires 128-bit alignment.
if runtime.GOOS == "netbsd" && runtime.GOARCH == "arm64" {
salign = 16
}
}
return (salen + salign - 1) & ^(salign - 1)

View file

@ -106,15 +106,24 @@ func IoctlGetRTCTime(fd int) (*RTCTime, error) {
return &value, err
}
// IoctlGetWatchdogInfo fetches information about a watchdog device from the
// Linux watchdog API. For more information, see:
// https://www.kernel.org/doc/html/latest/watchdog/watchdog-api.html.
func IoctlGetWatchdogInfo(fd int) (*WatchdogInfo, error) {
var value WatchdogInfo
err := ioctl(fd, WDIOC_GETSUPPORT, uintptr(unsafe.Pointer(&value)))
return &value, err
}
func IoctlGetRTCWkAlrm(fd int) (*RTCWkAlrm, error) {
var value RTCWkAlrm
err := ioctl(fd, RTC_WKALM_RD, uintptr(unsafe.Pointer(&value)))
return &value, err
}
// IoctlFileClone performs an FICLONERANGE ioctl operation to clone the range of
// data conveyed in value to the file associated with the file descriptor
// destFd. See the ioctl_ficlonerange(2) man page for details.
// IoctlFileCloneRange performs an FICLONERANGE ioctl operation to clone the
// range of data conveyed in value to the file associated with the file
// descriptor destFd. See the ioctl_ficlonerange(2) man page for details.
func IoctlFileCloneRange(destFd int, value *FileCloneRange) error {
err := ioctl(destFd, FICLONERANGE, uintptr(unsafe.Pointer(value)))
runtime.KeepAlive(value)
@ -128,15 +137,22 @@ func IoctlFileClone(destFd, srcFd int) error {
return ioctl(destFd, FICLONE, uintptr(srcFd))
}
// IoctlFileClone performs an FIDEDUPERANGE ioctl operation to share the range of
// data conveyed in value with the file associated with the file descriptor
// destFd. See the ioctl_fideduperange(2) man page for details.
// IoctlFileDedupeRange performs an FIDEDUPERANGE ioctl operation to share the
// range of data conveyed in value with the file associated with the file
// descriptor destFd. See the ioctl_fideduperange(2) man page for details.
func IoctlFileDedupeRange(destFd int, value *FileDedupeRange) error {
err := ioctl(destFd, FIDEDUPERANGE, uintptr(unsafe.Pointer(value)))
runtime.KeepAlive(value)
return err
}
// IoctlWatchdogKeepalive issues a keepalive ioctl to a watchdog device. For
// more information, see:
// https://www.kernel.org/doc/html/latest/watchdog/watchdog-api.html.
func IoctlWatchdogKeepalive(fd int) error {
return ioctl(fd, WDIOC_KEEPALIVE, 0)
}
//sys Linkat(olddirfd int, oldpath string, newdirfd int, newpath string, flags int) (err error)
func Link(oldpath string, newpath string) (err error) {

View file

@ -13,6 +13,7 @@
package unix
import (
"runtime"
"syscall"
"unsafe"
)
@ -554,7 +555,9 @@ func Minor(dev uint64) uint32 {
//sys ioctl(fd int, req uint, arg uintptr) (err error)
func IoctlSetTermio(fd int, req uint, value *Termio) (err error) {
return ioctl(fd, req, uintptr(unsafe.Pointer(value)))
err := ioctl(fd, req, uintptr(unsafe.Pointer(value)))
runtime.KeepAlive(value)
return err
}
func IoctlGetTermio(fd int, req uint) (*Termio, error) {

View file

@ -1508,6 +1508,92 @@ const (
PARITY_DEFAULT = 0x0
PARITY_NONE = 0x1
PARMRK = 0x8
PERF_ATTR_SIZE_VER0 = 0x40
PERF_ATTR_SIZE_VER1 = 0x48
PERF_ATTR_SIZE_VER2 = 0x50
PERF_ATTR_SIZE_VER3 = 0x60
PERF_ATTR_SIZE_VER4 = 0x68
PERF_ATTR_SIZE_VER5 = 0x70
PERF_ATTR_SIZE_VER6 = 0x78
PERF_AUX_FLAG_COLLISION = 0x8
PERF_AUX_FLAG_OVERWRITE = 0x2
PERF_AUX_FLAG_PARTIAL = 0x4
PERF_AUX_FLAG_TRUNCATED = 0x1
PERF_FLAG_FD_CLOEXEC = 0x8
PERF_FLAG_FD_NO_GROUP = 0x1
PERF_FLAG_FD_OUTPUT = 0x2
PERF_FLAG_PID_CGROUP = 0x4
PERF_MAX_CONTEXTS_PER_STACK = 0x8
PERF_MAX_STACK_DEPTH = 0x7f
PERF_MEM_LOCK_LOCKED = 0x2
PERF_MEM_LOCK_NA = 0x1
PERF_MEM_LOCK_SHIFT = 0x18
PERF_MEM_LVLNUM_ANY_CACHE = 0xb
PERF_MEM_LVLNUM_L1 = 0x1
PERF_MEM_LVLNUM_L2 = 0x2
PERF_MEM_LVLNUM_L3 = 0x3
PERF_MEM_LVLNUM_L4 = 0x4
PERF_MEM_LVLNUM_LFB = 0xc
PERF_MEM_LVLNUM_NA = 0xf
PERF_MEM_LVLNUM_PMEM = 0xe
PERF_MEM_LVLNUM_RAM = 0xd
PERF_MEM_LVLNUM_SHIFT = 0x21
PERF_MEM_LVL_HIT = 0x2
PERF_MEM_LVL_IO = 0x1000
PERF_MEM_LVL_L1 = 0x8
PERF_MEM_LVL_L2 = 0x20
PERF_MEM_LVL_L3 = 0x40
PERF_MEM_LVL_LFB = 0x10
PERF_MEM_LVL_LOC_RAM = 0x80
PERF_MEM_LVL_MISS = 0x4
PERF_MEM_LVL_NA = 0x1
PERF_MEM_LVL_REM_CCE1 = 0x400
PERF_MEM_LVL_REM_CCE2 = 0x800
PERF_MEM_LVL_REM_RAM1 = 0x100
PERF_MEM_LVL_REM_RAM2 = 0x200
PERF_MEM_LVL_SHIFT = 0x5
PERF_MEM_LVL_UNC = 0x2000
PERF_MEM_OP_EXEC = 0x10
PERF_MEM_OP_LOAD = 0x2
PERF_MEM_OP_NA = 0x1
PERF_MEM_OP_PFETCH = 0x8
PERF_MEM_OP_SHIFT = 0x0
PERF_MEM_OP_STORE = 0x4
PERF_MEM_REMOTE_REMOTE = 0x1
PERF_MEM_REMOTE_SHIFT = 0x25
PERF_MEM_SNOOPX_FWD = 0x1
PERF_MEM_SNOOPX_SHIFT = 0x25
PERF_MEM_SNOOP_HIT = 0x4
PERF_MEM_SNOOP_HITM = 0x10
PERF_MEM_SNOOP_MISS = 0x8
PERF_MEM_SNOOP_NA = 0x1
PERF_MEM_SNOOP_NONE = 0x2
PERF_MEM_SNOOP_SHIFT = 0x13
PERF_MEM_TLB_HIT = 0x2
PERF_MEM_TLB_L1 = 0x8
PERF_MEM_TLB_L2 = 0x10
PERF_MEM_TLB_MISS = 0x4
PERF_MEM_TLB_NA = 0x1
PERF_MEM_TLB_OS = 0x40
PERF_MEM_TLB_SHIFT = 0x1a
PERF_MEM_TLB_WK = 0x20
PERF_RECORD_KSYMBOL_FLAGS_UNREGISTER = 0x1
PERF_RECORD_MISC_COMM_EXEC = 0x2000
PERF_RECORD_MISC_CPUMODE_MASK = 0x7
PERF_RECORD_MISC_CPUMODE_UNKNOWN = 0x0
PERF_RECORD_MISC_EXACT_IP = 0x4000
PERF_RECORD_MISC_EXT_RESERVED = 0x8000
PERF_RECORD_MISC_FORK_EXEC = 0x2000
PERF_RECORD_MISC_GUEST_KERNEL = 0x4
PERF_RECORD_MISC_GUEST_USER = 0x5
PERF_RECORD_MISC_HYPERVISOR = 0x3
PERF_RECORD_MISC_KERNEL = 0x1
PERF_RECORD_MISC_MMAP_DATA = 0x2000
PERF_RECORD_MISC_PROC_MAP_PARSE_TIMEOUT = 0x1000
PERF_RECORD_MISC_SWITCH_OUT = 0x2000
PERF_RECORD_MISC_SWITCH_OUT_PREEMPT = 0x4000
PERF_RECORD_MISC_USER = 0x2
PERF_SAMPLE_BRANCH_PLM_ALL = 0x7
PIPEFS_MAGIC = 0x50495045
PPC_CMM_MAGIC = 0xc7571590
PPPIOCGNPMODE = 0xc008744c
@ -2357,6 +2443,23 @@ const (
WCONTINUED = 0x8
WDIOC_SETPRETIMEOUT = 0xc0045708
WDIOC_SETTIMEOUT = 0xc0045706
WDIOF_ALARMONLY = 0x400
WDIOF_CARDRESET = 0x20
WDIOF_EXTERN1 = 0x4
WDIOF_EXTERN2 = 0x8
WDIOF_FANFAULT = 0x2
WDIOF_KEEPALIVEPING = 0x8000
WDIOF_MAGICCLOSE = 0x100
WDIOF_OVERHEAT = 0x1
WDIOF_POWEROVER = 0x40
WDIOF_POWERUNDER = 0x10
WDIOF_PRETIMEOUT = 0x200
WDIOF_SETTIMEOUT = 0x80
WDIOF_UNKNOWN = -0x1
WDIOS_DISABLECARD = 0x1
WDIOS_ENABLECARD = 0x2
WDIOS_TEMPPANIC = 0x4
WDIOS_UNKNOWN = -0x1
WEXITED = 0x4
WIN_ACKMEDIACHANGE = 0xdb
WIN_CHECKPOWERMODE1 = 0xe5

View file

@ -823,8 +823,6 @@ type SignalfdSiginfo struct {
_ [28]uint8
}
const PERF_IOC_FLAG_GROUP = 0x1
type Winsize struct {
Row uint16
Col uint16
@ -990,13 +988,13 @@ const (
)
const (
PERF_TYPE_HARDWARE = 0x0
PERF_TYPE_SOFTWARE = 0x1
PERF_TYPE_TRACEPOINT = 0x2
PERF_TYPE_HW_CACHE = 0x3
PERF_TYPE_RAW = 0x4
PERF_TYPE_BREAKPOINT = 0x5
PERF_FLAG_FD_CLOEXEF_TYPE_HARDWARE = 0x0
PERF_TYPE_SOFTWARE = 0x1
PERF_TYPE_TRACEPOINT = 0x2
PERF_TYPE_HW_CACHE = 0x3
PERF_TYPE_RAW = 0x4
PERF_TYPE_BREAKPOINT = 0x5
PERF_TYPE_MAX = 0x6
PERF_COUNT_HW_CPU_CYCLES = 0x0
PERF_COUNT_HW_INSTRUCTIONS = 0x1
PERF_COUNT_HW_CACHE_REFERENCES = 0x2
@ -1007,106 +1005,161 @@ const (
PERF_COUNT_HW_STALLED_CYCLES_FRONTEND = 0x7
PERF_COUNT_HW_STALLED_CYCLES_BACKEND = 0x8
PERF_COUNT_HW_REF_CPU_CYCLES = 0x9
PERF_COUNT_HW_CACHE_L1D = 0x0
PERF_COUNT_HW_CACHE_L1I = 0x1
PERF_COUNT_HW_CACHE_LL = 0x2
PERF_COUNT_HW_CACHE_DTLB = 0x3
PERF_COUNT_HW_CACHE_ITLB = 0x4
PERF_COUNT_HW_CACHE_BPU = 0x5
PERF_COUNT_HW_CACHE_NODE = 0x6
PERF_COUNT_HW_CACHE_OP_READ = 0x0
PERF_COUNT_HW_CACHE_OP_WRITE = 0x1
PERF_COUNT_HW_CACHE_OP_PREFETCH = 0x2
PERF_COUNT_HW_CACHE_RESULT_ACCESS = 0x0
PERF_COUNT_HW_CACHE_RESULT_MISS = 0x1
PERF_COUNT_SW_CPU_CLOCK = 0x0
PERF_COUNT_SW_TASK_CLOCK = 0x1
PERF_COUNT_SW_PAGE_FAULTS = 0x2
PERF_COUNT_SW_CONTEXT_SWITCHES = 0x3
PERF_COUNT_SW_CPU_MIGRATIONS = 0x4
PERF_COUNT_SW_PAGE_FAULTS_MIN = 0x5
PERF_COUNT_SW_PAGE_FAULTS_MAJ = 0x6
PERF_COUNT_SW_ALIGNMENT_FAULTS = 0x7
PERF_COUNT_SW_EMULATION_FAULTS = 0x8
PERF_COUNT_SW_DUMMY = 0x9
PERF_COUNT_SW_BPF_OUTPUT = 0xa
PERF_SAMPLE_IP = 0x1
PERF_SAMPLE_TID = 0x2
PERF_SAMPLE_TIME = 0x4
PERF_SAMPLE_ADDR = 0x8
PERF_SAMPLE_READ = 0x10
PERF_SAMPLE_CALLCHAIN = 0x20
PERF_SAMPLE_ID = 0x40
PERF_SAMPLE_CPU = 0x80
PERF_SAMPLE_PERIOD = 0x100
PERF_SAMPLE_STREAM_ID = 0x200
PERF_SAMPLE_RAW = 0x400
PERF_SAMPLE_BRANCH_STACK = 0x800
PERF_SAMPLE_REGS_USER = 0x1000
PERF_SAMPLE_STACK_USER = 0x2000
PERF_SAMPLE_WEIGHT = 0x4000
PERF_SAMPLE_DATA_SRC = 0x8000
PERF_SAMPLE_IDENTIFIER = 0x10000
PERF_SAMPLE_TRANSACTION = 0x20000
PERF_SAMPLE_REGS_INTR = 0x40000
PERF_SAMPLE_BRANCH_USER = 0x1
PERF_SAMPLE_BRANCH_KERNEL = 0x2
PERF_SAMPLE_BRANCH_HV = 0x4
PERF_SAMPLE_BRANCH_ANY = 0x8
PERF_SAMPLE_BRANCH_ANY_CALL = 0x10
PERF_SAMPLE_BRANCH_ANY_RETURN = 0x20
PERF_SAMPLE_BRANCH_IND_CALL = 0x40
PERF_SAMPLE_BRANCH_ABORT_TX = 0x80
PERF_SAMPLE_BRANCH_IN_TX = 0x100
PERF_SAMPLE_BRANCH_NO_TX = 0x200
PERF_SAMPLE_BRANCH_COND = 0x400
PERF_SAMPLE_BRANCH_CALL_STACK = 0x800
PERF_SAMPLE_BRANCH_IND_JUMP = 0x1000
PERF_SAMPLE_BRANCH_CALL = 0x2000
PERF_SAMPLE_BRANCH_NO_FLAGS = 0x4000
PERF_SAMPLE_BRANCH_NO_CYCLES = 0x8000
PERF_SAMPLE_BRANCH_TYPE_SAVE = 0x10000
PERF_FORMAT_TOTAL_TIME_ENABLED = 0x1
PERF_FORMAT_TOTAL_TIME_RUNNING = 0x2
PERF_FORMAT_ID = 0x4
PERF_FORMAT_GROUP = 0x8
PERF_RECORD_MMAP = 0x1
PERF_RECORD_LOST = 0x2
PERF_RECORD_COMM = 0x3
PERF_RECORD_EXIT = 0x4
PERF_RECORD_THROTTLE = 0x5
PERF_RECORD_UNTHROTTLE = 0x6
PERF_RECORD_FORK = 0x7
PERF_RECORD_READ = 0x8
PERF_RECORD_SAMPLE = 0x9
PERF_RECORD_MMAP2 = 0xa
PERF_RECORD_AUX = 0xb
PERF_RECORD_ITRACE_START = 0xc
PERF_RECORD_LOST_SAMPLES = 0xd
PERF_RECORD_SWITCH = 0xe
PERF_RECORD_SWITCH_CPU_WIDE = 0xf
PERF_RECORD_NAMESPACES = 0x10
PERF_CONTEXT_HV = -0x20
PERF_CONTEXT_KERNEL = -0x80
PERF_CONTEXT_USER = -0x200
PERF_CONTEXT_GUEST = -0x800
PERF_CONTEXT_GUEST_KERNEL = -0x880
PERF_CONTEXT_GUEST_USER = -0xa00
PERF_FLAG_FD_NO_GROUP = 0x1
PERF_FLAG_FD_OUTPUT = 0x2
PERF_FLAG_PID_CGROUP = 0x4
PERF_FLAG_FD_CLOEXEC = 0x8
PERF_COUNT_HW_MAX = 0xa
PERF_COUNT_HW_CACHE_L1D = 0x0
PERF_COUNT_HW_CACHE_L1I = 0x1
PERF_COUNT_HW_CACHE_LL = 0x2
PERF_COUNT_HW_CACHE_DTLB = 0x3
PERF_COUNT_HW_CACHE_ITLB = 0x4
PERF_COUNT_HW_CACHE_BPU = 0x5
PERF_COUNT_HW_CACHE_NODE = 0x6
PERF_COUNT_HW_CACHE_MAX = 0x7
PERF_COUNT_HW_CACHE_OP_READ = 0x0
PERF_COUNT_HW_CACHE_OP_WRITE = 0x1
PERF_COUNT_HW_CACHE_OP_PREFETCH = 0x2
PERF_COUNT_HW_CACHE_OP_MAX = 0x3
PERF_COUNT_HW_CACHE_RESULT_ACCESS = 0x0
PERF_COUNT_HW_CACHE_RESULT_MISS = 0x1
PERF_COUNT_HW_CACHE_RESULT_MAX = 0x2
PERF_COUNT_SW_CPU_CLOCK = 0x0
PERF_COUNT_SW_TASK_CLOCK = 0x1
PERF_COUNT_SW_PAGE_FAULTS = 0x2
PERF_COUNT_SW_CONTEXT_SWITCHES = 0x3
PERF_COUNT_SW_CPU_MIGRATIONS = 0x4
PERF_COUNT_SW_PAGE_FAULTS_MIN = 0x5
PERF_COUNT_SW_PAGE_FAULTS_MAJ = 0x6
PERF_COUNT_SW_ALIGNMENT_FAULTS = 0x7
PERF_COUNT_SW_EMULATION_FAULTS = 0x8
PERF_COUNT_SW_DUMMY = 0x9
PERF_COUNT_SW_BPF_OUTPUT = 0xa
PERF_COUNT_SW_MAX = 0xb
PERF_SAMPLE_IP = 0x1
PERF_SAMPLE_TID = 0x2
PERF_SAMPLE_TIME = 0x4
PERF_SAMPLE_ADDR = 0x8
PERF_SAMPLE_READ = 0x10
PERF_SAMPLE_CALLCHAIN = 0x20
PERF_SAMPLE_ID = 0x40
PERF_SAMPLE_CPU = 0x80
PERF_SAMPLE_PERIOD = 0x100
PERF_SAMPLE_STREAM_ID = 0x200
PERF_SAMPLE_RAW = 0x400
PERF_SAMPLE_BRANCH_STACK = 0x800
PERF_SAMPLE_REGS_USER = 0x1000
PERF_SAMPLE_STACK_USER = 0x2000
PERF_SAMPLE_WEIGHT = 0x4000
PERF_SAMPLE_DATA_SRC = 0x8000
PERF_SAMPLE_IDENTIFIER = 0x10000
PERF_SAMPLE_TRANSACTION = 0x20000
PERF_SAMPLE_REGS_INTR = 0x40000
PERF_SAMPLE_PHYS_ADDR = 0x80000
PERF_SAMPLE_AUX = 0x100000
PERF_SAMPLE_CGROUP = 0x200000
PERF_SAMPLE_MAX = 0x400000
PERF_SAMPLE_BRANCH_USER_SHIFT = 0x0
PERF_SAMPLE_BRANCH_KERNEL_SHIFT = 0x1
PERF_SAMPLE_BRANCH_HV_SHIFT = 0x2
PERF_SAMPLE_BRANCH_ANY_SHIFT = 0x3
PERF_SAMPLE_BRANCH_ANY_CALL_SHIFT = 0x4
PERF_SAMPLE_BRANCH_ANY_RETURN_SHIFT = 0x5
PERF_SAMPLE_BRANCH_IND_CALL_SHIFT = 0x6
PERF_SAMPLE_BRANCH_ABORT_TX_SHIFT = 0x7
PERF_SAMPLE_BRANCH_IN_TX_SHIFT = 0x8
PERF_SAMPLE_BRANCH_NO_TX_SHIFT = 0x9
PERF_SAMPLE_BRANCH_COND_SHIFT = 0xa
PERF_SAMPLE_BRANCH_CALL_STACK_SHIFT = 0xb
PERF_SAMPLE_BRANCH_IND_JUMP_SHIFT = 0xc
PERF_SAMPLE_BRANCH_CALL_SHIFT = 0xd
PERF_SAMPLE_BRANCH_NO_FLAGS_SHIFT = 0xe
PERF_SAMPLE_BRANCH_NO_CYCLES_SHIFT = 0xf
PERF_SAMPLE_BRANCH_TYPE_SAVE_SHIFT = 0x10
PERF_SAMPLE_BRANCH_HW_INDEX_SHIFT = 0x11
PERF_SAMPLE_BRANCH_MAX_SHIFT = 0x12
PERF_SAMPLE_BRANCH_USER = 0x1
PERF_SAMPLE_BRANCH_KERNEL = 0x2
PERF_SAMPLE_BRANCH_HV = 0x4
PERF_SAMPLE_BRANCH_ANY = 0x8
PERF_SAMPLE_BRANCH_ANY_CALL = 0x10
PERF_SAMPLE_BRANCH_ANY_RETURN = 0x20
PERF_SAMPLE_BRANCH_IND_CALL = 0x40
PERF_SAMPLE_BRANCH_ABORT_TX = 0x80
PERF_SAMPLE_BRANCH_IN_TX = 0x100
PERF_SAMPLE_BRANCH_NO_TX = 0x200
PERF_SAMPLE_BRANCH_COND = 0x400
PERF_SAMPLE_BRANCH_CALL_STACK = 0x800
PERF_SAMPLE_BRANCH_IND_JUMP = 0x1000
PERF_SAMPLE_BRANCH_CALL = 0x2000
PERF_SAMPLE_BRANCH_NO_FLAGS = 0x4000
PERF_SAMPLE_BRANCH_NO_CYCLES = 0x8000
PERF_SAMPLE_BRANCH_TYPE_SAVE = 0x10000
PERF_SAMPLE_BRANCH_HW_INDEX = 0x20000
PERF_SAMPLE_BRANCH_MAX = 0x40000
PERF_BR_UNKNOWN = 0x0
PERF_BR_COND = 0x1
PERF_BR_UNCOND = 0x2
PERF_BR_IND = 0x3
PERF_BR_CALL = 0x4
PERF_BR_IND_CALL = 0x5
PERF_BR_RET = 0x6
PERF_BR_SYSCALL = 0x7
PERF_BR_SYSRET = 0x8
PERF_BR_COND_CALL = 0x9
PERF_BR_COND_RET = 0xa
PERF_BR_MAX = 0xb
PERF_SAMPLE_REGS_ABI_NONE = 0x0
PERF_SAMPLE_REGS_ABI_32 = 0x1
PERF_SAMPLE_REGS_ABI_64 = 0x2
PERF_TXN_ELISION = 0x1
PERF_TXN_TRANSACTION = 0x2
PERF_TXN_SYNC = 0x4
PERF_TXN_ASYNC = 0x8
PERF_TXN_RETRY = 0x10
PERF_TXN_CONFLICT = 0x20
PERF_TXN_CAPACITY_WRITE = 0x40
PERF_TXN_CAPACITY_READ = 0x80
PERF_TXN_MAX = 0x100
PERF_TXN_ABORT_MASK = -0x100000000
PERF_TXN_ABORT_SHIFT = 0x20
PERF_FORMAT_TOTAL_TIME_ENABLED = 0x1
PERF_FORMAT_TOTAL_TIME_RUNNING = 0x2
PERF_FORMAT_ID = 0x4
PERF_FORMAT_GROUP = 0x8
PERF_FORMAT_MAX = 0x10
PERF_IOC_FLAG_GROUP = 0x1
PERF_RECORD_MMAP = 0x1
PERF_RECORD_LOST = 0x2
PERF_RECORD_COMM = 0x3
PERF_RECORD_EXIT = 0x4
PERF_RECORD_THROTTLE = 0x5
PERF_RECORD_UNTHROTTLE = 0x6
PERF_RECORD_FORK = 0x7
PERF_RECORD_READ = 0x8
PERF_RECORD_SAMPLE = 0x9
PERF_RECORD_MMAP2 = 0xa
PERF_RECORD_AUX = 0xb
PERF_RECORD_ITRACE_START = 0xc
PERF_RECORD_LOST_SAMPLES = 0xd
PERF_RECORD_SWITCH = 0xe
PERF_RECORD_SWITCH_CPU_WIDE = 0xf
PERF_RECORD_NAMESPACES = 0x10
PERF_RECORD_KSYMBOL = 0x11
PERF_RECORD_BPF_EVENT = 0x12
PERF_RECORD_CGROUP = 0x13
PERF_RECORD_MAX = 0x14
PERF_RECORD_KSYMBOL_TYPE_UNKNOWN = 0x0
PERF_RECORD_KSYMBOL_TYPE_BPF = 0x1
PERF_RECORD_KSYMBOL_TYPE_MAX = 0x2
PERF_BPF_EVENT_UNKNOWN = 0x0
PERF_BPF_EVENT_PROG_LOAD = 0x1
PERF_BPF_EVENT_PROG_UNLOAD = 0x2
PERF_BPF_EVENT_MAX = 0x3
PERF_CONTEXT_HV = -0x20
PERF_CONTEXT_KERNEL = -0x80
PERF_CONTEXT_USER = -0x200
PERF_CONTEXT_GUEST = -0x800
PERF_CONTEXT_GUEST_KERNEL = -0x880
PERF_CONTEXT_GUEST_USER = -0xa00
PERF_CONTEXT_MAX = -0xfff
)
type TCPMD5Sig struct {
@ -2567,3 +2620,9 @@ const (
CAN_RAW_FD_FRAMES = 0x5
CAN_RAW_JOIN_FILTERS = 0x6
)
type WatchdogInfo struct {
Options uint32
Version uint32
Identity [32]uint8
}

21
vendor/modules.txt vendored
View file

@ -1,4 +1,4 @@
# cloud.google.com/go v0.67.0
# cloud.google.com/go v0.68.0
cloud.google.com/go
cloud.google.com/go/compute/metadata
cloud.google.com/go/iam
@ -19,7 +19,7 @@ github.com/VictoriaMetrics/metrics
# github.com/VictoriaMetrics/metricsql v0.6.0
github.com/VictoriaMetrics/metricsql
github.com/VictoriaMetrics/metricsql/binaryop
# github.com/aws/aws-sdk-go v1.34.34
# github.com/aws/aws-sdk-go v1.35.3
github.com/aws/aws-sdk-go/aws
github.com/aws/aws-sdk-go/aws/arn
github.com/aws/aws-sdk-go/aws/awserr
@ -41,7 +41,9 @@ github.com/aws/aws-sdk-go/aws/session
github.com/aws/aws-sdk-go/aws/signer/v4
github.com/aws/aws-sdk-go/internal/context
github.com/aws/aws-sdk-go/internal/ini
github.com/aws/aws-sdk-go/internal/s3err
github.com/aws/aws-sdk-go/internal/s3shared
github.com/aws/aws-sdk-go/internal/s3shared/arn
github.com/aws/aws-sdk-go/internal/s3shared/s3err
github.com/aws/aws-sdk-go/internal/sdkio
github.com/aws/aws-sdk-go/internal/sdkmath
github.com/aws/aws-sdk-go/internal/sdkrand
@ -60,7 +62,6 @@ github.com/aws/aws-sdk-go/private/protocol/rest
github.com/aws/aws-sdk-go/private/protocol/restxml
github.com/aws/aws-sdk-go/private/protocol/xml/xmlutil
github.com/aws/aws-sdk-go/service/s3
github.com/aws/aws-sdk-go/service/s3/internal/arn
github.com/aws/aws-sdk-go/service/s3/s3iface
github.com/aws/aws-sdk-go/service/s3/s3manager
github.com/aws/aws-sdk-go/service/sts
@ -96,7 +97,7 @@ github.com/jmespath/go-jmespath
github.com/jstemmer/go-junit-report
github.com/jstemmer/go-junit-report/formatter
github.com/jstemmer/go-junit-report/parser
# github.com/klauspost/compress v1.11.0
# github.com/klauspost/compress v1.11.1
github.com/klauspost/compress/flate
github.com/klauspost/compress/fse
github.com/klauspost/compress/gzip
@ -180,7 +181,7 @@ golang.org/x/lint/golint
# golang.org/x/mod v0.3.0
golang.org/x/mod/module
golang.org/x/mod/semver
# golang.org/x/net v0.0.0-20200927032502-5d4f70055728
# golang.org/x/net v0.0.0-20201002202402-0a1ea396d57c
golang.org/x/net/context
golang.org/x/net/context/ctxhttp
golang.org/x/net/http/httpguts
@ -195,9 +196,9 @@ golang.org/x/oauth2/google
golang.org/x/oauth2/internal
golang.org/x/oauth2/jws
golang.org/x/oauth2/jwt
# golang.org/x/sync v0.0.0-20200625203802-6e8e738ad208
# golang.org/x/sync v0.0.0-20200930132711-30421366ff76
golang.org/x/sync/errgroup
# golang.org/x/sys v0.0.0-20200929083018-4d22bbb62b3c
# golang.org/x/sys v0.0.0-20201005172224-997123666555
golang.org/x/sys/internal/unsafeheader
golang.org/x/sys/unix
golang.org/x/sys/windows
@ -206,7 +207,7 @@ golang.org/x/text/secure/bidirule
golang.org/x/text/transform
golang.org/x/text/unicode/bidi
golang.org/x/text/unicode/norm
# golang.org/x/tools v0.0.0-20200929223013-bf155c11ec6f
# golang.org/x/tools v0.0.0-20201005185003-576e169c3de7
golang.org/x/tools/cmd/goimports
golang.org/x/tools/go/ast/astutil
golang.org/x/tools/go/gcexportdata
@ -247,7 +248,7 @@ google.golang.org/appengine/internal/modules
google.golang.org/appengine/internal/remote_api
google.golang.org/appengine/internal/urlfetch
google.golang.org/appengine/urlfetch
# google.golang.org/genproto v0.0.0-20200929141702-51c3e5b607fe
# google.golang.org/genproto v0.0.0-20201006033701-bcad7cf615f2
google.golang.org/genproto/googleapis/api/annotations
google.golang.org/genproto/googleapis/iam/v1
google.golang.org/genproto/googleapis/rpc/code