Commit graph

22 commits

Author SHA1 Message Date
Aliaksandr Valialkin
ed5f4a0c5a
lib/fs: add ReaderAt.Path() function
This function is going to be used in VictoriaLogs
2023-07-06 17:25:19 -07:00
Alexander Marshalov
d321ea91f2
fixed typos in documentation and commandline flags descriptions (#4275) 2023-05-10 02:22:06 -07:00
Aliaksandr Valialkin
ee8be138b9
lib/fs: improve error logging at ReaderAt.MustReadAt()
- Add 'BUG:' prefix to error messages related to programming errors aka bugs.
- Consistently log the path to the file in all the messages in order to improve debuggability.
2023-04-14 14:52:14 -07:00
Aliaksandr Valialkin
db3bcbe56a
app/vmselect/netstorage: reduce the contention at fs.ReaderAt stats collection on systems with big number of CPU cores
This optimization is based on the profile provided at https://github.com/VictoriaMetrics/VictoriaMetrics/issues/3966#issuecomment-1483208419
2023-03-25 16:38:39 -07:00
Aliaksandr Valialkin
81400c80f0
lib/fs: remove logging redundant path values in a single error message 2022-12-03 22:00:43 -08:00
Aliaksandr Valialkin
7f1302688f lib/fs: follow-up after f3a03c4164 2021-02-27 01:09:37 +02:00
Nikolay
d88fa5ebe4 Adds windows build (#1040)
* fixes windows compilation,
adds signal impl for windows,
adds free space usage for windows,
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/70
https://github.com/VictoriaMetrics/VictoriaMetrics/issues/1036

NOTE victoria metrics database still CANNOT work under windows system,
only vmagent is supported.
To completly port victoria metrics, you have to fix issues with separators,
parsing and posix file removall

* rollback separator

* Adds windows setInformation api,
it must behave like unix, need to test it.
changes procutil

* check for invlaid param

* Fixes posix delete semantic

* refactored a bit

* fixes openbsd build

* removed windows api call

* Fixes code after windows add

* Update lib/procutil/signal_windows.go

Co-authored-by: Aliaksandr Valialkin <valyala@gmail.com>
2021-02-27 01:06:22 +02:00
Aliaksandr Valialkin
082eabf51e lib/fs: remove the code for tracking whether the given memory region is in page cache
This code didn't give performance gains under production workload, so let's remove it in order to simplify the code.
2021-02-09 16:51:11 +02:00
Aliaksandr Valialkin
588090765c lib/fs: properly initialize cleaner for pageCache bitmaps
Previously it wasnt working because the timer was fired only once
2021-01-27 00:40:39 +02:00
Aliaksandr Valialkin
f4fd917e4f lib/fs: replace fs.OpenReaderAt with fs.MustOpenReaderAt
All the callers for fs.OpenReaderAt expect that the file will be opened.
So it is better to log fatal error inside fs.MustOpenReaderAt instead of leaving this to the caller.
2020-11-23 09:57:30 +02:00
Aliaksandr Valialkin
77bb0e6595 lib/fs: clarify description for -fs.disableMmap command-line flag 2020-07-06 14:28:57 +03:00
Aliaksandr Valialkin
d962568e93 all: use %w instead of %s for wrapping errors in fmt.Errorf
This will simplify examining the returned errors such as httpserver.ErrorWithStatusCode .
See https://blog.golang.org/go1.13-errors for details.
2020-06-30 23:33:46 +03:00
Aliaksandr Valialkin
fd7a3d880e lib/fs: go fmt 2020-06-23 23:03:08 +03:00
Aliaksandr Valialkin
08edb90814 lib/fs: fall back to cgo copy for copying the last 4KB of mmaped data
This probably should fix https://github.com/VictoriaMetrics/VictoriaMetrics/issues/581
2020-06-23 22:55:56 +03:00
Aliaksandr Valialkin
521c657f8d lib/fs: an attempt to fix SIGBUS error by rounding mmap`ed region to multiple of 4KB pages
Updates https://github.com/VictoriaMetrics/VictoriaMetrics/issues/581
2020-06-23 13:40:20 +03:00
Aliaksandr Valialkin
ba1f764b29 lib/fs: optimize queries that read recent samples for big number of time series
Use standard copy() func instead of mmap-aware copy func for reading recently touched mmap-ed data.
This improves read performance by up to 4x.
2020-06-05 19:10:22 +03:00
Aliaksandr Valialkin
c9ab6dc532 lib/fs: do not use mmap for 32-bit arches by default, since they cannot map files bigger than 4GB in RAM 2020-05-12 20:21:39 +03:00
Aliaksandr Valialkin
e933cbac16 lib/storage: postpone reading data from blocks during search
This eliminates the need for storing block data into temporary files on a single-node VictoriaMetrics
during heavy queries, which touch big number of time series over long time ranges.

This improves single-node VM performance on heavy queries by up to 2x.
2020-04-27 08:44:01 +03:00
Aliaksandr Valialkin
7a045125cc lib/fs: typo fix: read blocks bigger than 8KB via pread() call instead of using mmap 2020-02-25 18:04:06 +02:00
Aliaksandr Valialkin
7cde594696 all: do not clash flag description with back-quoted flag types
See https://golang.org/pkg/flag/#PrintDefaults for more details.
2020-02-04 15:56:01 +02:00
Aliaksandr Valialkin
9b25a2fb67 lib/fs: remove unused readerAt interface 2020-01-31 15:13:00 +02:00
Aliaksandr Valialkin
cb5c39ee70 lib/fs: optimize small reads for ReaderAt.MustReadAt by reading from memory-mapped space instead of reading from file descriptor
This should improve performance when reading many small blocks.
2020-01-30 15:16:16 +02:00