pmdk-1.8/0000775000000000000000000000000013615011435011014 5ustar rootrootpmdk-1.8/VERSION0000664000000000000000000000000413615011243012053 0ustar rootroot1.8 pmdk-1.8/doc/0000775000000000000000000000000013615011243011556 5ustar rootrootpmdk-1.8/doc/pmem_ctl/0000775000000000000000000000000013615011416013360 5ustar rootrootpmdk-1.8/doc/pmem_ctl/pmem_ctl.5.md0000664000000000000000000001313413615011243015645 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEM_CTL, 5) collection: pmem_ctl header: PMDK date: pmem_ctl API version 1.4 ... [comment]: <> (Copyright 2018-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmem_ctl.5 -- man page for CTL) [NAME](#name)
[DESCRIPTION](#description)
[CTL EXTERNAL CONFIGURATION](#ctl-external-configuration)
[SEE ALSO](#see-also)
# NAME # ctl - interface for examination and modification of the library's internal state. # DESCRIPTION # The CTL namespace is organized in a tree structure. Starting from the root, each node can be either internal, containing other elements, or a leaf. Internal nodes themselves can only contain other nodes and cannot be entry points. There are two types of those nodes: *named* and *indexed*. Named nodes have string identifiers. Indexed nodes represent an abstract array index and have an associated string identifier. The index itself is provided by the user. A collection of indexes present on the path of an entry point is provided to the handler functions as name and index pairs. Entry points are the leaves of the CTL namespace structure. Each entry point can read from the internal state, write to the internal state, exec a function or a combination of these operations. The entry points are listed in the following format: name | r(ead)w(rite)x(ecute) | global/- | read argument type | write argument type | exec argument type | config argument type A description of **pmem_ctl** functions can be found on the following manual pages: **libpmemblk_ctl_get**(3), **libpmemlog_ctl_get**(3), **libpmemobj_ctl_get**(3) # CTL EXTERNAL CONFIGURATION # In addition to direct function call, each write entry point can also be set using two alternative methods. The first method is to load a configuration directly from the **PMEMBLK_CONF**/ **PMEMLOG_CONF**/ **PMEMOBJ_CONF** environment variable. A properly formatted ctl config string is a single-line sequence of queries separated by ';': ``` query0;query1;...;queryN ``` A single query is constructed from the name of the ctl write entry point and the argument, separated by '=': ``` entry_point=entry_point_argument ``` The entry point argument type is defined by the entry point itself, but there are three predefined primitives: *) integer: represented by a sequence of [0-9] characters that form a single number. *) boolean: represented by a single character: y/n/Y/N/0/1, each corresponds to true or false. If the argument contains any trailing characters, they are ignored. *) string: a simple sequence of characters. There are also complex argument types that are formed from the primitives separated by a ',': ``` first_arg,second_arg ``` In summary, a full configuration sequence looks like this: ``` (first_entry_point)=(arguments, ...);...;(last_entry_point)=(arguments, ...); ``` As an example, to set both prefault at_open and at_create variables: ``` PMEMBLK_CONF="prefault.at_open=1;prefault.at_create=1" ``` The second method of loading an external configuration is to set the **PMEMBLK_CONF_FILE**/ **PMEMLOG_CONF_FILE**/ **PMEMOBJ_CONF_FILE** environment variable to point to a file that contains a sequence of ctl queries. The parsing rules are all the same, but the file can also contain white-spaces and comments. To create a comment, simply use '#' anywhere in a line and everything afterwards, until a new line, will be ignored. An example configuration file: ``` ######################### # My pmemblk configuration ######################### # # Global settings: prefault. # modify the behavior of pre-faulting at_open = 1; # prefault when the pool is opened prefault. at_create = 0; # but don't prefault when it's created # Per-pool settings: # ... ``` # SEE ALSO # **libpmemblk_ctl_get**(3), **libpmemlog_ctl_get**(3), **libpmemobj_ctl_get**(3) and **** pmdk-1.8/doc/pmem_ctl/pmem_ctl.50000644000000000000000000001276413615011416015256 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEM_CTL" "5" "2020-01-31" "PMDK - pmem_ctl API version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2018-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP ctl \- interface for examination and modification of the library's internal state. .SH DESCRIPTION .PP The CTL namespace is organized in a tree structure. Starting from the root, each node can be either internal, containing other elements, or a leaf. Internal nodes themselves can only contain other nodes and cannot be entry points. There are two types of those nodes: \f[I]named\f[] and \f[I]indexed\f[]. Named nodes have string identifiers. Indexed nodes represent an abstract array index and have an associated string identifier. The index itself is provided by the user. A collection of indexes present on the path of an entry point is provided to the handler functions as name and index pairs. .PP Entry points are the leaves of the CTL namespace structure. Each entry point can read from the internal state, write to the internal state, exec a function or a combination of these operations. .PP The entry points are listed in the following format: .PP name | r(ead)w(rite)x(ecute) | global/\- | read argument type | write argument type | exec argument type | config argument type .PP A description of \f[B]pmem_ctl\f[] functions can be found on the following manual pages: \f[B]libpmemblk_ctl_get\f[](3), \f[B]libpmemlog_ctl_get\f[](3), \f[B]libpmemobj_ctl_get\f[](3) .SH CTL EXTERNAL CONFIGURATION .PP In addition to direct function call, each write entry point can also be set using two alternative methods. .PP The first method is to load a configuration directly from the \f[B]PMEMBLK_CONF\f[]/ \f[B]PMEMLOG_CONF\f[]/ \f[B]PMEMOBJ_CONF\f[] environment variable. A properly formatted ctl config string is a single\-line sequence of queries separated by `;': .IP .nf \f[C] query0;query1;...;queryN \f[] .fi .PP A single query is constructed from the name of the ctl write entry point and the argument, separated by `=': .IP .nf \f[C] entry_point=entry_point_argument \f[] .fi .PP The entry point argument type is defined by the entry point itself, but there are three predefined primitives: .IP .nf \f[C] *)\ integer:\ represented\ by\ a\ sequence\ of\ [0\-9]\ characters\ that\ form \ \ \ \ a\ single\ number. *)\ boolean:\ represented\ by\ a\ single\ character:\ y/n/Y/N/0/1,\ each \ \ \ \ corresponds\ to\ true\ or\ false.\ If\ the\ argument\ contains\ any \ \ \ \ trailing\ characters,\ they\ are\ ignored. *)\ string:\ a\ simple\ sequence\ of\ characters. \f[] .fi .PP There are also complex argument types that are formed from the primitives separated by a `,': .IP .nf \f[C] first_arg,second_arg \f[] .fi .PP In summary, a full configuration sequence looks like this: .IP .nf \f[C] (first_entry_point)=(arguments,\ ...);...;(last_entry_point)=(arguments,\ ...); \f[] .fi .PP As an example, to set both prefault at_open and at_create variables: .IP .nf \f[C] PMEMBLK_CONF="prefault.at_open=1;prefault.at_create=1" \f[] .fi .PP The second method of loading an external configuration is to set the \f[B]PMEMBLK_CONF_FILE\f[]/ \f[B]PMEMLOG_CONF_FILE\f[]/ \f[B]PMEMOBJ_CONF_FILE\f[] environment variable to point to a file that contains a sequence of ctl queries. The parsing rules are all the same, but the file can also contain white\-spaces and comments. .PP To create a comment, simply use `#' anywhere in a line and everything afterwards, until a new line, will be ignored. .PP An example configuration file: .IP .nf \f[C] ######################### #\ My\ pmemblk\ configuration ######################### # #\ Global\ settings: prefault.\ #\ modify\ the\ behavior\ of\ pre\-faulting \ \ \ \ at_open\ =\ 1;\ #\ prefault\ when\ the\ pool\ is\ opened prefault. \ \ \ \ at_create\ =\ 0;\ #\ but\ don\[aq]t\ prefault\ when\ it\[aq]s\ created #\ Per\-pool\ settings: #\ ... \f[] .fi .SH SEE ALSO .PP \f[B]libpmemblk_ctl_get\f[](3), \f[B]libpmemlog_ctl_get\f[](3), \f[B]libpmemobj_ctl_get\f[](3) and \f[B]\f[] pmdk-1.8/doc/pmem_ctl/.gitignore0000664000000000000000000000001313615011243015340 0ustar rootrootpmem_ctl.5 pmdk-1.8/doc/libvmmalloc/0000775000000000000000000000000013615011243014057 5ustar rootrootpmdk-1.8/doc/libvmmalloc/README.md0000664000000000000000000000012613615011243015335 0ustar rootrootThis library has been moved to a [separate repository](https://github.com/pmem/vmem). pmdk-1.8/doc/pmempool/0000775000000000000000000000000013615011420013403 5ustar rootrootpmdk-1.8/doc/pmempool/pmempool-sync.10000644000000000000000000001142013615011420016263 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-SYNC" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-sync\f[] \- Synchronize replicas or their parts within a pool set. .SH SYNOPSIS .IP .nf \f[C] pmempool\ sync\ [options]\ \f[] .fi .PP NOTE: Only the pool set file used to create the pool should be used for syncing the pool. .SH DESCRIPTION .PP The \f[B]pmempool sync\f[] command synchronizes data between replicas within a pool set. It checks if metadata of all replicas in a pool set are consistent, i.e.\ all parts are healthy, and if any of them is not, the corrupted or missing parts are recreated and filled with data from one of the healthy replicas. Currently synchronizing data is allowed only for \f[B]pmemobj\f[] pools (see \f[B]libpmemobj\f[](7)). .PP If a pool set has the option \f[I]SINGLEHDR\f[] or \f[I]NOHDRS\f[] (see \f[B]poolset\f[](5)), \f[B]pmempool sync\f[] command has limited capability of checking its metadata. This is due to limited or no, respectively, internal metadata at the beginning of pool set parts in every replica when either of the options is used. In that cases, only missing parts or the ones which cannot be opened are recreated. .SS Available options: .TP .B \f[C]\-b,\ \-\-bad\-blocks\f[] Fix bad blocks \- it causes creating or reading special recovery files. When bad blocks are detected, special recovery files have to be created in order to fix them safely. A separate recovery file is created for each part of the pool. The recovery files are created in the same directory where the poolset file is located using the following name pattern: _r _p _badblocks.txt These recovery files are automatically removed if the sync operation finishes successfully. .RS .PP If the last sync operation was interrupted and not finished correctly (eg. the application crashed) and the bad blocks fixing procedure was in progress, the bad block recovery files may be left over. In such case bad blocks might have been cleared and zeroed, but the correct data from these blocks was not recovered (not copied from a healthy replica), so the recovery files MUST NOT be deleted manually, because it would cause a data loss. Pmempool\-sync should be run again with the `\-b' option set. It will finish the previously interrupted sync operation and copy correct data to zeroed bad blocks using the left\-over bad block recovery files (the bad blocks will be read from the saved recovery files). Pmempool will delete the recovery files automatically at the end of the sync operation. .PP Using this option may have limitations depending on the operating system. For details see description of the CHECK_BAD_BLOCKS feature in \f[B]pmempool\-feature\f[](1). .RE .TP .B \f[C]\-d,\ \-\-dry\-run\f[] Enable dry run mode. In this mode no changes are applied, only check for viability of synchronization. .RS .RE .TP .B \f[C]\-v,\ \-\-verbose\f[] Increase verbosity level. .RS .RE .TP .B \f[C]\-h,\ \-\-help\f[] Display help message and exit. .RS .RE .SH SEE ALSO .PP \f[B]pmempool(1)\f[], \f[B]libpmemblk(7)\f[], \f[B]libpmemlog(7)\f[], \f[B]libpmempool(7)\f[] and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-info.1.md0000664000000000000000000003263213615011243016656 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-INFO, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-info.1 -- man page for pmempool-info) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RANGE](#range)
[STATISTICS](#statistics)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **pmempool-info** - show information about persistent memory pool # SYNOPSIS # ``` $ pmempool info [] ``` # DESCRIPTION # The **pmempool** invoked with *info* command analyzes an existing pool created by **PMDK** libraries provided by **file** parameter. The **file** can be either existing pool file, a part file or a poolset file. The main task of this command is to print all usable information from pool headers and user data in human readable format. It automatically recognizes the pool type by parsing and analyzing the pool header. The recognition is done by checking the signature in the pool header. The main job of *info* command is to present internal data structures as they are stored in file but *not* for checking consistency. For this purpose there is the **pmempool-check**(1) command available. The **pmempool** with *info* command analyzes pool file as long as it is possible regarding *correctness* of internal meta-data (correct offsets, sizes etc.). If it is *not* possible to analyze the rest of the file, **pmempool** exits with error code and prints appropriate error message. Currently there is lack of interprocess synchronization for pool files, so the *info* command should be invoked off-line. Using **pmempool** on pool file which may be modified by another process may lead to unexpected errors in pool file. A poolset file passed to **pmempool info** may contain multiple replicas, also remote ones, but **pmempool** currently does not read any data from remote replicas. It prints only a remote node address and a remote replica descriptor. **pmempool info** opens pool file in *read-only* mode so the file will remain untouched after processing. The *info* command may collect and print basic statistics about data usage. The statistics are specific to the type of pool. See **STATISTICS** section for details. Although the pool consistency is *not* checked by the *info* command, it prints information about checksum errors and/or offsets errors. ##### Common options: ##### By default the *info* command of **pmempool** prints information about the most important internal data structures from pool. The particular set of headers and meta-data depend on pool type. The pool type is recognized automatically and appropriate information is displayed in human-readable format. To force processing specified file(s) as desired pool type use **-f** option with appropriate name of pool type. The valid names off pool types are **blk**, **log**, **obj** or **btt**. This option may be useful when the pool header is corrupted and automatic recognition of pool type fails. `-f, --force blk|log|obj|btt` Force parsing pool as specified pool type. >NOTE: By default only pool headers and internal meta-data are displayed. To display user data use **-d** option. Using **-r** option you can specify number of blocks/bytes/data chunks or objects using special text format. See **RANGE** section for details. The range refers to *block numbers* in case of pmem blk pool type, to *chunk numbers* in case of pmem log pool type and to *object numbers* in case of pmem obj pool type. See **EXAMPLES** section for an example of usage of these options. `-d, --data` Dump user data in hexadecimal format. In case of pmem *blk* pool type data is dumped in *blocks*. In case of pmem *log* pool type data is dumped as a wholeor in *chunks* if **-w** option is used (See **Options for PMEMLOG** section for details). `-r, --range ` Range of blocks/data chunks/objects/zone headers/chunk headers/lanes. See **RANGE** section for details about range format. `-n, --human` Print sizes in human-readable format with appropriate units (e.g. 4k, 8M, 16G) `-x, --headers-hex` Print pool's internal data in mixed format which consists of hexadecimal dump of header's data and parsed format displayed in human-readable format. This allows one to see how data is stored in file. `-s, --stats` Print pool's statistics. See **STATISTICS** section for details. `-k, --bad-blocks=` Print bad blocks found in the pool. `-h, --help` Display help message and exit. ##### Options for PMEMLOG: ##### `-w, --walk ` Use this option to walk through used data with fixed data chunk size. See **pmemlog_walk**(3) in **libpmemlog**(7) for details. ##### Options for PMEMBLK: ##### By default the *info* command displays the **pmemblk** header and BTT (Block Translation Table) Info header in case of **pmemblk** pool type. To display BTT Map and/or BTT FLOG (Free List and Log) use **-m** and **-g** options respectively or increase verbosity level. In order to display BTT Info header backup use **-B** option. `-m, --map` Print BTT Map entries. `-g, --flog` Print BTT FLOG entries. `-B, --backup` Print BTT Info header backup. >NOTE: By default the *info* command displays all data blocks when **-d** options is used. However it is possible to skip blocks marked with *zero* and/or *error* flags. It is also possible to skip blocks which are *not* marked with any flag. Skipping blocks has impact on blocks ranges (e.g. display 10 blocks marked with error flag in the range from 0 to 10000) and statistics. `-z, --skip-zeros` Skip blocks marked with *zero* flag. `-e, --skip-error` Skip blocks marked with *error* flag. `-u, --skip-no-flag` Skip blocks *not* marked with any flag. ##### Options for PMEMOBJ: ##### By default the *info* command displays pool header and **pmemobj** pool descriptor. In order to print information about other data structures one of the following options may be used. `-l, --lanes []` Print information about lanes. If range is not specified all lanes are displayed. The range can be specified using **-r** option right after the **-l** option. See **RANGE** section for details about range format. `-R, --recovery` Print information about only those lanes which require recovery process. This option requires **-l**, **--lanes** option. `-O, --object-store` Print information about all allocated objects. `-t, --types ` Print information about allocated objects only from specified range of type numbers. If **-s**, **--stats** option is specified the objects statistics refer to objects from specified range of type numbers. This option requires **-O**, **--object-store** or **-s**, **--stats** options. See **RANGE** section for details about range format. `-E, --no-empty` Ignore empty lists of objects. This option requires **-O**, **--object-store** option. `-o, --root` Print information about a root object. `-A, --alloc-header` Print object's allocation header. This option requires **-O**, **--object-store** or **-l**, **--lanes** or **-o**, **--root** options. `-a, --oob-header` Print object's out of band header. This option requires **-O**, **--object-store** or **-l**, **--lanes** or **-o**, **--root** options. `-H, --heap` Print information about **pmemobj** heap. By default only a heap header is displayed. `-Z, --zones` If the **-H**, **--heap** option is used, print information about zones from specified range. If the **-O**, **--object-store** option is used, print information about objects only from specified range of zones. This option requires **-O**, **--object-store**, **-H**, **--heap** or **-s**, **--stats** options. The range can be specified using **-r** option right after the **-Z** option. See **RANGE** section for details about range format. `-C, --chunks []` If the **-H, --heap** option is used, print information about chunks from specified range. By default information about chunks of types *used* , *free* and *run* are displayed. If the **-O, --object-store** option is used, print information about objects from specified range of chunks within a zone. This option requires **-O, --object-store**, **-H, --heap** or **-s, --stats** options. The range can be specified using **-r** option right after the **-C** option. See **RANGE** section for details about range format. `-T, --chunk-type used,free,run,footer` Print only specified type(s) of chunks. The multiple types may be specified separated by comma. This option requires **-H, --heap** and **-C, --chunks** options. `-b, --bitmap` Print bitmap of used blocks in chunks of type run. This option requires **-H, --heap** and **-C, --chunks** options. `-p, --replica ` Print information from *\* replica. The 0 value means the master pool file. # RANGE # Using **-r, --range** option it is possible to dump only a range of user data. This section describes valid format of *\* string. You can specify multiple ranges separated by commas. `-` All blocks/bytes/data chunks from *\* to *\* will be dumped. `-` All blocks/bytes/data chunks up to *\* will be dumped. `-` All blocks/bytes/data chunks starting from *\* will be dumped. `` Only *\* block/byte/data chunk will be dumped. # STATISTICS # Below is the description of statistical measures for specific pool types. ##### PMEMLOG ##### + **Total** - Total space in pool. + **Available** - Size and percentage of available space. + **Used** - Size and percentage of used space. ##### PMEMBLK ##### + **Total blocks** - Total number of blocks in pool. + **Zeroed blocks** - Number and percentage of blocks marked with *zero* flag. + **Error blocks** - Number and percentage of blocks marked with *error* flag. + **Blocks without any flag** - Number and percentage of blocks *not* marked with any flag. >NOTE: In case of pmemblk, statistics are evaluated for blocks which meet requirements regarding: *range* of blocks (**-r** option), *skipped* types of blocks (**-z**, **-e**, **-u** options). ##### PMEMOBJ ##### + **Object store** + **Number of objects** - Total number of objects and number of objects per type number. + **Number of bytes** - Total number of bytes and number of bytes per type number. + **Heap** + **Number of zones** - Total number of zones in the pool. + **Number of used zones** - Number of used zones in the pool. + **Zone** The zone's statistics are presented for each zone separately and the aggregated results from all zones. + **Number of chunks** - Total number of chunks in the zone and number of chunks of specified type. + **Chunks size** - Total size of all chunks in the zone and sum of sizes of chunks of specified type. + **Allocation classes** + **Units** - Total number of units of specified class. + **Used units** - Number of used units of specified class. + **Bytes** - Total number of bytes of specified class. + **Used bytes** - Number of used bytes of specified class. + **Total bytes** - Total number of bytes of all classes. + **Total used bytes** - Total number of used bytes of all classes. # EXAMPLE # ``` $ pmempool info ./pmemblk ``` Parse and print information about "pmemblk" pool file. ``` $ pmempool info -f blk ./pmempool ``` Force parsing "pmempool" file as **pmemblk** pool type. ``` $ pmempool info -d ./pmemlog ``` Print information and data in hexadecimal dump format for file "pmemlog". ``` $ pmempool info -d -r10-100 -eu ./pmemblk ``` Print information from "pmemblk" file. Dump data blocks from 10 to 100, skip blocks marked with error flag and not marked with any flag. # SEE ALSO # **pmempool**(1), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/pmempool/pmempool-transform.10000644000000000000000000001352313615011420017330 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-TRANSFORM" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-transform\f[] \- Modify internal structure of a pool set. .SH SYNOPSIS .IP .nf \f[C] pmempool\ transform\ [options]\ \ \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool transform\f[] command modifies internal structure of a pool set defined by the \f[C]poolset_file_src\f[] file, according to a structure described in the \f[C]poolset_file_dst\f[] file. .PP The following operations are supported: .IP \[bu] 2 adding replicas \- one or more new replicas can be added and synchronized with other replicas in the pool set, .IP \[bu] 2 removing replicas \- one or more replicas can be removed from the pool set , .IP \[bu] 2 adding or removing pool set options. .PP Only one of the above operations can be performed at a time. .PP Currently adding and removing replicas are allowed only for \f[B]pmemobj\f[] pools (see \f[B]libpmemobj\f[](7)). .PP The \f[I]poolset_file_src\f[] argument provides the source pool set to be changed. .PP The \f[I]poolset_file_dst\f[] argument points to the target pool set. .PP When adding or deleting replicas, the two pool set files can differ only in the definitions of replicas which are to be added or deleted. When adding or removing pool set options (see \f[B]poolset\f[](5)), the rest of both pool set files have to be of the same structure. The operation of adding/removing a pool set option can be performed on a pool set with local replicas only. To add/remove a pool set option to/from a pool set with remote replicas, one has to remove the remote replicas first, then add/remove the option, and finally recreate the remote replicas having added/removed the pool set option to/from the remote replicas' poolset files. To add a replica it is necessary for its effective size to match or exceed the pool size. Otherwise the whole operation fails and no changes are applied. If none of the poolset options is used, the effective size of a replica is the sum of sizes of all its part files decreased by 4096 bytes per each part file. The 4096 bytes of each part file is utilized for storing internal metadata of the pool part files. If the option \f[I]SINGLEHDR\f[] is used, the effective size of a replica is the sum of sizes of all its part files decreased once by 4096 bytes. In this case only the first part contains internal metadata. If the option \f[I]NOHDRS\f[] is used, the effective size of a replica is the sum of sizes of all its part files. In this case none of the parts contains internal metadata. .SS Available options: .TP .B \f[C]\-d,\ \-\-dry\-run\f[] Enable dry run mode. In this mode no changes are applied, only check for viability of the operation is performed. .RS .RE .TP .B \f[C]\-v,\ \-\-verbose\f[] Increase verbosity level. .RS .RE .TP .B \f[C]\-h,\ \-\-help\f[] Display help message and exit. .RS .RE .SH EXAMPLES .SS Example 1. .PP Let files \f[C]/path/poolset_file_src\f[] and \f[C]/path/poolset_file_dst\f[] have the following contents: .IP .nf \f[C] PMEMPOOLSET 20M\ /0/partfile1 20M\ /0/partfile2 25M\ /0/partfile3 REPLICA 40M\ /1/partfile1 20M\ /1/partfile2 \f[] .fi .IP .nf \f[C] PMEMPOOLSET 20M\ /0/partfile1 20M\ /0/partfile2 25M\ /0/partfile3 REPLICA 40M\ /1/partfile1 20M\ /1/partfile2 REPLICA 50M\ /2/partfile1 20M\ /2/partfile2 \f[] .fi .PP Then, the command .PP \f[C]pmempool\ transform\ /path/poolset_file_src\ /path/poolset_file_dst\f[] .PP adds a replica to the pool set. All other replicas remain unchanged and the size of the pool remains 60M. .SS Example 2. .PP Let files \f[C]/path/poolset_file_src\f[] and \f[C]/path/poolset_file_dst\f[] have the following contents: .IP .nf \f[C] PMEMPOOLSET 20M\ /0/partfile1 20M\ /0/partfile2 25M\ /0/partfile3 REPLICA 40M\ /1/partfile1 20M\ /1/partfile2 \f[] .fi .IP .nf \f[C] PMEMPOOLSET 20M\ /0/partfile1 20M\ /0/partfile2 25M\ /0/partfile3 \f[] .fi .PP Then .PP \f[C]pmempool_transform\ /path/poolset_file_src\ /path/poolset_file_dst\f[] .PP deletes the second replica from the pool set. The first replica remains unchanged and the size of the pool is still 60M. .SH SEE ALSO .PP \f[B]pmempool(1)\f[], \f[B]libpmemblk(7)\f[], \f[B]libpmemlog(7)\f[], \f[B]libpmempool(7)\f[] and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-rm.1.md0000664000000000000000000001051013615011243016330 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-RM, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-rm.1 -- man page for pmempool-rm) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **pmempool-rm** - remove a persistent memory pool # SYNOPSIS # ``` $ pmempool rm [] .. ``` # DESCRIPTION # The **pmempool rm** command removes each specified file. If the specified file is a pool set file, all pool files (single-file pool or part files) and remote replicas are removed. By default the **pmempool rm** does not remove pool set files. All local and remote pool files are removed using **unlink**(3) call, except the pools created on **device dax** which are zeroed instead. If specified file does not exist, the remote pool is broken or not accessible, the **pmempool rm** command terminates with an error code. By default it prompts before removing *write-protected* local files. See **REMOTE REPLICATION** section for more details about support for remote pools. See **EXAMPLES** section for example usage of the *rm* command. ##### Available options: ##### `-h, --help` Print help message `-v, --verbose` Be verbose and print all removing files. `-s, --only-pools` Remove only pool files and do not remove pool set files (default behaviour). `-a, --all` Remove all pool set files - local and remote. `-l, --local` Remove local pool set files. `-r, --remote` Remove remote pool set files. `-f, --force` Remove all specified files, ignore nonexistent files, never prompt. `-i, --interactive` Prompt before removing every single file or remote pool. # REMOTE REPLICATION # A remote pool is removed using **rpmem_remove**(3) function if **librpmem**(7) library is available. If a pool set file contains remote replication but **librpmem**(7) is not available, the **pmempool rm** command terminates with an error code, unless the **-f, --force** option is specified. # EXAMPLE # ``` $ pmempool rm pool.obj pool.blk ``` Remove specified pool files. ``` $ pmempool rm pool.set ``` Remove all pool files from the "pool.set", do not remove *pool.set* itself. ``` $ pmempool rm -a pool.set ``` Remove all pool files from the "pool.set", remove the local pool set file and all remote pool set files. # SEE ALSO # **pmempool**(1), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7), **librpmem**(7) and **** pmdk-1.8/doc/pmempool/pmempool-create.1.md0000664000000000000000000001307113615011243017162 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-CREATE, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-create.1 -- man page for pmempool-create) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **pmempool-create** - create a persistent memory pool # SYNOPSIS # ``` $ pmempool create [] [] [] ``` # DESCRIPTION # The **pmempool** invoked with *create* command creates a pool file of specified type. Depending on a pool type it is possible to provide more properties of pool. Valid pool types are: **blk**, **log** and **obj** which stands for *pmemblk*, *pmemlog* and *pmemobj* pools respectively. By default the pool file is created with *minimum* allowed size for specified pool type. The minimum sizes for **blk**, **log** and **obj** pool types are **PMEMBLK_MIN_POOL**, **PMEMLOG_MIN_POOL** and **PMEMOBJ_MIN_POOL** respectively. See **libpmemblk**(7), **libpmemlog**(7) and **libpmemobj**(7) for details. For *pmemblk* pool type block size *\* is a required argument. In order to set custom size of pool use **-s** option, or use **-M** option to create a pool of maximum available size on underlying file system. The *size* argument may be passed in format that permits only the upper-case character for byte - B as specified in IEC 80000-13, IEEE 1541 and the Metric Interchange Format. Standards accept SI units with obligatory B - kB, MB, GB, ... which means multiplier by 1000 and IEC units with optional "iB" - KiB, MiB, GiB, ..., K, M, G, ... - which means multiplier by 1024. ##### Available options: ##### `-s, --size ` Size of pool file. `-M, --max-size` Set size of pool to available space of underlying file system. `-m, --mode ` Set permissions to (the default is 0664) when creating the files. If the file already exist the permissions are not changed. `-i, --inherit ` Create a new pool of the same size and other properties as *\*. `-b, --clear-bad-blocks` Clear bad blocks in existing files. `-f, --force` Remove the pool before creating. `-v, --verbose` Increase verbosity level. `-h, --help` Display help message and exit. ##### Options for PMEMBLK: ##### By default when creating a pmem **blk** pool, the **BTT** layout is *not* written until the first *write operation* of block entry is performed. Using **-w** option you can force writing the **BTT** layout by writing zero data to specified block number. By default the *write operation* is performed to block number 0. Please refer to **libpmemblk**(7) for details. `-w, --write-layout` Force writing the **BTT** layout by performing *write operation* to block number zero. ##### Options for PMEMOBJ: ##### By default when creating a pmem **obj** pool, the layout name provided to the **libpmemobj** library is an empty string. Please refer to **libpmemobj**(7) for details. `-l, --layout ` Layout name of the **pmemobj** pool. # EXAMPLE # ``` $ pmempool create blk 512 pool.blk ``` Create a blk pool file of minimum allowed size and block size 512 bytes ``` $ pmempool create log -M pool.log ``` Create a log pool file of maximum allowed size ``` $ pmempool create blk --size=4G --write-layout 1K pool.blk ``` Create a blk pool file of size 4G, block size 1K and write the BTT layout ``` $ pmempool create --layout my_layout obj pool.obj ``` Create an obj pool file of minimum allowed size and layout "my_layout" ``` $ pmempool create --inherit=pool.log new_pool.log ``` Create a pool file based on pool.log file # SEE ALSO # **pmempool**(1), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/pmempool/pmempool-dump.1.md0000664000000000000000000001041413615011243016662 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-DUMP, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-dump.1 -- man page for pmempool-dump) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RANGE](#range)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **pmempool-dump** - dump user data from persistent memory pool # SYNOPSIS # ``` $ pmempool dump [] ``` # DESCRIPTION # The **pmempool** invoked with *dump* command dumps user data from specified pool file. The output format may be either binary or hexadecimal. By default the output format is hexadecimal. By default data is dumped to standard output. It is possible to dump data to other file by specifying **-o** option. In this case data will be appended to this file. Using **-r** option you can specify number of blocks/bytes/data chunks using special text format. See **RANGE** section for details. ##### Available options: ##### `-b, --binary` Dump data in binary format. `-r, --range ` Range of pool file to dump. This may be number of blocks for **blk** pool type or either number of bytes or number of data chunks for **log** pool type. `-c, --chunk ` Size of chunk for **log** pool type. See **pmemlog_walk**(3) in **libpmemlog**(7) for details. `-o, --output ` Name of output file. `-h, --help` Display help message and exit. # RANGE # Using **-r**, **--range** option it is possible to dump only a range of user data. This section describes valid format of *\* string. You can specify multiple ranges separated by commas. `-` All blocks/bytes/data chunks from *\* to *\* will be dumped. `-` All blocks/bytes/data chunks up to *\* will be dumped. `-` All blocks/bytes/data chunks starting from *\* will be dumped. `` Only *\* block/byte/data chunk will be dumped. # EXAMPLE # ``` $ pmempool dump pool.bin ``` Dump user data from pool.bin file to standard output ``` $ pmempool dump -o output.bin -r1,10-100 pool_blk.bin ``` Dump block number 1 and blocks from 10 to 100 from pool_blk.bin containing pmem blk pool to output.bin file ``` $ pmempool dump -r 1K-2K pool.bin ``` Dump data form 1K to 2K from pool.bin file. # SEE ALSO # **pmempool**(1), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/pmempool/pmempool-convert.10000644000000000000000000000411413615011420016771 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-CONVERT" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-convert\f[] \- this is a wrapper around pmdk\-convert tool. More information can be found in \f[B]pmdk\-convert\f[](1) man page. .SH SEE ALSO .PP \f[B]pmdk\-convert\f[](1), \f[B]pmempool\f[](1), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7), \f[B]libpmempool\f[](7) and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-check.1.md0000664000000000000000000001045613615011243017000 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-CHECK, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-check.1 -- man page for pmempool-check) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **pmempool-check** - check and repair persistent memory pool # SYNOPSIS # ``` $ pmempool check [] ``` # DESCRIPTION # The **pmempool** invoked with *check* command checks consistency of a given pool file. If the pool file is consistent **pmempool** exits with 0 value. If the pool file is not consistent non-zero error code is returned. In case of any errors, the proper message is printed. The verbosity level may be increased using **-v** option. The output messages may be also suppressed using **-q** option. It is possible to try to fix encountered problems using **-r** option. In order to be sure this will not corrupt your data you can either create backup of the pool file using **-b** option or just print what would be fixed without modifying original pool using **-N** option. > NOTE: Currently, checking the consistency of a *pmemobj* pool is **not** supported. ##### Available options: ##### `-r, --repair` Try to repair a pool file if possible. `-y, --yes` Answer yes on all questions. `-d, --dry-run` Don't execute, just show what would be done. Not supported on Device DAX. `-N, --no-exec` Deprecated alias for `dry-run`. `-b, --backup ` Create backup of a pool file before executing. Terminate if it is *not* possible to create a backup file. This option requires **-r** option. `-a, --advanced` Perform advanced repairs. This option enables more aggressive steps in attempts to repair a pool. This option requires `-r, --repair`. `-q, --quiet` Be quiet and don't print any messages. `-v, --verbose` Be more verbose. `-h, --help` Display help message and exit. # EXAMPLE # ``` $ pmempool check pool.bin ``` Check consistency of "pool.bin" pool file ``` $ pmempool check --repair --backup pool.bin.backup pool.bin ``` Check consistency of "pool.bin" pool file, create backup and repair if necessary. ``` $ pmempool check -rvN pool.bin ``` Check consistency of "pool.bin" pool file, print what would be repaired with increased verbosity level. # SEE ALSO # **pmempool**(1), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7), **libpmempool**(7) and **** pmdk-1.8/doc/pmempool/pmempool-check.10000644000000000000000000001036213615011420016370 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-CHECK" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-check\f[] \- check and repair persistent memory pool .SH SYNOPSIS .IP .nf \f[C] $\ pmempool\ check\ []\ \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool\f[] invoked with \f[I]check\f[] command checks consistency of a given pool file. If the pool file is consistent \f[B]pmempool\f[] exits with 0 value. If the pool file is not consistent non\-zero error code is returned. .PP In case of any errors, the proper message is printed. The verbosity level may be increased using \f[B]\-v\f[] option. The output messages may be also suppressed using \f[B]\-q\f[] option. .PP It is possible to try to fix encountered problems using \f[B]\-r\f[] option. In order to be sure this will not corrupt your data you can either create backup of the pool file using \f[B]\-b\f[] option or just print what would be fixed without modifying original pool using \f[B]\-N\f[] option. .RS .PP NOTE: Currently, checking the consistency of a \f[I]pmemobj\f[] pool is \f[B]not\f[] supported. .RE .SS Available options: .PP \f[C]\-r,\ \-\-repair\f[] .PP Try to repair a pool file if possible. .PP \f[C]\-y,\ \-\-yes\f[] .PP Answer yes on all questions. .PP \f[C]\-d,\ \-\-dry\-run\f[] .PP Don't execute, just show what would be done. Not supported on Device DAX. .PP \f[C]\-N,\ \-\-no\-exec\f[] .PP Deprecated alias for \f[C]dry\-run\f[]. .PP \f[C]\-b,\ \-\-backup\ \f[] .PP Create backup of a pool file before executing. Terminate if it is \f[I]not\f[] possible to create a backup file. This option requires \f[B]\-r\f[] option. .PP \f[C]\-a,\ \-\-advanced\f[] .PP Perform advanced repairs. This option enables more aggressive steps in attempts to repair a pool. This option requires \f[C]\-r,\ \-\-repair\f[]. .PP \f[C]\-q,\ \-\-quiet\f[] .PP Be quiet and don't print any messages. .PP \f[C]\-v,\ \-\-verbose\f[] .PP Be more verbose. .PP \f[C]\-h,\ \-\-help\f[] .PP Display help message and exit. .SH EXAMPLE .IP .nf \f[C] $\ pmempool\ check\ pool.bin \f[] .fi .PP Check consistency of \[lq]pool.bin\[rq] pool file .IP .nf \f[C] $\ pmempool\ check\ \-\-repair\ \-\-backup\ pool.bin.backup\ pool.bin \f[] .fi .PP Check consistency of \[lq]pool.bin\[rq] pool file, create backup and repair if necessary. .IP .nf \f[C] $\ pmempool\ check\ \-rvN\ pool.bin \f[] .fi .PP Check consistency of \[lq]pool.bin\[rq] pool file, print what would be repaired with increased verbosity level. .SH SEE ALSO .PP \f[B]pmempool\f[](1), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7), \f[B]libpmempool\f[](7) and \f[B]\f[] pmdk-1.8/doc/pmempool/.gitignore0000664000000000000000000000024713615011243015401 0ustar rootrootpmempool-check.1 pmempool-convert.1 pmempool-create.1 pmempool-dump.1 pmempool-feature.1 pmempool-info.1 pmempool-rm.1 pmempool-sync.1 pmempool-transform.1 pmempool.1 pmdk-1.8/doc/pmempool/pmempool-sync.1.md0000664000000000000000000001206013615011243016670 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-SYNC, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-sync.1 -- man page for pmempool-sync) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[EXAMPLES](#examples)
[SEE ALSO](#see-also)
# NAME # **pmempool-sync** - Synchronize replicas or their parts within a pool set. # SYNOPSIS # ``` pmempool sync [options] ``` NOTE: Only the pool set file used to create the pool should be used for syncing the pool. # DESCRIPTION # The **pmempool sync** command synchronizes data between replicas within a pool set. It checks if metadata of all replicas in a pool set are consistent, i.e. all parts are healthy, and if any of them is not, the corrupted or missing parts are recreated and filled with data from one of the healthy replicas. Currently synchronizing data is allowed only for **pmemobj** pools (see **libpmemobj**(7)). _WINUX(,=q=If a pool set has the option *SINGLEHDR* or *NOHDRS* (see **poolset**(5)), **pmempool sync** command has limited capability of checking its metadata. This is due to limited or no, respectively, internal metadata at the beginning of pool set parts in every replica when either of the options is used. In that cases, only missing parts or the ones which cannot be opened are recreated.=e=) ##### Available options: ##### `-b, --bad-blocks` : Fix bad blocks - it causes creating or reading special recovery files. When bad blocks are detected, special recovery files have to be created in order to fix them safely. A separate recovery file is created for each part of the pool. The recovery files are created in the same directory where the poolset file is located using the following name pattern: \ _r \ _p \ _badblocks.txt These recovery files are automatically removed if the sync operation finishes successfully. If the last sync operation was interrupted and not finished correctly (eg. the application crashed) and the bad blocks fixing procedure was in progress, the bad block recovery files may be left over. In such case bad blocks might have been cleared and zeroed, but the correct data from these blocks was not recovered (not copied from a healthy replica), so the recovery files MUST NOT be deleted manually, because it would cause a data loss. Pmempool-sync should be run again with the '-b' option set. It will finish the previously interrupted sync operation and copy correct data to zeroed bad blocks using the left-over bad block recovery files (the bad blocks will be read from the saved recovery files). Pmempool will delete the recovery files automatically at the end of the sync operation. Using this option may have limitations depending on the operating system. For details see description of the CHECK_BAD_BLOCKS feature in **pmempool-feature**(1). `-d, --dry-run` : Enable dry run mode. In this mode no changes are applied, only check for viability of synchronization. `-v, --verbose` : Increase verbosity level. `-h, --help` : Display help message and exit. # SEE ALSO # **pmempool(1)**, **libpmemblk(7)**, **libpmemlog(7)**, **libpmempool(7)** and **** pmdk-1.8/doc/pmempool/pmempool.1.md0000664000000000000000000001032713615011243015722 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool.1 -- man page for pmempool) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[OPTIONS](#options)
[COMMANDS](#commands)
[SEE ALSO](#see-also)
# NAME # **pmempool** - Persistent Memory Pool Management Tool # SYNOPSIS # ``` $ pmempool [--help] [--version] [] ``` # DESCRIPTION # The **pmempool** is a management tool for *Persistent Memory* pool files created by **PMDK** libraries. The main purpose of **pmempool** is to provide a user with a set of utilities for off-line analysis and manipulation of pools created by pmem libraries. The pmempool is a generic command which consists of subcommands for specific purposes. Some of subcommands are required to work *without* any impact on processed pool, but some of them *may* create a new or modify an existing one. The **pmempool** may be useful for troubleshooting by system administrators and for software developers who work on applications based on **PMDK**. The latter may find these tools useful for testing and debugging purposes also. # OPTIONS # `-V, --version` Prints the version of **pmempool**. `-h, --help` Prints synopsis and list of commands. # COMMANDS # Currently there is a following set of commands available: + **pmempool-info**(1) - Prints information and statistics in human-readable format about specified pool. + **pmempool-check**(1) - Checks pool's consistency and repairs pool if it is not consistent. + **pmempool-create**(1) - Creates a pool of specified type with additional properties specific for this type of pool. + **pmempool-dump**(1) - Dumps usable data from pool in hexadecimal or binary format. + **pmempool-rm**(1) - Removes pool file or all pool files listed in pool set configuration file. + **pmempool-convert**(1) - Updates the pool to the latest available layout version. + **pmempool-sync**(1) - Synchronizes replicas within a poolset. + **pmempool-transform**(1) - Modifies internal structure of a poolset. + **pmempool-feature**(1) - Toggle or query a poolset features. In order to get more information about specific *command* you can use **pmempool help .** # SEE ALSO # **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/pmempool/pmempool-info.10000644000000000000000000003557113615011420016257 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-INFO" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-info\f[] \- show information about persistent memory pool .SH SYNOPSIS .IP .nf \f[C] $\ pmempool\ info\ []\ \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool\f[] invoked with \f[I]info\f[] command analyzes an existing pool created by \f[B]PMDK\f[] libraries provided by \f[B]file\f[] parameter. The \f[B]file\f[] can be either existing pool file, a part file or a poolset file. .PP The main task of this command is to print all usable information from pool headers and user data in human readable format. It automatically recognizes the pool type by parsing and analyzing the pool header. The recognition is done by checking the signature in the pool header. The main job of \f[I]info\f[] command is to present internal data structures as they are stored in file but \f[I]not\f[] for checking consistency. For this purpose there is the \f[B]pmempool\-check\f[](1) command available. .PP The \f[B]pmempool\f[] with \f[I]info\f[] command analyzes pool file as long as it is possible regarding \f[I]correctness\f[] of internal meta\-data (correct offsets, sizes etc.). If it is \f[I]not\f[] possible to analyze the rest of the file, \f[B]pmempool\f[] exits with error code and prints appropriate error message. .PP Currently there is lack of interprocess synchronization for pool files, so the \f[I]info\f[] command should be invoked off\-line. Using \f[B]pmempool\f[] on pool file which may be modified by another process may lead to unexpected errors in pool file. .PP A poolset file passed to \f[B]pmempool info\f[] may contain multiple replicas, also remote ones, but \f[B]pmempool\f[] currently does not read any data from remote replicas. It prints only a remote node address and a remote replica descriptor. .PP \f[B]pmempool info\f[] opens pool file in \f[I]read\-only\f[] mode so the file will remain untouched after processing. .PP The \f[I]info\f[] command may collect and print basic statistics about data usage. The statistics are specific to the type of pool. See \f[B]STATISTICS\f[] section for details. .PP Although the pool consistency is \f[I]not\f[] checked by the \f[I]info\f[] command, it prints information about checksum errors and/or offsets errors. .SS Common options: .PP By default the \f[I]info\f[] command of \f[B]pmempool\f[] prints information about the most important internal data structures from pool. The particular set of headers and meta\-data depend on pool type. The pool type is recognized automatically and appropriate information is displayed in human\-readable format. .PP To force processing specified file(s) as desired pool type use \f[B]\-f\f[] option with appropriate name of pool type. The valid names off pool types are \f[B]blk\f[], \f[B]log\f[], \f[B]obj\f[] or \f[B]btt\f[]. This option may be useful when the pool header is corrupted and automatic recognition of pool type fails. .PP \f[C]\-f,\ \-\-force\ blk|log|obj|btt\f[] .PP Force parsing pool as specified pool type. .RS .PP NOTE: By default only pool headers and internal meta\-data are displayed. To display user data use \f[B]\-d\f[] option. Using \f[B]\-r\f[] option you can specify number of blocks/bytes/data chunks or objects using special text format. See \f[B]RANGE\f[] section for details. The range refers to \f[I]block numbers\f[] in case of pmem blk pool type, to \f[I]chunk numbers\f[] in case of pmem log pool type and to \f[I]object numbers\f[] in case of pmem obj pool type. See \f[B]EXAMPLES\f[] section for an example of usage of these options. .RE .PP \f[C]\-d,\ \-\-data\f[] .PP Dump user data in hexadecimal format. In case of pmem \f[I]blk\f[] pool type data is dumped in \f[I]blocks\f[]. In case of pmem \f[I]log\f[] pool type data is dumped as a wholeor in \f[I]chunks\f[] if \f[B]\-w\f[] option is used (See \f[B]Options for PMEMLOG\f[] section for details). .PP \f[C]\-r,\ \-\-range\ \f[] .PP Range of blocks/data chunks/objects/zone headers/chunk headers/lanes. See \f[B]RANGE\f[] section for details about range format. .PP \f[C]\-n,\ \-\-human\f[] .PP Print sizes in human\-readable format with appropriate units (e.g.\ 4k, 8M, 16G) .PP \f[C]\-x,\ \-\-headers\-hex\f[] .PP Print pool's internal data in mixed format which consists of hexadecimal dump of header's data and parsed format displayed in human\-readable format. This allows one to see how data is stored in file. .PP \f[C]\-s,\ \-\-stats\f[] .PP Print pool's statistics. See \f[B]STATISTICS\f[] section for details. .PP \f[C]\-k,\ \-\-bad\-blocks=\f[] .PP Print bad blocks found in the pool. .PP \f[C]\-h,\ \-\-help\f[] .PP Display help message and exit. .SS Options for PMEMLOG: .PP \f[C]\-w,\ \-\-walk\ \f[] .PP Use this option to walk through used data with fixed data chunk size. See \f[B]pmemlog_walk\f[](3) in \f[B]libpmemlog\f[](7) for details. .SS Options for PMEMBLK: .PP By default the \f[I]info\f[] command displays the \f[B]pmemblk\f[] header and BTT (Block Translation Table) Info header in case of \f[B]pmemblk\f[] pool type. .PP To display BTT Map and/or BTT FLOG (Free List and Log) use \f[B]\-m\f[] and \f[B]\-g\f[] options respectively or increase verbosity level. .PP In order to display BTT Info header backup use \f[B]\-B\f[] option. .PP \f[C]\-m,\ \-\-map\f[] .PP Print BTT Map entries. .PP \f[C]\-g,\ \-\-flog\f[] .PP Print BTT FLOG entries. .PP \f[C]\-B,\ \-\-backup\f[] .PP Print BTT Info header backup. .RS .PP NOTE: By default the \f[I]info\f[] command displays all data blocks when \f[B]\-d\f[] options is used. However it is possible to skip blocks marked with \f[I]zero\f[] and/or \f[I]error\f[] flags. It is also possible to skip blocks which are \f[I]not\f[] marked with any flag. Skipping blocks has impact on blocks ranges (e.g.\ display 10 blocks marked with error flag in the range from 0 to 10000) and statistics. .RE .PP \f[C]\-z,\ \-\-skip\-zeros\f[] .PP Skip blocks marked with \f[I]zero\f[] flag. .PP \f[C]\-e,\ \-\-skip\-error\f[] .PP Skip blocks marked with \f[I]error\f[] flag. .PP \f[C]\-u,\ \-\-skip\-no\-flag\f[] .PP Skip blocks \f[I]not\f[] marked with any flag. .SS Options for PMEMOBJ: .PP By default the \f[I]info\f[] command displays pool header and \f[B]pmemobj\f[] pool descriptor. In order to print information about other data structures one of the following options may be used. .PP \f[C]\-l,\ \-\-lanes\ []\f[] .PP Print information about lanes. If range is not specified all lanes are displayed. The range can be specified using \f[B]\-r\f[] option right after the \f[B]\-l\f[] option. See \f[B]RANGE\f[] section for details about range format. .PP \f[C]\-R,\ \-\-recovery\f[] .PP Print information about only those lanes which require recovery process. This option requires \f[B]\-l\f[], \f[B]\[en]lanes\f[] option. .PP \f[C]\-O,\ \-\-object\-store\f[] .PP Print information about all allocated objects. .PP \f[C]\-t,\ \-\-types\ \f[] .PP Print information about allocated objects only from specified range of type numbers. If \f[B]\-s\f[], \f[B]\[en]stats\f[] option is specified the objects statistics refer to objects from specified range of type numbers. This option requires \f[B]\-O\f[], \f[B]\[en]object\-store\f[] or \f[B]\-s\f[], \f[B]\[en]stats\f[] options. See \f[B]RANGE\f[] section for details about range format. .PP \f[C]\-E,\ \-\-no\-empty\f[] .PP Ignore empty lists of objects. This option requires \f[B]\-O\f[], \f[B]\[en]object\-store\f[] option. .PP \f[C]\-o,\ \-\-root\f[] .PP Print information about a root object. .PP \f[C]\-A,\ \-\-alloc\-header\f[] .PP Print object's allocation header. This option requires \f[B]\-O\f[], \f[B]\[en]object\-store\f[] or \f[B]\-l\f[], \f[B]\[en]lanes\f[] or \f[B]\-o\f[], \f[B]\[en]root\f[] options. .PP \f[C]\-a,\ \-\-oob\-header\f[] .PP Print object's out of band header. This option requires \f[B]\-O\f[], \f[B]\[en]object\-store\f[] or \f[B]\-l\f[], \f[B]\[en]lanes\f[] or \f[B]\-o\f[], \f[B]\[en]root\f[] options. .PP \f[C]\-H,\ \-\-heap\f[] .PP Print information about \f[B]pmemobj\f[] heap. By default only a heap header is displayed. .PP \f[C]\-Z,\ \-\-zones\f[] .PP If the \f[B]\-H\f[], \f[B]\[en]heap\f[] option is used, print information about zones from specified range. If the \f[B]\-O\f[], \f[B]\[en]object\-store\f[] option is used, print information about objects only from specified range of zones. This option requires \f[B]\-O\f[], \f[B]\[en]object\-store\f[], \f[B]\-H\f[], \f[B]\[en]heap\f[] or \f[B]\-s\f[], \f[B]\[en]stats\f[] options. The range can be specified using \f[B]\-r\f[] option right after the \f[B]\-Z\f[] option. See \f[B]RANGE\f[] section for details about range format. .PP \f[C]\-C,\ \-\-chunks\ []\f[] .PP If the \f[B]\-H, \[en]heap\f[] option is used, print information about chunks from specified range. By default information about chunks of types \f[I]used\f[] , \f[I]free\f[] and \f[I]run\f[] are displayed. If the \f[B]\-O, \[en]object\-store\f[] option is used, print information about objects from specified range of chunks within a zone. This option requires \f[B]\-O, \[en]object\-store\f[], \f[B]\-H, \[en]heap\f[] or \f[B]\-s, \[en]stats\f[] options. The range can be specified using \f[B]\-r\f[] option right after the \f[B]\-C\f[] option. See \f[B]RANGE\f[] section for details about range format. .PP \f[C]\-T,\ \-\-chunk\-type\ used,free,run,footer\f[] .PP Print only specified type(s) of chunks. The multiple types may be specified separated by comma. This option requires \f[B]\-H, \[en]heap\f[] and \f[B]\-C, \[en]chunks\f[] options. .PP \f[C]\-b,\ \-\-bitmap\f[] .PP Print bitmap of used blocks in chunks of type run. This option requires \f[B]\-H, \[en]heap\f[] and \f[B]\-C, \[en]chunks\f[] options. .PP \f[C]\-p,\ \-\-replica\ \f[] .PP Print information from \f[I]\f[] replica. The 0 value means the master pool file. .SH RANGE .PP Using \f[B]\-r, \[en]range\f[] option it is possible to dump only a range of user data. This section describes valid format of \f[I]\f[] string. .PP You can specify multiple ranges separated by commas. .PP \f[C]\-\f[] .PP All blocks/bytes/data chunks from \f[I]\f[] to \f[I]\f[] will be dumped. .PP \f[C]\-\f[] .PP All blocks/bytes/data chunks up to \f[I]\f[] will be dumped. .PP \f[C]\-\f[] .PP All blocks/bytes/data chunks starting from \f[I]\f[] will be dumped. .PP \f[C]\f[] .PP Only \f[I]\f[] block/byte/data chunk will be dumped. .SH STATISTICS .PP Below is the description of statistical measures for specific pool types. .SS PMEMLOG .IP \[bu] 2 \f[B]Total\f[] \- Total space in pool. .IP \[bu] 2 \f[B]Available\f[] \- Size and percentage of available space. .IP \[bu] 2 \f[B]Used\f[] \- Size and percentage of used space. .SS PMEMBLK .IP \[bu] 2 \f[B]Total blocks\f[] \- Total number of blocks in pool. .IP \[bu] 2 \f[B]Zeroed blocks\f[] \- Number and percentage of blocks marked with \f[I]zero\f[] flag. .IP \[bu] 2 \f[B]Error blocks\f[] \- Number and percentage of blocks marked with \f[I]error\f[] flag. .IP \[bu] 2 \f[B]Blocks without any flag\f[] \- Number and percentage of blocks \f[I]not\f[] marked with any flag. .RS .PP NOTE: In case of pmemblk, statistics are evaluated for blocks which meet requirements regarding: \f[I]range\f[] of blocks (\f[B]\-r\f[] option), \f[I]skipped\f[] types of blocks (\f[B]\-z\f[], \f[B]\-e\f[], \f[B]\-u\f[] options). .RE .SS PMEMOBJ .IP \[bu] 2 \f[B]Object store\f[] .RS 2 .IP \[bu] 2 \f[B]Number of objects\f[] \- Total number of objects and number of objects per type number. .IP \[bu] 2 \f[B]Number of bytes\f[] \- Total number of bytes and number of bytes per type number. .RE .IP \[bu] 2 \f[B]Heap\f[] .RS 2 .IP \[bu] 2 \f[B]Number of zones\f[] \- Total number of zones in the pool. .IP \[bu] 2 \f[B]Number of used zones\f[] \- Number of used zones in the pool. .RE .IP \[bu] 2 \f[B]Zone\f[] The zone's statistics are presented for each zone separately and the aggregated results from all zones. .RS 2 .IP \[bu] 2 \f[B]Number of chunks\f[] \- Total number of chunks in the zone and number of chunks of specified type. .IP \[bu] 2 \f[B]Chunks size\f[] \- Total size of all chunks in the zone and sum of sizes of chunks of specified type. .RE .IP \[bu] 2 \f[B]Allocation classes\f[] .RS 2 .IP \[bu] 2 \f[B]Units\f[] \- Total number of units of specified class. .IP \[bu] 2 \f[B]Used units\f[] \- Number of used units of specified class. .IP \[bu] 2 \f[B]Bytes\f[] \- Total number of bytes of specified class. .IP \[bu] 2 \f[B]Used bytes\f[] \- Number of used bytes of specified class. .IP \[bu] 2 \f[B]Total bytes\f[] \- Total number of bytes of all classes. .IP \[bu] 2 \f[B]Total used bytes\f[] \- Total number of used bytes of all classes. .RE .SH EXAMPLE .IP .nf \f[C] $\ pmempool\ info\ ./pmemblk \f[] .fi .PP Parse and print information about \[lq]pmemblk\[rq] pool file. .IP .nf \f[C] $\ pmempool\ info\ \-f\ blk\ ./pmempool \f[] .fi .PP Force parsing \[lq]pmempool\[rq] file as \f[B]pmemblk\f[] pool type. .IP .nf \f[C] $\ pmempool\ info\ \-d\ ./pmemlog \f[] .fi .PP Print information and data in hexadecimal dump format for file \[lq]pmemlog\[rq]. .IP .nf \f[C] $\ pmempool\ info\ \-d\ \-r10\-100\ \-eu\ ./pmemblk \f[] .fi .PP Print information from \[lq]pmemblk\[rq] file. Dump data blocks from 10 to 100, skip blocks marked with error flag and not marked with any flag. .SH SEE ALSO .PP \f[B]pmempool\f[](1), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-dump.10000644000000000000000000001023013615011420016252 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-DUMP" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-dump\f[] \- dump user data from persistent memory pool .SH SYNOPSIS .IP .nf \f[C] $\ pmempool\ dump\ []\ \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool\f[] invoked with \f[I]dump\f[] command dumps user data from specified pool file. The output format may be either binary or hexadecimal. .PP By default the output format is hexadecimal. .PP By default data is dumped to standard output. It is possible to dump data to other file by specifying \f[B]\-o\f[] option. In this case data will be appended to this file. .PP Using \f[B]\-r\f[] option you can specify number of blocks/bytes/data chunks using special text format. See \f[B]RANGE\f[] section for details. .SS Available options: .PP \f[C]\-b,\ \-\-binary\f[] .PP Dump data in binary format. .PP \f[C]\-r,\ \-\-range\ \f[] .PP Range of pool file to dump. This may be number of blocks for \f[B]blk\f[] pool type or either number of bytes or number of data chunks for \f[B]log\f[] pool type. .PP \f[C]\-c,\ \-\-chunk\ \f[] .PP Size of chunk for \f[B]log\f[] pool type. See \f[B]pmemlog_walk\f[](3) in \f[B]libpmemlog\f[](7) for details. .PP \f[C]\-o,\ \-\-output\ \f[] .PP Name of output file. .PP \f[C]\-h,\ \-\-help\f[] .PP Display help message and exit. .SH RANGE .PP Using \f[B]\-r\f[], \f[B]\[en]range\f[] option it is possible to dump only a range of user data. This section describes valid format of \f[I]\f[] string. .PP You can specify multiple ranges separated by commas. .PP \f[C]\-\f[] .PP All blocks/bytes/data chunks from \f[I]\f[] to \f[I]\f[] will be dumped. .PP \f[C]\-\f[] .PP All blocks/bytes/data chunks up to \f[I]\f[] will be dumped. .PP \f[C]\-\f[] .PP All blocks/bytes/data chunks starting from \f[I]\f[] will be dumped. .PP \f[C]\f[] .PP Only \f[I]\f[] block/byte/data chunk will be dumped. .SH EXAMPLE .IP .nf \f[C] $\ pmempool\ dump\ pool.bin \f[] .fi .PP Dump user data from pool.bin file to standard output .IP .nf \f[C] $\ pmempool\ dump\ \-o\ output.bin\ \-r1,10\-100\ pool_blk.bin \f[] .fi .PP Dump block number 1 and blocks from 10 to 100 from pool_blk.bin containing pmem blk pool to output.bin file .IP .nf \f[C] $\ pmempool\ dump\ \-r\ 1K\-2K\ pool.bin \f[] .fi .PP Dump data form 1K to 2K from pool.bin file. .SH SEE ALSO .PP \f[B]pmempool\f[](1), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-feature.10000644000000000000000000001112413615011420016743 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-FEATURE" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-feature\f[] \- toggle or query pool set features .SH SYNOPSIS .IP .nf \f[C] $\ pmempool\ feature\ (\-e|\-d|\-q\ feature\-name)\ [options]\ \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool feature\f[] command enables / disables or queries pool set features. .PP Available pool \f[I]feature\-names\f[] are: .IP \[bu] 2 \f[B]SINGLEHDR\f[] \- only the first part in each replica contains the pool part internal metadata. This value can be used only with \f[B]\-q\f[]. It can not be enabled or disabled. For details see \f[B]poolset\f[](5). .IP \[bu] 2 \f[B]CHECKSUM_2K\f[] \- only the first 2KiB of pool part internal metadata is checksummed. Other features may depend on this one to store additional metadata in otherwise unused second 2KiB part of a header. When \f[B]CHECKSUM_2K\f[] is disabled whole 4KiB is checksummed. .IP \[bu] 2 \f[B]SHUTDOWN_STATE\f[] \- enables additional check performed during pool open which verifies pool consistency in the presence of dirty shutdown. \f[B]CHECKSUM_2K\f[] has to be enabled prior to \f[B]SHUTDOWN_STATE\f[] otherwise enabling \f[B]SHUTDOWN_STATE\f[] will fail. .IP \[bu] 2 \f[B]CHECK_BAD_BLOCKS\f[] \- enables checking bad blocks performed during opening a pool and fixing bad blocks performed by pmempool\-sync during syncing a pool. Currently (Linux kernel v4.19, libndctl v62) checking and fixing bad blocks require read access to the following resource files (containing physical addresses) of NVDIMM devices which only root can read by default: .IP .nf \f[C] /sys/bus/nd/devices/ndbus*/region*/resource /sys/bus/nd/devices/ndbus*/region*/dax*/resource /sys/bus/nd/devices/ndbus*/region*/pfn*/resource /sys/bus/nd/devices/ndbus*/region*/namespace*/resource \f[] .fi .PP It is possible to use poolset as \f[I]file\f[] argument. But poolsets with remote replicas are not supported. .SS Available options: .PP \f[C]\-h,\ \-\-help\f[] .PP Print help message. .PP \f[C]\-v,\ \-\-verbose\f[] .PP Increase verbosity level. .PP \f[C]\-e,\ \-\-enable\ feature\-name\f[] .PP Enable feature for pool set. .PP \f[C]\-d,\ \-\-disable\ feature\-name\f[] .PP Disable feature for pool set. .PP \f[C]\-q,\ \-\-query\ feature\-name\f[] .PP Print feature status. .SH COMPATIBILITY .PP Poolsets with features not defined in this document (e.g.\ enabled by the newer software version) are not supported. .SH DISCLAIMER .PP \f[C]pmempool\ feature\f[] command is not fail safe. .SH EXAMPLE .IP .nf \f[C] $\ pmempool\ feature\ \-\-enable\ CHECKSUM_2K\ pool.set \f[] .fi .PP Enables POOL_FEAT_CKSUM_2K incompat feature flag. .IP .nf \f[C] $\ pmempool\ feature\ \-\-disable\ CHECKSUM_2K\ pool.set \f[] .fi .PP Disables POOL_FEAT_CKSUM_2K incompat feature flag. .IP .nf \f[C] $\ pmempool\ feature\ \-\-query\ CHECKSUM_2K\ pool.set 0 \f[] .fi .PP Prints POOL_FEAT_CKSUM_2K incompat feature flag value. .SH SEE ALSO .PP \f[B]poolset\f[](5) and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-convert.1.md0000664000000000000000000000475413615011243017407 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-CONVERT, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-convert.1 -- man page for pmempool-convert) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **pmempool-convert** - this is a wrapper around pmdk-convert tool. More information can be found in **pmdk-convert**(1) man page. # SEE ALSO # **pmdk-convert**(1), **pmempool**(1), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7), **libpmempool**(7) and **** pmdk-1.8/doc/pmempool/pmempool.10000644000000000000000000001003713615011420015314 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\f[] \- Persistent Memory Pool Management Tool .SH SYNOPSIS .IP .nf \f[C] $\ pmempool\ [\-\-help]\ [\-\-version]\ \ [] \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool\f[] is a management tool for \f[I]Persistent Memory\f[] pool files created by \f[B]PMDK\f[] libraries. .PP The main purpose of \f[B]pmempool\f[] is to provide a user with a set of utilities for off\-line analysis and manipulation of pools created by pmem libraries. The pmempool is a generic command which consists of subcommands for specific purposes. Some of subcommands are required to work \f[I]without\f[] any impact on processed pool, but some of them \f[I]may\f[] create a new or modify an existing one. .PP The \f[B]pmempool\f[] may be useful for troubleshooting by system administrators and for software developers who work on applications based on \f[B]PMDK\f[]. The latter may find these tools useful for testing and debugging purposes also. .SH OPTIONS .PP \f[C]\-V,\ \-\-version\f[] .PP Prints the version of \f[B]pmempool\f[]. .PP \f[C]\-h,\ \-\-help\f[] .PP Prints synopsis and list of commands. .SH COMMANDS .PP Currently there is a following set of commands available: .IP \[bu] 2 \f[B]pmempool\-info\f[](1) \- Prints information and statistics in human\-readable format about specified pool. .IP \[bu] 2 \f[B]pmempool\-check\f[](1) \- Checks pool's consistency and repairs pool if it is not consistent. .IP \[bu] 2 \f[B]pmempool\-create\f[](1) \- Creates a pool of specified type with additional properties specific for this type of pool. .IP \[bu] 2 \f[B]pmempool\-dump\f[](1) \- Dumps usable data from pool in hexadecimal or binary format. .IP \[bu] 2 \f[B]pmempool\-rm\f[](1) \- Removes pool file or all pool files listed in pool set configuration file. .IP \[bu] 2 \f[B]pmempool\-convert\f[](1) \- Updates the pool to the latest available layout version. .IP \[bu] 2 \f[B]pmempool\-sync\f[](1) \- Synchronizes replicas within a poolset. .IP \[bu] 2 \f[B]pmempool\-transform\f[](1) \- Modifies internal structure of a poolset. .IP \[bu] 2 \f[B]pmempool\-feature\f[](1) \- Toggle or query a poolset features. .PP In order to get more information about specific \f[I]command\f[] you can use \f[B]pmempool help .\f[] .SH SEE ALSO .PP \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-rm.10000644000000000000000000001034013615011420015725 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-RM" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-rm\f[] \- remove a persistent memory pool .SH SYNOPSIS .IP .nf \f[C] $\ pmempool\ rm\ []\ .. \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool rm\f[] command removes each specified file. If the specified file is a pool set file, all pool files (single\-file pool or part files) and remote replicas are removed. By default the \f[B]pmempool rm\f[] does not remove pool set files. All local and remote pool files are removed using \f[B]unlink\f[](3) call, except the pools created on \f[B]device dax\f[] which are zeroed instead. If specified file does not exist, the remote pool is broken or not accessible, the \f[B]pmempool rm\f[] command terminates with an error code. By default it prompts before removing \f[I]write\-protected\f[] local files. See \f[B]REMOTE REPLICATION\f[] section for more details about support for remote pools. See \f[B]EXAMPLES\f[] section for example usage of the \f[I]rm\f[] command. .SS Available options: .PP \f[C]\-h,\ \-\-help\f[] .PP Print help message .PP \f[C]\-v,\ \-\-verbose\f[] .PP Be verbose and print all removing files. .PP \f[C]\-s,\ \-\-only\-pools\f[] .PP Remove only pool files and do not remove pool set files (default behaviour). .PP \f[C]\-a,\ \-\-all\f[] .PP Remove all pool set files \- local and remote. .PP \f[C]\-l,\ \-\-local\f[] .PP Remove local pool set files. .PP \f[C]\-r,\ \-\-remote\f[] .PP Remove remote pool set files. .PP \f[C]\-f,\ \-\-force\f[] .PP Remove all specified files, ignore nonexistent files, never prompt. .PP \f[C]\-i,\ \-\-interactive\f[] .PP Prompt before removing every single file or remote pool. .SH REMOTE REPLICATION .PP A remote pool is removed using \f[B]rpmem_remove\f[](3) function if \f[B]librpmem\f[](7) library is available. If a pool set file contains remote replication but \f[B]librpmem\f[](7) is not available, the \f[B]pmempool rm\f[] command terminates with an error code, unless the \f[B]\-f, \[en]force\f[] option is specified. .SH EXAMPLE .IP .nf \f[C] $\ pmempool\ rm\ pool.obj\ pool.blk \f[] .fi .PP Remove specified pool files. .IP .nf \f[C] $\ pmempool\ rm\ pool.set \f[] .fi .PP Remove all pool files from the \[lq]pool.set\[rq], do not remove \f[I]pool.set\f[] itself. .IP .nf \f[C] $\ pmempool\ rm\ \-a\ pool.set \f[] .fi .PP Remove all pool files from the \[lq]pool.set\[rq], remove the local pool set file and all remote pool set files. .SH SEE ALSO .PP \f[B]pmempool\f[](1), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7), \f[B]librpmem\f[](7) and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-create.10000644000000000000000000001325313615011420016560 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL-CREATE" "1" "2020-01-31" "PMDK - pmem Tools version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool\-create\f[] \- create a persistent memory pool .SH SYNOPSIS .IP .nf \f[C] $\ pmempool\ create\ []\ []\ []\ \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool\f[] invoked with \f[I]create\f[] command creates a pool file of specified type. Depending on a pool type it is possible to provide more properties of pool. .PP Valid pool types are: \f[B]blk\f[], \f[B]log\f[] and \f[B]obj\f[] which stands for \f[I]pmemblk\f[], \f[I]pmemlog\f[] and \f[I]pmemobj\f[] pools respectively. By default the pool file is created with \f[I]minimum\f[] allowed size for specified pool type. The minimum sizes for \f[B]blk\f[], \f[B]log\f[] and \f[B]obj\f[] pool types are \f[B]PMEMBLK_MIN_POOL\f[], \f[B]PMEMLOG_MIN_POOL\f[] and \f[B]PMEMOBJ_MIN_POOL\f[] respectively. See \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7) and \f[B]libpmemobj\f[](7) for details. .PP For \f[I]pmemblk\f[] pool type block size \f[I]\f[] is a required argument. .PP In order to set custom size of pool use \f[B]\-s\f[] option, or use \f[B]\-M\f[] option to create a pool of maximum available size on underlying file system. .PP The \f[I]size\f[] argument may be passed in format that permits only the upper\-case character for byte \- B as specified in IEC 80000\-13, IEEE 1541 and the Metric Interchange Format. Standards accept SI units with obligatory B \- kB, MB, GB, \&... which means multiplier by 1000 and IEC units with optional \[lq]iB\[rq] \- KiB, MiB, GiB, \&..., K, M, G, \&... \- which means multiplier by 1024. .SS Available options: .PP \f[C]\-s,\ \-\-size\ \f[] .PP Size of pool file. .PP \f[C]\-M,\ \-\-max\-size\f[] .PP Set size of pool to available space of underlying file system. .PP \f[C]\-m,\ \-\-mode\ \f[] .PP Set permissions to (the default is 0664) when creating the files. If the file already exist the permissions are not changed. .PP \f[C]\-i,\ \-\-inherit\ \f[] .PP Create a new pool of the same size and other properties as \f[I]\f[]. .PP \f[C]\-b,\ \-\-clear\-bad\-blocks\f[] .PP Clear bad blocks in existing files. .PP \f[C]\-f,\ \-\-force\f[] .PP Remove the pool before creating. .PP \f[C]\-v,\ \-\-verbose\f[] .PP Increase verbosity level. .PP \f[C]\-h,\ \-\-help\f[] .PP Display help message and exit. .SS Options for PMEMBLK: .PP By default when creating a pmem \f[B]blk\f[] pool, the \f[B]BTT\f[] layout is \f[I]not\f[] written until the first \f[I]write operation\f[] of block entry is performed. Using \f[B]\-w\f[] option you can force writing the \f[B]BTT\f[] layout by writing zero data to specified block number. By default the \f[I]write operation\f[] is performed to block number 0. Please refer to \f[B]libpmemblk\f[](7) for details. .PP \f[C]\-w,\ \-\-write\-layout\f[] .PP Force writing the \f[B]BTT\f[] layout by performing \f[I]write operation\f[] to block number zero. .SS Options for PMEMOBJ: .PP By default when creating a pmem \f[B]obj\f[] pool, the layout name provided to the \f[B]libpmemobj\f[] library is an empty string. Please refer to \f[B]libpmemobj\f[](7) for details. .PP \f[C]\-l,\ \-\-layout\ \f[] .PP Layout name of the \f[B]pmemobj\f[] pool. .SH EXAMPLE .IP .nf \f[C] $\ pmempool\ create\ blk\ 512\ pool.blk \f[] .fi .PP Create a blk pool file of minimum allowed size and block size 512 bytes .IP .nf \f[C] $\ pmempool\ create\ log\ \-M\ pool.log \f[] .fi .PP Create a log pool file of maximum allowed size .IP .nf \f[C] $\ pmempool\ create\ blk\ \-\-size=4G\ \-\-write\-layout\ 1K\ pool.blk \f[] .fi .PP Create a blk pool file of size 4G, block size 1K and write the BTT layout .IP .nf \f[C] $\ pmempool\ create\ \-\-layout\ my_layout\ obj\ pool.obj \f[] .fi .PP Create an obj pool file of minimum allowed size and layout \[lq]my_layout\[rq] .IP .nf \f[C] $\ pmempool\ create\ \-\-inherit=pool.log\ new_pool.log \f[] .fi .PP Create a pool file based on pool.log file .SH SEE ALSO .PP \f[B]pmempool\f[](1), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/pmempool/pmempool-transform.1.md0000664000000000000000000001522013615011243017730 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-TRANSFORM, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-transform.1 -- man page for pmempool-transform) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[EXAMPLES](#examples)
[SEE ALSO](#see-also)
# NAME # **pmempool-transform** - Modify internal structure of a pool set. # SYNOPSIS # ``` pmempool transform [options] ``` # DESCRIPTION # The **pmempool transform** command modifies internal structure of a pool set defined by the `poolset_file_src` file, according to a structure described in the `poolset_file_dst` file. The following operations are supported: * adding replicas - one or more new replicas can be added and synchronized with other replicas in the pool set, * removing replicas - one or more replicas can be removed from the pool set _WINUX(.,=q=, * adding or removing pool set options.=e=) Only one of the above operations can be performed at a time. Currently adding and removing replicas are allowed only for **pmemobj** pools (see **libpmemobj**(7)). The *poolset_file_src* argument provides the source pool set to be changed. The *poolset_file_dst* argument points to the target pool set. _WINUX(=q=When adding or deleting replicas, the two pool set files can differ only in the definitions of replicas which are to be added or deleted. One cannot add and remove replicas in the same step. Only one of these operations can be performed at a time. Reordering replicas is not supported Also, to add a replica it is necessary for its effective size to match or exceed the pool size. Otherwise the whole operation fails and no changes are applied. Effective size of a replica is the sum of sizes of all its part files decreased by 4096 bytes per each part file. The 4096 bytes of each part file is utilized for storing internal metadata of the pool part files.=e=) _WINUX(,=q=When adding or deleting replicas, the two pool set files can differ only in the definitions of replicas which are to be added or deleted. When adding or removing pool set options (see **poolset**(5)), the rest of both pool set files have to be of the same structure. The operation of adding/removing a pool set option can be performed on a pool set with local replicas only. To add/remove a pool set option to/from a pool set with remote replicas, one has to remove the remote replicas first, then add/remove the option, and finally recreate the remote replicas having added/removed the pool set option to/from the remote replicas' poolset files. To add a replica it is necessary for its effective size to match or exceed the pool size. Otherwise the whole operation fails and no changes are applied. If none of the poolset options is used, the effective size of a replica is the sum of sizes of all its part files decreased by 4096 bytes per each part file. The 4096 bytes of each part file is utilized for storing internal metadata of the pool part files. If the option *SINGLEHDR* is used, the effective size of a replica is the sum of sizes of all its part files decreased once by 4096 bytes. In this case only the first part contains internal metadata. If the option *NOHDRS* is used, the effective size of a replica is the sum of sizes of all its part files. In this case none of the parts contains internal metadata.=e=) ##### Available options: ##### `-d, --dry-run` : Enable dry run mode. In this mode no changes are applied, only check for viability of the operation is performed. `-v, --verbose` : Increase verbosity level. `-h, --help` : Display help message and exit. # EXAMPLES # ##### Example 1. ##### Let files `/path/poolset_file_src` and `/path/poolset_file_dst` have the following contents: ``` PMEMPOOLSET 20M /0/partfile1 20M /0/partfile2 25M /0/partfile3 REPLICA 40M /1/partfile1 20M /1/partfile2 ``` ``` PMEMPOOLSET 20M /0/partfile1 20M /0/partfile2 25M /0/partfile3 REPLICA 40M /1/partfile1 20M /1/partfile2 REPLICA 50M /2/partfile1 20M /2/partfile2 ``` Then, the command `pmempool transform /path/poolset_file_src /path/poolset_file_dst` adds a replica to the pool set. All other replicas remain unchanged and the size of the pool remains 60M. ##### Example 2. ##### Let files `/path/poolset_file_src` and `/path/poolset_file_dst` have the following contents: ``` PMEMPOOLSET 20M /0/partfile1 20M /0/partfile2 25M /0/partfile3 REPLICA 40M /1/partfile1 20M /1/partfile2 ``` ``` PMEMPOOLSET 20M /0/partfile1 20M /0/partfile2 25M /0/partfile3 ``` Then `pmempool_transform /path/poolset_file_src /path/poolset_file_dst` deletes the second replica from the pool set. The first replica remains unchanged and the size of the pool is still 60M. # SEE ALSO # **pmempool(1)**, **libpmemblk(7)**, **libpmemlog(7)**, **libpmempool(7)** and **** pmdk-1.8/doc/pmempool/pmempool-feature.1.md0000664000000000000000000001143413615011243017353 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL-FEATURE, 1) collection: pmempool header: PMDK date: pmem Tools version 1.4 ... [comment]: <> (Copyright 2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool-feature.1 -- man page for pmempool-feature) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[COMPATIBILITY](#compatibility)
[DISCLAIMER](#disclaimer)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **pmempool-feature** - toggle or query pool set features # SYNOPSIS # ``` $ pmempool feature (-e|-d|-q feature-name) [options] ``` # DESCRIPTION # The **pmempool feature** command enables / disables or queries pool set features. Available pool *feature-names* are: + **SINGLEHDR** - only the first part in each replica contains the pool part internal metadata. This value can be used only with **-q**. It can not be enabled or disabled. For details see **poolset**(5). + **CHECKSUM_2K** - only the first 2KiB of pool part internal metadata is checksummed. Other features may depend on this one to store additional metadata in otherwise unused second 2KiB part of a header. When **CHECKSUM_2K** is disabled whole 4KiB is checksummed. + **SHUTDOWN_STATE** - enables additional check performed during pool open which verifies pool consistency in the presence of dirty shutdown. **CHECKSUM_2K** has to be enabled prior to **SHUTDOWN_STATE** otherwise enabling **SHUTDOWN_STATE** will fail. + **CHECK_BAD_BLOCKS** - enables checking bad blocks performed during opening a pool and fixing bad blocks performed by pmempool-sync during syncing a pool. Currently (Linux kernel v4.19, libndctl v62) checking and fixing bad blocks require read access to the following resource files (containing physical addresses) of NVDIMM devices which only root can read by default: ``` /sys/bus/nd/devices/ndbus*/region*/resource /sys/bus/nd/devices/ndbus*/region*/dax*/resource /sys/bus/nd/devices/ndbus*/region*/pfn*/resource /sys/bus/nd/devices/ndbus*/region*/namespace*/resource ``` It is possible to use poolset as *file* argument. But poolsets with remote replicas are not supported. ##### Available options: ##### `-h, --help` Print help message. `-v, --verbose` Increase verbosity level. `-e, --enable feature-name` Enable feature for pool set. `-d, --disable feature-name` Disable feature for pool set. `-q, --query feature-name` Print feature status. # COMPATIBILITY # Poolsets with features not defined in this document (e.g. enabled by the newer software version) are not supported. # DISCLAIMER # ```pmempool feature``` command is not fail safe. # EXAMPLE # ``` $ pmempool feature --enable CHECKSUM_2K pool.set ``` Enables POOL_FEAT_CKSUM_2K incompat feature flag. ``` $ pmempool feature --disable CHECKSUM_2K pool.set ``` Disables POOL_FEAT_CKSUM_2K incompat feature flag. ``` $ pmempool feature --query CHECKSUM_2K pool.set 0 ``` Prints POOL_FEAT_CKSUM_2K incompat feature flag value. # SEE ALSO # **poolset**(5) and **** pmdk-1.8/doc/libpmemblk/0000775000000000000000000000000013615011416013676 5ustar rootrootpmdk-1.8/doc/libpmemblk/pmemblk_open.30000664000000000000000000000002513615011243016425 0ustar rootroot.so pmemblk_create.3 pmdk-1.8/doc/libpmemblk/pmemblk_check.30000664000000000000000000000002513615011243016541 0ustar rootroot.so pmemblk_create.3 pmdk-1.8/doc/libpmemblk/libpmemblk.70000644000000000000000000003262213615011416016107 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "LIBPMEMBLK" "7" "2020-01-31" "PMDK - pmemblk API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]libpmemblk\f[] \- persistent memory resident array of blocks .SH SYNOPSIS .IP .nf \f[C] #include\ cc\ ...\ \-lpmemblk\ \-lpmem \f[] .fi .SS Library API versioning: .IP .nf \f[C] const\ char\ *pmemblk_check_version( \ \ \ \ unsigned\ major_required, \ \ \ \ unsigned\ minor_required); \f[] .fi .SS Managing library behavior: .IP .nf \f[C] void\ pmemblk_set_funcs( \ \ \ \ void\ *(*malloc_func)(size_t\ size), \ \ \ \ void\ (*free_func)(void\ *ptr), \ \ \ \ void\ *(*realloc_func)(void\ *ptr,\ size_t\ size), \ \ \ \ char\ *(*strdup_func)(const\ char\ *s)); \f[] .fi .SS Error handling: .IP .nf \f[C] const\ char\ *pmemblk_errormsg(void); \f[] .fi .SS Other library functions: .PP A description of other \f[B]libpmemblk\f[] functions can be found on the following manual pages: .PP \f[B]pmemblk_bsize\f[](3), \f[B]pmemblk_create\f[](3), \f[B]pmemblk_ctl_exec\f[](3), \f[B]pmemblk_ctl_get\f[](3), \f[B]pmemblk_ctl_set\f[](3), \f[B]pmemblk_read\f[](3), \f[B]pmemblk_set_zero\f[](3), .SH DESCRIPTION .PP \f[B]libpmemblk\f[] provides an array of blocks in \f[I]persistent memory\f[] (pmem) such that updates to a single block are atomic. This library is intended for applications using direct access storage (DAX), which is storage that supports load/store access without paging blocks from a block storage device. Some types of \f[I]non\-volatile memory DIMMs\f[] (NVDIMMs) provide this type of byte addressable access to storage. A \f[I]persistent memory aware file system\f[] is typically used to expose the direct access to applications. Memory mapping a file from this type of file system results in the load/store, non\-paged access to pmem. \f[B]libpmemblk\f[] builds on this type of memory mapped file. .PP This library is for applications that need a potentially large array of blocks, all the same size, where any given block is updated atomically (the update cannot be \f[I]torn\f[] by program interruption such as power failures). This library builds on the low\-level pmem support provided by \f[B]libpmem\f[](7), handling the transactional update of the blocks, flushing to persistence, and recovery for the application. \f[B]libpmemblk\f[] is one of a collection of persistent memory libraries available, the others are: .IP \[bu] 2 \f[B]libpmemobj\f[](7), a general use persistent memory API, providing memory allocation and transactional operations on variable\-sized objects. .IP \[bu] 2 \f[B]libpmemlog\f[](7), providing a pmem\-resident log file. .IP \[bu] 2 \f[B]libpmem\f[](7), low\-level persistent memory support. .PP Under normal usage, \f[B]libpmemblk\f[] will never print messages or intentionally cause the process to exit. The only exception to this is the debugging information, when enabled, as described under \f[B]DEBUGGING AND ERROR HANDLING\f[] below. .PP To use the atomic block arrays supplied by \f[B]libpmemblk\f[], a \f[I]memory pool\f[] is first created using the \f[B]pmemblk_create\f[]() function described in \f[B]pmemblk_create\f[](3). The other \f[B]libpmemblk\f[] functions operate on the resulting block memory pool using the opaque handle, of type \f[I]PMEMblkpool*\f[], that is returned by \f[B]pmemblk_create\f[]() or \f[B]pmemblk_open\f[](). Internally, \f[B]libpmemblk\f[] will use either \f[B]pmem_persist\f[](3) or \f[B]msync\f[](2) when it needs to flush changes, depending on whether the memory pool appears to be persistent memory or a regular file (see the \f[B]pmem_is_pmem\f[](3) function in \f[B]libpmem\f[](7) for more information). There is no need for applications to flush changes directly when using the block memory API provided by \f[B]libpmemblk\f[]. .SH CAVEATS .PP \f[B]libpmemblk\f[] relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. \f[B]dlclose\f[](3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. .SH LIBRARY API VERSIONING .PP This section describes how the library API is versioned, allowing applications to work with an evolving API. .PP The \f[B]pmemblk_check_version\f[]() function is used to determine whether the installed \f[B]libpmemblk\f[] supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile\-time version information, supplied by defines in \f[B]\f[], like this: .IP .nf \f[C] reason\ =\ pmemblk_check_version(PMEMBLK_MAJOR_VERSION, \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ PMEMBLK_MINOR_VERSION); if\ (reason\ !=\ NULL)\ { \ \ \ \ /*\ version\ check\ failed,\ reason\ string\ tells\ you\ why\ */ } \f[] .fi .PP Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. .PP An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text \f[I]introduced in version x.y\f[] in the section of this manual describing the feature. .PP When the version check performed by \f[B]pmemblk_check_version\f[]() is successful, the return value is NULL. Otherwise the return value is a static string describing the reason for failing the version check. The string returned by \f[B]pmemblk_check_version\f[]() must not be modified or freed. .SH MANAGING LIBRARY BEHAVIOR .PP The \f[B]pmemblk_set_funcs\f[]() function allows an application to override memory allocation calls used internally by \f[B]libpmemblk\f[]. Passing in NULL for any of the handlers will cause the \f[B]libpmemblk\f[] default function to be used. The library does not make heavy use of the system malloc functions, but it does allocate approximately 4\-8 kilobytes for each memory pool in use. .SH DEBUGGING AND ERROR HANDLING .PP The \f[B]pmemblk_errormsg\f[]() function returns a pointer to a static buffer containing the last error message logged for the current thread. If \f[I]errno\f[] was set, the error message may include a description of the corresponding error code, as returned by \f[B]strerror\f[](3). The error message buffer is thread\-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a \f[B]libpmemblk\f[] function indicated an error, or if \f[I]errno\f[] was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. .PP Two versions of \f[B]libpmemblk\f[] are typically available on a development system. The normal version, accessed when a program is linked using the \f[B]\-lpmemblk\f[] option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run\-time assertions. If an error is detected in a call to \f[B]libpmemblk\f[], the error message describing the failure may be retrieved with \f[B]pmemblk_errormsg\f[]() as described above. .PP A second version of \f[B]libpmemblk\f[], accessed when a program uses the libraries under \f[B]/usr/lib/pmdk_debug\f[], contains run\-time assertions and trace points. The typical way to access the debug version is to set the \f[B]LD_LIBRARY_PATH\f[] environment variable to \f[B]/usr/lib/pmdk_debug\f[] or \f[B]/usr/lib64/pmdk_debug\f[], as appropriate. Debugging output is controlled using the following environment variables. These variables have no effect on the non\-debug version of the library. .IP \[bu] 2 \f[B]PMEMBLK_LOG_LEVEL\f[] .PP The value of \f[B]PMEMBLK_LOG_LEVEL\f[] enables trace points in the debug version of the library, as follows: .IP \[bu] 2 \f[B]0\f[] \- This is the default level when \f[B]PMEMBLK_LOG_LEVEL\f[] is not set. No log messages are emitted at this level. .IP \[bu] 2 \f[B]1\f[] \- Additional details on any errors detected are logged, in addition to returning the \f[I]errno\f[]\-based errors as usual. The same information may be retrieved using \f[B]pmemblk_errormsg\f[](). .IP \[bu] 2 \f[B]2\f[] \- A trace of basic operations is logged. .IP \[bu] 2 \f[B]3\f[] \- Enables a very verbose amount of function call tracing in the library. .IP \[bu] 2 \f[B]4\f[] \- Enables voluminous and fairly obscure tracing information that is likely only useful to the \f[B]libpmemblk\f[] developers. .PP Unless \f[B]PMEMBLK_LOG_FILE\f[] is set, debugging output is written to \f[I]stderr\f[]. .IP \[bu] 2 \f[B]PMEMBLK_LOG_FILE\f[] .PP Specifies the name of a file where all logging information should be written. If the last character in the name is \[lq]\-\[rq], the \f[I]PID\f[] of the current process will be appended to the file name when the log file is created. If \f[B]PMEMBLK_LOG_FILE\f[] is not set, the logging output is written to \f[I]stderr\f[]. .PP See also \f[B]libpmem\f[](7) for information on other environment variables that may affect \f[B]libpmemblk\f[] behavior. .SH EXAMPLE .PP The following example illustrates how the \f[B]libpmemblk\f[] API is used. .IP .nf \f[C] #include\ #include\ #include\ #include\ #include\ #include\ /*\ size\ of\ the\ pmemblk\ pool\ \-\-\ 1\ GB\ */ #define\ POOL_SIZE\ ((size_t)(1\ <<\ 30)) /*\ size\ of\ each\ element\ in\ the\ pmem\ pool\ */ #define\ ELEMENT_SIZE\ 1024 int main(int\ argc,\ char\ *argv[]) { \ \ \ \ const\ char\ path[]\ =\ "/pmem\-fs/myfile"; \ \ \ \ PMEMblkpool\ *pbp; \ \ \ \ size_t\ nelements; \ \ \ \ char\ buf[ELEMENT_SIZE]; \ \ \ \ /*\ create\ the\ pmemblk\ pool\ or\ open\ it\ if\ it\ already\ exists\ */ \ \ \ \ pbp\ =\ pmemblk_create(path,\ ELEMENT_SIZE,\ POOL_SIZE,\ 0666); \ \ \ \ if\ (pbp\ ==\ NULL) \ \ \ \ \ \ \ \ pbp\ =\ pmemblk_open(path,\ ELEMENT_SIZE); \ \ \ \ if\ (pbp\ ==\ NULL)\ { \ \ \ \ \ \ \ \ perror(path); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ /*\ how\ many\ elements\ fit\ into\ the\ file?\ */ \ \ \ \ nelements\ =\ pmemblk_nblock(pbp); \ \ \ \ printf("file\ holds\ %zu\ elements",\ nelements); \ \ \ \ /*\ store\ a\ block\ at\ index\ 5\ */ \ \ \ \ strcpy(buf,\ "hello,\ world"); \ \ \ \ if\ (pmemblk_write(pbp,\ buf,\ 5)\ <\ 0)\ { \ \ \ \ \ \ \ \ perror("pmemblk_write"); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ /*\ read\ the\ block\ at\ index\ 10\ (reads\ as\ zeros\ initially)\ */ \ \ \ \ if\ (pmemblk_read(pbp,\ buf,\ 10)\ <\ 0)\ { \ \ \ \ \ \ \ \ perror("pmemblk_read"); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ /*\ zero\ out\ the\ block\ at\ index\ 5\ */ \ \ \ \ if\ (pmemblk_set_zero(pbp,\ 5)\ <\ 0)\ { \ \ \ \ \ \ \ \ perror("pmemblk_set_zero"); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ /*\ ...\ */ \ \ \ \ pmemblk_close(pbp); } \f[] .fi .PP See for more examples using the \f[B]libpmemblk\f[] API. .SH BUGS .PP Unlike \f[B]libpmemobj\f[](7), data replication is not supported in \f[B]libpmemblk\f[]. Thus, specifying replica sections in pool set files is not allowed. .SH ACKNOWLEDGEMENTS .PP \f[B]libpmemblk\f[] builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: .SH SEE ALSO .PP \f[B]msync\f[](2), \f[B]dlclose\f[](3), \f[B]pmemblk_bsize\f[](3), \f[B]pmemblk_create\f[](3), \f[B]pmemblk_ctl_exec\f[](3), \f[B]pmemblk_ctl_get\f[](3), \f[B]pmemblk_ctl_set\f[](3), \f[B]pmemblk_read\f[](3), \f[B]pmemblk_set_zero\f[](3), \f[B]pmem_is_pmem\f[](3), \f[B]pmem_persist\f[](3), \f[B]strerror\f[](3), \f[B]libpmem\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemblk/pmemblk_nblock.30000664000000000000000000000002413615011243016733 0ustar rootroot.so pmemblk_bsize.3 pmdk-1.8/doc/libpmemblk/pmemblk_create.30000644000000000000000000002142113615011416016732 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMBLK_CREATE" "3" "2020-01-31" "PMDK - pmemblk API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemblk_create\f[](), \f[B]pmemblk_open\f[](), \f[B]pmemblk_close\f[](), \f[B]pmemblk_check\f[]() \- create, open, close and validate block pool .SH SYNOPSIS .IP .nf \f[C] #include\ PMEMblkpool\ *pmemblk_create(const\ char\ *path,\ size_t\ bsize, \ \ \ \ \ \ \ \ size_t\ poolsize,\ mode_t\ mode); PMEMblkpool\ *pmemblk_open(const\ char\ *path,\ size_t\ bsize); void\ pmemblk_close(PMEMblkpool\ *pbp); int\ pmemblk_check(const\ char\ *path,\ size_t\ bsize); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemblk_create\f[]() function creates a block memory pool with the given total \f[I]poolsize\f[], divided into as many elements of size \f[I]bsize\f[] as will fit in the pool. Since the transactional nature of a block memory pool requires some space overhead in the memory pool, the resulting number of available blocks is less than \f[I]poolsize\f[]/\f[I]bsize\f[], and is made available to the caller via the \f[B]pmemblk_nblock\f[](3) function. Given the specifics of the implementation, the number of available blocks for the user cannot be less than 256. This translates to at least 512 internal blocks. \f[I]path\f[] specifies the name of the memory pool file to be created. \f[I]mode\f[] specifies the permissions to use when creating the file, as described by \f[B]creat\f[](2). The memory pool file is fully allocated to the size \f[I]poolsize\f[] using \f[B]posix_fallocate\f[](3). The caller may choose to take responsibility for creating the memory pool file by creating it before calling \f[B]pmemblk_create\f[](), and then specifying \f[I]poolsize\f[] as zero. In this case \f[B]pmemblk_create\f[]() will take the pool size from the size of the existing file, and will verify that the file appears to be empty by searching for any non\-zero data in the pool header at the beginning of the file. The net pool size of a pool file is equal to the file size. The minimum net pool size allowed by the library for a block pool is defined in \f[B]\f[] as \f[B]PMEMBLK_MIN_POOL\f[]. \f[I]bsize\f[] can be any non\-zero value; however, \f[B]libpmemblk\f[] will silently round up the given size to \f[B]PMEMBLK_MIN_BLK\f[], as defined in \f[B]\f[]. .PP Depending on the configuration of the system, the available non\-volatile memory space may be divided into multiple memory devices. In such case, the maximum size of the pmemblk memory pool could be limited by the capacity of a single memory device. \f[B]libpmemblk\f[](7) allows building a persistent memory resident array spanning multiple memory devices by creation of persistent memory pools consisting of multiple files, where each part of such a \f[I]pool set\f[] may be stored on a different memory device or pmem\-aware filesystem. .PP Creation of all the parts of the pool set can be done with \f[B]pmemblk_create\f[](); however, the recommended method for creating pool sets is by using the \f[B]pmempool\f[](1) utility. .PP When creating a pool set consisting of multiple files, the \f[I]path\f[] argument passed to \f[B]pmemblk_create\f[]() must point to the special \f[I]set\f[] file that defines the pool layout and the location of all the parts of the pool set. The \f[I]poolsize\f[] argument must be 0. The meaning of the \f[I]mode\f[] argument does not change, except that the same \f[I]mode\f[] is used for creation of all the parts of the pool set. .PP For more information on pool set format, see \f[B]poolset\f[](5). .PP The \f[B]pmemblk_open\f[]() function opens an existing block memory pool. As with \f[B]pmemblk_create\f[](), \f[I]path\f[] must identify either an existing block memory pool file, or the \f[I]set\f[] file used to create a pool set. The application must have permission to open the file and memory map the file or pool set with read/write permissions. If \f[I]bsize\f[] is non\-zero, \f[B]pmemblk_open\f[]() will verify that the given block size matches the block size used when the pool was created. Otherwise, \f[B]pmemblk_open\f[]() will open the pool without verifying the block size. The \f[I]bsize\f[] can be determined using the \f[B]pmemblk_bsize\f[](3) function. .PP Be aware that if the pool contains bad blocks inside, opening can be aborted by the SIGBUS signal, because currently the pool is not checked against bad blocks during opening. It can be turned on by setting the CHECK_BAD_BLOCKS compat feature. For details see description of this feature in \f[B]pmempool\-feature\f[](1). .PP The \f[B]pmemblk_close\f[]() function closes the memory pool indicated by \f[I]pbp\f[] and deletes the memory pool handle. The block memory pool itself lives on in the file that contains it and may be re\-opened at a later time using \f[B]pmemblk_open\f[]() as described above. .PP The \f[B]pmemblk_check\f[]() function performs a consistency check of the file indicated by \f[I]path\f[], and returns 1 if the memory pool is found to be consistent. If the pool is found not to be consistent, further use of the file with \f[B]libpmemblk\f[] will result in undefined behavior. The debug version of \f[B]libpmemblk\f[] will provide additional details on inconsistencies when \f[B]PMEMBLK_LOG_LEVEL\f[] is at least 1, as described in the \f[B]DEBUGGING AND ERROR HANDLING\f[] section in \f[B]libpmemblk\f[](7). \f[B]pmemblk_check\f[]() opens the given \f[I]path\f[] read\-only so it never makes any changes to the file. This function is not supported on Device DAX. .SH RETURN VALUE .PP On success, \f[B]pmemblk_create\f[]() returns a \f[I]PMEMblkpool*\f[] handle to the block memory pool. On error, it returns NULL and sets \f[I]errno\f[] appropriately. .PP On success, \f[B]pmemblk_open\f[]() returns a \f[I]PMEMblkpool*\f[] handle that can be used with most of the functions in \f[B]libpmemblk\f[](7). On error, it returns NULL and sets \f[I]errno\f[] appropriately. Possible errors include: .IP \[bu] 2 failure to open \f[I]path\f[] .IP \[bu] 2 \f[I]path\f[] specifies a \f[I]set\f[] file and any of the pool set files cannot be opened .IP \[bu] 2 \f[I]path\f[] specifies a \f[I]set\f[] file and the actual size of any file does not match the corresponding part size defined in the \f[I]set\f[] file .IP \[bu] 2 \f[I]bsize\f[] is non\-zero and does not match the block size given when the pool was created. \f[I]errno\f[] is set to \f[B]EINVAL\f[] in this case. .PP The \f[B]pmemblk_close\f[]() function returns no value. .PP \f[B]pmemblk_check\f[]() returns 1 if the memory pool is found to be consistent. If the check is successfully performed but the pool is found to be inconsistent, \f[B]pmemblk_check\f[]() returns 0. This includes the case where \f[I]bsize\f[] is non\-zero and does not match the block size given when the pool was created. If the consistency check cannot be performed, \f[B]pmemblk_check\f[]() returns \-1 and sets \f[I]errno\f[] appropriately. .SH CAVEATS .PP Not all file systems support \f[B]posix_fallocate\f[](3). \f[B]pmemblk_create\f[]() will fail if the underlying file system does not support \f[B]posix_fallocate\f[](3). .SH SEE ALSO .PP \f[B]pmempool\f[](1), \f[B]creat\f[](2), \f[B]pmemblk_nblock\f[](3), \f[B]posix_fallocate\f[](3), \f[B]poolset\f[](5), \f[B]libpmemblk\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemblk/pmemblk_ctl_exec.30000664000000000000000000000002613615011243017253 0ustar rootroot.so pmemblk_ctl_get.3 pmdk-1.8/doc/libpmemblk/pmemblk_check_version.30000664000000000000000000000002613615011243020307 0ustar rootroot.so man7/libpmemblk.7 pmdk-1.8/doc/libpmemblk/pmemblk_write.30000664000000000000000000000002313615011243016614 0ustar rootroot.so pmemblk_read.3 pmdk-1.8/doc/libpmemblk/pmemblk_set_error.30000664000000000000000000000002713615011243017472 0ustar rootroot.so pmemblk_set_zero.3 pmdk-1.8/doc/libpmemblk/pmemblk_set_zero.30000644000000000000000000000560213615011416017324 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMBLK_SET_ZERO" "3" "2020-01-31" "PMDK - pmemblk API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemblk_set_zero\f[](), \f[B]pmemblk_set_error\f[]() \- block management functions .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmemblk_set_zero(PMEMblkpool\ *pbp,\ long\ long\ blockno); int\ pmemblk_set_error(PMEMblkpool\ *pbp,\ long\ long\ blockno); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemblk_set_zero\f[]() function writes zeros to block number \f[I]blockno\f[] in persistent memory resident array of blocks \f[I]pbp\f[]. Using this function is faster than actually writing a block of zeros since \f[B]libpmemblk\f[](7) uses metadata to indicate the block should read back as zero. .PP The \f[B]pmemblk_set_error\f[]() function sets the error state for block number \f[I]blockno\f[] in persistent memory resident array of blocks \f[I]pbp\f[]. A block in the error state returns \f[I]errno\f[] \f[B]EIO\f[] when read. Writing the block clears the error state and returns the block to normal use. .SH RETURN VALUE .PP On success, \f[B]pmemblk_set_zero\f[]() and \f[B]pmemblk_set_error\f[]() return 0. On error, they return \-1 and set \f[I]errno\f[] appropriately. .SH SEE ALSO .PP \f[B]libpmemblk\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemblk/pmemblk_errormsg.30000664000000000000000000000002613615011243017325 0ustar rootroot.so man7/libpmemblk.7 pmdk-1.8/doc/libpmemblk/.gitignore0000664000000000000000000000014213615011243015661 0ustar rootrootlibpmemblk.7 pmemblk_bsize.3 pmemblk_create.3 pmemblk_ctl_get.3 pmemblk_read.3 pmemblk_set_zero.3 pmdk-1.8/doc/libpmemblk/pmemblk_ctl_get.3.md0000664000000000000000000001262313615011243017513 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMBLK_CTL_GET, 3) collection: libpmemblk header: PMDK date: pmemblk API version 1.1 ... [comment]: <> (Copyright 2018-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemblk_ctl_get.3 -- man page for libpmemblk CTL) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[CTL NAMESPACE](#ctl-namespace)
[CTL EXTERNAL CONFIGURATION](#ctl-external-configuration)
[SEE ALSO](#see-also)
# NAME # _UW(pmemblk_ctl_get), _UW(pmemblk_ctl_set), _UW(pmemblk_ctl_exec) - Query and modify libpmemblk internal behavior (EXPERIMENTAL) # SYNOPSIS # ```c #include _UWFUNCR2(int, pmemblk_ctl_get, PMEMblkpool *pbp, *name, void *arg, =q= (EXPERIMENTAL)=e=) _UWFUNCR2(int, pmemblk_ctl_set, PMEMblkpool *pbp, *name, void *arg, =q= (EXPERIMENTAL)=e=) _UWFUNCR2(int, pmemblk_ctl_exec, PMEMblkpool *pbp, *name, void *arg, =q= (EXPERIMENTAL)=e=) ``` _UNICODE() # DESCRIPTION # The _UW(pmemblk_ctl_get), _UW(pmemblk_ctl_set) and _UW(pmemblk_ctl_exec) functions provide a uniform interface for querying and modifying the internal behavior of **libpmemblk**(7) through the control (CTL) namespace. The *name* argument specifies an entry point as defined in the CTL namespace specification. The entry point description specifies whether the extra *arg* is required. Those two parameters together create a CTL query. The functions and the entry points are thread-safe unless indicated otherwise below. If there are special conditions for calling an entry point, they are explicitly stated in its description. The functions propagate the return value of the entry point. If either *name* or *arg* is invalid, -1 is returned. If the provided ctl query is valid, the CTL functions will always return 0 on success and -1 on failure, unless otherwise specified in the entry point description. See more in **pmem_ctl**(5) man page. # CTL NAMESPACE # prefault.at_create | rw | global | int | int | - | boolean If set, every page of the pool will be touched and written to when the pool is created, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the _UW(pmemblk_create) function. Always returns 0. prefault.at_open | rw | global | int | int | - | boolean If set, every page of the pool will be touched and written to when the pool is opened, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the _UW(pmemblk_open) function. Always returns 0. sds.at_create | rw | global | int | int | - | boolean If set, force-enables or force-disables SDS feature during pool creation. Affects only the _UW(pmemblk_create) function. See **pmempool_feature_query**(3) for information about SDS (SHUTDOWN_STATE) feature. Always returns 0. copy_on_write.at_open | rw | global | int | int | - | boolean If set, pool is mapped in such a way that modifications don't reach the underlying medium. From the user's perspective this means that when the pool is closed all changes are reverted. This feature is not supported for pools located on Device DAX. Always returns 0. # CTL EXTERNAL CONFIGURATION # In addition to direct function call, each write entry point can also be set using two alternative methods. The first method is to load a configuration directly from the **PMEMBLK_CONF** environment variable. The second method of loading an external configuration is to set the **PMEMBLK_CONF_FILE** environment variable to point to a file that contains a sequence of ctl queries. See more in **pmem_ctl**(5) man page. # SEE ALSO # **libpmemblk**(7), **pmem_ctl**(5) and **** pmdk-1.8/doc/libpmemblk/pmemblk_read.30000644000000000000000000000567013615011416016412 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMBLK_READ" "3" "2020-01-31" "PMDK - pmemblk API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemblk_read\f[](), \f[B]pmemblk_write\f[]() \- read or write a block from a block memory pool .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmemblk_read(PMEMblkpool\ *pbp,\ void\ *buf,\ long\ long\ blockno); int\ pmemblk_write(PMEMblkpool\ *pbp,\ const\ void\ *buf,\ long\ long\ blockno); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemblk_read\f[]() function reads the block with block number \f[I]blockno\f[] from memory pool \f[I]pbp\f[] into the buffer \f[I]buf\f[]. Reading a block that has never been written by \f[B]pmemblk_write\f[]() will return a block of zeroes. .PP The \f[B]pmemblk_write\f[]() function writes a block from \f[I]buf\f[] to block number \f[I]blockno\f[] in the memory pool \f[I]pbp\f[]. The write is atomic with respect to other reads and writes. In addition, the write cannot be torn by program failure or system crash; on recovery the block is guaranteed to contain either the old data or the new data, never a mixture of both. .SH RETURN VALUE .PP On success, the \f[B]pmemblk_read\f[]() and \f[B]pmemblk_write\f[]() functions return 0. On error, they return \-1 and set \f[I]errno\f[] appropriately. .SH SEE ALSO .PP \f[B]libpmemblk\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemblk/pmemblk_set_zero.3.md0000664000000000000000000000635713615011243017733 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMBLK_SET_ZERO, 3) collection: libpmemblk header: PMDK date: pmemblk API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemblk_set_zero.3 -- man page for block management functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemblk_set_zero**(), **pmemblk_set_error**() - block management functions # SYNOPSIS # ```c #include int pmemblk_set_zero(PMEMblkpool *pbp, long long blockno); int pmemblk_set_error(PMEMblkpool *pbp, long long blockno); ``` # DESCRIPTION # The **pmemblk_set_zero**() function writes zeros to block number *blockno* in persistent memory resident array of blocks *pbp*. Using this function is faster than actually writing a block of zeros since **libpmemblk**(7) uses metadata to indicate the block should read back as zero. The **pmemblk_set_error**() function sets the error state for block number *blockno* in persistent memory resident array of blocks *pbp*. A block in the error state returns *errno* **EIO** when read. Writing the block clears the error state and returns the block to normal use. # RETURN VALUE # On success, **pmemblk_set_zero**() and **pmemblk_set_error**() return 0. On error, they return -1 and set *errno* appropriately. # SEE ALSO # **libpmemblk**(7) and **** pmdk-1.8/doc/libpmemblk/pmemblk_set_funcs.30000664000000000000000000000002613615011243017456 0ustar rootroot.so man7/libpmemblk.7 pmdk-1.8/doc/libpmemblk/pmemblk_bsize.30000644000000000000000000000563413615011416016613 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMBLK_BSIZE" "3" "2020-01-31" "PMDK - pmemblk API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemblk_bsize\f[](), \f[B]pmemblk_nblock\f[]() \- check number of available blocks or usable space in block memory pool .SH SYNOPSIS .IP .nf \f[C] #include\ size_t\ pmemblk_bsize(PMEMblkpool\ *pbp); size_t\ pmemblk_nblock(PMEMblkpool\ *pbp); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemblk_bsize\f[]() function returns the block size of the specified block memory pool, that is, the value which was passed as \f[I]bsize\f[] to \f[B]pmemblk_create\f[](). \f[I]pbp\f[] must be a block memory pool handle as returned by \f[B]pmemblk_open\f[](3) or \f[B]pmemblk_create\f[](3). .PP The \f[B]pmemblk_nblock\f[]() function returns the usable space in the block memory pool. \f[I]pbp\f[] must be a block memory pool handle as returned by \f[B]pmemblk_open\f[](3) or \f[B]pmemblk_create\f[](3). .SH RETURN VALUE .PP The \f[B]pmemblk_bsize\f[]() function returns the block size of the specified block memory pool. .PP The \f[B]pmemblk_nblock\f[]() function returns the usable space in the block memory pool, expressed as the number of blocks available. .SH SEE ALSO .PP \f[B]pmemblk_create\f[](3), \f[B]pmemblk_open\f[](3), \f[B]libpmemblk\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemblk/pmemblk_ctl_set.30000664000000000000000000000002613615011243017122 0ustar rootroot.so pmemblk_ctl_get.3 pmdk-1.8/doc/libpmemblk/pmemblk_close.30000664000000000000000000000002513615011243016571 0ustar rootroot.so pmemblk_create.3 pmdk-1.8/doc/libpmemblk/pmemblk_ctl_get.30000644000000000000000000001206613615011416017115 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMBLK_CTL_GET" "3" "2020-01-31" "PMDK - pmemblk API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2018-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemblk_ctl_get\f[](), \f[B]pmemblk_ctl_set\f[](), \f[B]pmemblk_ctl_exec\f[]() \- Query and modify libpmemblk internal behavior (EXPERIMENTAL) .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmemblk_ctl_get(PMEMblkpool\ *pbp,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) int\ pmemblk_ctl_set(PMEMblkpool\ *pbp,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) int\ pmemblk_ctl_exec(PMEMblkpool\ *pbp,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemblk_ctl_get\f[](), \f[B]pmemblk_ctl_set\f[]() and \f[B]pmemblk_ctl_exec\f[]() functions provide a uniform interface for querying and modifying the internal behavior of \f[B]libpmemblk\f[](7) through the control (CTL) namespace. .PP The \f[I]name\f[] argument specifies an entry point as defined in the CTL namespace specification. The entry point description specifies whether the extra \f[I]arg\f[] is required. Those two parameters together create a CTL query. The functions and the entry points are thread\-safe unless indicated otherwise below. If there are special conditions for calling an entry point, they are explicitly stated in its description. The functions propagate the return value of the entry point. If either \f[I]name\f[] or \f[I]arg\f[] is invalid, \-1 is returned. .PP If the provided ctl query is valid, the CTL functions will always return 0 on success and \-1 on failure, unless otherwise specified in the entry point description. .PP See more in \f[B]pmem_ctl\f[](5) man page. .SH CTL NAMESPACE .PP prefault.at_create | rw | global | int | int | \- | boolean .PP If set, every page of the pool will be touched and written to when the pool is created, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the \f[B]pmemblk_create\f[]() function. .PP Always returns 0. .PP prefault.at_open | rw | global | int | int | \- | boolean .PP If set, every page of the pool will be touched and written to when the pool is opened, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the \f[B]pmemblk_open\f[]() function. .PP Always returns 0. .PP sds.at_create | rw | global | int | int | \- | boolean .PP If set, force\-enables or force\-disables SDS feature during pool creation. Affects only the \f[B]pmemblk_create\f[]() function. See \f[B]pmempool_feature_query\f[](3) for information about SDS (SHUTDOWN_STATE) feature. .PP Always returns 0. .PP copy_on_write.at_open | rw | global | int | int | \- | boolean .PP If set, pool is mapped in such a way that modifications don't reach the underlying medium. From the user's perspective this means that when the pool is closed all changes are reverted. This feature is not supported for pools located on Device DAX. .PP Always returns 0. .SH CTL EXTERNAL CONFIGURATION .PP In addition to direct function call, each write entry point can also be set using two alternative methods. .PP The first method is to load a configuration directly from the \f[B]PMEMBLK_CONF\f[] environment variable. .PP The second method of loading an external configuration is to set the \f[B]PMEMBLK_CONF_FILE\f[] environment variable to point to a file that contains a sequence of ctl queries. .PP See more in \f[B]pmem_ctl\f[](5) man page. .SH SEE ALSO .PP \f[B]libpmemblk\f[](7), \f[B]pmem_ctl\f[](5) and \f[B]\f[] pmdk-1.8/doc/libpmemblk/libpmemblk.7.md0000664000000000000000000003142613615011243016507 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(LIBPMEMBLK, 7) collection: libpmemblk header: PMDK date: pmemblk API version 1.1 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (libpmemblk.7 -- man page for libpmemblk) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[LIBRARY API VERSIONING](#library-api-versioning-1)
[MANAGING LIBRARY BEHAVIOR](#managing-library-behavior-1)
[DEBUGGING AND ERROR HANDLING](#debugging-and-error-handling)
[EXAMPLE](#example)
[BUGS](#bugs)
[ACKNOWLEDGEMENTS](#acknowledgements)
[SEE ALSO](#see-also)
# NAME # **libpmemblk** - persistent memory resident array of blocks # SYNOPSIS # ```c #include cc ... -lpmemblk -lpmem ``` _UNICODE() ##### Library API versioning: ##### ```c _UWFUNC(pmemblk_check_version, =q= unsigned major_required, unsigned minor_required=e=) ``` ##### Managing library behavior: ##### ```c void pmemblk_set_funcs( void *(*malloc_func)(size_t size), void (*free_func)(void *ptr), void *(*realloc_func)(void *ptr, size_t size), char *(*strdup_func)(const char *s)); ``` ##### Error handling: ##### ```c _UWFUNC(pmemblk_errormsg, void) ``` ##### Other library functions: ##### A description of other **libpmemblk** functions can be found on the following manual pages: **pmemblk_bsize**(3), **pmemblk_create**(3), **pmemblk_ctl_exec**(3), **pmemblk_ctl_get**(3), **pmemblk_ctl_set**(3), **pmemblk_read**(3), **pmemblk_set_zero**(3), # DESCRIPTION # **libpmemblk** provides an array of blocks in *persistent memory* (pmem) such that updates to a single block are atomic. This library is intended for applications using direct access storage (DAX), which is storage that supports load/store access without paging blocks from a block storage device. Some types of *non-volatile memory DIMMs* (NVDIMMs) provide this type of byte addressable access to storage. A *persistent memory aware file system* is typically used to expose the direct access to applications. Memory mapping a file from this type of file system results in the load/store, non-paged access to pmem. **libpmemblk** builds on this type of memory mapped file. This library is for applications that need a potentially large array of blocks, all the same size, where any given block is updated atomically (the update cannot be *torn* by program interruption such as power failures). This library builds on the low-level pmem support provided by **libpmem**(7), handling the transactional update of the blocks, flushing to persistence, and recovery for the application. **libpmemblk** is one of a collection of persistent memory libraries available, the others are: + **libpmemobj**(7), a general use persistent memory API, providing memory allocation and transactional operations on variable-sized objects. + **libpmemlog**(7), providing a pmem-resident log file. + **libpmem**(7), low-level persistent memory support. Under normal usage, **libpmemblk** will never print messages or intentionally cause the process to exit. The only exception to this is the debugging information, when enabled, as described under **DEBUGGING AND ERROR HANDLING** below. To use the atomic block arrays supplied by **libpmemblk**, a *memory pool* is first created using the _UW(pmemblk_create) function described in **pmemblk_create**(3). The other **libpmemblk** functions operate on the resulting block memory pool using the opaque handle, of type *PMEMblkpool\**, that is returned by _UW(pmemblk_create) or _UW(pmemblk_open). Internally, **libpmemblk** will use either **pmem_persist**(3) or **msync**(2) when it needs to flush changes, depending on whether the memory pool appears to be persistent memory or a regular file (see the **pmem_is_pmem**(3) function in **libpmem**(7) for more information). There is no need for applications to flush changes directly when using the block memory API provided by **libpmemblk**. # CAVEATS # **libpmemblk** relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. **dlclose**(3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. # LIBRARY API VERSIONING # This section describes how the library API is versioned, allowing applications to work with an evolving API. The _UW(pmemblk_check_version) function is used to determine whether the installed **libpmemblk** supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile-time version information, supplied by defines in **\**, like this: ```c reason = _U(pmemblk_check_version)(PMEMBLK_MAJOR_VERSION, PMEMBLK_MINOR_VERSION); if (reason != NULL) { /* version check failed, reason string tells you why */ } ``` Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text *introduced in version x.y* in the section of this manual describing the feature. When the version check performed by _UW(pmemblk_check_version) is successful, the return value is NULL. Otherwise the return value is a static string describing the reason for failing the version check. The string returned by _UW(pmemblk_check_version) must not be modified or freed. # MANAGING LIBRARY BEHAVIOR # The **pmemblk_set_funcs**() function allows an application to override memory allocation calls used internally by **libpmemblk**. Passing in NULL for any of the handlers will cause the **libpmemblk** default function to be used. The library does not make heavy use of the system malloc functions, but it does allocate approximately 4-8 kilobytes for each memory pool in use. # DEBUGGING AND ERROR HANDLING # The _UW(pmemblk_errormsg) function returns a pointer to a static buffer containing the last error message logged for the current thread. If *errno* was set, the error message may include a description of the corresponding error code, as returned by **strerror**(3). The error message buffer is thread-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a **libpmemblk** function indicated an error, or if *errno* was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. Two versions of **libpmemblk** are typically available on a development system. The normal version, accessed when a program is linked using the **-lpmemblk** option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run-time assertions. If an error is detected in a call to **libpmemblk**, the error message describing the failure may be retrieved with _UW(pmemblk_errormsg) as described above. A second version of **libpmemblk**, accessed when a program uses the libraries under _DEBUGLIBPATH(), contains run-time assertions and trace points. The typical way to access the debug version is to set the **LD_LIBRARY_PATH** environment variable to _LDLIBPATH(). Debugging output is controlled using the following environment variables. These variables have no effect on the non-debug version of the library. + **PMEMBLK_LOG_LEVEL** The value of **PMEMBLK_LOG_LEVEL** enables trace points in the debug version of the library, as follows: + **0** - This is the default level when **PMEMBLK_LOG_LEVEL** is not set. No log messages are emitted at this level. + **1** - Additional details on any errors detected are logged, in addition to returning the *errno*-based errors as usual. The same information may be retrieved using _UW(pmemblk_errormsg). + **2** - A trace of basic operations is logged. + **3** - Enables a very verbose amount of function call tracing in the library. + **4** - Enables voluminous and fairly obscure tracing information that is likely only useful to the **libpmemblk** developers. Unless **PMEMBLK_LOG_FILE** is set, debugging output is written to *stderr*. + **PMEMBLK_LOG_FILE** Specifies the name of a file where all logging information should be written. If the last character in the name is "-", the *PID* of the current process will be appended to the file name when the log file is created. If **PMEMBLK_LOG_FILE** is not set, the logging output is written to *stderr*. See also **libpmem**(7) for information on other environment variables that may affect **libpmemblk** behavior. # EXAMPLE # The following example illustrates how the **libpmemblk** API is used. ```c #include #include #include #include #include #include /* size of the pmemblk pool -- 1 GB */ #define POOL_SIZE ((size_t)(1 << 30)) /* size of each element in the pmem pool */ #define ELEMENT_SIZE 1024 int main(int argc, char *argv[]) { const char path[] = "/pmem-fs/myfile"; PMEMblkpool *pbp; size_t nelements; char buf[ELEMENT_SIZE]; /* create the pmemblk pool or open it if it already exists */ pbp = _U(pmemblk_create)(path, ELEMENT_SIZE, POOL_SIZE, 0666); if (pbp == NULL) pbp = _U(pmemblk_open)(path, ELEMENT_SIZE); if (pbp == NULL) { perror(path); exit(1); } /* how many elements fit into the file? */ nelements = pmemblk_nblock(pbp); printf("file holds %zu elements", nelements); /* store a block at index 5 */ strcpy(buf, "hello, world"); if (pmemblk_write(pbp, buf, 5) < 0) { perror("pmemblk_write"); exit(1); } /* read the block at index 10 (reads as zeros initially) */ if (pmemblk_read(pbp, buf, 10) < 0) { perror("pmemblk_read"); exit(1); } /* zero out the block at index 5 */ if (pmemblk_set_zero(pbp, 5) < 0) { perror("pmemblk_set_zero"); exit(1); } /* ... */ pmemblk_close(pbp); } ``` See for more examples using the **libpmemblk** API. # BUGS # Unlike **libpmemobj**(7), data replication is not supported in **libpmemblk**. Thus, specifying replica sections in pool set files is not allowed. # ACKNOWLEDGEMENTS # **libpmemblk** builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: # SEE ALSO # **msync**(2), **dlclose**(3), **pmemblk_bsize**(3), **pmemblk_create**(3), **pmemblk_ctl_exec**(3), **pmemblk_ctl_get**(3), **pmemblk_ctl_set**(3), **pmemblk_read**(3), **pmemblk_set_zero**(3), **pmem_is_pmem**(3), **pmem_persist**(3), **strerror**(3), **libpmem**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemblk/pmemblk_bsize.3.md0000664000000000000000000000647613615011243017217 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMBLK_BSIZE, 3) collection: libpmemblk header: PMDK date: pmemblk API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemblk_bsize.3 -- man page for functions that check number of available blocks or usable space in block memory pool) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemblk_bsize**(), **pmemblk_nblock**() - check number of available blocks or usable space in block memory pool # SYNOPSIS # ```c #include size_t pmemblk_bsize(PMEMblkpool *pbp); size_t pmemblk_nblock(PMEMblkpool *pbp); ``` # DESCRIPTION # The **pmemblk_bsize**() function returns the block size of the specified block memory pool, that is, the value which was passed as *bsize* to _UW(pmemblk_create). *pbp* must be a block memory pool handle as returned by **pmemblk_open**(3) or **pmemblk_create**(3). The **pmemblk_nblock**() function returns the usable space in the block memory pool. *pbp* must be a block memory pool handle as returned by **pmemblk_open**(3) or **pmemblk_create**(3). # RETURN VALUE # The **pmemblk_bsize**() function returns the block size of the specified block memory pool. The **pmemblk_nblock**() function returns the usable space in the block memory pool, expressed as the number of blocks available. # SEE ALSO # **pmemblk_create**(3), **pmemblk_open**(3), **libpmemblk**(7) and **** pmdk-1.8/doc/libpmemblk/pmemblk_read.3.md0000664000000000000000000000644213615011243017007 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMBLK_READ, 3) collection: libpmemblk header: PMDK date: pmemblk API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemblk_read.3 -- man page for libpmemblk read and write functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemblk_read**(), **pmemblk_write**() - read or write a block from a block memory pool # SYNOPSIS # ```c #include int pmemblk_read(PMEMblkpool *pbp, void *buf, long long blockno); int pmemblk_write(PMEMblkpool *pbp, const void *buf, long long blockno); ``` # DESCRIPTION # The **pmemblk_read**() function reads the block with block number *blockno* from memory pool *pbp* into the buffer *buf*. Reading a block that has never been written by **pmemblk_write**() will return a block of zeroes. The **pmemblk_write**() function writes a block from *buf* to block number *blockno* in the memory pool *pbp*. The write is atomic with respect to other reads and writes. In addition, the write cannot be torn by program failure or system crash; on recovery the block is guaranteed to contain either the old data or the new data, never a mixture of both. # RETURN VALUE # On success, the **pmemblk_read**() and **pmemblk_write**() functions return 0. On error, they return -1 and set *errno* appropriately. # SEE ALSO # **libpmemblk**(7) and **** pmdk-1.8/doc/libpmemblk/pmemblk_create.3.md0000664000000000000000000002121113615011243017326 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMBLK_CREATE, 3) collection: libpmemblk header: PMDK date: pmemblk API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemblk_create.3 -- man page for libpmemblk create, open, close and validate functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # _UW(pmemblk_create), _UW(pmemblk_open), **pmemblk_close**(), _UW(pmemblk_check) - create, open, close and validate block pool # SYNOPSIS # ```c #include _UWFUNCR1(PMEMblkpool, *pmemblk_create, *path, =q=size_t bsize, size_t poolsize, mode_t mode=e=) _UWFUNCR1(PMEMblkpool, *pmemblk_open, *path, size_t bsize) void pmemblk_close(PMEMblkpool *pbp); _UWFUNCR1(int, pmemblk_check, *path, size_t bsize) ``` _UNICODE() # DESCRIPTION # The _UW(pmemblk_create) function creates a block memory pool with the given total *poolsize*, divided into as many elements of size *bsize* as will fit in the pool. Since the transactional nature of a block memory pool requires some space overhead in the memory pool, the resulting number of available blocks is less than *poolsize*/*bsize*, and is made available to the caller via the **pmemblk_nblock**(3) function. Given the specifics of the implementation, the number of available blocks for the user cannot be less than 256. This translates to at least 512 internal blocks. *path* specifies the name of the memory pool file to be created. *mode* specifies the permissions to use when creating the file, as described by **creat**(2). The memory pool file is fully allocated to the size *poolsize* using **posix_fallocate**(3). The caller may choose to take responsibility for creating the memory pool file by creating it before calling _UW(pmemblk_create), and then specifying *poolsize* as zero. In this case _UW(pmemblk_create) will take the pool size from the size of the existing file, and will verify that the file appears to be empty by searching for any non-zero data in the pool header at the beginning of the file. The net pool size of a pool file is equal to the file size. The minimum net pool size allowed by the library for a block pool is defined in **\** as **PMEMBLK_MIN_POOL**. *bsize* can be any non-zero value; however, **libpmemblk** will silently round up the given size to **PMEMBLK_MIN_BLK**, as defined in **\**. Depending on the configuration of the system, the available non-volatile memory space may be divided into multiple memory devices. In such case, the maximum size of the pmemblk memory pool could be limited by the capacity of a single memory device. **libpmemblk**(7) allows building a persistent memory resident array spanning multiple memory devices by creation of persistent memory pools consisting of multiple files, where each part of such a *pool set* may be stored on a different memory device or pmem-aware filesystem. Creation of all the parts of the pool set can be done with _UW(pmemblk_create); however, the recommended method for creating pool sets is by using the **pmempool**(1) utility. When creating a pool set consisting of multiple files, the *path* argument passed to _UW(pmemblk_create) must point to the special *set* file that defines the pool layout and the location of all the parts of the pool set. The *poolsize* argument must be 0. The meaning of the *mode* argument does not change, except that the same *mode* is used for creation of all the parts of the pool set. For more information on pool set format, see **poolset**(5). The _UW(pmemblk_open) function opens an existing block memory pool. As with _UW(pmemblk_create), *path* must identify either an existing block memory pool file, or the *set* file used to create a pool set. The application must have permission to open the file and memory map the file or pool set with read/write permissions. If *bsize* is non-zero, _UW(pmemblk_open) will verify that the given block size matches the block size used when the pool was created. Otherwise, _UW(pmemblk_open) will open the pool without verifying the block size. The *bsize* can be determined using the **pmemblk_bsize**(3) function. Be aware that if the pool contains bad blocks inside, opening can be aborted by the SIGBUS signal, because currently the pool is not checked against bad blocks during opening. It can be turned on by setting the CHECK_BAD_BLOCKS compat feature. For details see description of this feature in **pmempool-feature**(1). The **pmemblk_close**() function closes the memory pool indicated by *pbp* and deletes the memory pool handle. The block memory pool itself lives on in the file that contains it and may be re-opened at a later time using _UW(pmemblk_open) as described above. The _UW(pmemblk_check) function performs a consistency check of the file indicated by *path*, and returns 1 if the memory pool is found to be consistent. If the pool is found not to be consistent, further use of the file with **libpmemblk** will result in undefined behavior. The debug version of **libpmemblk** will provide additional details on inconsistencies when **PMEMBLK_LOG_LEVEL** is at least 1, as described in the **DEBUGGING AND ERROR HANDLING** section in **libpmemblk**(7). _UW(pmemblk_check) opens the given *path* read-only so it never makes any changes to the file. This function is not supported on Device DAX. # RETURN VALUE # On success, _UW(pmemblk_create) returns a *PMEMblkpool\** handle to the block memory pool. On error, it returns NULL and sets *errno* appropriately. On success, _UW(pmemblk_open) returns a *PMEMblkpool\** handle that can be used with most of the functions in **libpmemblk**(7). On error, it returns NULL and sets *errno* appropriately. Possible errors include: + failure to open *path* + *path* specifies a *set* file and any of the pool set files cannot be opened + *path* specifies a *set* file and the actual size of any file does not match the corresponding part size defined in the *set* file + *bsize* is non-zero and does not match the block size given when the pool was created. *errno* is set to **EINVAL** in this case. The **pmemblk_close**() function returns no value. _UW(pmemblk_check) returns 1 if the memory pool is found to be consistent. If the check is successfully performed but the pool is found to be inconsistent, _UW(pmemblk_check) returns 0. This includes the case where *bsize* is non-zero and does not match the block size given when the pool was created. If the consistency check cannot be performed, _UW(pmemblk_check) returns -1 and sets *errno* appropriately. # CAVEATS # Not all file systems support **posix_fallocate**(3). _UW(pmemblk_create) will fail if the underlying file system does not support **posix_fallocate**(3). # SEE ALSO # **pmempool**(1), **creat**(2), **pmemblk_nblock**(3), **posix_fallocate**(3), **poolset**(5), **libpmemblk**(7) and **** pmdk-1.8/doc/daxio/0000775000000000000000000000000013615011420012657 5ustar rootrootpmdk-1.8/doc/daxio/daxio.10000644000000000000000000001075713615011420014055 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "DAXIO" "1" "2020-01-31" "PMDK - daxio version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]daxio\f[] \- Perform I/O on Device DAX devices or zero a Device DAX device .SH SYNOPSIS .IP .nf \f[C] $\ daxio\ [] \f[] .fi .SH DESCRIPTION .PP The daxio utility performs I/O on Device DAX devices or zero a Device DAX device. Since the standard I/O APIs (read/write) cannot be used with Device DAX, data transfer is performed on a memory\-mapped device. The \f[B]daxio\f[] may be used to dump Device DAX data to a file, restore data from a backup copy, move/copy data to another device or to erase data from a device. .PP There must be at least one Device DAX device involved either as the input or output. If input or output is not specified, it will default to stdin or stdout respectively. .PP No length specified will default to input file/device length or to the output file/device length, if input is a special char file or stdin. .PP For a Device DAX device, \f[B]daxio\f[] will attempt to clear bad blocks within the range of writes before performing the I/O (it can be turned off using the `\[en]clear\-bad\-blocks=no' option). .SH OPTIONS .PP \f[C]\-i,\ \-\-input\f[] Input device or file to read from. .PP \f[C]\-o,\ \-\-output\f[] Output device or file to write to. .PP \f[C]\-z,\ \-\-zero\f[] Zero the output device for \f[I]len\f[] size, or the entire device if no length was provided. The output device must be a Device DAX device. .PP \f[C]\-b,\ \-\-clear\-bad\-blocks=\f[] Clear bad blocks within the range of writes before performing the I/O (default: yes). .PP \f[C]\-l,\ \-\-len\f[] The length in bytes to perform the I/O. To make passing in size easier for kibi, mebi, gibi, and tebi bytes, \f[I]len\f[] may include unit suffix. The \f[I]len\f[] format must be compliant with the format specified in IEC 80000\-13, IEEE 1541 or the Metric Interchange Format. These standards accept SI units with obligatory B \- kB, MB, GB, \&... (multiplier by 1000) suffixes, and IEC units with optional \[lq]iB\[rq] \- KiB, MiB, GiB, \&..., K, M, G, \&... (multiplier by 1024) suffixes. .PP \f[C]\-s,\ \-\-seek\f[] The number of bytes to skip over on the output before performing a write. The same suffixes are accepted as for \f[I]len\f[]. .PP \f[C]\-k,\ \-\-skip\f[] The number of bytes to skip over on the input before performing a read. The same suffixes are accepted as for \f[I]len\f[]. .PP \f[C]\-V,\ \-\-version\f[] .PP Prints the version of \f[B]daxio\f[]. .PP \f[C]\-h,\ \-\-help\f[] .PP Prints synopsis and list of options. .SH EXAMPLE .IP .nf \f[C] #\ daxio\ \-\-zero\ /dev/dax1.0 #\ daxio\ \-\-input=/dev/dax1.0\ \-\-output=/home/myfile\ \-\-len=2M\ \-\-seek=4096 #\ cat\ /dev/zero\ |\ daxio\ \-\-output=/dev/dax1.0 #\ daxio\ \-\-input=/dev/zero\ \-\-output=/dev/dax1.0\ \-\-skip=4096 \f[] .fi .SH SEE ALSO .PP \f[B]daxctl\f[](1), \f[B]ndctl\f[](1) and \f[B]\f[] pmdk-1.8/doc/daxio/.gitignore0000664000000000000000000000001013615011243014641 0ustar rootrootdaxio.1 pmdk-1.8/doc/daxio/daxio.1.md0000664000000000000000000001123213615011243014446 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(DAXIO, 1) collection: daxio header: PMDK date: daxio version 1.4 ... [comment]: <> (Copyright 2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (daxio.1 -- man page for daxio) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[OPTIONS](#options)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **daxio** - Perform I/O on Device DAX devices or zero a Device DAX device # SYNOPSIS # ``` $ daxio [] ``` # DESCRIPTION # The daxio utility performs I/O on Device DAX devices or zero a Device DAX device. Since the standard I/O APIs (read/write) cannot be used with Device DAX, data transfer is performed on a memory-mapped device. The **daxio** may be used to dump Device DAX data to a file, restore data from a backup copy, move/copy data to another device or to erase data from a device. There must be at least one Device DAX device involved either as the input or output. If input or output is not specified, it will default to stdin or stdout respectively. No length specified will default to input file/device length or to the output file/device length, if input is a special char file or stdin. For a Device DAX device, **daxio** will attempt to clear bad blocks within the range of writes before performing the I/O (it can be turned off using the '--clear-bad-blocks=no' option). # OPTIONS # `-i, --input` Input device or file to read from. `-o, --output` Output device or file to write to. `-z, --zero` Zero the output device for *len* size, or the entire device if no length was provided. The output device must be a Device DAX device. `-b, --clear-bad-blocks=` Clear bad blocks within the range of writes before performing the I/O (default: yes). `-l, --len` The length in bytes to perform the I/O. To make passing in size easier for kibi, mebi, gibi, and tebi bytes, *len* may include unit suffix. The *len* format must be compliant with the format specified in IEC 80000-13, IEEE 1541 or the Metric Interchange Format. These standards accept SI units with obligatory B - kB, MB, GB, ... (multiplier by 1000) suffixes, and IEC units with optional "iB" - KiB, MiB, GiB, ..., K, M, G, ... (multiplier by 1024) suffixes. `-s, --seek` The number of bytes to skip over on the output before performing a write. The same suffixes are accepted as for *len*. `-k, --skip` The number of bytes to skip over on the input before performing a read. The same suffixes are accepted as for *len*. `-V, --version` Prints the version of **daxio**. `-h, --help` Prints synopsis and list of options. # EXAMPLE # ``` # daxio --zero /dev/dax1.0 # daxio --input=/dev/dax1.0 --output=/home/myfile --len=2M --seek=4096 # cat /dev/zero | daxio --output=/dev/dax1.0 # daxio --input=/dev/zero --output=/dev/dax1.0 --skip=4096 ``` # SEE ALSO # **daxctl**(1), **ndctl**(1) and **** pmdk-1.8/doc/librpmem/0000775000000000000000000000000013615011420013362 5ustar rootrootpmdk-1.8/doc/librpmem/librpmem.70000644000000000000000000004477013615011416015300 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "LIBRPMEM" "7" "2020-01-31" "PMDK - rpmem API version 1.3" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]librpmem\f[] \- remote persistent memory support library (EXPERIMENTAL) .SH SYNOPSIS .IP .nf \f[C] #include\ cc\ ...\ \-lrpmem \f[] .fi .SS Library API versioning: .IP .nf \f[C] const\ char\ *rpmem_check_version( \ \ \ \ unsigned\ major_required, \ \ \ \ unsigned\ minor_required); \f[] .fi .SS Error handling: .IP .nf \f[C] const\ char\ *rpmem_errormsg(void); \f[] .fi .SS Other library functions: .PP A description of other \f[B]librpmem\f[] functions can be found on the following manual pages: .IP \[bu] 2 \f[B]rpmem_create\f[](3), \f[B]rpmem_persist\f[](3) .SH DESCRIPTION .PP \f[B]librpmem\f[] provides low\-level support for remote access to \f[I]persistent memory\f[] (pmem) utilizing RDMA\-capable RNICs. The library can be used to remotely replicate a memory region over the RDMA protocol. It utilizes an appropriate persistency mechanism based on the remote node's platform capabilities. \f[B]librpmem\f[] utilizes the \f[B]ssh\f[](1) client to authenticate a user on the remote node, and for encryption of the connection's out\-of\-band configuration data. See \f[B]SSH\f[], below, for details. .PP The maximum replicated memory region size can not be bigger than the maximum locked\-in\-memory address space limit. See \f[B]memlock\f[] in \f[B]limits.conf\f[](5) for more details. .PP This library is for applications that use remote persistent memory directly, without the help of any library\-supplied transactions or memory allocation. Higher\-level libraries that build on \f[B]libpmem\f[](7) are available and are recommended for most applications, see: .IP \[bu] 2 \f[B]libpmemobj\f[](7), a general use persistent memory API, providing memory allocation and transactional operations on variable\-sized objects. .SH TARGET NODE ADDRESS FORMAT .IP .nf \f[C] [\@][:] \f[] .fi .PP The target node address is described by the \f[I]hostname\f[] which the client connects to, with an optional \f[I]user\f[] name. The user must be authorized to authenticate to the remote machine without querying for password/passphrase. The optional \f[I]port\f[] number is used to establish the SSH connection. The default port number is 22. .SH REMOTE POOL ATTRIBUTES .PP The \f[I]rpmem_pool_attr\f[] structure describes a remote pool and is stored in remote pool's metadata. This structure must be passed to the \f[B]rpmem_create\f[](3) function by caller when creating a pool on remote node. When opening the pool using \f[B]rpmem_open\f[](3) function the appropriate fields are read from pool's metadata and returned back to the caller. .IP .nf \f[C] #define\ RPMEM_POOL_HDR_SIG_LEN\ \ \ \ 8 #define\ RPMEM_POOL_HDR_UUID_LEN\ \ \ 16 #define\ RPMEM_POOL_USER_FLAGS_LEN\ 16 struct\ rpmem_pool_attr\ { \ \ \ \ char\ signature[RPMEM_POOL_HDR_SIG_LEN]; \ \ \ \ uint32_t\ major; \ \ \ \ uint32_t\ compat_features; \ \ \ \ uint32_t\ incompat_features; \ \ \ \ uint32_t\ ro_compat_features; \ \ \ \ unsigned\ char\ poolset_uuid[RPMEM_POOL_HDR_UUID_LEN]; \ \ \ \ unsigned\ char\ uuid[RPMEM_POOL_HDR_UUID_LEN]; \ \ \ \ unsigned\ char\ next_uuid[RPMEM_POOL_HDR_UUID_LEN]; \ \ \ \ unsigned\ char\ prev_uuid[RPMEM_POOL_HDR_UUID_LEN]; \ \ \ \ unsigned\ char\ user_flags[RPMEM_POOL_USER_FLAGS_LEN]; }; \f[] .fi .PP The \f[I]signature\f[] field is an 8\-byte field which describes the pool's on\-media format. .PP The \f[I]major\f[] field is a major version number of the pool's on\-media format. .PP The \f[I]compat_features\f[] field is a mask describing compatibility of pool's on\-media format optional features. .PP The \f[I]incompat_features\f[] field is a mask describing compatibility of pool's on\-media format required features. .PP The \f[I]ro_compat_features\f[] field is a mask describing compatibility of pool's on\-media format features. If these features are not available, the pool shall be opened in read\-only mode. .PP The \f[I]poolset_uuid\f[] field is an UUID of the pool which the remote pool is associated with. .PP The \f[I]uuid\f[] field is an UUID of a first part of the remote pool. This field can be used to connect the remote pool with other pools in a list. .PP The \f[I]next_uuid\f[] and \f[I]prev_uuid\f[] fields are UUIDs of next and previous replicas respectively. These fields can be used to connect the remote pool with other pools in a list. .PP The \f[I]user_flags\f[] field is a 16\-byte user\-defined flags. .SH SSH .PP \f[B]librpmem\f[] utilizes the \f[B]ssh\f[](1) client to login and execute the \f[B]rpmemd\f[](1) process on the remote node. By default, \f[B]ssh\f[](1) is executed with the \f[B]\-4\f[] option, which forces using \f[B]IPv4\f[] addressing. .PP For debugging purposes, both the ssh client and the commands executed on the remote node may be overridden by setting the \f[B]RPMEM_SSH\f[] and \f[B]RPMEM_CMD\f[] environment variables, respectively. See \f[B]ENVIRONMENT\f[] for details. .SH FORK .PP The \f[B]ssh\f[](1) client is executed by \f[B]rpmem_open\f[](3) and \f[B]rpmem_create\f[](3) after forking a child process using \f[B]fork\f[](2). The application must take this into account when using \f[B]wait\f[](2) and \f[B]waitpid\f[](2), which may return the \f[I]PID\f[] of the \f[B]ssh\f[](1) process executed by \f[B]librpmem\f[]. .PP If \f[B]fork\f[](2) support is not enabled in \f[B]libibverbs\f[], \f[B]rpmem_open\f[](3) and \f[B]rpmem_create\f[](3) will fail. By default, \f[B]fabric\f[](7) initializes \f[B]libibverbs\f[] with \f[B]fork\f[](2) support by calling the \f[B]ibv_fork_init\f[](3) function. See \f[B]fi_verbs\f[](7) for more details. .SH CAVEATS .PP \f[B]librpmem\f[] relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. \f[B]dlclose\f[](3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. .PP \f[B]librpmem\f[] registers a pool as a single memory region. A Chelsio T4 and T5 hardware can not handle a memory region greater than or equal to 8GB due to a hardware bug. So \f[I]pool_size\f[] value for \f[B]rpmem_create\f[](3) and \f[B]rpmem_open\f[](3) using this hardware can not be greater than or equal to 8GB. .SH LIBRARY API VERSIONING .PP This section describes how the library API is versioned, allowing applications to work with an evolving API. .PP The \f[B]rpmem_check_version\f[]() function is used to see if the installed \f[B]librpmem\f[] supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile\-time version information, supplied by defines in \f[B]\f[], like this: .IP .nf \f[C] reason\ =\ rpmem_check_version(RPMEM_MAJOR_VERSION, \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ RPMEM_MINOR_VERSION); if\ (reason\ !=\ NULL)\ { \ \ \ \ /*\ version\ check\ failed,\ reason\ string\ tells\ you\ why\ */ } \f[] .fi .PP Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. .PP An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text \f[I]introduced in version x.y\f[] in the section of this manual describing the feature. .PP When the version check performed by \f[B]rpmem_check_version\f[]() is successful, the return value is NULL. Otherwise the return value is a static string describing the reason for failing the version check. The string returned by \f[B]rpmem_check_version\f[]() must not be modified or freed. .SH ENVIRONMENT .PP \f[B]librpmem\f[] can change its default behavior based on the following environment variables. These are largely intended for testing and are not normally required. .IP \[bu] 2 \f[B]RPMEM_SSH\f[]=\f[I]ssh_client\f[] .PP Setting this environment variable overrides the default \f[B]ssh\f[](1) client command name. .IP \[bu] 2 \f[B]RPMEM_CMD\f[]=\f[I]cmd\f[] .PP Setting this environment variable overrides the default command executed on the remote node using either \f[B]ssh\f[](1) or the alternative remote shell command specified by \f[B]RPMEM_SSH\f[]. .PP \f[B]RPMEM_CMD\f[] can contain multiple commands separated by a vertical bar (\f[C]|\f[]). Each consecutive command is executed on the remote node in order read from a pool set file. This environment variable is read when the library is initialized, so \f[B]RPMEM_CMD\f[] must be set prior to application launch (or prior to \f[B]dlopen\f[](3) if \f[B]librpmem\f[] is being dynamically loaded). .IP \[bu] 2 \f[B]RPMEM_ENABLE_SOCKETS\f[]=0|1 .PP Setting this variable to 1 enables using \f[B]fi_sockets\f[](7) provider for in\-band RDMA connection. The \f[I]sockets\f[] provider does not support IPv6. It is required to disable IPv6 system wide if \f[B]RPMEM_ENABLE_SOCKETS\f[] == 1 and \f[I]target\f[] == localhost (or any other loopback interface address) and \f[B]SSH_CONNECTION\f[] variable (see \f[B]ssh\f[](1) for more details) contains IPv6 address after ssh to loopback interface. By default the \f[I]sockets\f[] provider is disabled. .IP \[bu] 2 \f[B]RPMEM_ENABLE_VERBS\f[]=0|1 .PP Setting this variable to 0 disables using \f[B]fi_verbs\f[](7) provider for in\-band RDMA connection. The \f[I]verbs\f[] provider is enabled by default. .IP \[bu] 2 \f[B]RPMEM_MAX_NLANES\f[]=\f[I]num\f[] .PP Limit the maximum number of lanes to \f[I]num\f[]. See \f[B]LANES\f[], in \f[B]rpmem_create\f[](3), for details. .IP \[bu] 2 \f[B]RPMEM_WORK_QUEUE_SIZE\f[]=\f[I]size\f[] .PP Suggest the work queue size. The effective work queue size can be greater than suggested if \f[B]librpmem\f[] requires it or it can be smaller if underlying hardware does not support the suggested size. The work queue size affects the performance of communication to the remote node. \f[B]rpmem_flush\f[](3) operations can be added to the work queue up to the size of this queue. When work queue is full any subsequent call has to wait till the work queue will be drained. \f[B]rpmem_drain\f[](3) and \f[B]rpmem_persist\f[](3) among other things also drain the work queue. .SH DEBUGGING AND ERROR HANDLING .PP If an error is detected during the call to a \f[B]librpmem\f[] function, the application may retrieve an error message describing the reason for the failure from \f[B]rpmem_errormsg\f[](). This function returns a pointer to a static buffer containing the last error message logged for the current thread. If \f[I]errno\f[] was set, the error message may include a description of the corresponding error code as returned by \f[B]strerror\f[](3). The error message buffer is thread\-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a \f[B]librpmem\f[] function indicated an error, or if \f[I]errno\f[] was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. .PP Two versions of \f[B]librpmem\f[] are typically available on a development system. The normal version, accessed when a program is linked using the \f[B]\-lrpmem\f[] option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run\-time assertions. .PP A second version of \f[B]librpmem\f[], accessed when a program uses the libraries under \f[B]/usr/lib/pmdk_debug\f[], contains run\-time assertions and trace points. The typical way to access the debug version is to set the environment variable \f[B]LD_LIBRARY_PATH\f[] to \f[B]/usr/lib/pmdk_debug\f[] or \f[B]/usr/lib64/pmdk_debug\f[], as appropriate. Debugging output is controlled using the following environment variables. These variables have no effect on the non\-debug version of the library. .IP \[bu] 2 \f[B]RPMEM_LOG_LEVEL\f[] .PP The value of \f[B]RPMEM_LOG_LEVEL\f[] enables trace points in the debug version of the library, as follows: .IP \[bu] 2 \f[B]0\f[] \- This is the default level when \f[B]RPMEM_LOG_LEVEL\f[] is not set. No log messages are emitted at this level. .IP \[bu] 2 \f[B]1\f[] \- Additional details on any errors detected are logged (in addition to returning the \f[I]errno\f[]\-based errors as usual). The same information may be retrieved using \f[B]rpmem_errormsg\f[](). .IP \[bu] 2 \f[B]2\f[] \- A trace of basic operations is logged. .IP \[bu] 2 \f[B]3\f[] \- Enables a very verbose amount of function call tracing in the library. .IP \[bu] 2 \f[B]4\f[] \- Enables voluminous and fairly obscure tracing information that is likely only useful to the \f[B]librpmem\f[] developers. .PP Unless \f[B]RPMEM_LOG_FILE\f[] is set, debugging output is written to \f[I]stderr\f[]. .IP \[bu] 2 \f[B]RPMEM_LOG_FILE\f[] .PP Specifies the name of a file where all logging information should be written. If the last character in the name is \[lq]\-\[rq], the \f[I]PID\f[] of the current process will be appended to the file name when the log file is created. If \f[B]RPMEM_LOG_FILE\f[] is not set, logging output is written to \f[I]stderr\f[]. .SH EXAMPLE .PP The following example uses \f[B]librpmem\f[] to create a remote pool on given target node identified by given pool set name. The associated local memory pool is zeroed and the data is made persistent on remote node. Upon success the remote pool is closed. .IP .nf \f[C] #include\ #include\ #include\ #include\ #include\ #include\ #define\ POOL_SIGNATURE\ \ "MANPAGE" #define\ POOL_SIZE\ \ \ (32\ *\ 1024\ *\ 1024) #define\ NLANES\ \ \ \ \ \ 4 #define\ DATA_OFF\ \ \ \ 4096 #define\ DATA_SIZE\ \ \ (POOL_SIZE\ \-\ DATA_OFF) static\ void parse_args(int\ argc,\ char\ *argv[],\ const\ char\ **target,\ const\ char\ **poolset) { \ \ \ \ if\ (argc\ <\ 3)\ { \ \ \ \ \ \ \ \ fprintf(stderr,\ "usage:\\t%s\ \ \\n",\ argv[0]); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ *target\ =\ argv[1]; \ \ \ \ *poolset\ =\ argv[2]; } static\ void\ * alloc_memory() { \ \ \ \ long\ pagesize\ =\ sysconf(_SC_PAGESIZE); \ \ \ \ if\ (pagesize\ <\ 0)\ { \ \ \ \ \ \ \ \ perror("sysconf"); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ /*\ allocate\ a\ page\ size\ aligned\ local\ memory\ pool\ */ \ \ \ \ void\ *mem; \ \ \ \ int\ ret\ =\ posix_memalign(&mem,\ pagesize,\ POOL_SIZE); \ \ \ \ if\ (ret)\ { \ \ \ \ \ \ \ \ fprintf(stderr,\ "posix_memalign:\ %s\\n",\ strerror(ret)); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ assert(mem\ !=\ NULL); \ \ \ \ return\ mem; } int main(int\ argc,\ char\ *argv[]) { \ \ \ \ const\ char\ *target,\ *poolset; \ \ \ \ parse_args(argc,\ argv,\ &target,\ &poolset); \ \ \ \ unsigned\ nlanes\ =\ NLANES; \ \ \ \ void\ *pool\ =\ alloc_memory(); \ \ \ \ int\ ret; \ \ \ \ /*\ fill\ pool_attributes\ */ \ \ \ \ struct\ rpmem_pool_attr\ pool_attr; \ \ \ \ memset(&pool_attr,\ 0,\ sizeof(pool_attr)); \ \ \ \ strncpy(pool_attr.signature,\ POOL_SIGNATURE,\ RPMEM_POOL_HDR_SIG_LEN); \ \ \ \ /*\ create\ a\ remote\ pool\ */ \ \ \ \ RPMEMpool\ *rpp\ =\ rpmem_create(target,\ poolset,\ pool,\ POOL_SIZE, \ \ \ \ \ \ \ \ \ \ \ \ &nlanes,\ &pool_attr); \ \ \ \ if\ (!rpp)\ { \ \ \ \ \ \ \ \ fprintf(stderr,\ "rpmem_create:\ %s\\n",\ rpmem_errormsg()); \ \ \ \ \ \ \ \ return\ 1; \ \ \ \ } \ \ \ \ /*\ store\ data\ on\ local\ pool\ */ \ \ \ \ memset(pool,\ 0,\ POOL_SIZE); \ \ \ \ /*\ make\ local\ data\ persistent\ on\ remote\ node\ */ \ \ \ \ ret\ =\ rpmem_persist(rpp,\ DATA_OFF,\ DATA_SIZE,\ 0,\ 0); \ \ \ \ if\ (ret)\ { \ \ \ \ \ \ \ \ fprintf(stderr,\ "rpmem_persist:\ %s\\n",\ rpmem_errormsg()); \ \ \ \ \ \ \ \ return\ 1; \ \ \ \ } \ \ \ \ /*\ close\ the\ remote\ pool\ */ \ \ \ \ ret\ =\ rpmem_close(rpp); \ \ \ \ if\ (ret)\ { \ \ \ \ \ \ \ \ fprintf(stderr,\ "rpmem_close:\ %s\\n",\ rpmem_errormsg()); \ \ \ \ \ \ \ \ return\ 1; \ \ \ \ } \ \ \ \ free(pool); \ \ \ \ return\ 0; } \f[] .fi .SH NOTE .PP The \f[B]librpmem\f[] API is experimental and may be subject to change in the future. However, using the remote replication in \f[B]libpmemobj\f[](7) is safe and backward compatibility will be preserved. .SH ACKNOWLEDGEMENTS .PP \f[B]librpmem\f[] builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: .SH SEE ALSO .PP \f[B]rpmemd\f[](1), \f[B]ssh\f[](1), \f[B]fork\f[](2), \f[B]dlclose\f[](3), \f[B]dlopen\f[](3), \f[B]ibv_fork_init\f[](3), \f[B]rpmem_create\f[](3), \f[B]rpmem_drain\f[](3), \f[B]rpmem_flush\f[](3), \f[B]rpmem_open\f[](3), \f[B]rpmem_persist\f[](3), \f[B]strerror\f[](3), \f[B]limits.conf\f[](5), \f[B]fabric\f[](7), \f[B]fi_sockets\f[](7), \f[B]fi_verbs\f[](7), \f[B]libpmem\f[](7), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/librpmem/rpmem_check_version.30000664000000000000000000000002413615011243017467 0ustar rootroot.so man7/librpmem.7 pmdk-1.8/doc/librpmem/rpmem_create.3.md0000664000000000000000000002340013615011243016512 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(RPMEM_CREATE, 3) collection: librpmem header: PMDK date: rpmem API version 1.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (rpmem_create.3 -- man page for most commonly used librpmem functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[NOTES](#notes)
[SEE ALSO](#see-also)
# NAME # **rpmem_create**(), **rpmem_open**(), **rpmem_set_attr**(), **rpmem_close**(), **rpmem_remove**() - most commonly used functions for remote access to *persistent memory* # SYNOPSIS # ```c #include RPMEMpool *rpmem_create(const char *target, const char *pool_set_name, void *pool_addr, size_t pool_size, unsigned *nlanes, const struct rpmem_pool_attr *create_attr); RPMEMpool *rpmem_open(const char *target, const char *pool_set_name, void *pool_addr, size_t pool_size, unsigned *nlanes, struct rpmem_pool_attr *open_attr); int rpmem_set_attr(RPMEMpool *rpp, const struct rpmem_pool_attr *attr); int rpmem_close(RPMEMpool *rpp); int rpmem_remove(const char *target, const char *pool_set_name, int flags); ``` # DESCRIPTION # The **rpmem_create**() function creates a remote pool on a given *target* node, using pool *set* file *pool_set_name* to map the remote pool. *pool_set_name* is a relative path in the root config directory on the *target* node. For pool set file format and options see **poolset**(5). *pool_addr* is a pointer to the associated local memory pool with size *pool_size*. Both *pool_addr* and *pool_size* must be aligned to the system's page size (see **sysconf**(3)). The size of the remote pool must be at least *pool_size*. See **REMOTE POOL SIZE**, below, for details. *nlanes* points to the maximum number of lanes which the caller is requesting. Upon successful creation of the remote pool, \**nlanes* is set to the maximum number of lanes supported by both the local and remote nodes. See **LANES**, below, for details. The *create_attr* structure contains the attributes used for creating the remote pool. If the *create_attr* structure is not NULL, a pool with internal metadata is created. The metadata is stored in the first 4096 bytes of the pool and can be read when opening the remote pool with **rpmem_open**(). To prevent user from overwriting the pool metadata, this region is not accessible to the user via **rpmem_persist**(). If *create_attr* is NULL or zeroed, remote pool set file must contain the *NOHDRS* option. In that case the remote pool is created without internal metadata in it and the entire pool space is available to the user. See **rpmem_persist**(3) for details. The **rpmem_open**() function opens the existing remote pool with *set* file *pool_set_name* on remote node *target*. *pool_set_name* is a relative path in the root config directory on the *target* node. *pool_addr* is a pointer to the associated local memory pool of size *pool_size*. Both *pool_addr* and *pool_size* must be aligned to the system's page size (see **sysconf**(3)). The size of the remote pool must be at least *pool_size*. See **REMOTE POOL SIZE**, below, for details. *nlanes* points to the maximum number of lanes which the caller is requesting. Upon successful opening of the remote pool, \**nlanes* is set to the maximum number of lanes supported by both the local and remote nodes. See **LANES**, below, for details. The **rpmem_set_attr**() function overwrites the pool's attributes. The *attr* structure contains the attributes used for overwriting the remote pool attributes that were passed to **rpmem_create**() at pool creation. If *attr* is NULL, a zeroed structure with attributes will be used. New attributes are stored in the pool's metadata. The **rpmem_close**() function closes the remote pool *rpp*. All resources are released on both the local and remote nodes. The remote pool itself persists on the remote node and may be re-opened at a later time using **rpmem_open**(). The **rpmem_remove**() function removes the remote pool with *set* file name *pool_set_name* from node *target*. The *pool_set_name* is a relative path in the root config directory on the *target* node. By default only the pool part files are removed; the pool *set* file is left untouched. If the pool is not consistent, the **rpmem_remove**() function fails. The *flags* argument determines the behavior of **rpmem_remove**(). *flags* may be either 0 or the bitwise OR of one or more of the following flags: + **RPMEM_REMOVE_FORCE** - Ignore errors when opening an inconsistent pool. The pool *set* file must still be in appropriate format for the pool to be removed. + **RPMEM_REMOVE_POOL_SET** - Remove the pool *set* file after removing the pool described by this pool set. # RETURN VALUE # On success, **rpmem_create**() returns an opaque handle to the remote pool for use in subsequent **librpmem** calls. If any error prevents the remote pool from being created, **rpmem_create**() returns NULL and sets *errno* appropriately. On success, **rpmem_open**() returns an opaque handle to the remote pool for subsequent **librpmem** calls. If the *open_attr* argument is not NULL, the remote pool attributes are returned in the provided structure. If the remote pool was created without internal metadata, zeroes are returned in the *open_attr* structure on successful call to **rpmem_open**(). If any error prevents the remote pool from being opened, **rpmem_open**() returns NULL and sets *errno* appropriately. On success, **rpmem_set_attr**() returns 0. On error, it returns -1 and sets *errno* appropriately. On success, **rpmem_close**() returns 0. On error, it returns a non-zero value and sets *errno* appropriately. On success, **rpmem_remove**() returns 0. On error, it returns a non-zero value and sets *errno* appropriately. # NOTES # ## REMOTE POOL SIZE ## The size of a remote pool depends on the configuration in the pool set file on the remote node (see **poolset**(5)). If no pool set options is used in the remote pool set file, the remote pool size is the sum of the sizes of all part files, decreased by 4096 bytes per part file. 4096 bytes of each part file are utilized for storing internal metadata. If the *SINGLEHDR* option is used in the remote pool set file, the remote pool size is the sum of sizes of all part files, decreased once by 4096 bytes. In this case only the first part contains internal metadata. If a remote pool set file contains the *NOHDRS* option, the remote pool size is the sum of sizes of all its part files. In this case none of the parts contains internal metadata. For other consequences of using the *SINGLEHDR* and *NOHDRS* options see **rpmem_persist**(3). **RPMEM_MIN_PART** and **RPMEM_MIN_POOL** in **\** define the minimum size allowed by **librpmem** for a part file and a remote pool, respectively. ## LANES ## The term *lane* means an isolated path of execution. The underlying hardware utilized by both local and remote nodes may have limited resources that restrict the maximum number of parallel **rpmem_persist**(3) operations. The maximum number of supported lanes is returned by the **rpmem_open**() and **rpmem_create**() function calls. The caller passes the maximum number of lanes requested in \**nlanes*. If the pool is successfully created or opened, \**nlanes* is updated to reflect the minimum of the number of lanes requested by the caller and the maximum number of lanes supported by underlying hardware. The application is obligated to use at most the returned number of lanes in parallel. **rpmem_persist**(3) does not provide any locking mechanism; thus any serialization of calls must be performed by the application if required. Each lane requires a separate connection, represented by a file descriptor. If the system runs out of free file descriptors during **rpmem_create**() or **rpmem_open**(), these functions will fail. See **nofile** in **limits.conf**(5) for more details. # SEE ALSO # **rpmem_persist**(3), **sysconf**(3), **limits.conf**(5), **libpmemobj**(7), **librpmem**(7) and **** pmdk-1.8/doc/librpmem/rpmem_create.30000644000000000000000000002404713615011420016116 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "RPMEM_CREATE" "3" "2020-01-31" "PMDK - rpmem API version 1.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]rpmem_create\f[](), \f[B]rpmem_open\f[](), \f[B]rpmem_set_attr\f[](), \f[B]rpmem_close\f[](), \f[B]rpmem_remove\f[]() \- most commonly used functions for remote access to \f[I]persistent memory\f[] .SH SYNOPSIS .IP .nf \f[C] #include\ RPMEMpool\ *rpmem_create(const\ char\ *target,\ const\ char\ *pool_set_name, \ \ \ \ void\ *pool_addr,\ size_t\ pool_size,\ unsigned\ *nlanes, \ \ \ \ const\ struct\ rpmem_pool_attr\ *create_attr); RPMEMpool\ *rpmem_open(const\ char\ *target,\ const\ char\ *pool_set_name, \ \ \ \ void\ *pool_addr,\ size_t\ pool_size,\ unsigned\ *nlanes, \ \ \ \ struct\ rpmem_pool_attr\ *open_attr); int\ rpmem_set_attr(RPMEMpool\ *rpp,\ const\ struct\ rpmem_pool_attr\ *attr); int\ rpmem_close(RPMEMpool\ *rpp); int\ rpmem_remove(const\ char\ *target,\ const\ char\ *pool_set_name,\ int\ flags); \f[] .fi .SH DESCRIPTION .PP The \f[B]rpmem_create\f[]() function creates a remote pool on a given \f[I]target\f[] node, using pool \f[I]set\f[] file \f[I]pool_set_name\f[] to map the remote pool. \f[I]pool_set_name\f[] is a relative path in the root config directory on the \f[I]target\f[] node. For pool set file format and options see \f[B]poolset\f[](5). \f[I]pool_addr\f[] is a pointer to the associated local memory pool with size \f[I]pool_size\f[]. Both \f[I]pool_addr\f[] and \f[I]pool_size\f[] must be aligned to the system's page size (see \f[B]sysconf\f[](3)). The size of the remote pool must be at least \f[I]pool_size\f[]. See \f[B]REMOTE POOL SIZE\f[], below, for details. \f[I]nlanes\f[] points to the maximum number of lanes which the caller is requesting. Upon successful creation of the remote pool, *\f[I]nlanes\f[] is set to the maximum number of lanes supported by both the local and remote nodes. See \f[B]LANES\f[], below, for details. The \f[I]create_attr\f[] structure contains the attributes used for creating the remote pool. If the \f[I]create_attr\f[] structure is not NULL, a pool with internal metadata is created. The metadata is stored in the first 4096 bytes of the pool and can be read when opening the remote pool with \f[B]rpmem_open\f[](). To prevent user from overwriting the pool metadata, this region is not accessible to the user via \f[B]rpmem_persist\f[](). If \f[I]create_attr\f[] is NULL or zeroed, remote pool set file must contain the \f[I]NOHDRS\f[] option. In that case the remote pool is created without internal metadata in it and the entire pool space is available to the user. See \f[B]rpmem_persist\f[](3) for details. .PP The \f[B]rpmem_open\f[]() function opens the existing remote pool with \f[I]set\f[] file \f[I]pool_set_name\f[] on remote node \f[I]target\f[]. \f[I]pool_set_name\f[] is a relative path in the root config directory on the \f[I]target\f[] node. \f[I]pool_addr\f[] is a pointer to the associated local memory pool of size \f[I]pool_size\f[]. Both \f[I]pool_addr\f[] and \f[I]pool_size\f[] must be aligned to the system's page size (see \f[B]sysconf\f[](3)). The size of the remote pool must be at least \f[I]pool_size\f[]. See \f[B]REMOTE POOL SIZE\f[], below, for details. \f[I]nlanes\f[] points to the maximum number of lanes which the caller is requesting. Upon successful opening of the remote pool, *\f[I]nlanes\f[] is set to the maximum number of lanes supported by both the local and remote nodes. See \f[B]LANES\f[], below, for details. .PP The \f[B]rpmem_set_attr\f[]() function overwrites the pool's attributes. The \f[I]attr\f[] structure contains the attributes used for overwriting the remote pool attributes that were passed to \f[B]rpmem_create\f[]() at pool creation. If \f[I]attr\f[] is NULL, a zeroed structure with attributes will be used. New attributes are stored in the pool's metadata. .PP The \f[B]rpmem_close\f[]() function closes the remote pool \f[I]rpp\f[]. All resources are released on both the local and remote nodes. The remote pool itself persists on the remote node and may be re\-opened at a later time using \f[B]rpmem_open\f[](). .PP The \f[B]rpmem_remove\f[]() function removes the remote pool with \f[I]set\f[] file name \f[I]pool_set_name\f[] from node \f[I]target\f[]. The \f[I]pool_set_name\f[] is a relative path in the root config directory on the \f[I]target\f[] node. By default only the pool part files are removed; the pool \f[I]set\f[] file is left untouched. If the pool is not consistent, the \f[B]rpmem_remove\f[]() function fails. The \f[I]flags\f[] argument determines the behavior of \f[B]rpmem_remove\f[](). \f[I]flags\f[] may be either 0 or the bitwise OR of one or more of the following flags: .IP \[bu] 2 \f[B]RPMEM_REMOVE_FORCE\f[] \- Ignore errors when opening an inconsistent pool. The pool \f[I]set\f[] file must still be in appropriate format for the pool to be removed. .IP \[bu] 2 \f[B]RPMEM_REMOVE_POOL_SET\f[] \- Remove the pool \f[I]set\f[] file after removing the pool described by this pool set. .SH RETURN VALUE .PP On success, \f[B]rpmem_create\f[]() returns an opaque handle to the remote pool for use in subsequent \f[B]librpmem\f[] calls. If any error prevents the remote pool from being created, \f[B]rpmem_create\f[]() returns NULL and sets \f[I]errno\f[] appropriately. .PP On success, \f[B]rpmem_open\f[]() returns an opaque handle to the remote pool for subsequent \f[B]librpmem\f[] calls. If the \f[I]open_attr\f[] argument is not NULL, the remote pool attributes are returned in the provided structure. If the remote pool was created without internal metadata, zeroes are returned in the \f[I]open_attr\f[] structure on successful call to \f[B]rpmem_open\f[](). If any error prevents the remote pool from being opened, \f[B]rpmem_open\f[]() returns NULL and sets \f[I]errno\f[] appropriately. .PP On success, \f[B]rpmem_set_attr\f[]() returns 0. On error, it returns \-1 and sets \f[I]errno\f[] appropriately. .PP On success, \f[B]rpmem_close\f[]() returns 0. On error, it returns a non\-zero value and sets \f[I]errno\f[] appropriately. .PP On success, \f[B]rpmem_remove\f[]() returns 0. On error, it returns a non\-zero value and sets \f[I]errno\f[] appropriately. .SH NOTES .SS REMOTE POOL SIZE .PP The size of a remote pool depends on the configuration in the pool set file on the remote node (see \f[B]poolset\f[](5)). If no pool set options is used in the remote pool set file, the remote pool size is the sum of the sizes of all part files, decreased by 4096 bytes per part file. 4096 bytes of each part file are utilized for storing internal metadata. If the \f[I]SINGLEHDR\f[] option is used in the remote pool set file, the remote pool size is the sum of sizes of all part files, decreased once by 4096 bytes. In this case only the first part contains internal metadata. If a remote pool set file contains the \f[I]NOHDRS\f[] option, the remote pool size is the sum of sizes of all its part files. In this case none of the parts contains internal metadata. For other consequences of using the \f[I]SINGLEHDR\f[] and \f[I]NOHDRS\f[] options see \f[B]rpmem_persist\f[](3). \f[B]RPMEM_MIN_PART\f[] and \f[B]RPMEM_MIN_POOL\f[] in \f[B]\f[] define the minimum size allowed by \f[B]librpmem\f[] for a part file and a remote pool, respectively. .SS LANES .PP The term \f[I]lane\f[] means an isolated path of execution. The underlying hardware utilized by both local and remote nodes may have limited resources that restrict the maximum number of parallel \f[B]rpmem_persist\f[](3) operations. The maximum number of supported lanes is returned by the \f[B]rpmem_open\f[]() and \f[B]rpmem_create\f[]() function calls. The caller passes the maximum number of lanes requested in *\f[I]nlanes\f[]. If the pool is successfully created or opened, *\f[I]nlanes\f[] is updated to reflect the minimum of the number of lanes requested by the caller and the maximum number of lanes supported by underlying hardware. The application is obligated to use at most the returned number of lanes in parallel. .PP \f[B]rpmem_persist\f[](3) does not provide any locking mechanism; thus any serialization of calls must be performed by the application if required. .PP Each lane requires a separate connection, represented by a file descriptor. If the system runs out of free file descriptors during \f[B]rpmem_create\f[]() or \f[B]rpmem_open\f[](), these functions will fail. See \f[B]nofile\f[] in \f[B]limits.conf\f[](5) for more details. .SH SEE ALSO .PP \f[B]rpmem_persist\f[](3), \f[B]sysconf\f[](3), \f[B]limits.conf\f[](5), \f[B]libpmemobj\f[](7), \f[B]librpmem\f[](7) and \f[B]\f[] pmdk-1.8/doc/librpmem/rpmem_set_attr.30000664000000000000000000000002313615011243016471 0ustar rootroot.so rpmem_create.3 pmdk-1.8/doc/librpmem/rpmem_errormsg.30000664000000000000000000000002413615011243016505 0ustar rootroot.so man7/librpmem.7 pmdk-1.8/doc/librpmem/rpmem_flush.30000664000000000000000000000002413615011243015766 0ustar rootroot.so rpmem_persist.3 pmdk-1.8/doc/librpmem/rpmem_persist.3.md0000664000000000000000000001641713615011243016752 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(RPMEM_PERSIST, 3) collection: librpmem header: PMDK date: rpmem API version 1.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (rpmem_persist.3 -- man page for rpmem persist, flush, drain and read functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[CAVEATS](#caveats)
[SEE ALSO](#see-also)
# NAME # **rpmem_persist**(), **rpmem_deep_persist**(), **rpmem_flush**(), **rpmem_drain**(), **rpmem_read**() - functions to copy and read remote pools # SYNOPSIS # ```c #include int rpmem_persist(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane, unsigned flags); int rpmem_deep_persist(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane); int rpmem_flush(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane, unsigned flags); int rpmem_drain(RPMEMpool *rpp, unsigned lane, unsigned flags); int rpmem_read(RPMEMpool *rpp, void *buff, size_t offset, size_t length, unsigned lane); ``` # DESCRIPTION # The **rpmem_persist**() function copies data of given *length* at given *offset* from the associated local memory pool and makes sure the data is persistent on the remote node before the function returns. The remote node is identified by the *rpp* handle which must be returned from either **rpmem_open**(3) or **rpmem_create**(3). The *offset* is relative to the *pool_addr* specified in the **rpmem_open**(3) or **rpmem_create**(3) call. If the remote pool was created using **rpmem_create**() with non-NULL *create_attr* argument, *offset* has to be greater or equal to 4096. In that case the first 4096 bytes of the pool is used for storing the pool metadata and cannot be overwritten. If the pool was created with NULL *create_attr* argument, the pool metadata is not stored with the pool and *offset* can be any nonnegative number. The *offset* and *length* combined must not exceed the *pool_size* passed to **rpmem_open**(3) or **rpmem_create**(3). The **rpmem_persist**() operation is performed using the given *lane* number. The lane must be less than the value returned by **rpmem_open**(3) or **rpmem_create**(3) through the *nlanes* argument (so it can take a value from 0 to *nlanes* - 1). The *flags* argument can be 0 or RPMEM_PERSIST_RELAXED which means the persist operation will be done without any guarantees regarding atomicity of memory transfer. The **rpmem_deep_persist**() function works in the same way as **rpmem_persist**(3) function, but additionally it flushes the data to the lowest possible persistency domain available from software. Please see **pmem_deep_persist**(3) for details. The **rpmem_flush**() and **rpmem_drain**() functions are two halves of the single **rpmem_persist**(). The **rpmem_persist**() copies data and makes it persistent in the one shot, where **rpmem_flush**() and **rpmem_drain**() split this operation into two stages. The **rpmem_flush**() copies data of given *length* at a given *offset* from the associated local memory pool to the remote node. The **rpmem_drain**() makes sure the data copied in all preceding **rpmem_flush**() calls is persistent on the remote node before the function returns. Data copied using **rpmem_flush**() can not be considered persistent on the remote node before return from following **rpmem_drain**(). Single **rpmem_drain**() confirms persistence on the remote node of data copied by all **rpmem_flush**() functions called before it and using the same *lane*. The last **rpmem_flush**() + **rpmem_drain**() can be replaced with **rpmem_persist**() at no cost. The *flags* argument for **rpmem_flush**() can be 0 or RPMEM_FLUSH_RELAXED which means the flush operation will be done without any guarantees regarding atomicity of memory transfer. The *flags* argument for **rpmem_drain**() must be 0. The **rpmem_flush**() function performance is affected by **RPMEM_WORK_QUEUE_SIZE** environment variable (see **librpmem**(7) for more details). The **rpmem_read**() function reads *length* bytes of data from a remote pool at *offset* and copies it to the buffer *buff*. The operation is performed on the specified *lane*. The lane must be less than the value returned by **rpmem_open**(3) or **rpmem_create**(3) through the *nlanes* argument (so it can take a value from 0 to *nlanes* - 1). The *rpp* must point to a remote pool opened or created previously by **rpmem_open**(3) or **rpmem_create**(3). # RETURN VALUE # The **rpmem_persist**() function returns 0 if the entire memory area was made persistent on the remote node. Otherwise it returns a non-zero value and sets *errno* appropriately. The **rpmem_flush**() function returns 0 if duplication of the memory area to the remote node was initialized successfully. Otherwise, it returns a non-zero value and sets *errno* appropriately. The **rpmem_drain**() function returns 0 if the memory areas duplicated by all **rpmem_flush**() calls preceding the **rpmem_drain**() are made persistent on the remote node. Otherwise, it returns a non-zero value and sets *errno* appropriately. The **rpmem_read**() function returns 0 if the data was read entirely. Otherwise it returns a non-zero value and sets *errno* appropriately. # CAVEATS # Ordering of **rpmem_flush**() and **rpmem_persist**() operations which are using different *lane* values is not guaranteed. # SEE ALSO # **rpmem_create**(3), **rpmem_open**(3), **rpmem_persist**(3), **sysconf**(3), **limits.conf**(5), **libpmemobj**(7) and **** pmdk-1.8/doc/librpmem/rpmem_deep_persist.30000664000000000000000000000002413615011243017333 0ustar rootroot.so rpmem_persist.3 pmdk-1.8/doc/librpmem/librpmem.7.md0000664000000000000000000004237613615011243015677 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(LIBRPMEM, 7) collection: librpmem header: PMDK date: rpmem API version 1.3 ... [comment]: <> (Copyright 2016-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (librpmem.7 -- man page for librpmem) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[TARGET NODE ADDRESS FORMAT](#target-node-address-format)
[REMOTE POOL ATTRIBUTES](#remote-pool-attributes)
[SSH](#ssh)
[FORK](#fork)
[CAVEATS](#caveats)
[LIBRARY API VERSIONING](#library-api-versioning-1)
[ENVIRONMENT](#environment)
[DEBUGGING AND ERROR HANDLING](#debugging-and-error-handling)
[EXAMPLE](#example)
[ACKNOWLEDGEMENTS](#acknowledgements)
[SEE ALSO](#see-also) # NAME # **librpmem** - remote persistent memory support library (EXPERIMENTAL) # SYNOPSIS # ```c #include cc ... -lrpmem ``` ##### Library API versioning: ##### ```c const char *rpmem_check_version( unsigned major_required, unsigned minor_required); ``` ##### Error handling: ##### ```c const char *rpmem_errormsg(void); ``` ##### Other library functions: ##### A description of other **librpmem** functions can be found on the following manual pages: + **rpmem_create**(3), **rpmem_persist**(3) # DESCRIPTION # **librpmem** provides low-level support for remote access to *persistent memory* (pmem) utilizing RDMA-capable RNICs. The library can be used to remotely replicate a memory region over the RDMA protocol. It utilizes an appropriate persistency mechanism based on the remote node's platform capabilities. **librpmem** utilizes the **ssh**(1) client to authenticate a user on the remote node, and for encryption of the connection's out-of-band configuration data. See **SSH**, below, for details. The maximum replicated memory region size can not be bigger than the maximum locked-in-memory address space limit. See **memlock** in **limits.conf**(5) for more details. This library is for applications that use remote persistent memory directly, without the help of any library-supplied transactions or memory allocation. Higher-level libraries that build on **libpmem**(7) are available and are recommended for most applications, see: + **libpmemobj**(7), a general use persistent memory API, providing memory allocation and transactional operations on variable-sized objects. # TARGET NODE ADDRESS FORMAT # ``` [@][:] ``` The target node address is described by the *hostname* which the client connects to, with an optional *user* name. The user must be authorized to authenticate to the remote machine without querying for password/passphrase. The optional *port* number is used to establish the SSH connection. The default port number is 22. # REMOTE POOL ATTRIBUTES # The *rpmem_pool_attr* structure describes a remote pool and is stored in remote pool's metadata. This structure must be passed to the **rpmem_create**(3) function by caller when creating a pool on remote node. When opening the pool using **rpmem_open**(3) function the appropriate fields are read from pool's metadata and returned back to the caller. ```c #define RPMEM_POOL_HDR_SIG_LEN 8 #define RPMEM_POOL_HDR_UUID_LEN 16 #define RPMEM_POOL_USER_FLAGS_LEN 16 struct rpmem_pool_attr { char signature[RPMEM_POOL_HDR_SIG_LEN]; uint32_t major; uint32_t compat_features; uint32_t incompat_features; uint32_t ro_compat_features; unsigned char poolset_uuid[RPMEM_POOL_HDR_UUID_LEN]; unsigned char uuid[RPMEM_POOL_HDR_UUID_LEN]; unsigned char next_uuid[RPMEM_POOL_HDR_UUID_LEN]; unsigned char prev_uuid[RPMEM_POOL_HDR_UUID_LEN]; unsigned char user_flags[RPMEM_POOL_USER_FLAGS_LEN]; }; ``` The *signature* field is an 8-byte field which describes the pool's on-media format. The *major* field is a major version number of the pool's on-media format. The *compat_features* field is a mask describing compatibility of pool's on-media format optional features. The *incompat_features* field is a mask describing compatibility of pool's on-media format required features. The *ro_compat_features* field is a mask describing compatibility of pool's on-media format features. If these features are not available, the pool shall be opened in read-only mode. The *poolset_uuid* field is an UUID of the pool which the remote pool is associated with. The *uuid* field is an UUID of a first part of the remote pool. This field can be used to connect the remote pool with other pools in a list. The *next_uuid* and *prev_uuid* fields are UUIDs of next and previous replicas respectively. These fields can be used to connect the remote pool with other pools in a list. The *user_flags* field is a 16-byte user-defined flags. # SSH # **librpmem** utilizes the **ssh**(1) client to login and execute the **rpmemd**(1) process on the remote node. By default, **ssh**(1) is executed with the **-4** option, which forces using **IPv4** addressing. For debugging purposes, both the ssh client and the commands executed on the remote node may be overridden by setting the **RPMEM_SSH** and **RPMEM_CMD** environment variables, respectively. See **ENVIRONMENT** for details. # FORK # The **ssh**(1) client is executed by **rpmem_open**(3) and **rpmem_create**(3) after forking a child process using **fork**(2). The application must take this into account when using **wait**(2) and **waitpid**(2), which may return the *PID* of the **ssh**(1) process executed by **librpmem**. If **fork**(2) support is not enabled in **libibverbs**, **rpmem_open**(3) and **rpmem_create**(3) will fail. By default, **fabric**(7) initializes **libibverbs** with **fork**(2) support by calling the **ibv_fork_init**(3) function. See **fi_verbs**(7) for more details. # CAVEATS # **librpmem** relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. **dlclose**(3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. **librpmem** registers a pool as a single memory region. A Chelsio T4 and T5 hardware can not handle a memory region greater than or equal to 8GB due to a hardware bug. So *pool_size* value for **rpmem_create**(3) and **rpmem_open**(3) using this hardware can not be greater than or equal to 8GB. # LIBRARY API VERSIONING # This section describes how the library API is versioned, allowing applications to work with an evolving API. The **rpmem_check_version**() function is used to see if the installed **librpmem** supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile-time version information, supplied by defines in **\**, like this: ```c reason = rpmem_check_version(RPMEM_MAJOR_VERSION, RPMEM_MINOR_VERSION); if (reason != NULL) { /* version check failed, reason string tells you why */ } ``` Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text *introduced in version x.y* in the section of this manual describing the feature. When the version check performed by **rpmem_check_version**() is successful, the return value is NULL. Otherwise the return value is a static string describing the reason for failing the version check. The string returned by **rpmem_check_version**() must not be modified or freed. # ENVIRONMENT # **librpmem** can change its default behavior based on the following environment variables. These are largely intended for testing and are not normally required. + **RPMEM_SSH**=*ssh_client* Setting this environment variable overrides the default **ssh**(1) client command name. + **RPMEM_CMD**=*cmd* Setting this environment variable overrides the default command executed on the remote node using either **ssh**(1) or the alternative remote shell command specified by **RPMEM_SSH**. **RPMEM_CMD** can contain multiple commands separated by a vertical bar (`|`). Each consecutive command is executed on the remote node in order read from a pool set file. This environment variable is read when the library is initialized, so **RPMEM_CMD** must be set prior to application launch (or prior to **dlopen**(3) if **librpmem** is being dynamically loaded). + **RPMEM_ENABLE_SOCKETS**=0\|1 Setting this variable to 1 enables using **fi_sockets**(7) provider for in-band RDMA connection. The *sockets* provider does not support IPv6. It is required to disable IPv6 system wide if **RPMEM_ENABLE_SOCKETS** == 1 and *target* == localhost (or any other loopback interface address) and **SSH_CONNECTION** variable (see **ssh**(1) for more details) contains IPv6 address after ssh to loopback interface. By default the *sockets* provider is disabled. * **RPMEM_ENABLE_VERBS**=0\|1 Setting this variable to 0 disables using **fi_verbs**(7) provider for in-band RDMA connection. The *verbs* provider is enabled by default. * **RPMEM_MAX_NLANES**=*num* Limit the maximum number of lanes to *num*. See **LANES**, in **rpmem_create**(3), for details. * **RPMEM_WORK_QUEUE_SIZE**=*size* Suggest the work queue size. The effective work queue size can be greater than suggested if **librpmem** requires it or it can be smaller if underlying hardware does not support the suggested size. The work queue size affects the performance of communication to the remote node. **rpmem_flush**(3) operations can be added to the work queue up to the size of this queue. When work queue is full any subsequent call has to wait till the work queue will be drained. **rpmem_drain**(3) and **rpmem_persist**(3) among other things also drain the work queue. # DEBUGGING AND ERROR HANDLING # If an error is detected during the call to a **librpmem** function, the application may retrieve an error message describing the reason for the failure from **rpmem_errormsg**(). This function returns a pointer to a static buffer containing the last error message logged for the current thread. If *errno* was set, the error message may include a description of the corresponding error code as returned by **strerror**(3). The error message buffer is thread-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a **librpmem** function indicated an error, or if *errno* was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. Two versions of **librpmem** are typically available on a development system. The normal version, accessed when a program is linked using the **-lrpmem** option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run-time assertions. A second version of **librpmem**, accessed when a program uses the libraries under _DEBUGLIBPATH(), contains run-time assertions and trace points. The typical way to access the debug version is to set the environment variable **LD_LIBRARY_PATH** to _LDLIBPATH(). Debugging output is controlled using the following environment variables. These variables have no effect on the non-debug version of the library. + **RPMEM_LOG_LEVEL** The value of **RPMEM_LOG_LEVEL** enables trace points in the debug version of the library, as follows: + **0** - This is the default level when **RPMEM_LOG_LEVEL** is not set. No log messages are emitted at this level. + **1** - Additional details on any errors detected are logged (in addition to returning the *errno*-based errors as usual). The same information may be retrieved using **rpmem_errormsg**(). + **2** - A trace of basic operations is logged. + **3** - Enables a very verbose amount of function call tracing in the library. + **4** - Enables voluminous and fairly obscure tracing information that is likely only useful to the **librpmem** developers. Unless **RPMEM_LOG_FILE** is set, debugging output is written to *stderr*. + **RPMEM_LOG_FILE** Specifies the name of a file where all logging information should be written. If the last character in the name is "-", the *PID* of the current process will be appended to the file name when the log file is created. If **RPMEM_LOG_FILE** is not set, logging output is written to *stderr*. # EXAMPLE # The following example uses **librpmem** to create a remote pool on given target node identified by given pool set name. The associated local memory pool is zeroed and the data is made persistent on remote node. Upon success the remote pool is closed. ```c #include #include #include #include #include #include #define POOL_SIGNATURE "MANPAGE" #define POOL_SIZE (32 * 1024 * 1024) #define NLANES 4 #define DATA_OFF 4096 #define DATA_SIZE (POOL_SIZE - DATA_OFF) static void parse_args(int argc, char *argv[], const char **target, const char **poolset) { if (argc < 3) { fprintf(stderr, "usage:\t%s \n", argv[0]); exit(1); } *target = argv[1]; *poolset = argv[2]; } static void * alloc_memory() { long pagesize = sysconf(_SC_PAGESIZE); if (pagesize < 0) { perror("sysconf"); exit(1); } /* allocate a page size aligned local memory pool */ void *mem; int ret = posix_memalign(&mem, pagesize, POOL_SIZE); if (ret) { fprintf(stderr, "posix_memalign: %s\n", strerror(ret)); exit(1); } assert(mem != NULL); return mem; } int main(int argc, char *argv[]) { const char *target, *poolset; parse_args(argc, argv, &target, &poolset); unsigned nlanes = NLANES; void *pool = alloc_memory(); int ret; /* fill pool_attributes */ struct rpmem_pool_attr pool_attr; memset(&pool_attr, 0, sizeof(pool_attr)); strncpy(pool_attr.signature, POOL_SIGNATURE, RPMEM_POOL_HDR_SIG_LEN); /* create a remote pool */ RPMEMpool *rpp = rpmem_create(target, poolset, pool, POOL_SIZE, &nlanes, &pool_attr); if (!rpp) { fprintf(stderr, "rpmem_create: %s\n", rpmem_errormsg()); return 1; } /* store data on local pool */ memset(pool, 0, POOL_SIZE); /* make local data persistent on remote node */ ret = rpmem_persist(rpp, DATA_OFF, DATA_SIZE, 0, 0); if (ret) { fprintf(stderr, "rpmem_persist: %s\n", rpmem_errormsg()); return 1; } /* close the remote pool */ ret = rpmem_close(rpp); if (ret) { fprintf(stderr, "rpmem_close: %s\n", rpmem_errormsg()); return 1; } free(pool); return 0; } ``` # NOTE # The **librpmem** API is experimental and may be subject to change in the future. However, using the remote replication in **libpmemobj**(7) is safe and backward compatibility will be preserved. # ACKNOWLEDGEMENTS # **librpmem** builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: # SEE ALSO # **rpmemd**(1), **ssh**(1), **fork**(2), **dlclose**(3), **dlopen**(3), **ibv_fork_init**(3), **rpmem_create**(3), **rpmem_drain**(3), **rpmem_flush**(3), **rpmem_open**(3), **rpmem_persist**(3), **strerror**(3), **limits.conf**(5), **fabric**(7), **fi_sockets**(7), **fi_verbs**(7), **libpmem**(7), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/librpmem/.gitignore0000664000000000000000000000005213615011243015352 0ustar rootrootlibrpmem.7 rpmem_create.3 rpmem_persist.3 pmdk-1.8/doc/librpmem/rpmem_persist.30000644000000000000000000001661013615011420016341 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "RPMEM_PERSIST" "3" "2020-01-31" "PMDK - rpmem API version 1.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]rpmem_persist\f[](), \f[B]rpmem_deep_persist\f[](), \f[B]rpmem_flush\f[](), \f[B]rpmem_drain\f[](), \f[B]rpmem_read\f[]() \- functions to copy and read remote pools .SH SYNOPSIS .IP .nf \f[C] #include\ int\ rpmem_persist(RPMEMpool\ *rpp,\ size_t\ offset, \ \ \ \ size_t\ length,\ unsigned\ lane,\ unsigned\ flags); int\ rpmem_deep_persist(RPMEMpool\ *rpp,\ size_t\ offset, \ \ \ \ size_t\ length,\ unsigned\ lane); int\ rpmem_flush(RPMEMpool\ *rpp,\ size_t\ offset, \ \ \ \ size_t\ length,\ unsigned\ lane,\ unsigned\ flags); int\ rpmem_drain(RPMEMpool\ *rpp,\ unsigned\ lane,\ unsigned\ flags); int\ rpmem_read(RPMEMpool\ *rpp,\ void\ *buff,\ size_t\ offset, \ \ \ \ size_t\ length,\ unsigned\ lane); \f[] .fi .SH DESCRIPTION .PP The \f[B]rpmem_persist\f[]() function copies data of given \f[I]length\f[] at given \f[I]offset\f[] from the associated local memory pool and makes sure the data is persistent on the remote node before the function returns. The remote node is identified by the \f[I]rpp\f[] handle which must be returned from either \f[B]rpmem_open\f[](3) or \f[B]rpmem_create\f[](3). The \f[I]offset\f[] is relative to the \f[I]pool_addr\f[] specified in the \f[B]rpmem_open\f[](3) or \f[B]rpmem_create\f[](3) call. If the remote pool was created using \f[B]rpmem_create\f[]() with non\-NULL \f[I]create_attr\f[] argument, \f[I]offset\f[] has to be greater or equal to 4096. In that case the first 4096 bytes of the pool is used for storing the pool metadata and cannot be overwritten. If the pool was created with NULL \f[I]create_attr\f[] argument, the pool metadata is not stored with the pool and \f[I]offset\f[] can be any nonnegative number. The \f[I]offset\f[] and \f[I]length\f[] combined must not exceed the \f[I]pool_size\f[] passed to \f[B]rpmem_open\f[](3) or \f[B]rpmem_create\f[](3). The \f[B]rpmem_persist\f[]() operation is performed using the given \f[I]lane\f[] number. The lane must be less than the value returned by \f[B]rpmem_open\f[](3) or \f[B]rpmem_create\f[](3) through the \f[I]nlanes\f[] argument (so it can take a value from 0 to \f[I]nlanes\f[] \- 1). The \f[I]flags\f[] argument can be 0 or RPMEM_PERSIST_RELAXED which means the persist operation will be done without any guarantees regarding atomicity of memory transfer. .PP The \f[B]rpmem_deep_persist\f[]() function works in the same way as \f[B]rpmem_persist\f[](3) function, but additionally it flushes the data to the lowest possible persistency domain available from software. Please see \f[B]pmem_deep_persist\f[](3) for details. .PP The \f[B]rpmem_flush\f[]() and \f[B]rpmem_drain\f[]() functions are two halves of the single \f[B]rpmem_persist\f[](). The \f[B]rpmem_persist\f[]() copies data and makes it persistent in the one shot, where \f[B]rpmem_flush\f[]() and \f[B]rpmem_drain\f[]() split this operation into two stages. The \f[B]rpmem_flush\f[]() copies data of given \f[I]length\f[] at a given \f[I]offset\f[] from the associated local memory pool to the remote node. The \f[B]rpmem_drain\f[]() makes sure the data copied in all preceding \f[B]rpmem_flush\f[]() calls is persistent on the remote node before the function returns. Data copied using \f[B]rpmem_flush\f[]() can not be considered persistent on the remote node before return from following \f[B]rpmem_drain\f[](). Single \f[B]rpmem_drain\f[]() confirms persistence on the remote node of data copied by all \f[B]rpmem_flush\f[]() functions called before it and using the same \f[I]lane\f[]. The last \f[B]rpmem_flush\f[]() + \f[B]rpmem_drain\f[]() can be replaced with \f[B]rpmem_persist\f[]() at no cost. .PP The \f[I]flags\f[] argument for \f[B]rpmem_flush\f[]() can be 0 or RPMEM_FLUSH_RELAXED which means the flush operation will be done without any guarantees regarding atomicity of memory transfer. The \f[I]flags\f[] argument for \f[B]rpmem_drain\f[]() must be 0. .PP The \f[B]rpmem_flush\f[]() function performance is affected by \f[B]RPMEM_WORK_QUEUE_SIZE\f[] environment variable (see \f[B]librpmem\f[](7) for more details). .PP The \f[B]rpmem_read\f[]() function reads \f[I]length\f[] bytes of data from a remote pool at \f[I]offset\f[] and copies it to the buffer \f[I]buff\f[]. The operation is performed on the specified \f[I]lane\f[]. The lane must be less than the value returned by \f[B]rpmem_open\f[](3) or \f[B]rpmem_create\f[](3) through the \f[I]nlanes\f[] argument (so it can take a value from 0 to \f[I]nlanes\f[] \- 1). The \f[I]rpp\f[] must point to a remote pool opened or created previously by \f[B]rpmem_open\f[](3) or \f[B]rpmem_create\f[](3). .SH RETURN VALUE .PP The \f[B]rpmem_persist\f[]() function returns 0 if the entire memory area was made persistent on the remote node. Otherwise it returns a non\-zero value and sets \f[I]errno\f[] appropriately. .PP The \f[B]rpmem_flush\f[]() function returns 0 if duplication of the memory area to the remote node was initialized successfully. Otherwise, it returns a non\-zero value and sets \f[I]errno\f[] appropriately. .PP The \f[B]rpmem_drain\f[]() function returns 0 if the memory areas duplicated by all \f[B]rpmem_flush\f[]() calls preceding the \f[B]rpmem_drain\f[]() are made persistent on the remote node. Otherwise, it returns a non\-zero value and sets \f[I]errno\f[] appropriately. .PP The \f[B]rpmem_read\f[]() function returns 0 if the data was read entirely. Otherwise it returns a non\-zero value and sets \f[I]errno\f[] appropriately. .SH CAVEATS .PP Ordering of \f[B]rpmem_flush\f[]() and \f[B]rpmem_persist\f[]() operations which are using different \f[I]lane\f[] values is not guaranteed. .SH SEE ALSO .PP \f[B]rpmem_create\f[](3), \f[B]rpmem_open\f[](3), \f[B]rpmem_persist\f[](3), \f[B]sysconf\f[](3), \f[B]limits.conf\f[](5), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/librpmem/rpmem_open.30000664000000000000000000000002313615011243015605 0ustar rootroot.so rpmem_create.3 pmdk-1.8/doc/librpmem/rpmem_close.30000664000000000000000000000002313615011243015751 0ustar rootroot.so rpmem_create.3 pmdk-1.8/doc/librpmem/rpmem_read.30000664000000000000000000000002413615011243015560 0ustar rootroot.so rpmem_persist.3 pmdk-1.8/doc/librpmem/rpmem_remove.30000664000000000000000000000002113615011243016137 0ustar rootroot.so rpmem_open.3 pmdk-1.8/doc/librpmem/rpmem_drain.30000664000000000000000000000002413615011243015742 0ustar rootroot.so rpmem_persist.3 pmdk-1.8/doc/libpmem2/0000775000000000000000000000000013615011420013262 5ustar rootrootpmdk-1.8/doc/libpmem2/pmem2_errormsg.3.md0000664000000000000000000000651713615011243016721 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEM2_ERRORMSG, 3) collection: libpmem2 header: PMDK date: pmem2 API version 1.0 ... [comment]: <> (Copyright 2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmem2_errormsg.3 -- man page for error handling in libpmem2 [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # _UW(pmem2_errormsg) - returns last error message # SYNOPSIS # ```c #include _UWFUNC(pmem2_errormsg, void) ``` _UNICODE() # DESCRIPTION # If an error is detected during the call to a **libpmem2**(7) function, the application may retrieve an error message describing the reason of the failure from _UW(pmem2_errormsg). The error message buffer is thread-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a **libpmem2**(7) function indicated an error. The application must not modify or free the error message string. Subsequent calls to other library functions may modify the previous message. # RETURN VALUE # The _UW(pmem2_errormsg) function returns a pointer to a static buffer containing the last error message logged for the current thread. If *errno* was set, the error message may include a description of the corresponding error code as returned by **strerror**(3). # SEE ALSO # **strerror**(3), **libpmem2**(7) and **** pmdk-1.8/doc/libpmem2/pmem2_mapping.3.md0000664000000000000000000000473213615011243016511 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEM2_MAPPING, 3) collection: libpmem2 header: PMDK date: pmem2 API version 1.0 ... [comment]: <> (Copyright 2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmem2_mapping.3 -- man page for libpmem2 mapping operations [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmem2_map**() - create mapping on the file # SYNOPSIS # ```c #include int pmem2_map(const struct pmem2_config *cfg, struct pmem2_map **map_ptr); ``` # DESCRIPTION # # RETURN VALUE # # SEE ALSO # **libpmem2**(7) and **** pmdk-1.8/doc/libpmem2/pmem2_config_set_handle.30000664000000000000000000000002313615011243020077 0ustar rootroot.so pmem2_config.3 pmdk-1.8/doc/libpmem2/pmem2_config.3.md0000664000000000000000000000527613615011243016327 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEM2_CONFIG, 3) collection: libpmem2 header: PMDK date: pmem2 API version 1.0 ... [comment]: <> (Copyright 2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmem2_config.3 -- man page for libpmem2 config API [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmem2_config_new**(), **pmem2_config_delete**(), **pmem2_config_set_fd**(), **pmem2_config_set_handle**() # SYNOPSIS # ```c #include int pmem2_config_new(struct pmem2_config **cfg); int pmem2_config_delete(struct pmem2_config **cfg); int pmem2_config_set_fd(struct pmem2_config *cfg, int fd); int pmem2_config_set_handle(struct pmem2_config *cfg, HANDLE handle); (XXX: WINDOWS ONLY) ``` # DESCRIPTION # # RETURN VALUE # # SEE ALSO # **libpmem2**(7) and **** pmdk-1.8/doc/libpmem2/libpmem2.7.md0000664000000000000000000001104013615011243015457 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(LIBPMEM2, 7) collection: libpmem2 header: PMDK date: pmem2 API version 1.0 ... [comment]: <> (Copyright 2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (libpmem2.7 -- man page for libpmem2) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[CAVEATS](#caveats)
[ENVIRONMENT](#environment)
[DEBUGGING](#debugging)
[EXAMPLE](#example)
[ACKNOWLEDGEMENTS](#acknowledgements)
[SEE ALSO](#see-also) # NAME # **libpmem2** - persistent memory support library (EXPERIMENTAL) # SYNOPSIS # ```c #include cc ... -lpmem2 ``` # DESCRIPTION # # CAVEATS # # ENVIRONMENT # # DEBUGGING # Two versions of **libpmem2** are typically available on a development system. The normal version, accessed when a program is linked using the **-lpmem2** option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run-time assertions. A second version of **libpmem2**, accessed when a program uses the libraries under _DEBUGLIBPATH(), contains run-time assertions and trace points. The typical way to access the debug version is to set the environment variable **LD_LIBRARY_PATH** to _LDLIBPATH(). Debugging output is controlled using the following environment variables. These variables have no effect on the non-debug version of the library. + **PMEM2_LOG_LEVEL** The value of **PMEM2_LOG_LEVEL** enables trace points in the debug version of the library, as follows: + **0** - This is the default level when **PMEM2_LOG_LEVEL** is not set. No log messages are emitted at this level. + **1** - Additional details on any errors detected are logged, in addition to returning the *errno*-based errors as usual. The same information may be retrieved using _UW(pmem2_errormsg). + **2** - A trace of basic operations is logged. + **3** - Enables a very verbose amount of function call tracing in the library. + **4** - Enables voluminous and fairly obscure tracing information that is likely only useful to the **libpmem2** developers. Unless **PMEM2_LOG_FILE** is set, debugging output is written to *stderr*. + **PMEM2_LOG_FILE** Specifies the name of a file where all logging information should be written. If the last character in the name is "-", the *PID* of the current process will be appended to the file name when the log file is created. If **PMEM2_LOG_FILE** is not set, output is written to *stderr*. # EXAMPLE # # ACKNOWLEDGEMENTS # **libpmem2** builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: # SEE ALSO # **libpmem**(7), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmem2/.gitignore0000664000000000000000000000007313615011243015255 0ustar rootrootlibpmem2.7 pmem2_config.3 pmem2_errormsg.3 pmem2_mapping.3 pmdk-1.8/doc/libpmem2/pmem2_config_new.30000664000000000000000000000002313615011243016562 0ustar rootroot.so pmem2_config.3 pmdk-1.8/doc/libpmem2/pmem2_config_set_fd.30000664000000000000000000000002313615011243017235 0ustar rootroot.so pmem2_config.3 pmdk-1.8/doc/libpmem2/pmem2_config.30000644000000000000000000000444013615011420015713 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEM2_CONFIG" "3" "2020-01-31" "PMDK - pmem2 API version 1.0" "PMDK Programmer's Manual" .hy .\" Copyright 2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmem2_config_new\f[](), \f[B]pmem2_config_delete\f[](), \f[B]pmem2_config_set_fd\f[](), \f[B]pmem2_config_set_handle\f[]() .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmem2_config_new(struct\ pmem2_config\ **cfg); int\ pmem2_config_delete(struct\ pmem2_config\ **cfg); int\ pmem2_config_set_fd(struct\ pmem2_config\ *cfg,\ int\ fd); int\ pmem2_config_set_handle(struct\ pmem2_config\ *cfg,\ HANDLE\ handle);\ (XXX:\ WINDOWS\ ONLY) \f[] .fi .SH DESCRIPTION .SH RETURN VALUE .SH SEE ALSO .PP \f[B]libpmem2\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmem2/pmem2_errormsg.30000644000000000000000000000566013615011420016313 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEM2_ERRORMSG" "3" "2020-01-31" "PMDK - pmem2 API version 1.0" "PMDK Programmer's Manual" .hy .\" Copyright 2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmem2_errormsg\f[]() \- returns last error message .SH SYNOPSIS .IP .nf \f[C] #include\ const\ char\ *pmem2_errormsg(void); \f[] .fi .SH DESCRIPTION .PP If an error is detected during the call to a \f[B]libpmem2\f[](7) function, the application may retrieve an error message describing the reason of the failure from \f[B]pmem2_errormsg\f[](). The error message buffer is thread\-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a \f[B]libpmem2\f[](7) function indicated an error. The application must not modify or free the error message string. Subsequent calls to other library functions may modify the previous message. .SH RETURN VALUE .PP The \f[B]pmem2_errormsg\f[]() function returns a pointer to a static buffer containing the last error message logged for the current thread. If \f[I]errno\f[] was set, the error message may include a description of the corresponding error code as returned by \f[B]strerror\f[](3). .SH SEE ALSO .PP \f[B]strerror\f[](3), \f[B]libpmem2\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmem2/pmem2_mapping.30000644000000000000000000000403113615011420016075 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEM2_MAPPING" "3" "2020-01-31" "PMDK - pmem2 API version 1.0" "PMDK Programmer's Manual" .hy .\" Copyright 2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmem2_map\f[]() \- create mapping on the file .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmem2_map(const\ struct\ pmem2_config\ *cfg,\ struct\ pmem2_map\ **map_ptr); \f[] .fi .SH DESCRIPTION .SH RETURN VALUE .SH SEE ALSO .PP \f[B]libpmem2\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmem2/libpmem2.70000644000000000000000000001046113615011420015061 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "LIBPMEM2" "7" "2020-01-31" "PMDK - pmem2 API version 1.0" "PMDK Programmer's Manual" .hy .\" Copyright 2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]libpmem2\f[] \- persistent memory support library (EXPERIMENTAL) .SH SYNOPSIS .IP .nf \f[C] #include\ cc\ ...\ \-lpmem2 \f[] .fi .SH DESCRIPTION .SH CAVEATS .SH ENVIRONMENT .SH DEBUGGING .PP Two versions of \f[B]libpmem2\f[] are typically available on a development system. The normal version, accessed when a program is linked using the \f[B]\-lpmem2\f[] option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run\-time assertions. .PP A second version of \f[B]libpmem2\f[], accessed when a program uses the libraries under \f[B]/usr/lib/pmdk_debug\f[], contains run\-time assertions and trace points. The typical way to access the debug version is to set the environment variable \f[B]LD_LIBRARY_PATH\f[] to \f[B]/usr/lib/pmdk_debug\f[] or \f[B]/usr/lib64/pmdk_debug\f[], as appropriate. Debugging output is controlled using the following environment variables. These variables have no effect on the non\-debug version of the library. .IP \[bu] 2 \f[B]PMEM2_LOG_LEVEL\f[] .PP The value of \f[B]PMEM2_LOG_LEVEL\f[] enables trace points in the debug version of the library, as follows: .IP \[bu] 2 \f[B]0\f[] \- This is the default level when \f[B]PMEM2_LOG_LEVEL\f[] is not set. No log messages are emitted at this level. .IP \[bu] 2 \f[B]1\f[] \- Additional details on any errors detected are logged, in addition to returning the \f[I]errno\f[]\-based errors as usual. The same information may be retrieved using \f[B]pmem2_errormsg\f[](). .IP \[bu] 2 \f[B]2\f[] \- A trace of basic operations is logged. .IP \[bu] 2 \f[B]3\f[] \- Enables a very verbose amount of function call tracing in the library. .IP \[bu] 2 \f[B]4\f[] \- Enables voluminous and fairly obscure tracing information that is likely only useful to the \f[B]libpmem2\f[] developers. .PP Unless \f[B]PMEM2_LOG_FILE\f[] is set, debugging output is written to \f[I]stderr\f[]. .IP \[bu] 2 \f[B]PMEM2_LOG_FILE\f[] .PP Specifies the name of a file where all logging information should be written. If the last character in the name is \[lq]\-\[rq], the \f[I]PID\f[] of the current process will be appended to the file name when the log file is created. If \f[B]PMEM2_LOG_FILE\f[] is not set, output is written to \f[I]stderr\f[]. .SH EXAMPLE .SH ACKNOWLEDGEMENTS .PP \f[B]libpmem2\f[] builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: .SH SEE ALSO .PP \f[B]libpmem\f[](7), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmem2/pmem2_map.30000664000000000000000000000002413615011243015222 0ustar rootroot.so pmem2_mapping.3 pmdk-1.8/doc/libpmem2/pmem2_config_delete.30000664000000000000000000000002313615011243017233 0ustar rootroot.so pmem2_config.3 pmdk-1.8/doc/libpmemlog/0000775000000000000000000000000013615011417013710 5ustar rootrootpmdk-1.8/doc/libpmemlog/pmemlog_ctl_get.30000644000000000000000000001206613615011417017140 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMLOG_CTL_GET" "3" "2020-01-31" "PMDK - pmemlog API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2018-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemlog_ctl_get\f[](), \f[B]pmemlog_ctl_set\f[](), \f[B]pmemlog_ctl_exec\f[]() \- Query and modify libpmemlog internal behavior (EXPERIMENTAL) .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmemlog_ctl_get(PMEMlogpool\ *plp,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) int\ pmemlog_ctl_set(PMEMlogpool\ *plp,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) int\ pmemlog_ctl_exec(PMEMlogpool\ *plp,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemlog_ctl_get\f[](), \f[B]pmemlog_ctl_set\f[]() and \f[B]pmemlog_ctl_exec\f[]() functions provide a uniform interface for querying and modifying the internal behavior of \f[B]libpmemlog\f[](7) through the control (CTL) namespace. .PP The \f[I]name\f[] argument specifies an entry point as defined in the CTL namespace specification. The entry point description specifies whether the extra \f[I]arg\f[] is required. Those two parameters together create a CTL query. The functions and the entry points are thread\-safe unless indicated otherwise below. If there are special conditions for calling an entry point, they are explicitly stated in its description. The functions propagate the return value of the entry point. If either \f[I]name\f[] or \f[I]arg\f[] is invalid, \-1 is returned. .PP If the provided ctl query is valid, the CTL functions will always return 0 on success and \-1 on failure, unless otherwise specified in the entry point description. .PP See more in \f[B]pmem_ctl\f[](5) man page. .SH CTL NAMESPACE .PP prefault.at_create | rw | global | int | int | \- | boolean .PP If set, every page of the pool will be touched and written to when the pool is created, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the \f[B]pmemlog_create\f[]() function. .PP Always returns 0. .PP prefault.at_open | rw | global | int | int | \- | boolean .PP If set, every page of the pool will be touched and written to when the pool is opened, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the \f[B]pmemlog_open\f[]() function. .PP Always returns 0. .PP sds.at_create | rw | global | int | int | \- | boolean .PP If set, force\-enables or force\-disables SDS feature during pool creation. Affects only the \f[B]pmemlog_create\f[]() function. See \f[B]pmempool_feature_query\f[](3) for information about SDS (SHUTDOWN_STATE) feature. .PP Always returns 0. .PP copy_on_write.at_open | rw | global | int | int | \- | boolean .PP If set, pool is mapped in such a way that modifications don't reach the underlying medium. From the user's perspective this means that when the pool is closed all changes are reverted. This feature is not supported for pools located on Device DAX. .PP Always returns 0. .SH CTL EXTERNAL CONFIGURATION .PP In addition to direct function call, each write entry point can also be set using two alternative methods. .PP The first method is to load a configuration directly from the \f[B]PMEMLOG_CONF\f[] environment variable. .PP The second method of loading an external configuration is to set the \f[B]PMEMLOG_CONF_FILE\f[] environment variable to point to a file that contains a sequence of ctl queries. .PP See more in \f[B]pmem_ctl\f[](5) man page. .SH SEE ALSO .PP \f[B]libpmemlog\f[](7), \f[B]pmem_ctl\f[](5) and \f[B]\f[] pmdk-1.8/doc/libpmemlog/pmemlog_appendv.30000664000000000000000000000002513615011243017143 0ustar rootroot.so pmemlog_append.3 pmdk-1.8/doc/libpmemlog/pmemlog_append.3.md0000664000000000000000000000751313615011243017365 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMLOG_APPEND, 3) collection: libpmemlog header: PMDK date: pmemlog API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemlog_append.3 -- man page for pmemlog_append and pmemlog_appendv functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[NOTES](#notes)
[SEE ALSO](#see-also)
# NAME # **pmemlog_append**(), **pmemlog_appendv**() - append bytes to the persistent memory resident log file # SYNOPSIS # ```c #include int pmemlog_append(PMEMlogpool *plp, const void *buf, size_t count); int pmemlog_appendv(PMEMlogpool *plp, const struct iovec *iov, int iovcnt); ``` # DESCRIPTION # The **pmemlog_append**() function appends *count* bytes from *buf* to the current write offset in the log memory pool *plp*. Calling this function is analogous to appending to a file. The append is atomic and cannot be torn by a program failure or system crash. The **pmemlog_appendv**() function appends to the log memory pool *plp* from the scatter/gather list *iov* in a manner similar to **writev**(2). The entire list of buffers is appended atomically, as if the buffers in *iov* were concatenated in order. The append is atomic and cannot be torn by a program failure or system crash. # RETURN VALUE # On success, **pmemlog_append**() and **pmemlog_appendv**() return 0. On error, they return -1 and set *errno* appropriately. # ERRORS # **EINVAL** The vector count *iovcnt* is less than zero. **ENOSPC** There is no room for the data in the log file. **EROFS** The log file is open in read-only mode. # NOTES # Since **libpmemlog**(7) is designed as a low-latency code path, many of the checks routinely done by the operating system for **writev**(2) are not practical in the library's implementation of **pmemlog_appendv**(). No attempt is made to detect NULL or incorrect pointers, for example. # SEE ALSO # **writev**(2), **libpmemlog**(7) and **** pmdk-1.8/doc/libpmemlog/pmemlog_append.30000644000000000000000000000676613615011416016777 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMLOG_APPEND" "3" "2020-01-31" "PMDK - pmemlog API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemlog_append\f[](), \f[B]pmemlog_appendv\f[]() \- append bytes to the persistent memory resident log file .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmemlog_append(PMEMlogpool\ *plp,\ const\ void\ *buf,\ size_t\ count); int\ pmemlog_appendv(PMEMlogpool\ *plp,\ const\ struct\ iovec\ *iov,\ int\ iovcnt); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemlog_append\f[]() function appends \f[I]count\f[] bytes from \f[I]buf\f[] to the current write offset in the log memory pool \f[I]plp\f[]. Calling this function is analogous to appending to a file. The append is atomic and cannot be torn by a program failure or system crash. .PP The \f[B]pmemlog_appendv\f[]() function appends to the log memory pool \f[I]plp\f[] from the scatter/gather list \f[I]iov\f[] in a manner similar to \f[B]writev\f[](2). The entire list of buffers is appended atomically, as if the buffers in \f[I]iov\f[] were concatenated in order. The append is atomic and cannot be torn by a program failure or system crash. .SH RETURN VALUE .PP On success, \f[B]pmemlog_append\f[]() and \f[B]pmemlog_appendv\f[]() return 0. On error, they return \-1 and set \f[I]errno\f[] appropriately. .SH ERRORS .PP \f[B]EINVAL\f[] The vector count \f[I]iovcnt\f[] is less than zero. .PP \f[B]ENOSPC\f[] There is no room for the data in the log file. .PP \f[B]EROFS\f[] The log file is open in read\-only mode. .SH NOTES .PP Since \f[B]libpmemlog\f[](7) is designed as a low\-latency code path, many of the checks routinely done by the operating system for \f[B]writev\f[](2) are not practical in the library's implementation of \f[B]pmemlog_appendv\f[](). No attempt is made to detect NULL or incorrect pointers, for example. .SH SEE ALSO .PP \f[B]writev\f[](2), \f[B]libpmemlog\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemlog/libpmemlog.7.md0000664000000000000000000003112613615011243016526 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(LIBPMEMLOG, 7) collection: libpmemlog header: PMDK date: pmemlog API version 1.1 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (libpmemlog.7 -- man page for libpmemlog) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[CAVEATS](#caveats)
[LIBRARY API VERSIONING](#library-api-versioning-1)
[MANAGING LIBRARY BEHAVIOR](#managing-library-behavior-1)
[DEBUGGING AND ERROR HANDLING](#debugging-and-error-handling)
[EXAMPLE](#example)
[BUGS](#bugs)
[ACKNOWLEDGEMENTS](#acknowledgements)
[SEE ALSO](#see-also) # NAME # **libpmemlog** - persistent memory resident log file # SYNOPSIS # ```c #include cc ... -lpmemlog -lpmem ``` _UNICODE() ##### Library API versioning: ##### ```c _UWFUNC(pmemlog_check_version, =q= unsigned major_required, unsigned minor_required=e=) ``` ##### Managing library behavior: ##### ```c void pmemlog_set_funcs( void *(*malloc_func)(size_t size), void (*free_func)(void *ptr), void *(*realloc_func)(void *ptr, size_t size), char *(*strdup_func)(const char *s)); ``` ##### Error handling: ##### ```c _UWFUNCR(int, pmemlog_check, *path) ``` ##### Other library functions: ##### A description of other **libpmemlog** functions can be found on the following manual pages: **pmemlog_append**(3), **pmemlog_create**(3), **pmemlog_ctl_exec**(3), **pmemlog_ctl_get**(3), **pmemlog_ctl_set**(3), **pmemlog_nbyte**(3), **pmemlog_tell**(3) # DESCRIPTION # **libpmemlog** provides a log file in *persistent memory* (pmem) such that additions to the log are appended atomically. This library is intended for applications using direct access storage (DAX), which is storage that supports load/store access without paging blocks from a block storage device. Some types of *non-volatile memory DIMMs* (NVDIMMs) provide this type of byte addressable access to storage. A *persistent memory aware file system* is typically used to expose the direct access to applications. Memory mapping a file from this type of file system results in the load/store, non-paged access to pmem. **libpmemlog** builds on thistype of memory mapped file. This library is for applications that need a persistent log file updated atomically (the updates cannot be *torn* by program interruption such as power failures). This library builds on the low-level pmem support provided by **libpmem**(7), handling the transactional update of the log, flushing to persistence, and recovery for the application. **libpmemlog** is one of a collection of persistent memory libraries available. The others are: + **libpmemobj**(7), a general use persistent memory API, providing memory allocation and transactional operations on variable-sized objects. + **libpmemblk**(7), providing pmem-resident arrays of fixed-sized blocks with atomic updates. + **libpmem**(7), low-level persistent memory support. Under normal usage, **libpmemlog** will never print messages or intentionally cause the process to exit. The only exception to this is the debugging information, when enabled, as described under **DEBUGGING AND ERROR HANDLING** below. To use the pmem-resident log file provided by **libpmemlog**, a *memory pool* is first created. This is done with the **pmemlog_create**(3) function. The other functions mentioned above in SYNOPSIS section then operate on the resulting log memory pool. Once created, the memory pool is represented by an opaque handle, of type *PMEMlogpool\**, which is passed to most of the other functions from **libpmemlog**. Internally, **libpmemlog** will use either **pmem_persist**(3) or **msync**(2) when it needs to flush changes, depending on whether the memory pool appears to be persistent memory or a regular file (see the **pmem_is_pmem**(3) function in **libpmem**(7) for more information). There is no need for applications to flush changes directly when using the log memory API provided by **libpmemlog**. # CAVEATS # **libpmemlog** relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. **dlclose**(3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. # LIBRARY API VERSIONING # This section describes how the library API is versioned, allowing applications to work with an evolving API. The _UW(pmemlog_check_version) function is used to determine whether the installed **libpmemlog** supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile-time version information provided by defines in **\**, like this: ```c reason = _U(pmemlog_check_version)(PMEMLOG_MAJOR_VERSION, PMEMLOG_MINOR_VERSION); if (reason != NULL) { /* version check failed, reason string tells you why */ } ``` Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text *introduced in version x.y* in the section of this manual describing the feature. On success, _UW(pmemlog_check_version) returns NULL. Otherwise, the return value is a static string describing the reason the version check failed. The string returned by _UW(pmemlog_check_version) must not be modified or freed. # MANAGING LIBRARY BEHAVIOR # The **pmemlog_set_funcs**() function allows an application to override memory allocation calls used internally by **libpmemlog**. Passing in NULL for any of the handlers will cause the **libpmemlog** default function to be used. The library does not make heavy use of the system malloc functions, but it does allocate approximately 4-8 kilobytes for each memory pool in use. # DEBUGGING AND ERROR HANDLING # The _UW(pmemlog_errormsg) function returns a pointer to a static buffer containing the last error message logged for the current thread. If *errno* was set, the error message may include a description of the corresponding error code as returned by **strerror**(3). The error message buffer is thread-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a **libpmemlog** function indicated an error, or if *errno* was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. Two versions of **libpmemlog** are typically available on a development system. The normal version, accessed when a program is linked using the **-lpmemlog** option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run-time assertions. A second version of **libpmemlog**, accessed when a program uses the libraries under _DEBUGLIBPATH(), contains run-time assertions and trace points. The typical way to access the debug version is to set the environment variable **LD_LIBRARY_PATH** to _LDLIBPATH(). Debugging output is controlled using the following environment variables. These variables have no effect on the non-debug version of the library. + **PMEMLOG_LOG_LEVEL** The value of **PMEMLOG_LOG_LEVEL** enables trace points in the debug version of the library, as follows: + **0** - This is the default level when **PMEMLOG_LOG_LEVEL** is not set. No log messages are emitted at this level. + **1** - Additional details on any errors detected are logged, in addition to returning the *errno*-based errors as usual. The same information may be retrieved using _UW(pmemlog_errormsg). + **2** - A trace of basic operations is logged. + **3** - Enables a very verbose amount of function call tracing in the library. + **4** - Enables voluminous and fairly obscure tracing information that is likely only useful to the **libpmemlog** developers. Unless **PMEMLOG_LOG_FILE** is set, debugging output is written to *stderr*. + **PMEMLOG_LOG_FILE** Specifies the name of a file name where all logging information should be written. If the last character in the name is "-", the *PID* of the current process will be appended to the file name when the log file is created. If **PMEMLOG_LOG_FILE** is not set, logging output is written to *stderr*. See also **libpmem**(7) for information about other environment variables affecting **libpmemlog** behavior. # EXAMPLE # The following example illustrates how the **libpmemlog** API is used. ```c #include #include #include #include #include #include #include /* size of the pmemlog pool -- 1 GB */ #define POOL_SIZE ((size_t)(1 << 30)) /* * printit -- log processing callback for use with pmemlog_walk() */ int printit(const void *buf, size_t len, void *arg) { fwrite(buf, len, 1, stdout); return 0; } int main(int argc, char *argv[]) { const char path[] = "/pmem-fs/myfile"; PMEMlogpool *plp; size_t nbyte; char *str; /* create the pmemlog pool or open it if it already exists */ plp = _U(pmemlog_create)(path, POOL_SIZE, 0666); if (plp == NULL) plp = _U(pmemlog_open)(path); if (plp == NULL) { perror(path); exit(1); } /* how many bytes does the log hold? */ nbyte = pmemlog_nbyte(plp); printf("log holds %zu bytes", nbyte); /* append to the log... */ str = "This is the first string appended"; if (pmemlog_append(plp, str, strlen(str)) < 0) { perror("pmemlog_append"); exit(1); } str = "This is the second string appended"; if (pmemlog_append(plp, str, strlen(str)) < 0) { perror("pmemlog_append"); exit(1); } /* print the log contents */ printf("log contains:"); pmemlog_walk(plp, 0, printit, NULL); pmemlog_close(plp); } ``` See for more examples using the **libpmemlog** API. # BUGS # Unlike **libpmemobj**(7), data replication is not supported in **libpmemlog**. Thus, specifying replica sections in pool set files is not allowed. # ACKNOWLEDGEMENTS # **libpmemlog** builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: # SEE ALSO # **msync**(2), **pmemlog_append**(3), **pmemlog_create**(3), **pmemlog_ctl_exec**(3), **pmemlog_ctl_get**(3), **pmemlog_ctl_set**(3), **pmemlog_nbyte**(3), **pmemlog_tell**(3), **strerror**(3), **libpmem**(7), **libpmemblk**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemlog/pmemlog_ctl_get.3.md0000664000000000000000000001262313615011243017535 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMLOG_CTL_GET, 3) collection: libpmemlog header: PMDK date: pmemlog API version 1.1 ... [comment]: <> (Copyright 2018-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemlog_ctl_get.3 -- man page for libpmemlog CTL) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[CTL NAMESPACE](#ctl-namespace)
[CTL EXTERNAL CONFIGURATION](#ctl-external-configuration)
[SEE ALSO](#see-also)
# NAME # _UW(pmemlog_ctl_get), _UW(pmemlog_ctl_set), _UW(pmemlog_ctl_exec) - Query and modify libpmemlog internal behavior (EXPERIMENTAL) # SYNOPSIS # ```c #include _UWFUNCR2(int, pmemlog_ctl_get, PMEMlogpool *plp, *name, void *arg, =q= (EXPERIMENTAL)=e=) _UWFUNCR2(int, pmemlog_ctl_set, PMEMlogpool *plp, *name, void *arg, =q= (EXPERIMENTAL)=e=) _UWFUNCR2(int, pmemlog_ctl_exec, PMEMlogpool *plp, *name, void *arg, =q= (EXPERIMENTAL)=e=) ``` _UNICODE() # DESCRIPTION # The _UW(pmemlog_ctl_get), _UW(pmemlog_ctl_set) and _UW(pmemlog_ctl_exec) functions provide a uniform interface for querying and modifying the internal behavior of **libpmemlog**(7) through the control (CTL) namespace. The *name* argument specifies an entry point as defined in the CTL namespace specification. The entry point description specifies whether the extra *arg* is required. Those two parameters together create a CTL query. The functions and the entry points are thread-safe unless indicated otherwise below. If there are special conditions for calling an entry point, they are explicitly stated in its description. The functions propagate the return value of the entry point. If either *name* or *arg* is invalid, -1 is returned. If the provided ctl query is valid, the CTL functions will always return 0 on success and -1 on failure, unless otherwise specified in the entry point description. See more in **pmem_ctl**(5) man page. # CTL NAMESPACE # prefault.at_create | rw | global | int | int | - | boolean If set, every page of the pool will be touched and written to when the pool is created, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the _UW(pmemlog_create) function. Always returns 0. prefault.at_open | rw | global | int | int | - | boolean If set, every page of the pool will be touched and written to when the pool is opened, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the _UW(pmemlog_open) function. Always returns 0. sds.at_create | rw | global | int | int | - | boolean If set, force-enables or force-disables SDS feature during pool creation. Affects only the _UW(pmemlog_create) function. See **pmempool_feature_query**(3) for information about SDS (SHUTDOWN_STATE) feature. Always returns 0. copy_on_write.at_open | rw | global | int | int | - | boolean If set, pool is mapped in such a way that modifications don't reach the underlying medium. From the user's perspective this means that when the pool is closed all changes are reverted. This feature is not supported for pools located on Device DAX. Always returns 0. # CTL EXTERNAL CONFIGURATION # In addition to direct function call, each write entry point can also be set using two alternative methods. The first method is to load a configuration directly from the **PMEMLOG_CONF** environment variable. The second method of loading an external configuration is to set the **PMEMLOG_CONF_FILE** environment variable to point to a file that contains a sequence of ctl queries. See more in **pmem_ctl**(5) man page. # SEE ALSO # **libpmemlog**(7), **pmem_ctl**(5) and **** pmdk-1.8/doc/libpmemlog/pmemlog_nbyte.3.md0000664000000000000000000000543713615011243017242 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMLOG_NBYTE, 3) collection: libpmemlog header: PMDK date: pmemlog API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemlog_nbyte.3 -- man page for pmemlog_nbyte function) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemlog_nbyte**() - checks the amount of usable space in the log pool. # SYNOPSIS # ```c #include size_t pmemlog_nbyte(PMEMlogpool *plp); ``` # DESCRIPTION # The **pmemlog_nbyte**() function checks the amount of usable space in the log *plp*. This function may be used on a log to determine how much usable space is available after **libpmemlog**(7) has added its metadata to the memory pool. # RETURN VALUE # The **pmemlog_nbyte**() function returns the amount of usable space in the log *plp*. # SEE ALSO # **libpmemlog**(7) and **** pmdk-1.8/doc/libpmemlog/pmemlog_close.30000664000000000000000000000002513615011243016613 0ustar rootroot.so pmemlog_create.3 pmdk-1.8/doc/libpmemlog/pmemlog_tell.30000644000000000000000000000770013615011417016456 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMLOG_TELL" "3" "2020-01-31" "PMDK - pmemlog API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemlog_tell\f[](), \f[B]pmemlog_rewind\f[](), \f[B]pmemlog_walk\f[]() \- checks current write point for the log or walks through the log .SH SYNOPSIS .IP .nf \f[C] #include\ long\ long\ pmemlog_tell(PMEMlogpool\ *plp); void\ pmemlog_rewind(PMEMlogpool\ *plp); void\ pmemlog_walk(PMEMlogpool\ *plp,\ size_t\ chunksize, \ \ \ \ int\ (*process_chunk)(const\ void\ *buf,\ size_t\ len,\ void\ *arg), \ \ \ \ void\ *arg); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemlog_tell\f[]() function returns the current write point for the log, expressed as a byte offset into the usable log space in the memory pool. This offset starts off as zero on a newly\-created log, and is incremented by each successful append operation. This function can be used to determine how much data is currently in the log. .PP The \f[B]pmemlog_rewind\f[]() function resets the current write point for the log to zero. After this call, the next append adds to the beginning of the log. .PP The \f[B]pmemlog_walk\f[]() function walks through the log \f[I]plp\f[], from beginning to end, calling the callback function \f[I]process_chunk\f[] for each \f[I]chunksize\f[] block of data found. The argument \f[I]arg\f[] is also passed to the callback to help avoid the need for global state. The \f[I]chunksize\f[] argument is useful for logs with fixed\-length records and may be specified as 0 to cause a single call to the callback with the entire log contents passed as the \f[I]buf\f[] argument. The \f[I]len\f[] argument tells the \f[I]process_chunk\f[] function how much data \f[I]buf\f[] is holding. The callback function should return 1 if \f[B]pmemlog_walk\f[]() should continue walking through the log, or 0 to terminate the walk. The callback function is called while holding \f[B]libpmemlog\f[](7) internal locks that make calls atomic, so the callback function must not try to append to the log itself or deadlock will occur. .SH RETURN VALUE .PP On success, \f[B]pmemlog_tell\f[]() returns the current write point for the log. On error, it returns \-1 and sets \f[I]errno\f[] appropriately. .PP The \f[B]pmemlog_rewind\f[]() and \f[B]pmemlog_walk\f[]() functions return no value. .SH SEE ALSO .PP \f[B]libpmemlog\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemlog/pmemlog_ctl_exec.30000664000000000000000000000002613615011243017275 0ustar rootroot.so pmemlog_ctl_get.3 pmdk-1.8/doc/libpmemlog/pmemlog_nbyte.30000644000000000000000000000457613615011417016647 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMLOG_NBYTE" "3" "2020-01-31" "PMDK - pmemlog API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemlog_nbyte\f[]() \- checks the amount of usable space in the log pool. .SH SYNOPSIS .IP .nf \f[C] #include\ size_t\ pmemlog_nbyte(PMEMlogpool\ *plp); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemlog_nbyte\f[]() function checks the amount of usable space in the log \f[I]plp\f[]. This function may be used on a log to determine how much usable space is available after \f[B]libpmemlog\f[](7) has added its metadata to the memory pool. .SH RETURN VALUE .PP The \f[B]pmemlog_nbyte\f[]() function returns the amount of usable space in the log \f[I]plp\f[]. .SH SEE ALSO .PP \f[B]libpmemlog\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemlog/.gitignore0000664000000000000000000000014013615011243015670 0ustar rootrootlibpmemlog.7 pmemlog_append.3 pmemlog_create.3 pmemlog_ctl_get.3 pmemlog_nbyte.3 pmemlog_tell.3 pmdk-1.8/doc/libpmemlog/pmemlog_open.30000664000000000000000000000002513615011243016447 0ustar rootroot.so pmemlog_create.3 pmdk-1.8/doc/libpmemlog/pmemlog_walk.30000664000000000000000000000002313615011243016442 0ustar rootroot.so pmemlog_tell.3 pmdk-1.8/doc/libpmemlog/libpmemlog.70000644000000000000000000003220713615011416016130 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "LIBPMEMLOG" "7" "2020-01-31" "PMDK - pmemlog API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]libpmemlog\f[] \- persistent memory resident log file .SH SYNOPSIS .IP .nf \f[C] #include\ cc\ ...\ \-lpmemlog\ \-lpmem \f[] .fi .SS Library API versioning: .IP .nf \f[C] const\ char\ *pmemlog_check_version( \ \ \ \ unsigned\ major_required, \ \ \ \ unsigned\ minor_required); \f[] .fi .SS Managing library behavior: .IP .nf \f[C] void\ pmemlog_set_funcs( \ \ \ \ void\ *(*malloc_func)(size_t\ size), \ \ \ \ void\ (*free_func)(void\ *ptr), \ \ \ \ void\ *(*realloc_func)(void\ *ptr,\ size_t\ size), \ \ \ \ char\ *(*strdup_func)(const\ char\ *s)); \f[] .fi .SS Error handling: .IP .nf \f[C] int\ pmemlog_check(const\ char\ *path); \f[] .fi .SS Other library functions: .PP A description of other \f[B]libpmemlog\f[] functions can be found on the following manual pages: .PP \f[B]pmemlog_append\f[](3), \f[B]pmemlog_create\f[](3), \f[B]pmemlog_ctl_exec\f[](3), \f[B]pmemlog_ctl_get\f[](3), \f[B]pmemlog_ctl_set\f[](3), \f[B]pmemlog_nbyte\f[](3), \f[B]pmemlog_tell\f[](3) .SH DESCRIPTION .PP \f[B]libpmemlog\f[] provides a log file in \f[I]persistent memory\f[] (pmem) such that additions to the log are appended atomically. This library is intended for applications using direct access storage (DAX), which is storage that supports load/store access without paging blocks from a block storage device. Some types of \f[I]non\-volatile memory DIMMs\f[] (NVDIMMs) provide this type of byte addressable access to storage. A \f[I]persistent memory aware file system\f[] is typically used to expose the direct access to applications. Memory mapping a file from this type of file system results in the load/store, non\-paged access to pmem. \f[B]libpmemlog\f[] builds on thistype of memory mapped file. .PP This library is for applications that need a persistent log file updated atomically (the updates cannot be \f[I]torn\f[] by program interruption such as power failures). This library builds on the low\-level pmem support provided by \f[B]libpmem\f[](7), handling the transactional update of the log, flushing to persistence, and recovery for the application. .PP \f[B]libpmemlog\f[] is one of a collection of persistent memory libraries available. The others are: .IP \[bu] 2 \f[B]libpmemobj\f[](7), a general use persistent memory API, providing memory allocation and transactional operations on variable\-sized objects. .IP \[bu] 2 \f[B]libpmemblk\f[](7), providing pmem\-resident arrays of fixed\-sized blocks with atomic updates. .IP \[bu] 2 \f[B]libpmem\f[](7), low\-level persistent memory support. .PP Under normal usage, \f[B]libpmemlog\f[] will never print messages or intentionally cause the process to exit. The only exception to this is the debugging information, when enabled, as described under \f[B]DEBUGGING AND ERROR HANDLING\f[] below. .PP To use the pmem\-resident log file provided by \f[B]libpmemlog\f[], a \f[I]memory pool\f[] is first created. This is done with the \f[B]pmemlog_create\f[](3) function. The other functions mentioned above in SYNOPSIS section then operate on the resulting log memory pool. .PP Once created, the memory pool is represented by an opaque handle, of type \f[I]PMEMlogpool*\f[], which is passed to most of the other functions from \f[B]libpmemlog\f[]. Internally, \f[B]libpmemlog\f[] will use either \f[B]pmem_persist\f[](3) or \f[B]msync\f[](2) when it needs to flush changes, depending on whether the memory pool appears to be persistent memory or a regular file (see the \f[B]pmem_is_pmem\f[](3) function in \f[B]libpmem\f[](7) for more information). There is no need for applications to flush changes directly when using the log memory API provided by \f[B]libpmemlog\f[]. .SH CAVEATS .PP \f[B]libpmemlog\f[] relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. \f[B]dlclose\f[](3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. .SH LIBRARY API VERSIONING .PP This section describes how the library API is versioned, allowing applications to work with an evolving API. .PP The \f[B]pmemlog_check_version\f[]() function is used to determine whether the installed \f[B]libpmemlog\f[] supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile\-time version information provided by defines in \f[B]\f[], like this: .IP .nf \f[C] reason\ =\ pmemlog_check_version(PMEMLOG_MAJOR_VERSION, \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ PMEMLOG_MINOR_VERSION); if\ (reason\ !=\ NULL)\ { \ \ \ \ /*\ version\ check\ failed,\ reason\ string\ tells\ you\ why\ */ } \f[] .fi .PP Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. .PP An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text \f[I]introduced in version x.y\f[] in the section of this manual describing the feature. .PP On success, \f[B]pmemlog_check_version\f[]() returns NULL. Otherwise, the return value is a static string describing the reason the version check failed. The string returned by \f[B]pmemlog_check_version\f[]() must not be modified or freed. .SH MANAGING LIBRARY BEHAVIOR .PP The \f[B]pmemlog_set_funcs\f[]() function allows an application to override memory allocation calls used internally by \f[B]libpmemlog\f[]. Passing in NULL for any of the handlers will cause the \f[B]libpmemlog\f[] default function to be used. The library does not make heavy use of the system malloc functions, but it does allocate approximately 4\-8 kilobytes for each memory pool in use. .SH DEBUGGING AND ERROR HANDLING .PP The \f[B]pmemlog_errormsg\f[]() function returns a pointer to a static buffer containing the last error message logged for the current thread. If \f[I]errno\f[] was set, the error message may include a description of the corresponding error code as returned by \f[B]strerror\f[](3). The error message buffer is thread\-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a \f[B]libpmemlog\f[] function indicated an error, or if \f[I]errno\f[] was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. .PP Two versions of \f[B]libpmemlog\f[] are typically available on a development system. The normal version, accessed when a program is linked using the \f[B]\-lpmemlog\f[] option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run\-time assertions. .PP A second version of \f[B]libpmemlog\f[], accessed when a program uses the libraries under \f[B]/usr/lib/pmdk_debug\f[], contains run\-time assertions and trace points. The typical way to access the debug version is to set the environment variable \f[B]LD_LIBRARY_PATH\f[] to \f[B]/usr/lib/pmdk_debug\f[] or \f[B]/usr/lib64/pmdk_debug\f[], as appropriate. Debugging output is controlled using the following environment variables. These variables have no effect on the non\-debug version of the library. .IP \[bu] 2 \f[B]PMEMLOG_LOG_LEVEL\f[] .PP The value of \f[B]PMEMLOG_LOG_LEVEL\f[] enables trace points in the debug version of the library, as follows: .IP \[bu] 2 \f[B]0\f[] \- This is the default level when \f[B]PMEMLOG_LOG_LEVEL\f[] is not set. No log messages are emitted at this level. .IP \[bu] 2 \f[B]1\f[] \- Additional details on any errors detected are logged, in addition to returning the \f[I]errno\f[]\-based errors as usual. The same information may be retrieved using \f[B]pmemlog_errormsg\f[](). .IP \[bu] 2 \f[B]2\f[] \- A trace of basic operations is logged. .IP \[bu] 2 \f[B]3\f[] \- Enables a very verbose amount of function call tracing in the library. .IP \[bu] 2 \f[B]4\f[] \- Enables voluminous and fairly obscure tracing information that is likely only useful to the \f[B]libpmemlog\f[] developers. .PP Unless \f[B]PMEMLOG_LOG_FILE\f[] is set, debugging output is written to \f[I]stderr\f[]. .IP \[bu] 2 \f[B]PMEMLOG_LOG_FILE\f[] .PP Specifies the name of a file name where all logging information should be written. If the last character in the name is \[lq]\-\[rq], the \f[I]PID\f[] of the current process will be appended to the file name when the log file is created. If \f[B]PMEMLOG_LOG_FILE\f[] is not set, logging output is written to \f[I]stderr\f[]. .PP See also \f[B]libpmem\f[](7) for information about other environment variables affecting \f[B]libpmemlog\f[] behavior. .SH EXAMPLE .PP The following example illustrates how the \f[B]libpmemlog\f[] API is used. .IP .nf \f[C] #include\ #include\ #include\ #include\ #include\ #include\ #include\ /*\ size\ of\ the\ pmemlog\ pool\ \-\-\ 1\ GB\ */ #define\ POOL_SIZE\ ((size_t)(1\ <<\ 30)) /* \ *\ printit\ \-\-\ log\ processing\ callback\ for\ use\ with\ pmemlog_walk() \ */ int printit(const\ void\ *buf,\ size_t\ len,\ void\ *arg) { \ \ \ \ fwrite(buf,\ len,\ 1,\ stdout); \ \ \ \ return\ 0; } int main(int\ argc,\ char\ *argv[]) { \ \ \ \ const\ char\ path[]\ =\ "/pmem\-fs/myfile"; \ \ \ \ PMEMlogpool\ *plp; \ \ \ \ size_t\ nbyte; \ \ \ \ char\ *str; \ \ \ \ /*\ create\ the\ pmemlog\ pool\ or\ open\ it\ if\ it\ already\ exists\ */ \ \ \ \ plp\ =\ pmemlog_create(path,\ POOL_SIZE,\ 0666); \ \ \ \ if\ (plp\ ==\ NULL) \ \ \ \ \ \ \ \ plp\ =\ pmemlog_open(path); \ \ \ \ if\ (plp\ ==\ NULL)\ { \ \ \ \ \ \ \ \ perror(path); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ /*\ how\ many\ bytes\ does\ the\ log\ hold?\ */ \ \ \ \ nbyte\ =\ pmemlog_nbyte(plp); \ \ \ \ printf("log\ holds\ %zu\ bytes",\ nbyte); \ \ \ \ /*\ append\ to\ the\ log...\ */ \ \ \ \ str\ =\ "This\ is\ the\ first\ string\ appended"; \ \ \ \ if\ (pmemlog_append(plp,\ str,\ strlen(str))\ <\ 0)\ { \ \ \ \ \ \ \ \ perror("pmemlog_append"); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ str\ =\ "This\ is\ the\ second\ string\ appended"; \ \ \ \ if\ (pmemlog_append(plp,\ str,\ strlen(str))\ <\ 0)\ { \ \ \ \ \ \ \ \ perror("pmemlog_append"); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ /*\ print\ the\ log\ contents\ */ \ \ \ \ printf("log\ contains:"); \ \ \ \ pmemlog_walk(plp,\ 0,\ printit,\ NULL); \ \ \ \ pmemlog_close(plp); } \f[] .fi .PP See for more examples using the \f[B]libpmemlog\f[] API. .SH BUGS .PP Unlike \f[B]libpmemobj\f[](7), data replication is not supported in \f[B]libpmemlog\f[]. Thus, specifying replica sections in pool set files is not allowed. .SH ACKNOWLEDGEMENTS .PP \f[B]libpmemlog\f[] builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: .SH SEE ALSO .PP \f[B]msync\f[](2), \f[B]pmemlog_append\f[](3), \f[B]pmemlog_create\f[](3), \f[B]pmemlog_ctl_exec\f[](3), \f[B]pmemlog_ctl_get\f[](3), \f[B]pmemlog_ctl_set\f[](3), \f[B]pmemlog_nbyte\f[](3), \f[B]pmemlog_tell\f[](3), \f[B]strerror\f[](3), \f[B]libpmem\f[](7), \f[B]libpmemblk\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemlog/pmemlog_errormsg.30000664000000000000000000000002613615011243017347 0ustar rootroot.so man7/libpmemlog.7 pmdk-1.8/doc/libpmemlog/pmemlog_set_funcs.30000664000000000000000000000002613615011243017500 0ustar rootroot.so man7/libpmemlog.7 pmdk-1.8/doc/libpmemlog/pmemlog_rewind.30000664000000000000000000000002313615011243016774 0ustar rootroot.so pmemlog_tell.3 pmdk-1.8/doc/libpmemlog/pmemlog_create.30000644000000000000000000001717413615011416016766 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMLOG_CREATE" "3" "2020-01-31" "PMDK - pmemlog API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemlog_create\f[](), \f[B]pmemlog_open\f[](), \f[B]pmemlog_close\f[](), \f[B]pmemlog_check\f[]() \- create, open, close and validate persistent memory resident log file .SH SYNOPSIS .IP .nf \f[C] #include\ PMEMlogpool\ *pmemlog_open(const\ char\ *path); PMEMlogpool\ *pmemlog_create(const\ char\ *path,\ size_t\ poolsize,\ mode_t\ mode); void\ pmemlog_close(PMEMlogpool\ *plp); int\ pmemlog_check(const\ char\ *path); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemlog_create\f[]() function creates a log memory pool with the given total \f[I]poolsize\f[]. Since the transactional nature of a log memory pool requires some space overhead in the memory pool, the resulting available log size is less than \f[I]poolsize\f[], and is made available to the caller via the \f[B]pmemlog_nbyte\f[](3) function. \f[I]path\f[] specifies the name of the memory pool file to be created. \f[I]mode\f[] specifies the permissions to use when creating the file as described by \f[B]creat\f[](2). The memory pool file is fully allocated to the size \f[I]poolsize\f[] using \f[B]posix_fallocate\f[](3). The caller may choose to take responsibility for creating the memory pool file by creating it before calling \f[B]pmemlog_create\f[]() and then specifying \f[I]poolsize\f[] as zero. In this case \f[B]pmemlog_create\f[]() will take the pool size from the size of the existing file and will verify that the file appears to be empty by searching for any non\-zero data in the pool header at the beginning of the file. The net pool size of a pool file is equal to the file size. The minimum net pool size allowed by the library for a log pool is defined in \f[B]\f[] as \f[B]PMEMLOG_MIN_POOL\f[]. .PP Depending on the configuration of the system, the available non\-volatile memory space may be divided into multiple memory devices. In such case, the maximum size of the pmemlog memory pool could be limited by the capacity of a single memory device. \f[B]libpmemlog\f[](7) allows building persistent memory resident logs spanning multiple memory devices by creation of persistent memory pools consisting of multiple files, where each part of such a \f[I]pool set\f[] may be stored on a different memory device or pmem\-aware filesystem. .PP Creation of all the parts of the pool set can be done with \f[B]pmemlog_create\f[](); however, the recommended method for creating pool sets is with the \f[B]pmempool\f[](1) utility. .PP When creating a pool set consisting of multiple files, the \f[I]path\f[] argument passed to \f[B]pmemlog_create\f[]() must point to the special \f[I]set\f[] file that defines the pool layout and the location of all the parts of the pool set. The \f[I]poolsize\f[] argument must be 0. The meaning of the \f[I]mode\f[] argument does not change, except that the same \f[I]mode\f[] is used for creation of all the parts of the pool set. .PP The set file is a plain text file, the structure of which is described in \f[B]poolset\f[](5). .PP The \f[B]pmemlog_open\f[]() function opens an existing log memory pool. Similar to \f[B]pmemlog_create\f[](), \f[I]path\f[] must identify either an existing log memory pool file, or the \f[I]set\f[] file used to create a pool set. The application must have permission to open the file and memory map the file or pool set with read/write permissions. .PP Be aware that if the pool contains bad blocks inside, opening can be aborted by the SIGBUS signal, because currently the pool is not checked against bad blocks during opening. It can be turned on by setting the CHECK_BAD_BLOCKS compat feature. For details see description of this feature in \f[B]pmempool\-feature\f[](1). .PP The \f[B]pmemlog_close\f[]() function closes the memory pool indicated by \f[I]plp\f[] and deletes the memory pool handle. The log memory pool itself lives on in the file that contains it and may be re\-opened at a later time using \f[B]pmemlog_open\f[]() as described above. .PP The \f[B]pmemlog_check\f[]() function performs a consistency check of the file indicated by \f[I]path\f[]. \f[B]pmemlog_check\f[]() opens the given \f[I]path\f[] read\-only so it never makes any changes to the file. This function is not supported on Device DAX. .SH RETURN VALUE .PP On success, \f[B]pmemlog_create\f[]() returns a \f[I]PMEMlogpool*\f[] handle to the memory pool that is used with most of the functions from \f[B]libpmemlog\f[](7). If an error prevents any of the pool set files from being created, it returns NULL and sets \f[I]errno\f[] appropriately. .PP On success, \f[B]pmemlog_open\f[]() returns a \f[I]PMEMlogpool*\f[] handle to the memory pool that is used with most of the functions from \f[B]libpmemlog\f[](7). If an error prevents the pool from being opened, or a pool set is being opened and the actual size of any file does not match the corresponding part size defined in the \f[I]set\f[] file, \f[B]pmemlog_open\f[]() returns NULL and sets \f[I]errno\f[] appropriately. .PP The \f[B]pmemlog_close\f[]() function returns no value. .PP The \f[B]pmemlog_check\f[]() function returns 1 if the persistent memory resident log file is found to be consistent. Any inconsistencies will cause \f[B]pmemlog_check\f[]() to return 0, in which case the use of the file with \f[B]libpmemlog\f[] will result in undefined behavior. The debug version of \f[B]libpmemlog\f[] will provide additional details on inconsistencies when \f[B]PMEMLOG_LOG_LEVEL\f[] is at least 1, as described in the \f[B]DEBUGGING AND ERROR HANDLING\f[] section in \f[B]libpmemlog\f[](7). \f[B]pmemlog_check\f[]() will return \-1 and set \f[I]errno\f[] if it cannot perform the consistency check due to other errors. .SH CAVEATS .PP Not all file systems support \f[B]posix_fallocate\f[](3). \f[B]pmemlog_create\f[]() will fail if the underlying file system does not support \f[B]posix_fallocate\f[](3). .SH SEE ALSO .PP \f[B]pmempool\f[](1), \f[B]creat\f[](2), \f[B]posix_fallocate\f[](3), \f[B]pmemlog_nbyte\f[](3), \f[B]poolset\f[](5), \f[B]libpmemlog\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemlog/pmemlog_check.30000664000000000000000000000002513615011243016563 0ustar rootroot.so pmemlog_create.3 pmdk-1.8/doc/libpmemlog/pmemlog_check_version.30000664000000000000000000000002613615011243020331 0ustar rootroot.so man7/libpmemlog.7 pmdk-1.8/doc/libpmemlog/pmemlog_create.3.md0000664000000000000000000001726213615011243017363 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMLOG_CREATE, 3) collection: libpmemlog header: PMDK date: pmemlog API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemlog_create.3 -- man page for libpmemlog create, open, close and validate) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[CAVEATS](#caveats)
[SEE ALSO](#see-also)
# NAME # _UW(pmemlog_create), _UW(pmemlog_open), **pmemlog_close**(), _UW(pmemlog_check) - create, open, close and validate persistent memory resident log file # SYNOPSIS # ```c #include _UWFUNCR(PMEMlogpool, *pmemlog_open, *path) _UWFUNCR1(PMEMlogpool, *pmemlog_create, *path, =q=size_t poolsize, mode_t mode=e=) void pmemlog_close(PMEMlogpool *plp); _UWFUNCR(int, pmemlog_check, *path) ``` _UNICODE() # DESCRIPTION # The _UW(pmemlog_create) function creates a log memory pool with the given total *poolsize*. Since the transactional nature of a log memory pool requires some space overhead in the memory pool, the resulting available log size is less than *poolsize*, and is made available to the caller via the **pmemlog_nbyte**(3) function. *path* specifies the name of the memory pool file to be created. *mode* specifies the permissions to use when creating the file as described by **creat**(2). The memory pool file is fully allocated to the size *poolsize* using **posix_fallocate**(3). The caller may choose to take responsibility for creating the memory pool file by creating it before calling _UW(pmemlog_create) and then specifying *poolsize* as zero. In this case _UW(pmemlog_create) will take the pool size from the size of the existing file and will verify that the file appears to be empty by searching for any non-zero data in the pool header at the beginning of the file. The net pool size of a pool file is equal to the file size. The minimum net pool size allowed by the library for a log pool is defined in **\** as **PMEMLOG_MIN_POOL**. Depending on the configuration of the system, the available non-volatile memory space may be divided into multiple memory devices. In such case, the maximum size of the pmemlog memory pool could be limited by the capacity of a single memory device. **libpmemlog**(7) allows building persistent memory resident logs spanning multiple memory devices by creation of persistent memory pools consisting of multiple files, where each part of such a *pool set* may be stored on a different memory device or pmem-aware filesystem. Creation of all the parts of the pool set can be done with _UW(pmemlog_create); however, the recommended method for creating pool sets is with the **pmempool**(1) utility. When creating a pool set consisting of multiple files, the *path* argument passed to _UW(pmemlog_create) must point to the special *set* file that defines the pool layout and the location of all the parts of the pool set. The *poolsize* argument must be 0. The meaning of the *mode* argument does not change, except that the same *mode* is used for creation of all the parts of the pool set. The set file is a plain text file, the structure of which is described in **poolset**(5). The _UW(pmemlog_open) function opens an existing log memory pool. Similar to _UW(pmemlog_create), *path* must identify either an existing log memory pool file, or the *set* file used to create a pool set. The application must have permission to open the file and memory map the file or pool set with read/write permissions. Be aware that if the pool contains bad blocks inside, opening can be aborted by the SIGBUS signal, because currently the pool is not checked against bad blocks during opening. It can be turned on by setting the CHECK_BAD_BLOCKS compat feature. For details see description of this feature in **pmempool-feature**(1). The **pmemlog_close**() function closes the memory pool indicated by *plp* and deletes the memory pool handle. The log memory pool itself lives on in the file that contains it and may be re-opened at a later time using _UW(pmemlog_open) as described above. The _UW(pmemlog_check) function performs a consistency check of the file indicated by *path*. _UW(pmemlog_check) opens the given *path* read-only so it never makes any changes to the file. This function is not supported on Device DAX. # RETURN VALUE # On success, _UW(pmemlog_create) returns a *PMEMlogpool\** handle to the memory pool that is used with most of the functions from **libpmemlog**(7). If an error prevents any of the pool set files from being created, it returns NULL and sets *errno* appropriately. On success, _UW(pmemlog_open) returns a *PMEMlogpool\** handle to the memory pool that is used with most of the functions from **libpmemlog**(7). If an error prevents the pool from being opened, or a pool set is being opened and the actual size of any file does not match the corresponding part size defined in the *set* file, _UW(pmemlog_open) returns NULL and sets *errno* appropriately. The **pmemlog_close**() function returns no value. The _UW(pmemlog_check) function returns 1 if the persistent memory resident log file is found to be consistent. Any inconsistencies will cause _UW(pmemlog_check) to return 0, in which case the use of the file with **libpmemlog** will result in undefined behavior. The debug version of **libpmemlog** will provide additional details on inconsistencies when **PMEMLOG_LOG_LEVEL** is at least 1, as described in the **DEBUGGING AND ERROR HANDLING** section in **libpmemlog**(7). _UW(pmemlog_check) will return -1 and set *errno* if it cannot perform the consistency check due to other errors. # CAVEATS # Not all file systems support **posix_fallocate**(3). _UW(pmemlog_create) will fail if the underlying file system does not support **posix_fallocate**(3). # SEE ALSO # **pmempool**(1), **creat**(2), **posix_fallocate**(3), **pmemlog_nbyte**(3), **poolset**(5), **libpmemlog**(7) and **** pmdk-1.8/doc/libpmemlog/pmemlog_ctl_set.30000664000000000000000000000002613615011243017144 0ustar rootroot.so pmemlog_ctl_get.3 pmdk-1.8/doc/libpmemlog/pmemlog_tell.3.md0000664000000000000000000001037513615011243017056 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMLOG_TELL, 3) collection: libpmemlog header: PMDK date: pmemlog API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemlog_tell.3 -- man page for pmemlog_tell, pmemlog_rewind and pmemlog_walk functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemlog_tell**(), **pmemlog_rewind**(), **pmemlog_walk**() - checks current write point for the log or walks through the log # SYNOPSIS # ```c #include long long pmemlog_tell(PMEMlogpool *plp); void pmemlog_rewind(PMEMlogpool *plp); void pmemlog_walk(PMEMlogpool *plp, size_t chunksize, int (*process_chunk)(const void *buf, size_t len, void *arg), void *arg); ``` # DESCRIPTION # The **pmemlog_tell**() function returns the current write point for the log, expressed as a byte offset into the usable log space in the memory pool. This offset starts off as zero on a newly-created log, and is incremented by each successful append operation. This function can be used to determine how much data is currently in the log. The **pmemlog_rewind**() function resets the current write point for the log to zero. After this call, the next append adds to the beginning of the log. The **pmemlog_walk**() function walks through the log *plp*, from beginning to end, calling the callback function *process_chunk* for each *chunksize* block of data found. The argument *arg* is also passed to the callback to help avoid the need for global state. The *chunksize* argument is useful for logs with fixed-length records and may be specified as 0 to cause a single call to the callback with the entire log contents passed as the *buf* argument. The *len* argument tells the *process_chunk* function how much data *buf* is holding. The callback function should return 1 if **pmemlog_walk**() should continue walking through the log, or 0 to terminate the walk. The callback function is called while holding **libpmemlog**(7) internal locks that make calls atomic, so the callback function must not try to append to the log itself or deadlock will occur. # RETURN VALUE # On success, **pmemlog_tell**() returns the current write point for the log. On error, it returns -1 and sets *errno* appropriately. The **pmemlog_rewind**() and **pmemlog_walk**() functions return no value. # SEE ALSO # **libpmemlog**(7) and **** pmdk-1.8/doc/libvmem/0000775000000000000000000000000013615011243013211 5ustar rootrootpmdk-1.8/doc/libvmem/README.md0000664000000000000000000000012613615011243014467 0ustar rootrootThis library has been moved to a [separate repository](https://github.com/pmem/vmem). pmdk-1.8/doc/generated/0000775000000000000000000000000013615011243013514 5ustar rootrootpmdk-1.8/doc/generated/.gitignore0000664000000000000000000000000613615011243015500 0ustar rootroot*.yml pmdk-1.8/doc/generated/Makefile0000664000000000000000000000324513615011243015160 0ustar rootroot# # Copyright 2017, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # generated/Makefile -- Makefile for generate pmem.io aliases map # all: ../../utils/get_aliases.sh clean: $(RM) -rf libs_map.yml pmdk-1.8/doc/libpmempool/0000775000000000000000000000000013615011420014072 5ustar rootrootpmdk-1.8/doc/libpmempool/libpmempool.7.md0000664000000000000000000002503713615011243017112 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(LIBPMEMPOOL, 7) collection: libpmempool header: PMDK date: pmempool API version 1.3 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (libpmempool.7 -- man page for libpmempool) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[LIBRARY API VERSIONING](#library-api-versioning-1)
[DEBUGGING AND ERROR HANDLING](#debugging-and-error-handling)
[EXAMPLE](#example)
[ACKNOWLEDGEMENTS](#acknowledgements)
[SEE ALSO](#see-also)
# NAME # **libpmempool** - persistent memory pool management library # SYNOPSIS # ```c #include cc _WINUX(,-std=gnu99) ... -lpmempool -lpmem ``` _UNICODE() ##### Library API versioning: ##### ```c _UWFUNC(pmempool_check_version, =q= unsigned major_required, unsigned minor_required=e=) ``` ##### Error handling: ##### ```c _UWFUNC(pmempool_errormsg, void) ``` ##### Other library functions: ##### A description of other **libpmempool** functions can be found on the following manual pages: + health check functions: **pmempool_check_init**(3) + pool set synchronization and transformation: **pmempool_sync**(3) + pool set management functions: **pmempool_rm**(3) + toggle or query pool set features: **pmempool_feature_query**(3) # DESCRIPTION # **libpmempool** provides a set of utilities for off-line analysis and manipulation of a *pool*. A *pool* in this manpage means a pmemobj pool, pmemblk pool, pmemlog pool or BTT layout, independent of the underlying storage. Some **libpmempool** functions are required to work without any impact on the *pool* but some may create a new or modify an existing *pool*. **libpmempool** is for applications that need high reliability or built-in troubleshooting. It may be useful for testing and debugging purposes also. **libpmempool** introduces functionality of pool set health check, synchronization, transformation and removal. # CAVEATS # **libpmempool** relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. **dlclose**(3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. _WINUX(,=q=**libpmempool** requires the **-std=gnu99** compilation flag to build properly.=e=) # LIBRARY API VERSIONING # This section describes how the library API is versioned, allowing applications to work with an evolving API. The _UW(pmempool_check_version) function is used to see if the installed **libpmempool** supports the version of the library API required by an application. The easiest way to do this for the application is to supply the compile-time version information, supplied by defines in **\**, like this: ```c reason = _U(pmempool_check_version)(PMEMPOOL_MAJOR_VERSION, PMEMPOOL_MINOR_VERSION); if (reason != NULL) { /* version check failed, reason string tells you why */ } ``` Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text *introduced in version x.y* in the section of this manual describing the feature. When the version check performed by _UW(pmempool_check_version) is successful, the return value is NULL. Otherwise the return value is a static string describing the reason for failing the version check. The string returned by _UW(pmempool_check_version) must not be modified or freed. # DEBUGGING AND ERROR HANDLING # If an error is detected during the call to a **libpmempool** function, the application may retrieve an error message describing the reason for the failure from _UW(pmempool_errormsg). This function returns a pointer to a static buffer containing the last error message logged for the current thread. If *errno* was set, the error message may include a description of the corresponding error code as returned by **strerror**(3). The error message buffer is thread-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a **libpmempool** function indicated an error, or if *errno* was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. Two versions of **libpmempool** are typically available on a development system. The normal version, accessed when a program is linked using the **-lpmempool** option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run-time assertions. A second version of **libpmempool**, accessed when a program uses the libraries under _DEBUGLIBPATH(), contains run-time assertions and trace points. The typical way to access the debug version is to set the environment variable **LD_LIBRARY_PATH** to _LDLIBPATH(). Debugging output is controlled using the following environment variables. These variables have no effect on the non-debug version of the library. + **PMEMPOOL_LOG_LEVEL** The value of **PMEMPOOL_LOG_LEVEL** enables trace points in the debug version of the library, as follows: + **0** - This is the default level when **PMEMPOOL_LOG_LEVEL** is not set. No log messages are emitted at this level. + **1** - Additional details on any errors detected are logged (in addition to returning the *errno*-based errors as usual). The same information may be retrieved using _UW(pmempool_errormsg). + **2** - A trace of basic operations is logged. + **3** - Enables a very verbose amount of function call tracing in the library. + **4** - Enables voluminous and fairly obscure tracing information that is likely only useful to the **libpmempool** developers. Unless **PMEMPOOL_LOG_FILE** is set, debugging output is written to *stderr*. + **PMEMPOOL_LOG_FILE** Specifies the name of a file where all logging information should be written. If the last character in the name is "-", the *PID* of the current process will be appended to the file name when the log file is created. If **PMEMPOOL_LOG_FILE** is not set, output is written to *stderr*. # EXAMPLE # The following example illustrates how the **libpmempool** API is used. The program detects the type and checks consistency of given pool. If there are any issues detected, the pool is automatically repaired. ```c #include _WINUX(,=q= #include =e=) #include #include #include #define PATH "./pmem-fs/myfile" #define CHECK_FLAGS (PMEMPOOL_CHECK_FORMAT_STR|PMEMPOOL_CHECK_REPAIR|\ PMEMPOOL_CHECK_VERBOSE) int main(int argc, char *argv[]) { PMEMpoolcheck *ppc; struct _U(pmempool_check_status) *status; enum pmempool_check_result ret; /* arguments for check */ struct _U(pmempool_check_args) args = { .path = PATH, .backup_path = NULL, .pool_type = PMEMPOOL_POOL_TYPE_DETECT, .flags = CHECK_FLAGS }; /* initialize check context */ if ((ppc = _U(pmempool_check_init)(&args, sizeof(args))) == NULL) { perror("_U(pmempool_check_init)"); exit(EXIT_FAILURE); } /* perform check and repair, answer 'yes' for each question */ while ((status = _U(pmempool_check)(ppc)) != NULL) { switch (status->type) { case PMEMPOOL_CHECK_MSG_TYPE_ERROR: printf("%s\n", status->str.msg); break; case PMEMPOOL_CHECK_MSG_TYPE_INFO: printf("%s\n", status->str.msg); break; case PMEMPOOL_CHECK_MSG_TYPE_QUESTION: printf("%s\n", status->str.msg); status->str.answer = "yes"; break; default: pmempool_check_end(ppc); exit(EXIT_FAILURE); } } /* finalize the check and get the result */ ret = pmempool_check_end(ppc); switch (ret) { case PMEMPOOL_CHECK_RESULT_CONSISTENT: case PMEMPOOL_CHECK_RESULT_REPAIRED: return 0; default: return 1; } } ``` See for more examples using the **libpmempool** API. # ACKNOWLEDGEMENTS # **libpmempool** builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: # SEE ALSO # **dlclose**(3), **pmempool_check_init**(3), **pmempool_feature_query**(3), **pmempool_rm**(3), **pmempool_sync**(3), **strerror**(3), **libpmem**(7), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7)** and **** pmdk-1.8/doc/libpmempool/pmempool_check_version.30000664000000000000000000000002713615011243020712 0ustar rootroot.so man7/libpmempool.7 pmdk-1.8/doc/libpmempool/pmempool_feature_disable.30000664000000000000000000000003513615011243021205 0ustar rootroot.so pmempool_feature_query.3 pmdk-1.8/doc/libpmempool/pmempool_rm.3.md0000664000000000000000000000662013615011243017112 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL_RM, 3) collection: libpmempool header: PMDK date: pmempool API version 1.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool_rm.3 -- man page for pool set management functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # _UW(pmempool_rm) - remove persistent memory pool # SYNOPSIS # ```c #include _UWFUNCR1(int, pmempool_rm, *path, int flags) ``` _UNICODE() # DESCRIPTION # The _UW(pmempool_rm) function removes the pool pointed to by *path*. The *path* can point to a regular file, device dax or pool set file. If *path* is a pool set file, _UW(pmempool_rm) will remove all part files from local replicas using **unlink**(2)_WINUX(,=q=, and all remote replicas using **rpmem_remove**(3) (see **librpmem**(7)),=e=) before removing the pool set file itself. The *flags* argument determines the behavior of _UW(pmempool_rm). It is either 0 or the bitwise OR of one or more of the following flags: + **PMEMPOOL_RM_FORCE** - Ignore all errors when removing part files from local _WINUX(,or remote )replicas. + **PMEMPOOL_RM_POOLSET_LOCAL** - Also remove local pool set file. _WINUX(, + **PMEMPOOL_RM_POOLSET_REMOTE** - Also remove remote pool set file.) # RETURN VALUE # On success, _UW(pmempool_rm) returns 0. On error, it returns -1 and sets *errno* accordingly. # SEE ALSO # **rpmem_remove**(3), **unlink**(3), **libpmemlog**(7), **libpmemobj**(7), **librpmem**(7) and **** pmdk-1.8/doc/libpmempool/pmempool_errormsg.30000664000000000000000000000002713615011243017730 0ustar rootroot.so man7/libpmempool.7 pmdk-1.8/doc/libpmempool/pmempool_feature_enable.30000664000000000000000000000003513615011243021030 0ustar rootroot.so pmempool_feature_query.3 pmdk-1.8/doc/libpmempool/pmempool_feature_query.30000644000000000000000000001145213615011420020747 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL_FEATURE_QUERY" "3" "2020-01-31" "PMDK - pmempool API version 1.3" "PMDK Programmer's Manual" .hy .\" Copyright 2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool_feature_query\f[](), \f[B]pmempool_feature_enable\f[](), \f[B]pmempool_feature_disable\f[]() \- toggle or query pool set features .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmempool_feature_query(const\ char\ *path,\ enum\ pmempool_feature\ feature,\ unsigned\ flags); int\ pmempool_feature_enable(const\ char\ *path,\ enum\ pmempool_feature\ feature,\ unsigned\ flags); int\ pmempool_feature_disable(const\ char\ *path,\ enum\ pmempool_feature\ feature,\ unsigned\ flags); \f[] .fi .SH DESCRIPTION .PP The \f[I]feature\f[] argument accepts following values: .IP \[bu] 2 \f[B]PMEMPOOL_FEAT_SINGLEHDR\f[] \- only the first part in each replica contains the pool part internal metadata. This value can be used only with \f[B]pmempool_feature_query\f[](). It can not be enabled or disabled. For details see \f[B]poolset\f[](5). .IP \[bu] 2 \f[B]PMEMPOOL_FEAT_CKSUM_2K\f[] \- only the first 2KiB of pool part internal metadata is checksummed. Other features may depend on this one to store additional metadata in otherwise unused second 2KiB part of a header. When \f[B]PMEMPOOL_FEAT_CKSUM_2K\f[] is disabled whole 4KiB is checksummed. .IP \[bu] 2 \f[B]PMEMPOOL_FEAT_SHUTDOWN_STATE\f[] \- enables additional check performed during pool open which verifies pool consistency in the presence of dirty shutdown. \f[B]PMEMPOOL_FEAT_CKSUM_2K\f[] has to be enabled prior to \f[B]PMEMPOOL_FEAT_SHUTDOWN_STATE\f[] otherwise enabling \f[B]PMEMPOOL_FEAT_SHUTDOWN_STATE\f[] will fail. .IP \[bu] 2 \f[B]PMEMPOOL_FEAT_CHECK_BAD_BLOCKS\f[] \- enables checking bad blocks performed during opening a pool and fixing bad blocks performed by pmempool\-sync during syncing a pool. For details see \f[B]pmempool\-feature\f[](1). .PP The \f[B]pmempool_feature_query\f[]() function checks state of \f[I]feature\f[] in the pool set pointed by \f[I]path\f[]. .PP The \f[B]pmempool_feature_enable\f[]() function enables \f[I]feature\f[] in the pool set pointed by \f[I]path\f[]. .PP The \f[B]pmempool_feature_disable\f[]() function disables \f[I]feature\f[] in the pool set pointed by \f[I]path\f[]. .SH COMPATIBILITY .PP Poolsets with features not defined in this document (e.g.\ enabled by the newer software version) are not supported. .SH DISCLAIMER .PP \f[B]pmempool_feature_query\f[](), \f[B]pmempool_feature_enable\f[]() and \f[B]pmempool_feature_disable\f[]() are not fail safe. .SH RETURN VALUE .PP On success, \f[B]pmempool_feature_query\f[]() returns 0 if \f[I]feature\f[] is disabled or 1 if it is enabled. On error, it returns \-1 and sets \f[I]errno\f[] accordingly. .PP On success, \f[B]pmempool_feature_enable\f[]() returns 0. On error, it returns \-1 and sets \f[I]errno\f[] accordingly. .PP On success, \f[B]pmempool_feature_disable\f[]() returns 0. On error, it returns \-1 and sets \f[I]errno\f[] accordingly. .PP If \f[I]path\f[] points poolset with remote replica \f[B]errno\f[] is set to EINVAL and function returns \-1. .PP If non zero \f[I]flags\f[] are provided \f[B]errno\f[] is set to EINVAL and function returns \-1. .SH SEE ALSO .PP \f[B]poolset\f[](5) and \f[B]\f[] pmdk-1.8/doc/libpmempool/pmempool_feature_query.3.md0000664000000000000000000001204413615011243021351 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL_FEATURE_QUERY, 3) collection: libpmempool header: PMDK date: pmempool API version 1.3 ... [comment]: <> (Copyright 2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool_feature_query.3 -- man page for toggle and query pool set features) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[COMPATIBILITY](#compatibility)
[DISCLAIMER](#disclaimer)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # _UW(pmempool_feature_query), _UW(pmempool_feature_enable), _UW(pmempool_feature_disable) - toggle or query pool set features # SYNOPSIS # ```c #include _UWFUNCR1(int, pmempool_feature_query, *path, =q=enum pmempool_feature feature, unsigned flags=e=) _UWFUNCR1(int, pmempool_feature_enable, *path, =q=enum pmempool_feature feature, unsigned flags=e=) _UWFUNCR1(int, pmempool_feature_disable, *path, =q=enum pmempool_feature feature, unsigned flags=e=) ``` _UNICODE() # DESCRIPTION # The *feature* argument accepts following values: + **PMEMPOOL_FEAT_SINGLEHDR** - only the first part in each replica contains the pool part internal metadata. This value can be used only with **pmempool_feature_query**(). It can not be enabled or disabled. For details see **poolset**(5). + **PMEMPOOL_FEAT_CKSUM_2K** - only the first 2KiB of pool part internal metadata is checksummed. Other features may depend on this one to store additional metadata in otherwise unused second 2KiB part of a header. When **PMEMPOOL_FEAT_CKSUM_2K** is disabled whole 4KiB is checksummed. + **PMEMPOOL_FEAT_SHUTDOWN_STATE** - enables additional check performed during pool open which verifies pool consistency in the presence of dirty shutdown. **PMEMPOOL_FEAT_CKSUM_2K** has to be enabled prior to **PMEMPOOL_FEAT_SHUTDOWN_STATE** otherwise enabling **PMEMPOOL_FEAT_SHUTDOWN_STATE** will fail. + **PMEMPOOL_FEAT_CHECK_BAD_BLOCKS** - enables checking bad blocks performed during opening a pool and fixing bad blocks performed by pmempool-sync during syncing a pool. For details see **pmempool-feature**(1). The _UW(pmempool_feature_query) function checks state of *feature* in the pool set pointed by *path*. The _UW(pmempool_feature_enable) function enables *feature* in the pool set pointed by *path*. The _UW(pmempool_feature_disable) function disables *feature* in the pool set pointed by *path*. # COMPATIBILITY # Poolsets with features not defined in this document (e.g. enabled by the newer software version) are not supported. # DISCLAIMER # _UW(pmempool_feature_query), _UW(pmempool_feature_enable) and _UW(pmempool_feature_disable) are not fail safe. # RETURN VALUE # On success, _UW(pmempool_feature_query) returns 0 if *feature* is disabled or 1 if it is enabled. On error, it returns -1 and sets *errno* accordingly. On success, _UW(pmempool_feature_enable) returns 0. On error, it returns -1 and sets *errno* accordingly. On success, _UW(pmempool_feature_disable) returns 0. On error, it returns -1 and sets *errno* accordingly. If *path* points poolset with remote replica **errno** is set to EINVAL and function returns -1. If non zero *flags* are provided **errno** is set to EINVAL and function returns -1. # SEE ALSO # **poolset**(5) and **** pmdk-1.8/doc/libpmempool/pmempool_check_init.3.md0000664000000000000000000002340213615011243020571 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL_CHECK_INIT, 3) collection: libpmempool header: PMDK date: pmempool API version 1.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool_check_init.3 -- man page for pmempool health check functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[EXAMPLE](#example)
[NOTES](#notes)
[SEE ALSO](#see-also)
# NAME # _UW(pmempool_check_init), _UW(pmempool_check), **pmempool_check_end**() - checks pmempool health # SYNOPSIS # ```c #include _UWFUNCR1UW(PMEMpoolcheck, *pmempool_check_init, struct pmempool_check_args, *args,=q= size_t args_size=e=) _UWFUNCRUW(struct pmempool_check_status, *pmempool_check, PMEMpoolcheck *ppc) enum pmempool_check_result pmempool_check_end(PMEMpoolcheck *ppc); ``` _UNICODE() # DESCRIPTION # To perform the checks provided by **libpmempool**, a *check context* must first be initialized using the _UW(pmempool_check_init) function described in this section. Once initialized, the *check context* is represented by an opaque handle of type *PMEMpoolcheck\**, which is passed to all of the other functions available in **libpmempool** To execute checks, _UW(pmempool_check) must be called iteratively. Each call generates a new check status, represented by a _UWS(pmempool_check_status) structure. Status messages are described later below. When the checks are completed, _UW(pmempool_check) returns NULL. The check must be finalized using **pmempool_check_end**(), which returns an *enum pmempool_check_result* describing the results of the entire check. _UW(pmempool_check_init) initializes the check context. *args* describes parameters of the check context. *args_size* should be equal to the size of the _UWS(pmempool_check_args). _UWS(pmempool_check_args) is defined as follows: _WINUX(=q= ```c struct pmempool_check_argsU { /* path to the pool to check */ const char *path; /* optional backup path */ const char *backup_path; /* type of the pool */ enum pmempool_pool_type pool_type; /* parameters */ int flags; }; struct pmempool_check_argsW { /* path to the pool to check */ const wchar_t *path; /* optional backup path */ const wchar_t *backup_path; /* type of the pool */ enum pmempool_pool_type pool_type; /* parameters */ int flags; }; ``` =e=,=q= ```c struct pmempool_check_args { /* path to the pool to check */ const char *path; /* optional backup path */ const char *backup_path; /* type of the pool */ enum pmempool_pool_type pool_type; /* parameters */ int flags; }; ``` =e=) The *flags* argument accepts any combination of the following values (ORed): + **PMEMPOOL_CHECK_REPAIR** - perform repairs + **PMEMPOOL_CHECK_DRY_RUN** - emulate repairs, not supported on Device DAX + **PMEMPOOL_CHECK_ADVANCED** - perform hazardous repairs + **PMEMPOOL_CHECK_ALWAYS_YES** - do not ask before repairs + **PMEMPOOL_CHECK_VERBOSE** - generate info statuses + **PMEMPOOL_CHECK_FORMAT_STR** - generate string format statuses *pool_type* must match the type of the *pool* being processed. Pool type detection may be enabled by setting *pool_type* to **PMEMPOOL_POOL_TYPE_DETECT**. A pool type detection failure ends the check. *backup_path* may be: + NULL. No backup will be performed. + a non-existent file: *backup_path* will be created and backup will be performed. *path* must be a single file *pool*. + an existing *pool set* file: Backup will be performed as defined by the *backup_path* pool set. *path* must be a pool set, and *backup_path* must have the same structure (the same number of parts with exactly the same size) as the *path* pool set. Backup is supported only if the source *pool set* has no defined replicas. Neither *path* nor *backup_path* may specify a pool set with remote replicas. The _UW(pmempool_check) function starts or resumes the check indicated by *ppc*. When the next status is generated, the check is paused and _UW(pmempool_check) returns a pointer to the _UWS(pmempool_check_status) structure: _WINUX(=q= { ```c struct pmempool_check_statusU { enum pmempool_check_msg_type type; /* type of the status */ struct { const char *msg; /* status message string */ const char *answer; /* answer to message if applicable */ } str; }; struct pmempool_check_statusW { enum pmempool_check_msg_type type; /* type of the status */ struct { const wchar_t *msg; /* status message string */ const wchar_t *answer; /* answer to message if applicable */ } str; }; ``` =e=,=q= ```c struct pmempool_check_status { enum pmempool_check_msg_type type; /* type of the status */ struct { const char *msg; /* status message string */ const char *answer; /* answer to message if applicable */ } str; }; ``` =e=) This structure can describe three types of statuses: + **PMEMPOOL_CHECK_MSG_TYPE_INFO** - detailed information about the check. Generated only if a **PMEMPOOL_CHECK_VERBOSE** flag was set. + **PMEMPOOL_CHECK_MSG_TYPE_ERROR** - An error was encountered. + **PMEMPOOL_CHECK_MSG_TYPE_QUESTION** - question. Generated only if an **PMEMPOOL_CHECK_ALWAYS_YES** flag was not set. It requires *answer* to be set to "yes" or "no" before continuing. After calling _UW(pmempool_check) again, the previously provided _UWS(pmempool_check_status) pointer must be considered invalid. The **pmempool_check_end**() function finalizes the check and releases all related resources. *ppc* is invalid after calling **pmempool_check_end**(). # RETURN VALUE # _UW(pmempool_check_init) returns an opaque handle of type *PMEMpoolcheck\**. If the provided parameters are invalid or the initialization process fails, _UW(pmempool_check_init) returns NULL and sets *errno* appropriately. Each call to _UW(pmempool_check) returns a pointer to a _UWS(pmempool_check_status) structure when a status is generated. When the check completes, _UW(pmempool_check) returns NULL. The **pmempool_check_end**() function returns an *enum pmempool_check_result* summarizing the results of the finalized check. **pmempool_check_end**() can return one of the following values: + **PMEMPOOL_CHECK_RESULT_CONSISTENT** - the *pool* is consistent + **PMEMPOOL_CHECK_RESULT_NOT_CONSISTENT** - the *pool* is not consistent + **PMEMPOOL_CHECK_RESULT_REPAIRED** - the *pool* has issues but all repair steps completed successfully + **PMEMPOOL_CHECK_RESULT_CANNOT_REPAIR** - the *pool* has issues which can not be repaired + **PMEMPOOL_CHECK_RESULT_ERROR** - the *pool* has errors or the check encountered an issue + **PMEMPOOL_CHECK_RESULT_SYNC_REQ** - the *pool* has single healthy replica. To fix remaining issues use **pmempool_sync**(3). # EXAMPLE # This is an example of a *check context* initialization: ```c struct _U(pmempool_check_args) args = { .path = "/path/to/blk.pool", .backup_path = NULL, .pool_type = PMEMPOOL_POOL_TYPE_BLK, .flags = PMEMPOOL_CHECK_REPAIR | PMEMPOOL_CHECK_DRY_RUN | PMEMPOOL_CHECK_VERBOSE | PMEMPOOL_CHECK_FORMAT_STR }; ``` ```c PMEMpoolcheck *ppc = _U(pmempool_check_init)(&args, sizeof(args)); ``` The check will process a *pool* of type **PMEMPOOL_POOL_TYPE_BLK** located in the path */path/to/blk.pool*. Before the check it will not create a backup of the *pool* (*backup_path == NULL*). If the check finds any issues it will try to perform repair steps (**PMEMPOOL_CHECK_REPAIR**), but it will not make any changes to the *pool* (**PMEMPOOL_CHECK_DRY_RUN**) and it will not perform any dangerous repair steps (no **PMEMPOOL_CHECK_ADVANCED**). The check will ask before performing any repair steps (no **PMEMPOOL_CHECK_ALWAYS_YES**). It will also generate detailed information about the check (**PMEMPOOL_CHECK_VERBOSE**). The **PMEMPOOL_CHECK_FORMAT_STR** flag indicates string format statuses (*struct pmempool_check_status*). Currently this is the only supported status format so this flag is required. # NOTES # Currently, checking the consistency of a *pmemobj* pool is **not** supported. # SEE ALSO # **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmempool/libpmempool.70000644000000000000000000002637113615011416016515 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "LIBPMEMPOOL" "7" "2020-01-31" "PMDK - pmempool API version 1.3" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]libpmempool\f[] \- persistent memory pool management library .SH SYNOPSIS .IP .nf \f[C] #include\ cc\ \-std=gnu99\ ...\ \-lpmempool\ \-lpmem \f[] .fi .SS Library API versioning: .IP .nf \f[C] const\ char\ *pmempool_check_version( \ \ \ \ unsigned\ major_required, \ \ \ \ unsigned\ minor_required); \f[] .fi .SS Error handling: .IP .nf \f[C] const\ char\ *pmempool_errormsg(void); \f[] .fi .SS Other library functions: .PP A description of other \f[B]libpmempool\f[] functions can be found on the following manual pages: .IP \[bu] 2 health check functions: \f[B]pmempool_check_init\f[](3) .IP \[bu] 2 pool set synchronization and transformation: \f[B]pmempool_sync\f[](3) .IP \[bu] 2 pool set management functions: \f[B]pmempool_rm\f[](3) .IP \[bu] 2 toggle or query pool set features: \f[B]pmempool_feature_query\f[](3) .SH DESCRIPTION .PP \f[B]libpmempool\f[] provides a set of utilities for off\-line analysis and manipulation of a \f[I]pool\f[]. A \f[I]pool\f[] in this manpage means a pmemobj pool, pmemblk pool, pmemlog pool or BTT layout, independent of the underlying storage. Some \f[B]libpmempool\f[] functions are required to work without any impact on the \f[I]pool\f[] but some may create a new or modify an existing \f[I]pool\f[]. .PP \f[B]libpmempool\f[] is for applications that need high reliability or built\-in troubleshooting. It may be useful for testing and debugging purposes also. .PP \f[B]libpmempool\f[] introduces functionality of pool set health check, synchronization, transformation and removal. .SH CAVEATS .PP \f[B]libpmempool\f[] relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. \f[B]dlclose\f[](3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. .PP \f[B]libpmempool\f[] requires the \f[B]\-std=gnu99\f[] compilation flag to build properly. .SH LIBRARY API VERSIONING .PP This section describes how the library API is versioned, allowing applications to work with an evolving API. .PP The \f[B]pmempool_check_version\f[]() function is used to see if the installed \f[B]libpmempool\f[] supports the version of the library API required by an application. The easiest way to do this for the application is to supply the compile\-time version information, supplied by defines in \f[B]\f[], like this: .IP .nf \f[C] reason\ =\ pmempool_check_version(PMEMPOOL_MAJOR_VERSION, \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ PMEMPOOL_MINOR_VERSION); if\ (reason\ !=\ NULL)\ { \ \ \ \ /*\ version\ check\ failed,\ reason\ string\ tells\ you\ why\ */ } \f[] .fi .PP Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. .PP An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text \f[I]introduced in version x.y\f[] in the section of this manual describing the feature. .PP When the version check performed by \f[B]pmempool_check_version\f[]() is successful, the return value is NULL. Otherwise the return value is a static string describing the reason for failing the version check. The string returned by \f[B]pmempool_check_version\f[]() must not be modified or freed. .SH DEBUGGING AND ERROR HANDLING .PP If an error is detected during the call to a \f[B]libpmempool\f[] function, the application may retrieve an error message describing the reason for the failure from \f[B]pmempool_errormsg\f[](). This function returns a pointer to a static buffer containing the last error message logged for the current thread. If \f[I]errno\f[] was set, the error message may include a description of the corresponding error code as returned by \f[B]strerror\f[](3). The error message buffer is thread\-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a \f[B]libpmempool\f[] function indicated an error, or if \f[I]errno\f[] was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. .PP Two versions of \f[B]libpmempool\f[] are typically available on a development system. The normal version, accessed when a program is linked using the \f[B]\-lpmempool\f[] option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run\-time assertions. .PP A second version of \f[B]libpmempool\f[], accessed when a program uses the libraries under \f[B]/usr/lib/pmdk_debug\f[], contains run\-time assertions and trace points. The typical way to access the debug version is to set the environment variable \f[B]LD_LIBRARY_PATH\f[] to \f[B]/usr/lib/pmdk_debug\f[] or \f[B]/usr/lib64/pmdk_debug\f[], as appropriate. Debugging output is controlled using the following environment variables. These variables have no effect on the non\-debug version of the library. .IP \[bu] 2 \f[B]PMEMPOOL_LOG_LEVEL\f[] .PP The value of \f[B]PMEMPOOL_LOG_LEVEL\f[] enables trace points in the debug version of the library, as follows: .IP \[bu] 2 \f[B]0\f[] \- This is the default level when \f[B]PMEMPOOL_LOG_LEVEL\f[] is not set. No log messages are emitted at this level. .IP \[bu] 2 \f[B]1\f[] \- Additional details on any errors detected are logged (in addition to returning the \f[I]errno\f[]\-based errors as usual). The same information may be retrieved using \f[B]pmempool_errormsg\f[](). .IP \[bu] 2 \f[B]2\f[] \- A trace of basic operations is logged. .IP \[bu] 2 \f[B]3\f[] \- Enables a very verbose amount of function call tracing in the library. .IP \[bu] 2 \f[B]4\f[] \- Enables voluminous and fairly obscure tracing information that is likely only useful to the \f[B]libpmempool\f[] developers. .PP Unless \f[B]PMEMPOOL_LOG_FILE\f[] is set, debugging output is written to \f[I]stderr\f[]. .IP \[bu] 2 \f[B]PMEMPOOL_LOG_FILE\f[] .PP Specifies the name of a file where all logging information should be written. If the last character in the name is \[lq]\-\[rq], the \f[I]PID\f[] of the current process will be appended to the file name when the log file is created. If \f[B]PMEMPOOL_LOG_FILE\f[] is not set, output is written to \f[I]stderr\f[]. .SH EXAMPLE .PP The following example illustrates how the \f[B]libpmempool\f[] API is used. The program detects the type and checks consistency of given pool. If there are any issues detected, the pool is automatically repaired. .IP .nf \f[C] #include\ #include\ #include\ #include\ #include\ #define\ PATH\ "./pmem\-fs/myfile" #define\ CHECK_FLAGS\ (PMEMPOOL_CHECK_FORMAT_STR|PMEMPOOL_CHECK_REPAIR|\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ PMEMPOOL_CHECK_VERBOSE) int main(int\ argc,\ char\ *argv[]) { \ \ \ \ PMEMpoolcheck\ *ppc; \ \ \ \ struct\ pmempool_check_status\ *status; \ \ \ \ enum\ pmempool_check_result\ ret; \ \ \ \ /*\ arguments\ for\ check\ */ \ \ \ \ struct\ pmempool_check_args\ args\ =\ { \ \ \ \ \ \ \ \ .path\ \ \ \ \ \ \ =\ PATH, \ \ \ \ \ \ \ \ .backup_path\ \ \ \ =\ NULL, \ \ \ \ \ \ \ \ .pool_type\ \ =\ PMEMPOOL_POOL_TYPE_DETECT, \ \ \ \ \ \ \ \ .flags\ \ \ \ \ \ =\ CHECK_FLAGS \ \ \ \ }; \ \ \ \ /*\ initialize\ check\ context\ */ \ \ \ \ if\ ((ppc\ =\ pmempool_check_init(&args,\ sizeof(args)))\ ==\ NULL)\ { \ \ \ \ \ \ \ \ perror("pmempool_check_init"); \ \ \ \ \ \ \ \ exit(EXIT_FAILURE); \ \ \ \ } \ \ \ \ /*\ perform\ check\ and\ repair,\ answer\ \[aq]yes\[aq]\ for\ each\ question\ */ \ \ \ \ while\ ((status\ =\ pmempool_check(ppc))\ !=\ NULL)\ { \ \ \ \ \ \ \ \ switch\ (status\->type)\ { \ \ \ \ \ \ \ \ case\ PMEMPOOL_CHECK_MSG_TYPE_ERROR: \ \ \ \ \ \ \ \ \ \ \ \ printf("%s\\n",\ status\->str.msg); \ \ \ \ \ \ \ \ \ \ \ \ break; \ \ \ \ \ \ \ \ case\ PMEMPOOL_CHECK_MSG_TYPE_INFO: \ \ \ \ \ \ \ \ \ \ \ \ printf("%s\\n",\ status\->str.msg); \ \ \ \ \ \ \ \ \ \ \ \ break; \ \ \ \ \ \ \ \ case\ PMEMPOOL_CHECK_MSG_TYPE_QUESTION: \ \ \ \ \ \ \ \ \ \ \ \ printf("%s\\n",\ status\->str.msg); \ \ \ \ \ \ \ \ \ \ \ \ status\->str.answer\ =\ "yes"; \ \ \ \ \ \ \ \ \ \ \ \ break; \ \ \ \ \ \ \ \ default: \ \ \ \ \ \ \ \ \ \ \ \ pmempool_check_end(ppc); \ \ \ \ \ \ \ \ \ \ \ \ exit(EXIT_FAILURE); \ \ \ \ \ \ \ \ } \ \ \ \ } \ \ \ \ /*\ finalize\ the\ check\ and\ get\ the\ result\ */ \ \ \ \ ret\ =\ pmempool_check_end(ppc); \ \ \ \ switch\ (ret)\ { \ \ \ \ \ \ \ \ case\ PMEMPOOL_CHECK_RESULT_CONSISTENT: \ \ \ \ \ \ \ \ case\ PMEMPOOL_CHECK_RESULT_REPAIRED: \ \ \ \ \ \ \ \ \ \ \ \ return\ 0; \ \ \ \ \ \ \ \ default: \ \ \ \ \ \ \ \ \ \ \ \ return\ 1; \ \ \ \ } } \f[] .fi .PP See for more examples using the \f[B]libpmempool\f[] API. .SH ACKNOWLEDGEMENTS .PP \f[B]libpmempool\f[] builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: .SH SEE ALSO .PP \f[B]dlclose\f[](3), \f[B]pmempool_check_init\f[](3), \f[B]pmempool_feature_query\f[](3), \f[B]pmempool_rm\f[](3), \f[B]pmempool_sync\f[](3), \f[B]strerror\f[](3), \f[B]libpmem\f[](7), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7)\f[B] and \f[]** pmdk-1.8/doc/libpmempool/pmempool_rm.30000644000000000000000000000606113615011420016505 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL_RM" "3" "2020-01-31" "PMDK - pmempool API version 1.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool_rm\f[]() \- remove persistent memory pool .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmempool_rm(const\ char\ *path,\ int\ flags); \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool_rm\f[]() function removes the pool pointed to by \f[I]path\f[]. The \f[I]path\f[] can point to a regular file, device dax or pool set file. If \f[I]path\f[] is a pool set file, \f[B]pmempool_rm\f[]() will remove all part files from local replicas using \f[B]unlink\f[](2), and all remote replicas using \f[B]rpmem_remove\f[](3) (see \f[B]librpmem\f[](7)), before removing the pool set file itself. .PP The \f[I]flags\f[] argument determines the behavior of \f[B]pmempool_rm\f[](). It is either 0 or the bitwise OR of one or more of the following flags: .IP \[bu] 2 \f[B]PMEMPOOL_RM_FORCE\f[] \- Ignore all errors when removing part files from local or remote replicas. .IP \[bu] 2 \f[B]PMEMPOOL_RM_POOLSET_LOCAL\f[] \- Also remove local pool set file. .IP \[bu] 2 \f[B]PMEMPOOL_RM_POOLSET_REMOTE\f[] \- Also remove remote pool set file. .SH RETURN VALUE .PP On success, \f[B]pmempool_rm\f[]() returns 0. On error, it returns \-1 and sets \f[I]errno\f[] accordingly. .SH SEE ALSO .PP \f[B]rpmem_remove\f[](3), \f[B]unlink\f[](3), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7), \f[B]librpmem\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmempool/pmempool_sync.30000644000000000000000000001654513615011420017053 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL_SYNC" "3" "2020-01-31" "PMDK - pmempool API version 1.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool_sync\f[](), \f[B]pmempool_transform\f[]() \- pool set synchronization and transformation .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmempool_sync(const\ char\ *poolset_file,\ \ \ \ \ unsigned\ flags);\ (EXPERIMENTAL) int\ pmempool_transform(const\ char\ *poolset_file_src, \ \ \ \ const\ char\ *poolset_file_dst,\ unsigned\ flags);\ (EXPERIMENTAL) \f[] .fi .SH DESCRIPTION .PP The \f[B]pmempool_sync\f[]() function synchronizes data between replicas within a pool set. .PP \f[B]pmempool_sync\f[]() accepts two arguments: .IP \[bu] 2 \f[I]poolset_file\f[] \- a path to a pool set file, .IP \[bu] 2 \f[I]flags\f[] \- a combination of flags (ORed) which modify how synchronization is performed. .RS .PP NOTE: Only the pool set file used to create the pool should be used for syncing the pool. .RE .RS .PP NOTE: The \f[B]pmempool_sync\f[]() cannot do anything useful if there are no replicas in the pool set. In such case, it fails with an error. .RE .RS .PP NOTE: At the moment, replication is only supported for \f[B]libpmemobj\f[](7) pools, so \f[B]pmempool_sync\f[]() cannot be used with other pool types (\f[B]libpmemlog\f[](7), \f[B]libpmemblk\f[](7)). .RE .PP The following flags are available: .IP \[bu] 2 \f[B]PMEMPOOL_SYNC_DRY_RUN\f[] \- do not apply changes, only check for viability of synchronization. .PP \f[B]pmempool_sync\f[]() checks that the metadata of all replicas in a pool set is consistent, i.e.\ all parts are healthy, and if any of them is not, the corrupted or missing parts are recreated and filled with data from one of the healthy replicas. .PP If a pool set has the option \f[I]SINGLEHDR\f[] (see \f[B]poolset\f[](5)), the internal metadata of each replica is limited to the beginning of the first part in the replica. If the option \f[I]NOHDRS\f[] is used, replicas contain no internal metadata. In both cases, only the missing parts or the ones which cannot be opened are recreated with the \f[B]pmempool_sync\f[]() function. .PP \f[B]pmempool_transform\f[]() modifies the internal structure of a pool set. It supports the following operations: .IP \[bu] 2 adding one or more replicas, .IP \[bu] 2 removing one or more replicas , .IP \[bu] 2 adding or removing pool set options. .PP Only one of the above operations can be performed at a time. .PP \f[B]pmempool_transform\f[]() accepts three arguments: .IP \[bu] 2 \f[I]poolset_file_src\f[] \- pathname of the pool \f[I]set\f[] file for the source pool set to be changed, .IP \[bu] 2 \f[I]poolset_file_dst\f[] \- pathname of the pool \f[I]set\f[] file that defines the new structure of the pool set, .IP \[bu] 2 \f[I]flags\f[] \- a combination of flags (ORed) which modify how synchronization is performed. .PP The following flags are available: .IP \[bu] 2 \f[B]PMEMPOOL_TRANSFORM_DRY_RUN\f[] \- do not apply changes, only check for viability of transformation. .PP When adding or deleting replicas, the two pool set files can differ only in the definitions of replicas which are to be added or deleted. When adding or removing pool set options (see \f[B]poolset\f[](5)), the rest of both pool set files have to be of the same structure. The operation of adding/removing a pool set option can be performed on a pool set with local replicas only. To add/remove a pool set option to/from a pool set with remote replicas, one has to remove the remote replicas first, then add/remove the option, and finally recreate the remote replicas having added/removed the pool set option to/from the remote replicas' poolset files. To add a replica it is necessary for its effective size to match or exceed the pool size. Otherwise the whole operation fails and no changes are applied. If none of the pool set options is used, the effective size of a replica is the sum of sizes of all its part files decreased by 4096 bytes per each part file. The 4096 bytes of each part file is utilized for storing internal metadata of the pool part files. If the option \f[I]SINGLEHDR\f[] is used, the effective size of a replica is the sum of sizes of all its part files decreased once by 4096 bytes. In this case only the first part contains internal metadata. If the option \f[I]NOHDRS\f[] is used, the effective size of a replica is the sum of sizes of all its part files. In this case none of the parts contains internal metadata. .RS .PP NOTE: At the moment, \f[I]transform\f[] operation is only supported for \f[B]libpmemobj\f[](7) pools, so \f[B]pmempool_transform\f[]() cannot be used with other pool types (\f[B]libpmemlog\f[](7), \f[B]libpmemblk\f[](7)). .RE .SH RETURN VALUE .PP \f[B]pmempool_sync\f[]() and \f[B]pmempool_transform\f[]() return 0 on success. Otherwise, they return \-1 and set \f[I]errno\f[] appropriately. .SH ERRORS .PP \f[B]EINVAL\f[] Invalid format of the input/output pool set file. .PP \f[B]EINVAL\f[] Unsupported \f[I]flags\f[] value. .PP \f[B]EINVAL\f[] There is only master replica defined in the input pool set passed to \f[B]pmempool_sync\f[](). .PP \f[B]EINVAL\f[] The source pool set passed to \f[B]pmempool_transform\f[]() is not a \f[B]libpmemobj\f[] pool. .PP \f[B]EINVAL\f[] The input and output pool sets passed to \f[B]pmempool_transform\f[]() are identical. .PP \f[B]EINVAL\f[] Attempt to perform more than one transform operation at a time. .PP \f[B]ENOTSUP\f[] The pool set contains a remote replica, but remote replication is not supported (\f[B]librpmem\f[](7) is not available). .SH NOTES .PP The \f[B]pmempool_sync\f[]() API is experimental and it may change in future versions of the library. .PP The \f[B]pmempool_transform\f[]() API is experimental and it may change in future versions of the library. .SH SEE ALSO .PP \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmempool/.gitignore0000664000000000000000000000013313615011243016062 0ustar rootrootlibpmempool.7 pmempool_check_init.3 pmempool_feature_query.3 pmempool_rm.3 pmempool_sync.3 pmdk-1.8/doc/libpmempool/pmempool_check.30000664000000000000000000000003213615011243017141 0ustar rootroot.so pmempool_check_init.3 pmdk-1.8/doc/libpmempool/pmempool_sync.3.md0000664000000000000000000002010513615011243017442 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMPOOL_SYNC, 3) collection: libpmempool header: PMDK date: pmempool API version 1.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmempool_sync.3 -- man page for pmempool sync and transform) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[ERRORS](#errors)
[NOTES](#notes)
[SEE ALSO](#see-also)
# NAME # _UW(pmempool_sync), _UW(pmempool_transform) - pool set synchronization and transformation # SYNOPSIS # ```c #include _UWFUNCR1(int, pmempool_sync, *poolset_file,=q= unsigned flags=e=, =q= (EXPERIMENTAL)=e=) _UWFUNCR12(int, pmempool_transform, *poolset_file_src, *poolset_file_dst, unsigned flags, =q= (EXPERIMENTAL)=e=) ``` _UNICODE() # DESCRIPTION # The _UW(pmempool_sync) function synchronizes data between replicas within a pool set. _UW(pmempool_sync) accepts two arguments: * *poolset_file* - a path to a pool set file, * *flags* - a combination of flags (ORed) which modify how synchronization is performed. >NOTE: Only the pool set file used to create the pool should be used for syncing the pool. >NOTE: The _UW(pmempool_sync) cannot do anything useful if there are no replicas in the pool set. In such case, it fails with an error. >NOTE: At the moment, replication is only supported for **libpmemobj**(7) pools, so _UW(pmempool_sync) cannot be used with other pool types (**libpmemlog**(7), **libpmemblk**(7)). The following flags are available: * **PMEMPOOL_SYNC_DRY_RUN** - do not apply changes, only check for viability of synchronization. _UW(pmempool_sync) checks that the metadata of all replicas in a pool set is consistent, i.e. all parts are healthy, and if any of them is not, the corrupted or missing parts are recreated and filled with data from one of the healthy replicas. _WINUX(,=q=If a pool set has the option *SINGLEHDR* (see **poolset**(5)), the internal metadata of each replica is limited to the beginning of the first part in the replica. If the option *NOHDRS* is used, replicas contain no internal metadata. In both cases, only the missing parts or the ones which cannot be opened are recreated with the _UW(pmempool_sync) function.=e=) _UW(pmempool_transform) modifies the internal structure of a pool set. It supports the following operations: * adding one or more replicas, * removing one or more replicas _WINUX(.,=q=, * adding or removing pool set options.=e=) Only one of the above operations can be performed at a time. _UW(pmempool_transform) accepts three arguments: * *poolset_file_src* - pathname of the pool *set* file for the source pool set to be changed, * *poolset_file_dst* - pathname of the pool *set* file that defines the new structure of the pool set, * *flags* - a combination of flags (ORed) which modify how synchronization is performed. The following flags are available: * **PMEMPOOL_TRANSFORM_DRY_RUN** - do not apply changes, only check for viability of transformation. _WINUX(=q=When adding or deleting replicas, the two pool set files can differ only in the definitions of replicas which are to be added or deleted. One cannot add and remove replicas in the same step. Only one of these operations can be performed at a time. Reordering replicas is not supported. Also, to add a replica it is necessary for its effective size to match or exceed the pool size. Otherwise the whole operation fails and no changes are applied. The effective size of a replica is the sum of sizes of all its part files decreased by 4096 bytes per each part file. The 4096 bytes of each part file is utilized for storing internal metadata of the pool part files.=e=) _WINUX(,=q=When adding or deleting replicas, the two pool set files can differ only in the definitions of replicas which are to be added or deleted. When adding or removing pool set options (see **poolset**(5)), the rest of both pool set files have to be of the same structure. The operation of adding/removing a pool set option can be performed on a pool set with local replicas only. To add/remove a pool set option to/from a pool set with remote replicas, one has to remove the remote replicas first, then add/remove the option, and finally recreate the remote replicas having added/removed the pool set option to/from the remote replicas' poolset files. To add a replica it is necessary for its effective size to match or exceed the pool size. Otherwise the whole operation fails and no changes are applied. If none of the pool set options is used, the effective size of a replica is the sum of sizes of all its part files decreased by 4096 bytes per each part file. The 4096 bytes of each part file is utilized for storing internal metadata of the pool part files. If the option *SINGLEHDR* is used, the effective size of a replica is the sum of sizes of all its part files decreased once by 4096 bytes. In this case only the first part contains internal metadata. If the option *NOHDRS* is used, the effective size of a replica is the sum of sizes of all its part files. In this case none of the parts contains internal metadata.=e=) >NOTE: At the moment, *transform* operation is only supported for **libpmemobj**(7) pools, so _UW(pmempool_transform) cannot be used with other pool types (**libpmemlog**(7), **libpmemblk**(7)). # RETURN VALUE # _UW(pmempool_sync) and _UW(pmempool_transform) return 0 on success. Otherwise, they return -1 and set *errno* appropriately. # ERRORS # **EINVAL** Invalid format of the input/output pool set file. **EINVAL** Unsupported *flags* value. **EINVAL** There is only master replica defined in the input pool set passed to _UW(pmempool_sync). **EINVAL** The source pool set passed to _UW(pmempool_transform) is not a **libpmemobj** pool. **EINVAL** The input and output pool sets passed to _UW(pmempool_transform) are identical. **EINVAL** Attempt to perform more than one transform operation at a time. **ENOTSUP** The pool set contains a remote replica, but remote replication is not supported (**librpmem**(7) is not available). # NOTES # The _UW(pmempool_sync) API is experimental and it may change in future versions of the library. The _UW(pmempool_transform) API is experimental and it may change in future versions of the library. # SEE ALSO # **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmempool/pmempool_check_init.30000644000000000000000000002253313615011420020171 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMPOOL_CHECK_INIT" "3" "2020-01-31" "PMDK - pmempool API version 1.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmempool_check_init\f[](), \f[B]pmempool_check\f[](), \f[B]pmempool_check_end\f[]() \- checks pmempool health .SH SYNOPSIS .IP .nf \f[C] #include\ PMEMpoolcheck\ *pmempool_check_init(struct\ pmempool_check_args\ *args,\ \ \ \ \ size_t\ args_size); struct\ pmempool_check_status\ *pmempool_check(PMEMpoolcheck\ *ppc); enum\ pmempool_check_result\ pmempool_check_end(PMEMpoolcheck\ *ppc); \f[] .fi .SH DESCRIPTION .PP To perform the checks provided by \f[B]libpmempool\f[], a \f[I]check context\f[] must first be initialized using the \f[B]pmempool_check_init\f[]() function described in this section. Once initialized, the \f[I]check context\f[] is represented by an opaque handle of type \f[I]PMEMpoolcheck*\f[], which is passed to all of the other functions available in \f[B]libpmempool\f[] .PP To execute checks, \f[B]pmempool_check\f[]() must be called iteratively. Each call generates a new check status, represented by a \f[I]struct pmempool_check_status\f[] structure. Status messages are described later below. .PP When the checks are completed, \f[B]pmempool_check\f[]() returns NULL. The check must be finalized using \f[B]pmempool_check_end\f[](), which returns an \f[I]enum pmempool_check_result\f[] describing the results of the entire check. .PP \f[B]pmempool_check_init\f[]() initializes the check context. \f[I]args\f[] describes parameters of the check context. \f[I]args_size\f[] should be equal to the size of the \f[I]struct pmempool_check_args\f[]. \f[I]struct pmempool_check_args\f[] is defined as follows: .IP .nf \f[C] struct\ pmempool_check_args { \ \ \ \ /*\ path\ to\ the\ pool\ to\ check\ */ \ \ \ \ const\ char\ *path; \ \ \ \ /*\ optional\ backup\ path\ */ \ \ \ \ const\ char\ *backup_path; \ \ \ \ /*\ type\ of\ the\ pool\ */ \ \ \ \ enum\ pmempool_pool_type\ pool_type; \ \ \ \ /*\ parameters\ */ \ \ \ \ int\ flags; }; \f[] .fi .PP The \f[I]flags\f[] argument accepts any combination of the following values (ORed): .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_REPAIR\f[] \- perform repairs .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_DRY_RUN\f[] \- emulate repairs, not supported on Device DAX .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_ADVANCED\f[] \- perform hazardous repairs .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_ALWAYS_YES\f[] \- do not ask before repairs .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_VERBOSE\f[] \- generate info statuses .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_FORMAT_STR\f[] \- generate string format statuses .PP \f[I]pool_type\f[] must match the type of the \f[I]pool\f[] being processed. Pool type detection may be enabled by setting \f[I]pool_type\f[] to \f[B]PMEMPOOL_POOL_TYPE_DETECT\f[]. A pool type detection failure ends the check. .PP \f[I]backup_path\f[] may be: .IP \[bu] 2 NULL. No backup will be performed. .IP \[bu] 2 a non\-existent file: \f[I]backup_path\f[] will be created and backup will be performed. \f[I]path\f[] must be a single file \f[I]pool\f[]. .IP \[bu] 2 an existing \f[I]pool set\f[] file: Backup will be performed as defined by the \f[I]backup_path\f[] pool set. \f[I]path\f[] must be a pool set, and \f[I]backup_path\f[] must have the same structure (the same number of parts with exactly the same size) as the \f[I]path\f[] pool set. .PP Backup is supported only if the source \f[I]pool set\f[] has no defined replicas. .PP Neither \f[I]path\f[] nor \f[I]backup_path\f[] may specify a pool set with remote replicas. .PP The \f[B]pmempool_check\f[]() function starts or resumes the check indicated by \f[I]ppc\f[]. When the next status is generated, the check is paused and \f[B]pmempool_check\f[]() returns a pointer to the \f[I]struct pmempool_check_status\f[] structure: .IP .nf \f[C] struct\ pmempool_check_status { \ \ \ \ enum\ pmempool_check_msg_type\ type;\ /*\ type\ of\ the\ status\ */ \ \ \ \ struct \ \ \ \ { \ \ \ \ \ \ \ \ const\ char\ *msg;\ /*\ status\ message\ string\ */ \ \ \ \ \ \ \ \ const\ char\ *answer;\ /*\ answer\ to\ message\ if\ applicable\ */ \ \ \ \ }\ str; }; \f[] .fi .PP This structure can describe three types of statuses: .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_MSG_TYPE_INFO\f[] \- detailed information about the check. Generated only if a \f[B]PMEMPOOL_CHECK_VERBOSE\f[] flag was set. .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_MSG_TYPE_ERROR\f[] \- An error was encountered. .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_MSG_TYPE_QUESTION\f[] \- question. Generated only if an \f[B]PMEMPOOL_CHECK_ALWAYS_YES\f[] flag was not set. It requires \f[I]answer\f[] to be set to \[lq]yes\[rq] or \[lq]no\[rq] before continuing. .PP After calling \f[B]pmempool_check\f[]() again, the previously provided \f[I]struct pmempool_check_status\f[] pointer must be considered invalid. .PP The \f[B]pmempool_check_end\f[]() function finalizes the check and releases all related resources. \f[I]ppc\f[] is invalid after calling \f[B]pmempool_check_end\f[](). .SH RETURN VALUE .PP \f[B]pmempool_check_init\f[]() returns an opaque handle of type \f[I]PMEMpoolcheck*\f[]. If the provided parameters are invalid or the initialization process fails, \f[B]pmempool_check_init\f[]() returns NULL and sets \f[I]errno\f[] appropriately. .PP Each call to \f[B]pmempool_check\f[]() returns a pointer to a \f[I]struct pmempool_check_status\f[] structure when a status is generated. When the check completes, \f[B]pmempool_check\f[]() returns NULL. .PP The \f[B]pmempool_check_end\f[]() function returns an \f[I]enum pmempool_check_result\f[] summarizing the results of the finalized check. \f[B]pmempool_check_end\f[]() can return one of the following values: .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_RESULT_CONSISTENT\f[] \- the \f[I]pool\f[] is consistent .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_RESULT_NOT_CONSISTENT\f[] \- the \f[I]pool\f[] is not consistent .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_RESULT_REPAIRED\f[] \- the \f[I]pool\f[] has issues but all repair steps completed successfully .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_RESULT_CANNOT_REPAIR\f[] \- the \f[I]pool\f[] has issues which can not be repaired .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_RESULT_ERROR\f[] \- the \f[I]pool\f[] has errors or the check encountered an issue .IP \[bu] 2 \f[B]PMEMPOOL_CHECK_RESULT_SYNC_REQ\f[] \- the \f[I]pool\f[] has single healthy replica. To fix remaining issues use \f[B]pmempool_sync\f[](3). .SH EXAMPLE .PP This is an example of a \f[I]check context\f[] initialization: .IP .nf \f[C] struct\ pmempool_check_args\ args\ = { \ \ \ \ .path\ =\ "/path/to/blk.pool", \ \ \ \ .backup_path\ =\ NULL, \ \ \ \ .pool_type\ =\ PMEMPOOL_POOL_TYPE_BLK, \ \ \ \ .flags\ =\ PMEMPOOL_CHECK_REPAIR\ |\ PMEMPOOL_CHECK_DRY_RUN\ | \ \ \ \ \ \ \ \ PMEMPOOL_CHECK_VERBOSE\ |\ PMEMPOOL_CHECK_FORMAT_STR }; \f[] .fi .IP .nf \f[C] PMEMpoolcheck\ *ppc\ =\ pmempool_check_init(&args,\ sizeof(args)); \f[] .fi .PP The check will process a \f[I]pool\f[] of type \f[B]PMEMPOOL_POOL_TYPE_BLK\f[] located in the path \f[I]/path/to/blk.pool\f[]. Before the check it will not create a backup of the \f[I]pool\f[] (\f[I]backup_path == NULL\f[]). If the check finds any issues it will try to perform repair steps (\f[B]PMEMPOOL_CHECK_REPAIR\f[]), but it will not make any changes to the \f[I]pool\f[] (\f[B]PMEMPOOL_CHECK_DRY_RUN\f[]) and it will not perform any dangerous repair steps (no \f[B]PMEMPOOL_CHECK_ADVANCED\f[]). The check will ask before performing any repair steps (no \f[B]PMEMPOOL_CHECK_ALWAYS_YES\f[]). It will also generate detailed information about the check (\f[B]PMEMPOOL_CHECK_VERBOSE\f[]). The \f[B]PMEMPOOL_CHECK_FORMAT_STR\f[] flag indicates string format statuses (\f[I]struct pmempool_check_status\f[]). Currently this is the only supported status format so this flag is required. .SH NOTES .PP Currently, checking the consistency of a \f[I]pmemobj\f[] pool is \f[B]not\f[] supported. .SH SEE ALSO .PP \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmempool/pmempool_check_end.30000664000000000000000000000003213615011243017767 0ustar rootroot.so pmempool_check_init.3 pmdk-1.8/doc/libpmempool/pmempool_transform.30000664000000000000000000000002413615011243020100 0ustar rootroot.so pmempool_sync.3 pmdk-1.8/doc/README0000664000000000000000000000476713615011243012454 0ustar rootrootPersistent Memory Development Kit This is doc/README. Subdirectories of this directory contain source for the man pages for the Persistent Memory Development Kit in markdown format (.md files). If you're looking for documentation to get you started using PMDK, start here: https://pmem.io/pmdk and follow the links to examples and man pages. Developers new to PMDK are probably looking for libpmemobj. To generate web-based documentation or Linux/FreeBSD man pages, you need to have groff and pandoc installed. m4 macros are used in the document sources to generate OS-specific variants of man pages and web-based documentation. The macros are defined in macros.man. Processing is performed by the ../utils/md2man.sh script. All files in the *generated* directory are automatically generated and updated by the pmdk-bot. **DO NOT MODIFY THE FILES IN THAT DIRECTORY**. All changes to the documentation must be made by modifying the *.md files in the following document subdirectories: libpmem -- low-level persistent memory support libpmem2 -- low-level persistent memory support (EXPERIMENTAL) libpmemblk -- pmem-resident arrays of blocks libpmemlog -- pmem-resident log files libpmemobj -- transactional object store libpmempool -- persistent memory pool management library pmempool -- Persistent Memory Pool Management Tool librpmem -- remote access to persistent memory (EXPERIMENTAL) rpmemd -- remote persistent memory daemon (EXPERIMENTAL) daxio -- perform I/O on Device DAX device These man pages provide the API specification for the corresponding libraries and commands in this source tree, so any updates to one should be tested, reviewed, and committed with changes to the other. To create more readable text files from the source, use: $ [g]make NOTE: This will write man page output into the *generated* subdirectory. Files in this directory MUST NOT be included in any pull requests. The man(1) command may be used to format generated man pages for viewing in a terminal window (includes bold, underline, etc.), for example: $ man generated/libpmem.7 $ man generated/pmemobj_create.3 In addition, for testing purposes ../utils/md2man.sh will generate a preprocessed markdown file with the headers stripped off if the TESTOPTS variable is set. For example: $ export TESTOPTS="-DWIN32 -UFREEBSD -UWEB" $ ../utils/md2man.sh libpmemobj/libpmemobj.7.md x libpmemobj.7.win.md will generate a version of the libpmemobj.7 man page for Windows in markdown format. The resulting file may be viewed with a markdown-enabled browser. pmdk-1.8/doc/default.man0000664000000000000000000000412413615011243013700 0ustar rootroot$if(has-tables)$ .\"t $endif$ $if(pandoc-version)$ .\" Automatically generated by Pandoc $pandoc-version$ .\" $endif$ $if(adjusting)$ .ad $adjusting$ $endif$ .TH "$title$" "$section$" "$date$" "PMDK - $version$" "PMDK Programmer's Manual" $if(hyphenate)$ .hy $else$ .nh \" Turn off hyphenation by default. $endif$ $for(header-includes)$ $header-includes$ $endfor$ .\" $copyright$ .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. $for(include-before)$ $include-before$ $endfor$ $body$ $for(include-after)$ $include-after$ $endfor$ $if(author)$ .SH AUTHORS $for(author)$$author$$sep$; $endfor$. $endif$ pmdk-1.8/doc/rpmemd/0000775000000000000000000000000013615011420013037 5ustar rootrootpmdk-1.8/doc/rpmemd/.gitignore0000664000000000000000000000001113615011243015022 0ustar rootrootrpmemd.1 pmdk-1.8/doc/rpmemd/rpmemd.1.md0000664000000000000000000002041613615011243015012 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(RPMEMD, 1) collection: rpmemd header: PMDK date: rpmemd version 1.4 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (rpmemd.1.md -- man page for rpmemd) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[OPTIONS](#options)
[CONFIGURATION FILES](#configuration-files)
[EXAMPLE](#example)
[DEFAULT CONFIGURATION](#default-configuration)
[PERSISTENCY METHODS](#persistency-methods)
[SEE ALSO](#see-also)
# NAME # **rpmemd** - librpmem target node process (EXPERIMENTAL) # SYNOPSIS # ``` $ rpmemd [--help] [--version] [] ``` # DESCRIPTION # The **rpmemd** process is executed on target node by **librpmem**(7) library over **ssh**(1) and facilitates access to persistent memory over RDMA. The **rpmemd** should not be run manually under normal conditions. # OPTIONS # Command line options overwrite the default **rpmemd** configuration, the global configuration file and the user configuration file. `-V, --version` Displays **rpmemd** version and exits. `-h, --help` Prints synopsis and list of parameters and exits. `-c, --config ` Custom configuration file location. If the custom configuration file is provided others are omitted. See **CONFIGURATION FILES** section for details. All options described in **CONFIGURATION FILES** section are common for both the configuration file and the command line - the equivalent of the following line in the config file: `option = value` is `--option value` in the command line. The following command line options: **--persist-apm**, **--persist-general** and **--use-syslog** should not be followed by any value. Presence of each of them in the command line turns on an appropriate option. See **CONFIGURATION FILES** section for details. `-r, --remove ` Remove a pool described by given pool set file descriptor. It is interpreted as a path to the pool set file relative to the pool set directory. `-f, --force` Ignore errors when removing a pool file using **--remove** option. # CONFIGURATION FILES # The **rpmemd** searches for the configuration files with following priorities: + The global configuration file located in **/etc/rpmemd/rpmemd.conf**. + The user configuration file located in the user home directory (**$HOME/.rpmemd.conf**). The **rpmemd** can also read configuration from the custom configuration file provided using **--config** command line option. See **OPTIONS** section for details. The default configuration is described in the **DEFAULT CONFIGURATION** section. The configuration file is a plain text file. Each line of the configuration file can store only one configuration option defined as a *key=value* pair. Empty lines and lines starting with *#* are omitted. The allowed options are: + `log-file = ` - log file location + `poolset-dir = ` - pool set files directory + `persist-apm = {yes|no}` - enable **The Appliance Persistency Method**. This option must be set only if the target platform has non-allocating writes IO enabled. See **PERSISTENCY METHODS** section for details. + `persist-general = {yes|no}` - enable **The General Purpose Server Persistency Method**. See **PERSISTENCY METHODS** section for details. + `use-syslog = {yes|no}` - use **syslog**(3) for logging messages instead of log file + `log-level = ` - set log level value. Accepted *\* values are: + **err** - error conditions + **warn** - warning conditions + **notice** - normal, but significant conditions + **info** - informational message + **debug** - debug-level message The **$HOME** sub-string in the *poolset-dir* path is replaced with the current user home directory. # EXAMPLE # Example of the configuration file: ``` # This is an example of configuration file log-file = $HOME/.logs/rpmemd.log poolset-dir = $HOME/poolsets/ persist-apm = yes persist-general = no use-syslog = no # Use log file instead of syslog log-level = info ``` # DEFAULT CONFIGURATION # The **rpmemd** default configuration is equivalent of the following configuration file: ``` log-file = /var/log/rpmemd.log poolset-dir = $HOME persist-apm = no persist-general = yes use-syslog = yes log-level = err ``` # PERSISTENCY METHODS # The **librpmem**(7) supports two methods for making data written to remote persistent memory durable. The difference between the use of the two mechanisms is based on whether **librpmem**(7) will make use of non-allocating writes on the remote node. + **The General Purpose Server Persistency Method** does not have any requirements for the platform on which the target daemon runs and can be enabled by administrator using the *persist-general* option. This method utilize **libpmem**(7) persistency mechanisms on remote node and requires additional communication between initiator and remote node using the in-band connection. + **The Appliance Persistency Method** requires non-allocating writes enabled on the platform and can be enabled by administrator using *persist-apm* option. This method requires to issue an RDMA READ operation after the RDMA WRITE operations performed on requested chunk of memory. "Non-allocating write requests" is the Intel Integrated IO Controller mode where all incoming PCIe writes will utilize non-allocating buffers for the write requests. Non-allocating writes are guaranteed to bypass all of the CPU caches and force the write requests to flow directly to the Integrated Memory Controller without delay. The **rpmemd** dynamically choose the appropriate persistency method and the flushing to persistence primitive for GPSPM for each opened pool set name depending on available persistency methods and whether all pool set parts are stored in the persistent memory. If the **Appliance Persistency Method** is enabled and the pool set is stored in the persistent memory **rpmemd** will use the **Appliance Persistency Method**. If the pool set is NOT stored in the persistent memory it will fallback to the **General Puropose Server Persistency Method** with **pmem_msync**(3). If the **General Puropose Server Persistency Method** is enabled and the pool set is stored in the persistent memory **rpmemd** will use **pmem_persist**(3). If the pool set is NOT stored in the persistent momory it will use **pmem_msync**(3). See **pmem_persist**(3) and **pmem_msync**(3) for more details. # SEE ALSO # **ssh**(1), **pmem_msync**(3), **pmem_persist**(3), **syslog**(3), **libpmem**(7), **libpmemobj**(7), **librpmem**(7) and **** pmdk-1.8/doc/rpmemd/rpmemd.10000644000000000000000000002073613615011420014413 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "RPMEMD" "1" "2020-01-31" "PMDK - rpmemd version 1.4" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]rpmemd\f[] \- librpmem target node process (EXPERIMENTAL) .SH SYNOPSIS .IP .nf \f[C] $\ rpmemd\ [\-\-help]\ [\-\-version]\ [] \f[] .fi .SH DESCRIPTION .PP The \f[B]rpmemd\f[] process is executed on target node by \f[B]librpmem\f[](7) library over \f[B]ssh\f[](1) and facilitates access to persistent memory over RDMA. The \f[B]rpmemd\f[] should not be run manually under normal conditions. .SH OPTIONS .PP Command line options overwrite the default \f[B]rpmemd\f[] configuration, the global configuration file and the user configuration file. .PP \f[C]\-V,\ \-\-version\f[] .PP Displays \f[B]rpmemd\f[] version and exits. .PP \f[C]\-h,\ \-\-help\f[] .PP Prints synopsis and list of parameters and exits. .PP \f[C]\-c,\ \-\-config\ \f[] .PP Custom configuration file location. If the custom configuration file is provided others are omitted. See \f[B]CONFIGURATION FILES\f[] section for details. .PP All options described in \f[B]CONFIGURATION FILES\f[] section are common for both the configuration file and the command line \- the equivalent of the following line in the config file: .PP \f[C]option\ =\ value\f[] .PP is .PP \f[C]\-\-option\ value\f[] .PP in the command line. .PP The following command line options: \f[B]\[en]persist\-apm\f[], \f[B]\[en]persist\-general\f[] and \f[B]\[en]use\-syslog\f[] should not be followed by any value. Presence of each of them in the command line turns on an appropriate option. See \f[B]CONFIGURATION FILES\f[] section for details. .PP \f[C]\-r,\ \-\-remove\ \f[] .PP Remove a pool described by given pool set file descriptor. It is interpreted as a path to the pool set file relative to the pool set directory. .PP \f[C]\-f,\ \-\-force\f[] .PP Ignore errors when removing a pool file using \f[B]\[en]remove\f[] option. .SH CONFIGURATION FILES .PP The \f[B]rpmemd\f[] searches for the configuration files with following priorities: .IP \[bu] 2 The global configuration file located in \f[B]/etc/rpmemd/rpmemd.conf\f[]. .IP \[bu] 2 The user configuration file located in the user home directory (\f[B]$HOME/.rpmemd.conf\f[]). .PP The \f[B]rpmemd\f[] can also read configuration from the custom configuration file provided using \f[B]\[en]config\f[] command line option. See \f[B]OPTIONS\f[] section for details. .PP The default configuration is described in the \f[B]DEFAULT CONFIGURATION\f[] section. .PP The configuration file is a plain text file. Each line of the configuration file can store only one configuration option defined as a \f[I]key=value\f[] pair. Empty lines and lines starting with \f[I]#\f[] are omitted. .PP The allowed options are: .IP \[bu] 2 \f[C]log\-file\ =\ \f[] \- log file location .IP \[bu] 2 \f[C]poolset\-dir\ =\ \f[] \- pool set files directory .IP \[bu] 2 \f[C]persist\-apm\ =\ {yes|no}\f[] \- enable \f[B]The Appliance Persistency Method\f[]. This option must be set only if the target platform has non\-allocating writes IO enabled. See \f[B]PERSISTENCY METHODS\f[] section for details. .IP \[bu] 2 \f[C]persist\-general\ =\ {yes|no}\f[] \- enable \f[B]The General Purpose Server Persistency Method\f[]. See \f[B]PERSISTENCY METHODS\f[] section for details. .IP \[bu] 2 \f[C]use\-syslog\ =\ {yes|no}\f[] \- use \f[B]syslog\f[](3) for logging messages instead of log file .IP \[bu] 2 \f[C]log\-level\ =\ \f[] \- set log level value. Accepted \f[I]\f[] values are: .RS 2 .IP \[bu] 2 \f[B]err\f[] \- error conditions .IP \[bu] 2 \f[B]warn\f[] \- warning conditions .IP \[bu] 2 \f[B]notice\f[] \- normal, but significant conditions .IP \[bu] 2 \f[B]info\f[] \- informational message .IP \[bu] 2 \f[B]debug\f[] \- debug\-level message .RE .PP The \f[B]$HOME\f[] sub\-string in the \f[I]poolset\-dir\f[] path is replaced with the current user home directory. .SH EXAMPLE .PP Example of the configuration file: .IP .nf \f[C] #\ This\ is\ an\ example\ of\ configuration\ file log\-file\ =\ $HOME/.logs/rpmemd.log poolset\-dir\ =\ $HOME/poolsets/ persist\-apm\ =\ yes persist\-general\ =\ no use\-syslog\ =\ no\ #\ Use\ log\ file\ instead\ of\ syslog log\-level\ =\ info \f[] .fi .SH DEFAULT CONFIGURATION .PP The \f[B]rpmemd\f[] default configuration is equivalent of the following configuration file: .IP .nf \f[C] log\-file\ =\ /var/log/rpmemd.log poolset\-dir\ =\ $HOME persist\-apm\ =\ no persist\-general\ =\ yes use\-syslog\ =\ yes log\-level\ =\ err \f[] .fi .SH PERSISTENCY METHODS .PP The \f[B]librpmem\f[](7) supports two methods for making data written to remote persistent memory durable. The difference between the use of the two mechanisms is based on whether \f[B]librpmem\f[](7) will make use of non\-allocating writes on the remote node. .IP \[bu] 2 \f[B]The General Purpose Server Persistency Method\f[] does not have any requirements for the platform on which the target daemon runs and can be enabled by administrator using the \f[I]persist\-general\f[] option. This method utilize \f[B]libpmem\f[](7) persistency mechanisms on remote node and requires additional communication between initiator and remote node using the in\-band connection. .IP \[bu] 2 \f[B]The Appliance Persistency Method\f[] requires non\-allocating writes enabled on the platform and can be enabled by administrator using \f[I]persist\-apm\f[] option. This method requires to issue an RDMA READ operation after the RDMA WRITE operations performed on requested chunk of memory. .PP \[lq]Non\-allocating write requests\[rq] is the Intel Integrated IO Controller mode where all incoming PCIe writes will utilize non\-allocating buffers for the write requests. Non\-allocating writes are guaranteed to bypass all of the CPU caches and force the write requests to flow directly to the Integrated Memory Controller without delay. .PP The \f[B]rpmemd\f[] dynamically choose the appropriate persistency method and the flushing to persistence primitive for GPSPM for each opened pool set name depending on available persistency methods and whether all pool set parts are stored in the persistent memory. .PP If the \f[B]Appliance Persistency Method\f[] is enabled and the pool set is stored in the persistent memory \f[B]rpmemd\f[] will use the \f[B]Appliance Persistency Method\f[]. If the pool set is NOT stored in the persistent memory it will fallback to the \f[B]General Puropose Server Persistency Method\f[] with \f[B]pmem_msync\f[](3). .PP If the \f[B]General Puropose Server Persistency Method\f[] is enabled and the pool set is stored in the persistent memory \f[B]rpmemd\f[] will use \f[B]pmem_persist\f[](3). If the pool set is NOT stored in the persistent momory it will use \f[B]pmem_msync\f[](3). .PP See \f[B]pmem_persist\f[](3) and \f[B]pmem_msync\f[](3) for more details. .SH SEE ALSO .PP \f[B]ssh\f[](1), \f[B]pmem_msync\f[](3), \f[B]pmem_persist\f[](3), \f[B]syslog\f[](3), \f[B]libpmem\f[](7), \f[B]libpmemobj\f[](7), \f[B]librpmem\f[](7) and \f[B]\f[] pmdk-1.8/doc/.gitignore0000664000000000000000000000006213615011243013544 0ustar rootroot*.txt *.html *.gz LICENSE web_linux/ web_windows/ pmdk-1.8/doc/libpmem/0000775000000000000000000000000013615011416013205 5ustar rootrootpmdk-1.8/doc/libpmem/pmem_is_pmem.30000644000000000000000000002127313615011416015743 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEM_IS_PMEM" "3" "2020-01-31" "PMDK - pmem API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmem_is_pmem\f[](), \f[B]pmem_map_file\f[](), \f[B]pmem_unmap\f[]() \- check persistency, create and delete mappings .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmem_is_pmem(const\ void\ *addr,\ size_t\ len); void\ *pmem_map_file(const\ char\ *path,\ size_t\ len,\ int\ flags, \ \ \ \ mode_t\ mode,\ size_t\ *mapped_lenp,\ int\ *is_pmemp); int\ pmem_unmap(void\ *addr,\ size_t\ len); \f[] .fi .SH DESCRIPTION .PP Most pmem\-aware applications will take advantage of higher level libraries that alleviate the need for the application to call into \f[B]libpmem\f[] directly. Application developers that wish to access raw memory mapped persistence directly (via \f[B]mmap\f[](2)) and that wish to take on the responsibility for flushing stores to persistence will find the functions described in this section to be the most commonly used. .PP The \f[B]pmem_is_pmem\f[]() function detects if the entire range [\f[I]addr\f[], \f[I]addr\f[]+\f[I]len\f[]) consists of persistent memory. Calling this function with a memory range that originates from a source different than \f[B]pmem_map_file()\f[] is undefined. The implementation of \f[B]pmem_is_pmem\f[]() requires a non\-trivial amount of work to determine if the given range is entirely persistent memory. For this reason, it is better to call \f[B]pmem_is_pmem\f[]() once when a range of memory is first encountered, save the result, and use the saved result to determine whether \f[B]pmem_persist\f[](3) or \f[B]msync\f[](2) is appropriate for flushing changes to persistence. Calling \f[B]pmem_is_pmem\f[]() each time changes are flushed to persistence will not perform well. .PP The \f[B]pmem_map_file\f[]() function creates a new read/write mapping for a file. If \f[B]PMEM_FILE_CREATE\f[] is not specified in \f[I]flags\f[], the entire existing file \f[I]path\f[] is mapped, \f[I]len\f[] must be zero, and \f[I]mode\f[] is ignored. Otherwise, \f[I]path\f[] is opened or created as specified by \f[I]flags\f[] and \f[I]mode\f[], and \f[I]len\f[] must be non\-zero. \f[B]pmem_map_file\f[]() maps the file using \f[B]mmap\f[](2), but it also takes extra steps to make large page mappings more likely. .PP On success, \f[B]pmem_map_file\f[]() returns a pointer to the mapped area. If \f[I]mapped_lenp\f[] is not NULL, the length of the mapping is stored into *\f[I]mapped_lenp\f[]. If \f[I]is_pmemp\f[] is not NULL, a flag indicating whether the mapped file is actual pmem, or if \f[B]msync\f[]() must be used to flush writes for the mapped range, is stored into *\f[I]is_pmemp\f[]. .PP The \f[I]flags\f[] argument is 0 or the bitwise OR of one or more of the following file creation flags: .IP \[bu] 2 \f[B]PMEM_FILE_CREATE\f[] \- Create the file named \f[I]path\f[] if it does not exist. \f[I]len\f[] must be non\-zero and specifies the size of the file to be created. If the file already exists, it will be extended or truncated to \f[I]len.\f[] The new or existing file is then fully allocated to size \f[I]len\f[] using \f[B]posix_fallocate\f[](3). \f[I]mode\f[] specifies the mode to use in case a new file is created (see \f[B]creat\f[](2)). .PP The remaining flags modify the behavior of \f[B]pmem_map_file\f[]() when \f[B]PMEM_FILE_CREATE\f[] is specified. .IP \[bu] 2 \f[B]PMEM_FILE_EXCL\f[] \- If specified in conjunction with \f[B]PMEM_FILE_CREATE\f[], and \f[I]path\f[] already exists, then \f[B]pmem_map_file\f[]() will fail with \f[B]EEXIST\f[]. Otherwise, has the same meaning as \f[B]O_EXCL\f[] on \f[B]open\f[](2), which is generally undefined. .IP \[bu] 2 \f[B]PMEM_FILE_SPARSE\f[] \- When specified in conjunction with \f[B]PMEM_FILE_CREATE\f[], create a sparse (holey) file using \f[B]ftruncate\f[](2) rather than allocating it using \f[B]posix_fallocate\f[](3). Otherwise ignored. .IP \[bu] 2 \f[B]PMEM_FILE_TMPFILE\f[] \- Create a mapping for an unnamed temporary file. Must be specified with \f[B]PMEM_FILE_CREATE\f[]. \f[I]len\f[] must be non\-zero, \f[I]mode\f[] is ignored (the temporary file is always created with mode 0600), and \f[I]path\f[] must specify an existing directory name. If the underlying file system supports \f[B]O_TMPFILE\f[], the unnamed temporary file is created in the filesystem containing the directory \f[I]path\f[]; if \f[B]PMEM_FILE_EXCL\f[] is also specified, the temporary file may not subsequently be linked into the filesystem (see \f[B]open\f[](2)). Otherwise, the file is created in \f[I]path\f[] and immediately unlinked. .PP The \f[I]path\f[] can point to a Device DAX. In this case only the \f[B]PMEM_FILE_CREATE\f[] and \f[B]PMEM_FILE_SPARSE\f[] flags are valid, but they are both ignored. For Device DAX mappings, \f[I]len\f[] must be equal to either 0 or the exact size of the device. .PP To delete mappings created with \f[B]pmem_map_file\f[](), use \f[B]pmem_unmap\f[](). .PP The \f[B]pmem_unmap\f[]() function deletes all the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. It will use the address specified by the parameter \f[I]addr\f[], where \f[I]addr\f[] must be a previously mapped region. \f[B]pmem_unmap\f[]() will delete the mappings using \f[B]munmap\f[](2). .SH RETURN VALUE .PP The \f[B]pmem_is_pmem\f[]() function returns true only if the entire range [\f[I]addr\f[], \f[I]addr\f[]+\f[I]len\f[]) consists of persistent memory. A true return from \f[B]pmem_is_pmem\f[]() means it is safe to use \f[B]pmem_persist\f[](3) and the related functions to make changes durable for that memory range. See also \f[B]CAVEATS\f[]. .PP On success, \f[B]pmem_map_file\f[]() returns a pointer to the memory\-mapped region and sets *\f[I]mapped_lenp\f[] and *\f[I]is_pmemp\f[] if they are not NULL. On error, it returns NULL, sets \f[I]errno\f[] appropriately, and does not modify *\f[I]mapped_lenp\f[] or *\f[I]is_pmemp\f[]. .PP On success, \f[B]pmem_unmap\f[]() returns 0. On error, it returns \-1 and sets \f[I]errno\f[] appropriately. .SH NOTES .PP On Linux, \f[B]pmem_is_pmem\f[]() returns true only if the entire range is mapped directly from Device DAX (/dev/daxX.Y) without an intervening file system. In the future, as file systems become available that support flushing with \f[B]pmem_persist\f[](3), \f[B]pmem_is_pmem\f[]() will return true as appropriate. .SH CAVEATS .PP The result of \f[B]pmem_is_pmem\f[]() query is only valid for the mappings created using \f[B]pmem_map_file\f[](). For other memory regions, in particular those created by a direct call to \f[B]mmap\f[](2), \f[B]pmem_is_pmem\f[]() always returns false, even if the queried range is entirely persistent memory. .PP Not all file systems support \f[B]posix_fallocate\f[](3). \f[B]pmem_map_file\f[]() will fail if \f[B]PMEM_FILE_CREATE\f[] is specified without \f[B]PMEM_FILE_SPARSE\f[] and the underlying file system does not support \f[B]posix_fallocate\f[](3). .SH SEE ALSO .PP \f[B]creat\f[](2), \f[B]ftruncate\f[](2), \f[B]mmap\f[](2), \f[B]msync\f[](2), \f[B]munmap\f[](2), \f[B]open\f[](2), \f[B]pmem_persist\f[](3), \f[B]posix_fallocate\f[](3), \f[B]libpmem\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmem/pmem_memset_persist.30000664000000000000000000000003313615011243017344 0ustar rootroot.so pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/pmem_memcpy.30000664000000000000000000000003313615011243015573 0ustar rootroot.so pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/pmem_is_pmem.3.md0000664000000000000000000002104013615011243016332 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEM_IS_PMEM, 3) collection: libpmem header: PMDK date: pmem API version 1.1 ... [comment]: <> (Copyright 2017-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmem_is_pmem.3 -- man page for libpmem persistence and mapping functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[NOTES](#notes)
[CAVEATS](#caveats)
[BUGS](#bugs)
[SEE ALSO](#see-also)
# NAME # **pmem_is_pmem**(), _UW(pmem_map_file), **pmem_unmap**() - check persistency, create and delete mappings # SYNOPSIS # ```c #include int pmem_is_pmem(const void *addr, size_t len); _UWFUNCR1(void, *pmem_map_file, *path, =q=size_t len, int flags, mode_t mode, size_t *mapped_lenp, int *is_pmemp=e=) int pmem_unmap(void *addr, size_t len); ``` _UNICODE() # DESCRIPTION # Most pmem-aware applications will take advantage of higher level libraries that alleviate the need for the application to call into **libpmem** directly. Application developers that wish to access raw memory mapped persistence directly (via **mmap**(2)) and that wish to take on the responsibility for flushing stores to persistence will find the functions described in this section to be the most commonly used. The **pmem_is_pmem**() function detects if the entire range \[*addr*, *addr*+*len*) consists of persistent memory. Calling this function with a memory range that originates from a source different than **pmem_map_file()** is undefined. The implementation of **pmem_is_pmem**() requires a non-trivial amount of work to determine if the given range is entirely persistent memory. For this reason, it is better to call **pmem_is_pmem**() once when a range of memory is first encountered, save the result, and use the saved result to determine whether **pmem_persist**(3) or **msync**(2) is appropriate for flushing changes to persistence. Calling **pmem_is_pmem**() each time changes are flushed to persistence will not perform well. The _UW(pmem_map_file) function creates a new read/write mapping for a file. If **PMEM_FILE_CREATE** is not specified in *flags*, the entire existing file *path* is mapped, *len* must be zero, and *mode* is ignored. Otherwise, *path* is opened or created as specified by *flags* and *mode*, and *len* must be non-zero. _UW(pmem_map_file) maps the file using **mmap**(2), but it also takes extra steps to make large page mappings more likely. On success, _UW(pmem_map_file) returns a pointer to the mapped area. If *mapped_lenp* is not NULL, the length of the mapping is stored into \**mapped_lenp*. If *is_pmemp* is not NULL, a flag indicating whether the mapped file is actual pmem, or if **msync**() must be used to flush writes for the mapped range, is stored into \**is_pmemp*. The *flags* argument is 0 or the bitwise OR of one or more of the following file creation flags: + **PMEM_FILE_CREATE** - Create the file named *path* if it does not exist. *len* must be non-zero and specifies the size of the file to be created. If the file already exists, it will be extended or truncated to *len.* The new or existing file is then fully allocated to size *len* using **posix_fallocate**(3). *mode* specifies the mode to use in case a new file is created (see **creat**(2)). The remaining flags modify the behavior of _UW(pmem_map_file) when **PMEM_FILE_CREATE** is specified. + **PMEM_FILE_EXCL** - If specified in conjunction with **PMEM_FILE_CREATE**, and *path* already exists, then _UW(pmem_map_file) will fail with **EEXIST**. Otherwise, has the same meaning as **O_EXCL** on **open**(2), which is generally undefined. + **PMEM_FILE_SPARSE** - When specified in conjunction with **PMEM_FILE_CREATE**, create a sparse (holey) file using **ftruncate**(2) rather than allocating it using **posix_fallocate**(3). Otherwise ignored. + **PMEM_FILE_TMPFILE** - Create a mapping for an unnamed temporary file. Must be specified with **PMEM_FILE_CREATE**. *len* must be non-zero, *mode* is ignored (the temporary file is always created with mode 0600), and *path* must specify an existing directory name. If the underlying file system supports **O_TMPFILE**, the unnamed temporary file is created in the filesystem containing the directory *path*; if **PMEM_FILE_EXCL** is also specified, the temporary file may not subsequently be linked into the filesystem (see **open**(2)). Otherwise, the file is created in *path* and immediately unlinked. The *path* can point to a Device DAX. In this case only the **PMEM_FILE_CREATE** and **PMEM_FILE_SPARSE** flags are valid, but they are both ignored. For Device DAX mappings, *len* must be equal to either 0 or the exact size of the device. To delete mappings created with _UW(pmem_map_file), use **pmem_unmap**(). The **pmem_unmap**() function deletes all the mappings for the specified address range, and causes further references to addresses within the range to generate invalid memory references. It will use the address specified by the parameter *addr*, where *addr* must be a previously mapped region. **pmem_unmap**() will delete the mappings using **munmap**(2). # RETURN VALUE # The **pmem_is_pmem**() function returns true only if the entire range \[*addr*, *addr*+*len*) consists of persistent memory. A true return from **pmem_is_pmem**() means it is safe to use **pmem_persist**(3) and the related functions to make changes durable for that memory range. See also **CAVEATS**. On success, _UW(pmem_map_file) returns a pointer to the memory-mapped region and sets \**mapped_lenp* and \**is_pmemp* if they are not NULL. On error, it returns NULL, sets *errno* appropriately, and does not modify \**mapped_lenp* or \**is_pmemp*. On success, **pmem_unmap**() returns 0. On error, it returns -1 and sets *errno* appropriately. # NOTES # On Linux, **pmem_is_pmem**() returns true only if the entire range is mapped directly from Device DAX (/dev/daxX.Y) without an intervening file system. In the future, as file systems become available that support flushing with **pmem_persist**(3), **pmem_is_pmem**() will return true as appropriate. # CAVEATS # The result of **pmem_is_pmem**() query is only valid for the mappings created using _UW(pmem_map_file). For other memory regions, in particular those created by a direct call to **mmap**(2), **pmem_is_pmem**() always returns false, even if the queried range is entirely persistent memory. Not all file systems support **posix_fallocate**(3). _UW(pmem_map_file) will fail if **PMEM_FILE_CREATE** is specified without **PMEM_FILE_SPARSE** and the underlying file system does not support **posix_fallocate**(3). # SEE ALSO # **creat**(2), **ftruncate**(2), **mmap**(2), **msync**(2), **munmap**(2), **open**(2), **pmem_persist**(3), **posix_fallocate**(3), **libpmem**(7) and **** pmdk-1.8/doc/libpmem/pmem_deep_drain.30000664000000000000000000000002113615011243016370 0ustar rootroot.so pmem_flush.3 pmdk-1.8/doc/libpmem/pmem_persist.30000664000000000000000000000002113615011243015767 0ustar rootroot.so pmem_flush.3 pmdk-1.8/doc/libpmem/pmem_memset.30000664000000000000000000000003313615011243015573 0ustar rootroot.so pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/pmem_memcpy_nodrain.30000664000000000000000000000003313615011243017305 0ustar rootroot.so pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/pmem_check_version.30000664000000000000000000000002313615011243017122 0ustar rootroot.so man7/libpmem.7 pmdk-1.8/doc/libpmem/pmem_deep_flush.30000664000000000000000000000002113615011243016414 0ustar rootroot.so pmem_flush.3 pmdk-1.8/doc/libpmem/pmem_memmove_persist.3.md0000664000000000000000000001710613615011243020127 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEM_MEMMOVE_PERSIST, 3) collection: libpmem header: PMDK date: pmem API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmem_memmove_persist.3 -- man page for functions that provide optimized copying to persistent memory [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmem_memmove**(), **pmem_memcpy**(), **pmem_memset**(), **pmem_memmove_persist**(), **pmem_memcpy_persist**(), **pmem_memset_persist**(), **pmem_memmove_nodrain**(), **pmem_memcpy_nodrain**(), **pmem_memset_nodrain**() - functions that provide optimized copying to persistent memory # SYNOPSIS # ```c #include void *pmem_memmove(void *pmemdest, const void *src, size_t len, unsigned flags); void *pmem_memcpy(void *pmemdest, const void *src, size_t len, unsigned flags); void *pmem_memset(void *pmemdest, int c, size_t len, unsigned flags); void *pmem_memmove_persist(void *pmemdest, const void *src, size_t len); void *pmem_memcpy_persist(void *pmemdest, const void *src, size_t len); void *pmem_memset_persist(void *pmemdest, int c, size_t len); void *pmem_memmove_nodrain(void *pmemdest, const void *src, size_t len); void *pmem_memcpy_nodrain(void *pmemdest, const void *src, size_t len); void *pmem_memset_nodrain(void *pmemdest, int c, size_t len); ``` # DESCRIPTION # **pmem_memmove**(), **pmem_memcpy**() and **pmem_memset**() functions provide the same memory copying as their namesakes **memmove**(3), **memcpy**(3) and **memset**(3), and ensure that the result has been flushed to persistence before returning (unless **PMEM_F_MEM_NOFLUSH** flag was used). For example, the following code is functionally equivalent to **pmem_memmove**() (with flags equal to 0): ```c memmove(dest, src, len); pmem_persist(dest, len); ``` Calling **pmem_memmove**() may out-perform the above code, because **libpmem**(7) implementation may take advantage of the fact that *pmemdest* is persistent memory and use instructions such as *non-temporal* stores to avoid the need to flush processor caches. >WARNING: Using these functions where **pmem_is_pmem**(3) returns false may not do anything useful. Use libc functions in that case. Unlike libc implementation, **libpmem** functions guarantee that if destination buffer address and length are 8 byte aligned then all stores will be performed using at least 8 byte store instructions. This means that a series of 8 byte stores followed by **pmem_persist**(3) can be safely replaced by a single call to one of the above functions. The *flags* argument of all of the above functions has the same meaning. It can be 0 or a bitwise OR of one or more of the following flags: + **PMEM_F_MEM_NODRAIN** - modifies the behavior to skip the final **pmem_drain**() step. This allows applications to optimize cases where several ranges are being copied to persistent memory, followed by a single call to **pmem_drain**(). The following example illustrates how this flag might be used to avoid multiple calls to **pmem_drain**() when copying several ranges of memory to pmem: ```c /* ... write several ranges to pmem ... */ pmem_memcpy(pmemdest1, src1, len1, PMEM_F_MEM_NODRAIN); pmem_memcpy(pmemdest2, src2, len2, PMEM_F_MEM_NODRAIN); /* ... */ /* wait for any pmem stores to drain from HW buffers */ pmem_drain(); ``` + **PMEM_F_MEM_NOFLUSH** - Don't flush anything. This implies **PMEM_F_MEM_NODRAIN**. Using this flag only makes sense when it's followed by any function that flushes data. The remaining flags say *how* the operation should be done, and are merely hints. + **PMEM_F_MEM_NONTEMPORAL** - Use non-temporal instructions. This flag is mutually exclusive with **PMEM_F_MEM_TEMPORAL**. On x86\_64 this flag is mutually exclusive with **PMEM_F_MEM_NOFLUSH**. + **PMEM_F_MEM_TEMPORAL** - Use temporal instructions. This flag is mutually exclusive with **PMEM_F_MEM_NONTEMPORAL**. + **PMEM_F_MEM_WC** - Use write combining mode. This flag is mutually exclusive with **PMEM_F_MEM_WB**. On x86\_64 this is an alias for **PMEM_F_MEM_NONTEMPORAL**. On x86\_64 this flag is mutually exclusive with **PMEM_F_MEM_NOFLUSH**. + **PMEM_F_MEM_WB** - Use write back mode. This flag is mutually exclusive with **PMEM_F_MEM_WC**. On x86\_64 this is an alias for **PMEM_F_MEM_TEMPORAL**. Using an invalid combination of flags has undefined behavior. Without any of the above flags **libpmem** will try to guess the best strategy based on size. See **PMEM_MOVNT_THRESHOLD** description in **libpmem**(7) for details. **pmem_memmove_persist**() is an alias for **pmem_memmove**() with flags equal to 0. **pmem_memcpy_persist**() is an alias for **pmem_memcpy**() with flags equal to 0. **pmem_memset_persist**() is an alias for **pmem_memset**() with flags equal to 0. **pmem_memmove_nodrain**() is an alias for **pmem_memmove**() with flags equal to **PMEM_F_MEM_NODRAIN**. **pmem_memcpy_nodrain**() is an alias for **pmem_memcpy**() with flags equal to **PMEM_F_MEM_NODRAIN**. **pmem_memset_nodrain**() is an alias for **pmem_memset**() with flags equal to **PMEM_F_MEM_NODRAIN**. # RETURN VALUE # All of the above functions return address of the destination buffer. # CAVEATS # After calling any of the functions with **PMEM_F_MEM_NODRAIN** flag you should not expect memory to be visible to other threads before calling **pmem_drain**(3) or any of the *\_persist* functions. This is because on x86\_64 those functions may use non-temporal store instructions, which are weakly ordered. See "Intel 64 and IA-32 Architectures Software Developer's Manual", Volume 1, "Caching of Temporal vs. Non-Temporal Data" section for details. # SEE ALSO # **memcpy**(3), **memmove**(3), **memset**(3), **libpmem**(7) and **** pmdk-1.8/doc/libpmem/pmem_flush.3.md0000664000000000000000000002157713615011243016041 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEM_FLUSH, 3) collection: libpmem header: PMDK date: pmem API version 1.1 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmem_flush.3 -- man page for partial flushing operations [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmem_flush**(), **pmem_drain**(), **pmem_persist**(), **pmem_msync**(), **pmem_deep_flush**(), **pmem_deep_drain**(), **pmem_deep_persist**(), **pmem_has_hw_drain**(), **pmem_has_auto_flush**() - check persistency, store persistent data and delete mappings # SYNOPSIS # ```c #include void pmem_persist(const void *addr, size_t len); int pmem_msync(const void *addr, size_t len); void pmem_flush(const void *addr, size_t len); void pmem_deep_flush(const void *addr, size_t len); (EXPERIMENTAL) int pmem_deep_drain(const void *addr, size_t len); (EXPERIMENTAL) int pmem_deep_persist(const void *addr, size_t len); (EXPERIMENTAL) void pmem_drain(void); int pmem_has_auto_flush(void); (EXPERIMENTAL) int pmem_has_hw_drain(void); ``` # DESCRIPTION # The functions in this section provide access to the stages of flushing to persistence, for the less common cases where an application needs more control of the flushing operations than the **pmem_persist**() function. >WARNING: Using **pmem_persist**() on a range where **pmem_is_pmem**(3) returns false may not do anything useful -- use **msync**(2) instead. The **pmem_persist**() function force any changes in the range \[*addr*, *addr*+*len*) to be stored durably in persistent memory. This is equivalent to calling **msync**(2) but may be more optimal and will avoid calling into the kernel if possible. There are no alignment restrictions on the range described by *addr* and *len*, but **pmem_persist**() may expand the range as necessary to meet platform alignment requirements. >WARNING: Like **msync**(2), there is nothing atomic or transactional about this call. Any unwritten stores in the given range will be written, but some stores may have already been written by virtue of normal cache eviction/replacement policies. Correctly written code must not depend on stores waiting until **pmem_persist**() is called to become persistent -- they can become persistent at any time before **pmem_persist**() is called. The **pmem_msync**() function is like **pmem_persist**() in that it forces any changes in the range \[*addr*, *addr*+*len*) to be stored durably. Since it calls **msync**(), this function works on either persistent memory or a memory mapped file on traditional storage. **pmem_msync**() takes steps to ensure the alignment of addresses and lengths passed to **msync**() meet the requirements of that system call. It calls **msync**() with the **MS_SYNC** flag as described in **msync**(2). Typically the application only checks for the existence of persistent memory once, and then uses that result throughout the program, for example: ```c /* do this call once, after the pmem is memory mapped */ int is_pmem = pmem_is_pmem(rangeaddr, rangelen); /* ... make changes to a range of pmem ... */ /* make the changes durable */ if (is_pmem) pmem_persist(subrangeaddr, subrangelen); else pmem_msync(subrangeaddr, subrangelen); /* ... */ ``` _WINUX(,=q= >WARNING: On Linux, **pmem_msync**() and **msync**(2) have no effect on memory ranges mapped from Device DAX. In case of memory ranges where **pmem_is_pmem**(3) returns true use **pmem_persist**() to force the changes to be stored durably in persistent memory. =e=) The **pmem_flush**() and **pmem_drain**() functions provide partial versions of the **pmem_persist**() function. **pmem_persist**() can be thought of as this: ```c void pmem_persist(const void *addr, size_t len) { /* flush the processor caches */ pmem_flush(addr, len); /* wait for any pmem stores to drain from HW buffers */ pmem_drain(); } ``` These functions allow advanced programs to create their own variations of **pmem_persist**(). For example, a program that needs to flush several discontiguous ranges can call **pmem_flush**() for each range and then follow up by calling **pmem_drain**() once. The semantics of **pmem_deep_flush**() function is the same as **pmem_flush**() function except that **pmem_deep_flush**() is indifferent to **PMEM_NO_FLUSH** environment variable (see **ENVIRONMENT** section in **libpmem**(7)) and always flushes processor caches. The behavior of **pmem_deep_persist**() function is the same as **pmem_persist**(), except that it provides higher reliability by flushing persistent memory stores to the most reliable persistence domain available to software rather than depending on automatic WPQ (write pending queue) flush on power failure (ADR). The **pmem_deep_flush**() and **pmem_deep_drain**() functions provide partial versions of **pmem_deep_persist**() function. **pmem_deep_persist**() can be thought of as this: ``` int pmem_deep_persist(const void *addr, size_t len) { /* flush the processor caches */ pmem_deep_flush(addr, len); /* wait for any pmem stores to drain from HW buffers */ return pmem_deep_drain(addr, len); } ``` Since this operation is usually much more expensive than **pmem_persist**(), it should be used rarely. Typically the application should use this function only to flush the most critical data, which are required to recover after the power failure. The **pmem_has_auto_flush**() function checks if the machine supports automatic CPU cache flush on power failure or system crash. Function returns true only when each NVDIMM in the system is covered by this mechanism. The **pmem_has_hw_drain**() function checks if the machine supports an explicit *hardware drain* instruction for persistent memory. # RETURN VALUE # The **pmem_persist**() function returns no value. The **pmem_msync**() return value is the return value of **msync**(), which can return -1 and set *errno* to indicate an error. The **pmem_flush**(), **pmem_drain**() and **pmem_deep_flush**() functions return no value. The **pmem_deep_persist**() and **pmem_deep_drain**() return 0 on success. Otherwise it returns -1 and sets *errno* appropriately. If *len* is equal zero **pmem_deep_persist**() and **pmem_deep_drain**() return 0 but no flushing take place. The **pmem_has_auto_flush**() function returns 1 if given platform supports processor cache flushing on a power loss event. Otherwise it returns 0. On error it returns -1 and sets *errno* appropriately. The **pmem_has_hw_drain**() function returns true if the machine supports an explicit *hardware drain* instruction for persistent memory. On Intel processors with persistent memory, stores to persistent memory are considered persistent once they are flushed from the CPU caches, so this function always returns false. Despite that, programs using **pmem_flush**() to flush ranges of memory should still follow up by calling **pmem_drain**() once to ensure the flushes are complete. As mentioned above, **pmem_persist**() handles calling both **pmem_flush**() and **pmem_drain**(). # SEE ALSO # **msync**(2), **pmem_is_pmem**(3), **libpmem**(7) and **** pmdk-1.8/doc/libpmem/pmem_memmove.30000664000000000000000000000003313615011243015746 0ustar rootroot.so pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/pmem_map_file.30000664000000000000000000000002313615011243016054 0ustar rootroot.so pmem_is_pmem.3 pmdk-1.8/doc/libpmem/.gitignore0000664000000000000000000000007513615011243015175 0ustar rootrootlibpmem.7 pmem_flush.3 pmem_is_pmem.3 pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/pmem_drain.30000664000000000000000000000002113615011243015373 0ustar rootroot.so pmem_flush.3 pmdk-1.8/doc/libpmem/pmem_errormsg.30000664000000000000000000000002313615011243016140 0ustar rootroot.so man7/libpmem.7 pmdk-1.8/doc/libpmem/pmem_memmove_nodrain.30000664000000000000000000000003313615011243017460 0ustar rootroot.so pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/pmem_memcpy_persist.30000664000000000000000000000003313615011243017344 0ustar rootroot.so pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/libpmem.7.md0000664000000000000000000003536713615011243015335 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(LIBPMEM, 7) collection: libpmem header: PMDK date: pmem API version 1.1 ... [comment]: <> (Copyright 2016-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (libpmem.7 -- man page for libpmem) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[CAVEATS](#caveats)
[LIBRARY API VERSIONING](#library-api-versioning-1)
[ENVIRONMENT](#environment)
[DEBUGGING AND ERROR HANDLING](#debugging-and-error-handling)
[EXAMPLE](#example)
[ACKNOWLEDGEMENTS](#acknowledgements)
[SEE ALSO](#see-also) # NAME # **libpmem** - persistent memory support library # SYNOPSIS # ```c #include cc ... -lpmem ``` _UNICODE() ##### Library API versioning: ##### ```c _UWFUNC(pmem_check_version, =q= unsigned major_required, unsigned minor_required=e=) ``` ##### Error handling: ##### ```c _UWFUNC(pmem_errormsg, void) ``` ##### Other library functions: ##### A description of other **libpmem** functions can be found on the following manual pages: + most commonly used functions: **pmem_is_pmem**(3) + partial flushing operations: **pmem_flush**(3) + copying to persistent memory: **pmem_memmove_persist**(3) # DESCRIPTION # **libpmem** provides low-level *persistent memory* (pmem) support for applications using direct access storage (DAX), which is storage that supports load/store access without paging blocks from a block storage device. Some types of *non-volatile memory DIMMs* (NVDIMMs) provide this type of byte addressable access to storage. A *persistent memory aware file system* is typically used to expose the direct access to applications. Memory mapping a file from this type of file system results in the load/store, non-paged access to pmem. This library is for applications that use persistent memory directly, without the help of any library-supplied transactions or memory allocation. Higher-level libraries that build on **libpmem** are available and are recommended for most applications, see: + **libpmemobj**(7), a general use persistent memory API, providing memory allocation and transactional operations on variable-sized objects. + **libpmemblk**(7), providing pmem-resident arrays of fixed-sized blocks with atomic updates. + **libpmemlog**(7), providing a pmem-resident log file. Under normal usage, **libpmem** will never print messages or intentionally cause the process to exit. The only exception to this is the debugging information, when enabled, as described under **DEBUGGING AND ERROR HANDLING** below. # CAVEATS # **libpmem** relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. **dlclose**(3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. # LIBRARY API VERSIONING # This section describes how the library API is versioned, allowing applications to work with an evolving API. The _UW(pmem_check_version) function is used to determine whether the installed **libpmem** supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile-time version information, supplied by defines in **\**, like this: ```c reason = _U(pmem_check_version)(PMEM_MAJOR_VERSION, PMEM_MINOR_VERSION); if (reason != NULL) { /* version check failed, reason string tells you why */ } ``` Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text *introduced in version x.y* in the section of this manual describing the feature. When the version check performed by _UW(pmem_check_version) is successful, the return value is NULL. Otherwise the return value is a static string describing the reason for failing the version check. The string returned by _UW(pmem_check_version) must not be modified or freed. # ENVIRONMENT # **libpmem** can change its default behavior based on the following environment variables. These are largely intended for testing and are not normally required. + **PMEM_IS_PMEM_FORCE**=*val* If *val* is 0 (zero), then **pmem_is_pmem**(3) will always return false. Setting *val* to 1 causes **pmem_is_pmem**(3) to always return true. This variable is mostly used for testing but can be used to force pmem behavior on a system where a range of pmem is not detectable as pmem for some reason. >NOTE: Unlike the other variables, the value of **PMEM_IS_PMEM_FORCE** is not queried (and cached) at library initialization time, but on the first call to **pmem_is_pmem**(3). This means that in case of **libpmemlog**(7), **libpmemblk**(7), and **libpmemobj**(7), **PMEM_IS_PMEM_FORCE** may still be set or modified by the program until the first attempt to create or open the persistent memory pool. + **PMEM_NO_CLWB**=1 Setting this environment variable to 1 forces **libpmem** to never issue the **CLWB** instruction on Intel hardware, falling back to other cache flush instructions instead (**CLFLUSHOPT** or **CLFLUSH** on Intel hardware). Without this environment variable, **libpmem** will always use the **CLWB** instruction for flushing processor caches on platforms that support the instruction. This variable is intended for use during library testing but may be required for some rare cases where using **CLWB** has a negative impact on performance. + **PMEM_NO_CLFLUSHOPT**=1 Setting this environment variable to 1 forces **libpmem** to never issue the **CLFLUSHOPT** instruction on Intel hardware, falling back to the **CLFLUSH** instructions instead. Without this environment variable, **libpmem** will always use the **CLFLUSHOPT** instruction for flushing processor caches on platforms that support the instruction, but where **CLWB** is not available. This variable is intended for use during library testing. + **PMEM_NO_FLUSH**=1 Setting this environment variable to 1 forces most **libpmem** functions to never issue any of **CLFLUSH**, **CLFLUSHOPT** or **CLWB** instructions on Intel hardware. The only exceptions are **pmem_deep_flush**(3) and **pmem_deep_persist**(3) functions. + **PMEM_NO_FLUSH**=0 Setting this environment variable to 0 forces to always flush CPU caches using one of **CLFLUSH**, **CLFLUSHOPT** or **CLWB** instructions even if **pmem_has_auto_flush**(3) function returns true and the platform supports flushing the processor caches on power loss or system crash. + **PMEM_NO_MOVNT**=1 Setting this environment variable to 1 forces **libpmem** to never use the *non-temporal* move instructions on Intel hardware. Without this environment variable, **libpmem** will use the non-temporal instructions for copying larger ranges to persistent memory on platforms that support the instructions. This variable is intended for use during library testing. + **PMEM_MOVNT_THRESHOLD**=*val* This environment variable allows overriding the minimum length of the **pmem_memmove_persist**(3) operations, for which **libpmem** uses *non-temporal* move instructions. Setting this environment variable to 0 forces **libpmem** to always use the *non-temporal* move instructions if available. It has no effect if **PMEM_NO_MOVNT** is set to 1. This variable is intended for use during library testing. + **PMEM_MMAP_HINT**=*val* This environment variable allows overriding the hint address used by _UW(pmem_map_file). If set, it also disables mapping address randomization. This variable is intended for use during library testing and debugging. Setting it to some fairly large value (i.e. 0x10000000000) will very likely result in mapping the file at the specified address (if not used) or at the first unused region above given address, without adding any random offset. When debugging, this makes it easier to calculate the actual address of the persistent memory block, based on its offset in the file. In case of **libpmemobj** it simplifies conversion of a persistent object identifier (OID) into a direct pointer to the object. >NOTE: **Setting this environment variable affects all the PMDK libraries,** disabling mapping address randomization and causing the specified address to be used as a hint about where to place the mapping. # DEBUGGING AND ERROR HANDLING # If an error is detected during the call to a **libpmem** function, the application may retrieve an error message describing the reason of the failure from _UW(pmem_errormsg). This function returns a pointer to a static buffer containing the last error message logged for the current thread. If *errno* was set, the error message may include a description of the corresponding error code as returned by **strerror**(3). The error message buffer is thread-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a **libpmem** function indicated an error. The application must not modify or free the error message string. Subsequent calls to other library functions may modify the previous message. Two versions of **libpmem** are typically available on a development system. The normal version, accessed when a program is linked using the **-lpmem** option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run-time assertions. A second version of **libpmem**, accessed when a program uses the libraries under _DEBUGLIBPATH(), contains run-time assertions and trace points. The typical way to access the debug version is to set the environment variable **LD_LIBRARY_PATH** to _LDLIBPATH(). Debugging output is controlled using the following environment variables. These variables have no effect on the non-debug version of the library. + **PMEM_LOG_LEVEL** The value of **PMEM_LOG_LEVEL** enables trace points in the debug version of the library, as follows: + **0** - This is the default level when **PMEM_LOG_LEVEL** is not set. No log messages are emitted at this level. + **1** - Additional details on any errors detected are logged, in addition to returning the *errno*-based errors as usual. The same information may be retrieved using _UW(pmem_errormsg). + **2** - A trace of basic operations is logged. + **3** - Enables a very verbose amount of function call tracing in the library. + **4** - Enables voluminous and fairly obscure tracing information that is likely only useful to the **libpmem** developers. Unless **PMEM_LOG_FILE** is set, debugging output is written to *stderr*. + **PMEM_LOG_FILE** Specifies the name of a file where all logging information should be written. If the last character in the name is "-", the *PID* of the current process will be appended to the file name when the log file is created. If **PMEM_LOG_FILE** is not set, output is written to *stderr*. # EXAMPLE # The following example uses **libpmem** to flush changes made to raw, memory-mapped persistent memory. >WARNING: There is nothing transactional about the **pmem_persist**(3) or **pmem_msync**(3) calls in this example. Interrupting the program may result in a partial write to pmem. Use a transactional library such as **libpmemobj**(7) to avoid torn updates. ```c #include #include #include #include #include #include #include #include #include /* using 4k of pmem for this example */ #define PMEM_LEN 4096 #define PATH "/pmem-fs/myfile" int main(int argc, char *argv[]) { char *pmemaddr; size_t mapped_len; int is_pmem; /* create a pmem file and memory map it */ if ((pmemaddr = _U(pmem_map_file)(PATH, PMEM_LEN, PMEM_FILE_CREATE, 0666, &mapped_len, &is_pmem)) == NULL) { perror("_U(pmem_map_file)"); exit(1); } /* store a string to the persistent memory */ strcpy(pmemaddr, "hello, persistent memory"); /* flush above strcpy to persistence */ if (is_pmem) pmem_persist(pmemaddr, mapped_len); else pmem_msync(pmemaddr, mapped_len); /* * Delete the mappings. The region is also * automatically unmapped when the process is * terminated. */ pmem_unmap(pmemaddr, mapped_len); } ``` See for more examples using the **libpmem** API. # ACKNOWLEDGEMENTS # **libpmem** builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: # SEE ALSO # **dlclose**(3), **pmem_flush**(3), **pmem_is_pmem**(3), **pmem_memmove_persist**(3), **pmem_msync**(3), **pmem_persist**(3), **strerror**(3), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmem/pmem_flush.30000644000000000000000000002224413615011416015432 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEM_FLUSH" "3" "2020-01-31" "PMDK - pmem API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmem_flush\f[](), \f[B]pmem_drain\f[](), \f[B]pmem_persist\f[](), \f[B]pmem_msync\f[](), \f[B]pmem_deep_flush\f[](), \f[B]pmem_deep_drain\f[](), \f[B]pmem_deep_persist\f[](), \f[B]pmem_has_hw_drain\f[](), \f[B]pmem_has_auto_flush\f[]() \- check persistency, store persistent data and delete mappings .SH SYNOPSIS .IP .nf \f[C] #include\ void\ pmem_persist(const\ void\ *addr,\ size_t\ len); int\ pmem_msync(const\ void\ *addr,\ size_t\ len); void\ pmem_flush(const\ void\ *addr,\ size_t\ len); void\ pmem_deep_flush(const\ void\ *addr,\ size_t\ len);\ (EXPERIMENTAL) int\ pmem_deep_drain(const\ void\ *addr,\ size_t\ len);\ (EXPERIMENTAL) int\ pmem_deep_persist(const\ void\ *addr,\ size_t\ len);\ (EXPERIMENTAL) void\ pmem_drain(void); int\ pmem_has_auto_flush(void);\ (EXPERIMENTAL) int\ pmem_has_hw_drain(void); \f[] .fi .SH DESCRIPTION .PP The functions in this section provide access to the stages of flushing to persistence, for the less common cases where an application needs more control of the flushing operations than the \f[B]pmem_persist\f[]() function. .RS .PP WARNING: Using \f[B]pmem_persist\f[]() on a range where \f[B]pmem_is_pmem\f[](3) returns false may not do anything useful \[en] use \f[B]msync\f[](2) instead. .RE .PP The \f[B]pmem_persist\f[]() function force any changes in the range [\f[I]addr\f[], \f[I]addr\f[]+\f[I]len\f[]) to be stored durably in persistent memory. This is equivalent to calling \f[B]msync\f[](2) but may be more optimal and will avoid calling into the kernel if possible. There are no alignment restrictions on the range described by \f[I]addr\f[] and \f[I]len\f[], but \f[B]pmem_persist\f[]() may expand the range as necessary to meet platform alignment requirements. .RS .PP WARNING: Like \f[B]msync\f[](2), there is nothing atomic or transactional about this call. Any unwritten stores in the given range will be written, but some stores may have already been written by virtue of normal cache eviction/replacement policies. Correctly written code must not depend on stores waiting until \f[B]pmem_persist\f[]() is called to become persistent \[en] they can become persistent at any time before \f[B]pmem_persist\f[]() is called. .RE .PP The \f[B]pmem_msync\f[]() function is like \f[B]pmem_persist\f[]() in that it forces any changes in the range [\f[I]addr\f[], \f[I]addr\f[]+\f[I]len\f[]) to be stored durably. Since it calls \f[B]msync\f[](), this function works on either persistent memory or a memory mapped file on traditional storage. \f[B]pmem_msync\f[]() takes steps to ensure the alignment of addresses and lengths passed to \f[B]msync\f[]() meet the requirements of that system call. It calls \f[B]msync\f[]() with the \f[B]MS_SYNC\f[] flag as described in \f[B]msync\f[](2). Typically the application only checks for the existence of persistent memory once, and then uses that result throughout the program, for example: .IP .nf \f[C] /*\ do\ this\ call\ once,\ after\ the\ pmem\ is\ memory\ mapped\ */ int\ is_pmem\ =\ pmem_is_pmem(rangeaddr,\ rangelen); /*\ ...\ make\ changes\ to\ a\ range\ of\ pmem\ ...\ */ /*\ make\ the\ changes\ durable\ */ if\ (is_pmem) \ \ \ \ pmem_persist(subrangeaddr,\ subrangelen); else \ \ \ \ pmem_msync(subrangeaddr,\ subrangelen); /*\ ...\ */ \f[] .fi .RS .PP WARNING: On Linux, \f[B]pmem_msync\f[]() and \f[B]msync\f[](2) have no effect on memory ranges mapped from Device DAX. In case of memory ranges where \f[B]pmem_is_pmem\f[](3) returns true use \f[B]pmem_persist\f[]() to force the changes to be stored durably in persistent memory. .RE .PP The \f[B]pmem_flush\f[]() and \f[B]pmem_drain\f[]() functions provide partial versions of the \f[B]pmem_persist\f[]() function. \f[B]pmem_persist\f[]() can be thought of as this: .IP .nf \f[C] void pmem_persist(const\ void\ *addr,\ size_t\ len) { \ \ \ \ /*\ flush\ the\ processor\ caches\ */ \ \ \ \ pmem_flush(addr,\ len); \ \ \ \ /*\ wait\ for\ any\ pmem\ stores\ to\ drain\ from\ HW\ buffers\ */ \ \ \ \ pmem_drain(); } \f[] .fi .PP These functions allow advanced programs to create their own variations of \f[B]pmem_persist\f[](). For example, a program that needs to flush several discontiguous ranges can call \f[B]pmem_flush\f[]() for each range and then follow up by calling \f[B]pmem_drain\f[]() once. .PP The semantics of \f[B]pmem_deep_flush\f[]() function is the same as \f[B]pmem_flush\f[]() function except that \f[B]pmem_deep_flush\f[]() is indifferent to \f[B]PMEM_NO_FLUSH\f[] environment variable (see \f[B]ENVIRONMENT\f[] section in \f[B]libpmem\f[](7)) and always flushes processor caches. .PP The behavior of \f[B]pmem_deep_persist\f[]() function is the same as \f[B]pmem_persist\f[](), except that it provides higher reliability by flushing persistent memory stores to the most reliable persistence domain available to software rather than depending on automatic WPQ (write pending queue) flush on power failure (ADR). .PP The \f[B]pmem_deep_flush\f[]() and \f[B]pmem_deep_drain\f[]() functions provide partial versions of \f[B]pmem_deep_persist\f[]() function. \f[B]pmem_deep_persist\f[]() can be thought of as this: .IP .nf \f[C] int\ pmem_deep_persist(const\ void\ *addr,\ size_t\ len) { \ \ \ \ /*\ flush\ the\ processor\ caches\ */ \ \ \ \ pmem_deep_flush(addr,\ len); \ \ \ \ /*\ wait\ for\ any\ pmem\ stores\ to\ drain\ from\ HW\ buffers\ */ \ \ \ \ return\ pmem_deep_drain(addr,\ len); } \f[] .fi .PP Since this operation is usually much more expensive than \f[B]pmem_persist\f[](), it should be used rarely. Typically the application should use this function only to flush the most critical data, which are required to recover after the power failure. .PP The \f[B]pmem_has_auto_flush\f[]() function checks if the machine supports automatic CPU cache flush on power failure or system crash. Function returns true only when each NVDIMM in the system is covered by this mechanism. .PP The \f[B]pmem_has_hw_drain\f[]() function checks if the machine supports an explicit \f[I]hardware drain\f[] instruction for persistent memory. .SH RETURN VALUE .PP The \f[B]pmem_persist\f[]() function returns no value. .PP The \f[B]pmem_msync\f[]() return value is the return value of \f[B]msync\f[](), which can return \-1 and set \f[I]errno\f[] to indicate an error. .PP The \f[B]pmem_flush\f[](), \f[B]pmem_drain\f[]() and \f[B]pmem_deep_flush\f[]() functions return no value. .PP The \f[B]pmem_deep_persist\f[]() and \f[B]pmem_deep_drain\f[]() return 0 on success. Otherwise it returns \-1 and sets \f[I]errno\f[] appropriately. If \f[I]len\f[] is equal zero \f[B]pmem_deep_persist\f[]() and \f[B]pmem_deep_drain\f[]() return 0 but no flushing take place. .PP The \f[B]pmem_has_auto_flush\f[]() function returns 1 if given platform supports processor cache flushing on a power loss event. Otherwise it returns 0. On error it returns \-1 and sets \f[I]errno\f[] appropriately. .PP The \f[B]pmem_has_hw_drain\f[]() function returns true if the machine supports an explicit \f[I]hardware drain\f[] instruction for persistent memory. On Intel processors with persistent memory, stores to persistent memory are considered persistent once they are flushed from the CPU caches, so this function always returns false. Despite that, programs using \f[B]pmem_flush\f[]() to flush ranges of memory should still follow up by calling \f[B]pmem_drain\f[]() once to ensure the flushes are complete. As mentioned above, \f[B]pmem_persist\f[]() handles calling both \f[B]pmem_flush\f[]() and \f[B]pmem_drain\f[](). .SH SEE ALSO .PP \f[B]msync\f[](2), \f[B]pmem_is_pmem\f[](3), \f[B]libpmem\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmem/pmem_has_hw_drain.30000664000000000000000000000002113615011243016724 0ustar rootroot.so pmem_flush.3 pmdk-1.8/doc/libpmem/pmem_memset_nodrain.30000664000000000000000000000003313615011243017305 0ustar rootroot.so pmem_memmove_persist.3 pmdk-1.8/doc/libpmem/pmem_memmove_persist.30000644000000000000000000001723713615011416017535 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEM_MEMMOVE_PERSIST" "3" "2020-01-31" "PMDK - pmem API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmem_memmove\f[](), \f[B]pmem_memcpy\f[](), \f[B]pmem_memset\f[](), \f[B]pmem_memmove_persist\f[](), \f[B]pmem_memcpy_persist\f[](), \f[B]pmem_memset_persist\f[](), \f[B]pmem_memmove_nodrain\f[](), \f[B]pmem_memcpy_nodrain\f[](), \f[B]pmem_memset_nodrain\f[]() \- functions that provide optimized copying to persistent memory .SH SYNOPSIS .IP .nf \f[C] #include\ void\ *pmem_memmove(void\ *pmemdest,\ const\ void\ *src,\ size_t\ len,\ unsigned\ flags); void\ *pmem_memcpy(void\ *pmemdest,\ const\ void\ *src,\ size_t\ len,\ unsigned\ flags); void\ *pmem_memset(void\ *pmemdest,\ int\ c,\ size_t\ len,\ unsigned\ flags); void\ *pmem_memmove_persist(void\ *pmemdest,\ const\ void\ *src,\ size_t\ len); void\ *pmem_memcpy_persist(void\ *pmemdest,\ const\ void\ *src,\ size_t\ len); void\ *pmem_memset_persist(void\ *pmemdest,\ int\ c,\ size_t\ len); void\ *pmem_memmove_nodrain(void\ *pmemdest,\ const\ void\ *src,\ size_t\ len); void\ *pmem_memcpy_nodrain(void\ *pmemdest,\ const\ void\ *src,\ size_t\ len); void\ *pmem_memset_nodrain(void\ *pmemdest,\ int\ c,\ size_t\ len); \f[] .fi .SH DESCRIPTION .PP \f[B]pmem_memmove\f[](), \f[B]pmem_memcpy\f[]() and \f[B]pmem_memset\f[]() functions provide the same memory copying as their namesakes \f[B]memmove\f[](3), \f[B]memcpy\f[](3) and \f[B]memset\f[](3), and ensure that the result has been flushed to persistence before returning (unless \f[B]PMEM_F_MEM_NOFLUSH\f[] flag was used). .PP For example, the following code is functionally equivalent to \f[B]pmem_memmove\f[]() (with flags equal to 0): .IP .nf \f[C] \ \ \ \ memmove(dest,\ src,\ len); \ \ \ \ pmem_persist(dest,\ len); \f[] .fi .PP Calling \f[B]pmem_memmove\f[]() may out\-perform the above code, because \f[B]libpmem\f[](7) implementation may take advantage of the fact that \f[I]pmemdest\f[] is persistent memory and use instructions such as \f[I]non\-temporal\f[] stores to avoid the need to flush processor caches. .RS .PP WARNING: Using these functions where \f[B]pmem_is_pmem\f[](3) returns false may not do anything useful. Use libc functions in that case. .RE .PP Unlike libc implementation, \f[B]libpmem\f[] functions guarantee that if destination buffer address and length are 8 byte aligned then all stores will be performed using at least 8 byte store instructions. This means that a series of 8 byte stores followed by \f[B]pmem_persist\f[](3) can be safely replaced by a single call to one of the above functions. .PP The \f[I]flags\f[] argument of all of the above functions has the same meaning. It can be 0 or a bitwise OR of one or more of the following flags: .IP \[bu] 2 \f[B]PMEM_F_MEM_NODRAIN\f[] \- modifies the behavior to skip the final \f[B]pmem_drain\f[]() step. This allows applications to optimize cases where several ranges are being copied to persistent memory, followed by a single call to \f[B]pmem_drain\f[](). The following example illustrates how this flag might be used to avoid multiple calls to \f[B]pmem_drain\f[]() when copying several ranges of memory to pmem: .IP .nf \f[C] /*\ ...\ write\ several\ ranges\ to\ pmem\ ...\ */ pmem_memcpy(pmemdest1,\ src1,\ len1,\ PMEM_F_MEM_NODRAIN); pmem_memcpy(pmemdest2,\ src2,\ len2,\ PMEM_F_MEM_NODRAIN); /*\ ...\ */ /*\ wait\ for\ any\ pmem\ stores\ to\ drain\ from\ HW\ buffers\ */ pmem_drain(); \f[] .fi .IP \[bu] 2 \f[B]PMEM_F_MEM_NOFLUSH\f[] \- Don't flush anything. This implies \f[B]PMEM_F_MEM_NODRAIN\f[]. Using this flag only makes sense when it's followed by any function that flushes data. .PP The remaining flags say \f[I]how\f[] the operation should be done, and are merely hints. .IP \[bu] 2 \f[B]PMEM_F_MEM_NONTEMPORAL\f[] \- Use non\-temporal instructions. This flag is mutually exclusive with \f[B]PMEM_F_MEM_TEMPORAL\f[]. On x86_64 this flag is mutually exclusive with \f[B]PMEM_F_MEM_NOFLUSH\f[]. .IP \[bu] 2 \f[B]PMEM_F_MEM_TEMPORAL\f[] \- Use temporal instructions. This flag is mutually exclusive with \f[B]PMEM_F_MEM_NONTEMPORAL\f[]. .IP \[bu] 2 \f[B]PMEM_F_MEM_WC\f[] \- Use write combining mode. This flag is mutually exclusive with \f[B]PMEM_F_MEM_WB\f[]. On x86_64 this is an alias for \f[B]PMEM_F_MEM_NONTEMPORAL\f[]. On x86_64 this flag is mutually exclusive with \f[B]PMEM_F_MEM_NOFLUSH\f[]. .IP \[bu] 2 \f[B]PMEM_F_MEM_WB\f[] \- Use write back mode. This flag is mutually exclusive with \f[B]PMEM_F_MEM_WC\f[]. On x86_64 this is an alias for \f[B]PMEM_F_MEM_TEMPORAL\f[]. .PP Using an invalid combination of flags has undefined behavior. .PP Without any of the above flags \f[B]libpmem\f[] will try to guess the best strategy based on size. See \f[B]PMEM_MOVNT_THRESHOLD\f[] description in \f[B]libpmem\f[](7) for details. .PP \f[B]pmem_memmove_persist\f[]() is an alias for \f[B]pmem_memmove\f[]() with flags equal to 0. .PP \f[B]pmem_memcpy_persist\f[]() is an alias for \f[B]pmem_memcpy\f[]() with flags equal to 0. .PP \f[B]pmem_memset_persist\f[]() is an alias for \f[B]pmem_memset\f[]() with flags equal to 0. .PP \f[B]pmem_memmove_nodrain\f[]() is an alias for \f[B]pmem_memmove\f[]() with flags equal to \f[B]PMEM_F_MEM_NODRAIN\f[]. .PP \f[B]pmem_memcpy_nodrain\f[]() is an alias for \f[B]pmem_memcpy\f[]() with flags equal to \f[B]PMEM_F_MEM_NODRAIN\f[]. .PP \f[B]pmem_memset_nodrain\f[]() is an alias for \f[B]pmem_memset\f[]() with flags equal to \f[B]PMEM_F_MEM_NODRAIN\f[]. .SH RETURN VALUE .PP All of the above functions return address of the destination buffer. .SH CAVEATS .PP After calling any of the functions with \f[B]PMEM_F_MEM_NODRAIN\f[] flag you should not expect memory to be visible to other threads before calling \f[B]pmem_drain\f[](3) or any of the \f[I]_persist\f[] functions. This is because on x86_64 those functions may use non\-temporal store instructions, which are weakly ordered. See \[lq]Intel 64 and IA\-32 Architectures Software Developer's Manual\[rq], Volume 1, \[lq]Caching of Temporal vs.\ Non\-Temporal Data\[rq] section for details. .SH SEE ALSO .PP \f[B]memcpy\f[](3), \f[B]memmove\f[](3), \f[B]memset\f[](3), \f[B]libpmem\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmem/pmem_deep_persist.30000664000000000000000000000002113615011243016764 0ustar rootroot.so pmem_flush.3 pmdk-1.8/doc/libpmem/pmem_msync.30000664000000000000000000000002113615011243015427 0ustar rootroot.so pmem_flush.3 pmdk-1.8/doc/libpmem/pmem_has_auto_flush.30000664000000000000000000000002113615011243017302 0ustar rootroot.so pmem_flush.3 pmdk-1.8/doc/libpmem/pmem_unmap.30000664000000000000000000000002313615011243015420 0ustar rootroot.so pmem_is_pmem.3 pmdk-1.8/doc/libpmem/libpmem.70000644000000000000000000003676013615011416014734 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "LIBPMEM" "7" "2020-01-31" "PMDK - pmem API version 1.1" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]libpmem\f[] \- persistent memory support library .SH SYNOPSIS .IP .nf \f[C] #include\ cc\ ...\ \-lpmem \f[] .fi .SS Library API versioning: .IP .nf \f[C] const\ char\ *pmem_check_version( \ \ \ \ unsigned\ major_required, \ \ \ \ unsigned\ minor_required); \f[] .fi .SS Error handling: .IP .nf \f[C] const\ char\ *pmem_errormsg(void); \f[] .fi .SS Other library functions: .PP A description of other \f[B]libpmem\f[] functions can be found on the following manual pages: .IP \[bu] 2 most commonly used functions: \f[B]pmem_is_pmem\f[](3) .IP \[bu] 2 partial flushing operations: \f[B]pmem_flush\f[](3) .IP \[bu] 2 copying to persistent memory: \f[B]pmem_memmove_persist\f[](3) .SH DESCRIPTION .PP \f[B]libpmem\f[] provides low\-level \f[I]persistent memory\f[] (pmem) support for applications using direct access storage (DAX), which is storage that supports load/store access without paging blocks from a block storage device. Some types of \f[I]non\-volatile memory DIMMs\f[] (NVDIMMs) provide this type of byte addressable access to storage. A \f[I]persistent memory aware file system\f[] is typically used to expose the direct access to applications. Memory mapping a file from this type of file system results in the load/store, non\-paged access to pmem. .PP This library is for applications that use persistent memory directly, without the help of any library\-supplied transactions or memory allocation. Higher\-level libraries that build on \f[B]libpmem\f[] are available and are recommended for most applications, see: .IP \[bu] 2 \f[B]libpmemobj\f[](7), a general use persistent memory API, providing memory allocation and transactional operations on variable\-sized objects. .IP \[bu] 2 \f[B]libpmemblk\f[](7), providing pmem\-resident arrays of fixed\-sized blocks with atomic updates. .IP \[bu] 2 \f[B]libpmemlog\f[](7), providing a pmem\-resident log file. .PP Under normal usage, \f[B]libpmem\f[] will never print messages or intentionally cause the process to exit. The only exception to this is the debugging information, when enabled, as described under \f[B]DEBUGGING AND ERROR HANDLING\f[] below. .SH CAVEATS .PP \f[B]libpmem\f[] relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. \f[B]dlclose\f[](3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. .SH LIBRARY API VERSIONING .PP This section describes how the library API is versioned, allowing applications to work with an evolving API. .PP The \f[B]pmem_check_version\f[]() function is used to determine whether the installed \f[B]libpmem\f[] supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile\-time version information, supplied by defines in \f[B]\f[], like this: .IP .nf \f[C] reason\ =\ pmem_check_version(PMEM_MAJOR_VERSION, \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ PMEM_MINOR_VERSION); if\ (reason\ !=\ NULL)\ { \ \ \ \ /*\ version\ check\ failed,\ reason\ string\ tells\ you\ why\ */ } \f[] .fi .PP Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. .PP An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text \f[I]introduced in version x.y\f[] in the section of this manual describing the feature. .PP When the version check performed by \f[B]pmem_check_version\f[]() is successful, the return value is NULL. Otherwise the return value is a static string describing the reason for failing the version check. The string returned by \f[B]pmem_check_version\f[]() must not be modified or freed. .SH ENVIRONMENT .PP \f[B]libpmem\f[] can change its default behavior based on the following environment variables. These are largely intended for testing and are not normally required. .IP \[bu] 2 \f[B]PMEM_IS_PMEM_FORCE\f[]=\f[I]val\f[] .PP If \f[I]val\f[] is 0 (zero), then \f[B]pmem_is_pmem\f[](3) will always return false. Setting \f[I]val\f[] to 1 causes \f[B]pmem_is_pmem\f[](3) to always return true. This variable is mostly used for testing but can be used to force pmem behavior on a system where a range of pmem is not detectable as pmem for some reason. .RS .PP NOTE: Unlike the other variables, the value of \f[B]PMEM_IS_PMEM_FORCE\f[] is not queried (and cached) at library initialization time, but on the first call to \f[B]pmem_is_pmem\f[](3). This means that in case of \f[B]libpmemlog\f[](7), \f[B]libpmemblk\f[](7), and \f[B]libpmemobj\f[](7), \f[B]PMEM_IS_PMEM_FORCE\f[] may still be set or modified by the program until the first attempt to create or open the persistent memory pool. .RE .IP \[bu] 2 \f[B]PMEM_NO_CLWB\f[]=1 .PP Setting this environment variable to 1 forces \f[B]libpmem\f[] to never issue the \f[B]CLWB\f[] instruction on Intel hardware, falling back to other cache flush instructions instead (\f[B]CLFLUSHOPT\f[] or \f[B]CLFLUSH\f[] on Intel hardware). Without this environment variable, \f[B]libpmem\f[] will always use the \f[B]CLWB\f[] instruction for flushing processor caches on platforms that support the instruction. This variable is intended for use during library testing but may be required for some rare cases where using \f[B]CLWB\f[] has a negative impact on performance. .IP \[bu] 2 \f[B]PMEM_NO_CLFLUSHOPT\f[]=1 .PP Setting this environment variable to 1 forces \f[B]libpmem\f[] to never issue the \f[B]CLFLUSHOPT\f[] instruction on Intel hardware, falling back to the \f[B]CLFLUSH\f[] instructions instead. Without this environment variable, \f[B]libpmem\f[] will always use the \f[B]CLFLUSHOPT\f[] instruction for flushing processor caches on platforms that support the instruction, but where \f[B]CLWB\f[] is not available. This variable is intended for use during library testing. .IP \[bu] 2 \f[B]PMEM_NO_FLUSH\f[]=1 .PP Setting this environment variable to 1 forces most \f[B]libpmem\f[] functions to never issue any of \f[B]CLFLUSH\f[], \f[B]CLFLUSHOPT\f[] or \f[B]CLWB\f[] instructions on Intel hardware. The only exceptions are \f[B]pmem_deep_flush\f[](3) and \f[B]pmem_deep_persist\f[](3) functions. .IP \[bu] 2 \f[B]PMEM_NO_FLUSH\f[]=0 .PP Setting this environment variable to 0 forces to always flush CPU caches using one of \f[B]CLFLUSH\f[], \f[B]CLFLUSHOPT\f[] or \f[B]CLWB\f[] instructions even if \f[B]pmem_has_auto_flush\f[](3) function returns true and the platform supports flushing the processor caches on power loss or system crash. .IP \[bu] 2 \f[B]PMEM_NO_MOVNT\f[]=1 .PP Setting this environment variable to 1 forces \f[B]libpmem\f[] to never use the \f[I]non\-temporal\f[] move instructions on Intel hardware. Without this environment variable, \f[B]libpmem\f[] will use the non\-temporal instructions for copying larger ranges to persistent memory on platforms that support the instructions. This variable is intended for use during library testing. .IP \[bu] 2 \f[B]PMEM_MOVNT_THRESHOLD\f[]=\f[I]val\f[] .PP This environment variable allows overriding the minimum length of the \f[B]pmem_memmove_persist\f[](3) operations, for which \f[B]libpmem\f[] uses \f[I]non\-temporal\f[] move instructions. Setting this environment variable to 0 forces \f[B]libpmem\f[] to always use the \f[I]non\-temporal\f[] move instructions if available. It has no effect if \f[B]PMEM_NO_MOVNT\f[] is set to 1. This variable is intended for use during library testing. .IP \[bu] 2 \f[B]PMEM_MMAP_HINT\f[]=\f[I]val\f[] .PP This environment variable allows overriding the hint address used by \f[B]pmem_map_file\f[](). If set, it also disables mapping address randomization. This variable is intended for use during library testing and debugging. Setting it to some fairly large value (i.e.\ 0x10000000000) will very likely result in mapping the file at the specified address (if not used) or at the first unused region above given address, without adding any random offset. When debugging, this makes it easier to calculate the actual address of the persistent memory block, based on its offset in the file. In case of \f[B]libpmemobj\f[] it simplifies conversion of a persistent object identifier (OID) into a direct pointer to the object. .RS .PP NOTE: \f[B]Setting this environment variable affects all the PMDK libraries,\f[] disabling mapping address randomization and causing the specified address to be used as a hint about where to place the mapping. .RE .SH DEBUGGING AND ERROR HANDLING .PP If an error is detected during the call to a \f[B]libpmem\f[] function, the application may retrieve an error message describing the reason of the failure from \f[B]pmem_errormsg\f[](). This function returns a pointer to a static buffer containing the last error message logged for the current thread. If \f[I]errno\f[] was set, the error message may include a description of the corresponding error code as returned by \f[B]strerror\f[](3). The error message buffer is thread\-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a \f[B]libpmem\f[] function indicated an error. The application must not modify or free the error message string. Subsequent calls to other library functions may modify the previous message. .PP Two versions of \f[B]libpmem\f[] are typically available on a development system. The normal version, accessed when a program is linked using the \f[B]\-lpmem\f[] option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run\-time assertions. .PP A second version of \f[B]libpmem\f[], accessed when a program uses the libraries under \f[B]/usr/lib/pmdk_debug\f[], contains run\-time assertions and trace points. The typical way to access the debug version is to set the environment variable \f[B]LD_LIBRARY_PATH\f[] to \f[B]/usr/lib/pmdk_debug\f[] or \f[B]/usr/lib64/pmdk_debug\f[], as appropriate. Debugging output is controlled using the following environment variables. These variables have no effect on the non\-debug version of the library. .IP \[bu] 2 \f[B]PMEM_LOG_LEVEL\f[] .PP The value of \f[B]PMEM_LOG_LEVEL\f[] enables trace points in the debug version of the library, as follows: .IP \[bu] 2 \f[B]0\f[] \- This is the default level when \f[B]PMEM_LOG_LEVEL\f[] is not set. No log messages are emitted at this level. .IP \[bu] 2 \f[B]1\f[] \- Additional details on any errors detected are logged, in addition to returning the \f[I]errno\f[]\-based errors as usual. The same information may be retrieved using \f[B]pmem_errormsg\f[](). .IP \[bu] 2 \f[B]2\f[] \- A trace of basic operations is logged. .IP \[bu] 2 \f[B]3\f[] \- Enables a very verbose amount of function call tracing in the library. .IP \[bu] 2 \f[B]4\f[] \- Enables voluminous and fairly obscure tracing information that is likely only useful to the \f[B]libpmem\f[] developers. .PP Unless \f[B]PMEM_LOG_FILE\f[] is set, debugging output is written to \f[I]stderr\f[]. .IP \[bu] 2 \f[B]PMEM_LOG_FILE\f[] .PP Specifies the name of a file where all logging information should be written. If the last character in the name is \[lq]\-\[rq], the \f[I]PID\f[] of the current process will be appended to the file name when the log file is created. If \f[B]PMEM_LOG_FILE\f[] is not set, output is written to \f[I]stderr\f[]. .SH EXAMPLE .PP The following example uses \f[B]libpmem\f[] to flush changes made to raw, memory\-mapped persistent memory. .RS .PP WARNING: There is nothing transactional about the \f[B]pmem_persist\f[](3) or \f[B]pmem_msync\f[](3) calls in this example. Interrupting the program may result in a partial write to pmem. Use a transactional library such as \f[B]libpmemobj\f[](7) to avoid torn updates. .RE .IP .nf \f[C] #include\ #include\ #include\ #include\ #include\ #include\ #include\ #include\ #include\ /*\ using\ 4k\ of\ pmem\ for\ this\ example\ */ #define\ PMEM_LEN\ 4096 #define\ PATH\ "/pmem\-fs/myfile" int main(int\ argc,\ char\ *argv[]) { \ \ \ \ char\ *pmemaddr; \ \ \ \ size_t\ mapped_len; \ \ \ \ int\ is_pmem; \ \ \ \ /*\ create\ a\ pmem\ file\ and\ memory\ map\ it\ */ \ \ \ \ if\ ((pmemaddr\ =\ pmem_map_file(PATH,\ PMEM_LEN,\ PMEM_FILE_CREATE, \ \ \ \ \ \ \ \ \ \ \ \ 0666,\ &mapped_len,\ &is_pmem))\ ==\ NULL)\ { \ \ \ \ \ \ \ \ perror("pmem_map_file"); \ \ \ \ \ \ \ \ exit(1); \ \ \ \ } \ \ \ \ /*\ store\ a\ string\ to\ the\ persistent\ memory\ */ \ \ \ \ strcpy(pmemaddr,\ "hello,\ persistent\ memory"); \ \ \ \ /*\ flush\ above\ strcpy\ to\ persistence\ */ \ \ \ \ if\ (is_pmem) \ \ \ \ \ \ \ \ pmem_persist(pmemaddr,\ mapped_len); \ \ \ \ else \ \ \ \ \ \ \ \ pmem_msync(pmemaddr,\ mapped_len); \ \ \ \ /* \ \ \ \ \ *\ Delete\ the\ mappings.\ The\ region\ is\ also \ \ \ \ \ *\ automatically\ unmapped\ when\ the\ process\ is \ \ \ \ \ *\ terminated. \ \ \ \ \ */ \ \ \ \ pmem_unmap(pmemaddr,\ mapped_len); } \f[] .fi .PP See for more examples using the \f[B]libpmem\f[] API. .SH ACKNOWLEDGEMENTS .PP \f[B]libpmem\f[] builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: .SH SEE ALSO .PP \f[B]dlclose\f[](3), \f[B]pmem_flush\f[](3), \f[B]pmem_is_pmem\f[](3), \f[B]pmem_memmove_persist\f[](3), \f[B]pmem_msync\f[](3), \f[B]pmem_persist\f[](3), \f[B]strerror\f[](3), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/macros.man0000664000000000000000000001430113615011243013536 0ustar rootroot# # Copyright 2017, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # These macros are defined for the m4 preprocessor and are controlled by # the FREEBSD, WIN32 and WEB variables. These MUST be explicitly defined or # undefined on the m4 command line. # # This solution allows the maintenance of Windows, Linux and FreeBSD # documentation in the same file. # # The macros are: # # _BSDWX(FreeBSD,WinLinux): # Choose text based on FREEBSD. Both arguments are optional # (although the comma must be present if FreeBSD is omitted). # Bracket string with (=q=, =e=) if it contains commas. # _DEBUGLIBPATH() # Inserts pathnames for debug libraries depending on WIN32 and # FREEBSD. # _LDLIBPATH() # Inserts suggested pathnames for LD_LIBRARY_PATH depending on # WIN32 and FREEBSD. # _MP(man_page_name, section): # Include the man page section number if not building for WEB. # _UNICODE(): # Inserts a standard note regarding UNICODE support if WIN32. # _U(func_name): # Append "U" to func_name if WIN32. # _UW(func_name): # Emit **func_nameU**()/**func_nameW**() if WIN32. # _UWFUNC(func_name, args): # Define U and W prototypes of char/wchar_t *func_name if WIN32. # Bracket args string with (=q=, =e=) if it is a comma-separated # list. # _UWFUNCR(ret_type, func_name, char_arg): # Define U and W prototypes of ret_type func_name if WIN32. # Single char/wchar_t argument is char_arg. # _UWFUNCRUW(ret_type, func_name, args): # Define U and W prototypes of ret_type[U/W] func_name if WIN32. # Bracket args string with (=q=, =e=) if it is a comma-separated # list. # _UWFUNCR1(ret_type, func_name, char_arg, rest_of_args, comment): # Define U and W prototypes of ret_type func_name if WIN32. # First char/wchar_t argument is char_arg. Bracket rest_of_args # string with (=q=, =e=) if it is a comma-separated list. # Comment is added after prototype definition if present. # _UWFUNCR12(ret_type, func_name, char_arg1, char_arg2, rest_of_args, # comment): # Define U and W prototypes of ret_type func_name if WIN32. # Two char/wchar_t arguments are char_arg1-2. Bracket # rest_of_args string with (=q=, =e=) if it is a comma-separated # list. Comment is added after prototype definition if present. # _UWFUNCR1UW(ret_type, func_name, arg1_type, arg1, rest_of_args): # Define U and W prototypes of ret_type func_name, append [U/W] # to arg1_type arg1. Bracket rest_of_args string with (=q=, =e=) # if it is a comma-separated list. # _UWFUNCR2(ret_type, func_name, arg1, char_arg, rest_of_args, comment): # Define U and W prototypes of ret_type func_name if WIN32. # Second char/wchar_t argument is char_arg. Bracket rest_of_args # string with (=q=, =e=) if it is a comma-separated list. # Comment is added after prototype definition if present. # _UWS(struct_name): # Emit *struct struct_nameU*/*struct struct_nameW* if WIN32. # _WINUX(Windows,UX): # Choose text based on WIN32. Both arguments are optional # (although the comma must be present if Windows is omitted). # Bracket string with (=q=, =e=) if it contains commas. changequote(=q=,=e=) changecom() define(_BSDWX, ifdef(=q=FREEBSD=e=,$1,$2)) define(_DEBUGLIBPATH, ifdef(=q=WIN32=e=,**/pmdk/src/x64/Debug**, ifdef(=q=FREEBSD=e=,**/usr/local/lib/pmdk_debug**, **/usr/lib/pmdk_debug**))) define(_LDLIBPATH, ifdef(=q=WIN32=e=,**/pmdk/src/x64/Debug**, ifdef(=q=FREEBSD=e=,**/usr/local/lib/pmdk_debug**, =q==q==q=**/usr/lib/pmdk_debug** or **/usr/lib64/pmdk_debug**, as appropriate=e==e==e=))) define(_MP, ifdef(=q=WEB=e=,$1,$1($2))) define(_UNICODE, ifdef(=q=WIN32=e=,=q==q= >NOTE: The PMDK API supports UNICODE. If the **PMDK_UTF8_API** macro is defined, basic API functions are expanded to the UTF-8 API with postfix *U*. Otherwise they are expanded to the UNICODE API with postfix *W*.=e==e=)) define(_U, ifdef(=q=WIN32=e=,$1U,$1)) define(_UW, ifdef(=q=WIN32=e=,**$1U**()/**$1W**(),**$1**())) define(_UWFUNC, ifdef(=q=WIN32=e=, const char *$1U($2); const wchar_t *$1W($2);, const char *$1($2);)) define(_UWFUNCR, ifdef(=q=WIN32=e=, $1 $2U(const char $3); $1 $2W(const wchar_t $3);, $1 $2(const char $3);)) define(_UWFUNCRUW, ifdef(=q=WIN32=e=, $1U $2U($3); $1W $2W($3);, $1 $2($3);)) define(_UWFUNCR1, ifdef(=q=WIN32=e=, $1 $2U(const char $3, $4);$5 $1 $2W(const wchar_t $3, $4);$5, $1 $2(const char $3, $4);$5)) define(_UWFUNCR12, ifdef(=q=WIN32=e=, $1 $2U(const char $3, const char $4, $5);$6 $1 $2W(const wchar_t $3, const wchar_t $4, $5);$6, $1 $2(const char $3, const char $4, $5);$6)) define(_UWFUNCR1UW, ifdef(=q=WIN32=e=, $1 $2U($3U $4, $5); $1 $2W($3W $4, $5);, $1 $2($3 $4, $5);)) define(_UWFUNCR2, ifdef(=q=WIN32=e=, $1 $2U($3, const char $4, $5);$6 $1 $2W($3, const wchar_t $4, $5);$6, $1 $2($3, const char $4, $5);$6)) define(_UWS, ifdef(=q=WIN32=e=,*struct $1U*/*struct $1W*,*struct $1*)) define(_WINUX, ifdef(=q=WIN32=e=,$1,$2)) pmdk-1.8/doc/libpmemobj/0000775000000000000000000000000013615011420013673 5ustar rootrootpmdk-1.8/doc/libpmemobj/pobj_list_head.3.md0000664000000000000000000002764013615011243017340 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(POBJ_LIST_HEAD, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pobj_list_head.3 -- man page for type-safe non-transactional persistent atomic lists) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[SEE ALSO](#see-also)
# NAME # **POBJ_LIST_HEAD**(), **POBJ_LIST_ENTRY**(), **POBJ_LIST_FIRST**(), **POBJ_LIST_LAST**(), **POBJ_LIST_EMPTY**(), **POBJ_LIST_NEXT**(), **POBJ_LIST_PREV**(), **POBJ_LIST_FOREACH**(), **POBJ_LIST_FOREACH_REVERSE**(), **POBJ_LIST_INSERT_HEAD**(), **POBJ_LIST_INSERT_TAIL**(), **POBJ_LIST_INSERT_AFTER**(), **POBJ_LIST_INSERT_BEFORE**(), **POBJ_LIST_INSERT_NEW_HEAD**(), **POBJ_LIST_INSERT_NEW_TAIL**(), **POBJ_LIST_INSERT_NEW_AFTER**(), **POBJ_LIST_INSERT_NEW_BEFORE**(), **POBJ_LIST_REMOVE**(), **POBJ_LIST_REMOVE_FREE**(), **POBJ_LIST_MOVE_ELEMENT_HEAD**(), **POBJ_LIST_MOVE_ELEMENT_TAIL**(), **POBJ_LIST_MOVE_ELEMENT_AFTER**(), **POBJ_LIST_MOVE_ELEMENT_BEFORE**() - type-safe non-transactional persistent atomic lists # SYNOPSIS # ```c #include POBJ_LIST_HEAD(HEADNAME, TYPE) POBJ_LIST_ENTRY(TYPE) POBJ_LIST_FIRST(POBJ_LIST_HEAD *head) POBJ_LIST_LAST(POBJ_LIST_HEAD *head, POBJ_LIST_ENTRY FIELD) POBJ_LIST_EMPTY(POBJ_LIST_HEAD *head) POBJ_LIST_NEXT(TOID elm, POBJ_LIST_ENTRY FIELD) POBJ_LIST_PREV(TOID elm, POBJ_LIST_ENTRY FIELD) POBJ_LIST_FOREACH(TOID var, POBJ_LIST_HEAD *head, POBJ_LIST_ENTRY FIELD) POBJ_LIST_FOREACH_REVERSE(TOID var, POBJ_LIST_HEAD *head, POBJ_LIST_ENTRY FIELD) POBJ_LIST_INSERT_HEAD(PMEMobjpool *pop, POBJ_LIST_HEAD *head, TOID elm, POBJ_LIST_ENTRY FIELD) POBJ_LIST_INSERT_TAIL(PMEMobjpool *pop, POBJ_LIST_HEAD *head, TOID elm, POBJ_LIST_ENTRY FIELD) POBJ_LIST_INSERT_AFTER(PMEMobjpool *pop, POBJ_LIST_HEAD *head, TOID listelm, TOID elm, POBJ_LIST_ENTRY FIELD) POBJ_LIST_INSERT_BEFORE(PMEMobjpool *pop, POBJ_LIST_HEAD *head, TOID listelm, TOID elm, POBJ_LIST_ENTRY FIELD) POBJ_LIST_INSERT_NEW_HEAD(PMEMobjpool *pop, POBJ_LIST_HEAD *head, POBJ_LIST_ENTRY FIELD, size_t size, pmemobj_constr constructor, void *arg) POBJ_LIST_INSERT_NEW_TAIL(PMEMobjpool *pop, POBJ_LIST_HEAD *head, POBJ_LIST_ENTRY FIELD, size_t size, pmemobj_constr constructor, void *arg) POBJ_LIST_INSERT_NEW_AFTER(PMEMobjpool *pop, POBJ_LIST_HEAD *head, TOID listelm, POBJ_LIST_ENTRY FIELD, size_t size, pmemobj_constr constructor, void *arg) POBJ_LIST_INSERT_NEW_BEFORE(PMEMobjpool *pop, POBJ_LIST_HEAD *head, TOID listelm, POBJ_LIST_ENTRY FIELD, size_t size, pmemobj_constr constructor, void *arg) POBJ_LIST_REMOVE(PMEMobjpool *pop, POBJ_LIST_HEAD *head, TOID elm, POBJ_LIST_ENTRY FIELD) POBJ_LIST_REMOVE_FREE(PMEMobjpool *pop, POBJ_LIST_HEAD *head, TOID elm, POBJ_LIST_ENTRY FIELD) POBJ_LIST_MOVE_ELEMENT_HEAD(PMEMobjpool *pop, POBJ_LIST_HEAD *head, POBJ_LIST_HEAD *head_new, TOID elm, POBJ_LIST_ENTRY FIELD, POBJ_LIST_ENTRY field_new) POBJ_LIST_MOVE_ELEMENT_TAIL(PMEMobjpool *pop, POBJ_LIST_HEAD *head, POBJ_LIST_HEAD *head_new, TOID elm, POBJ_LIST_ENTRY FIELD, POBJ_LIST_ENTRY field_new) POBJ_LIST_MOVE_ELEMENT_AFTER(PMEMobjpool *pop, POBJ_LIST_HEAD *head, POBJ_LIST_HEAD *head_new, TOID listelm, TOID elm, POBJ_LIST_ENTRY FIELD, POBJ_LIST_ENTRY field_new) POBJ_LIST_MOVE_ELEMENT_BEFORE(PMEMobjpool *pop, POBJ_LIST_HEAD *head, POBJ_LIST_HEAD *head_new, TOID listelm, TOID elm, POBJ_LIST_ENTRY FIELD, POBJ_LIST_ENTRY field_new) ``` # DESCRIPTION # The following macros define and operate on a type-safe persistent atomic circular doubly linked list data structure that consist of a set of persistent objects of a well-known type. Unlike the functions described in the previous section, these macros provide type enforcement by requiring declaration of type of the objects stored in given list, and not allowing mixing objects of different types in a single list. The functionality and semantics of those macros is similar to circular queues defined in **queue**(3). The majority of the macros must specify the handle to the memory pool *pop* and the name of the *field* in the user-defined structure, which must be of type *POBJ_LIST_ENTRY* and is used to connect the elements in the list. A list is headed by a structure defined by the **POBJ_LIST_HEAD**() macro. This structure contains an object handle of the first element on the list. The elements are doubly linked so that an arbitrary element can be removed without a need to traverse the list. New elements can be added to the list before or after an existing element, at the head of the list, or at the end of the list. A list may be traversed in either direction. A *POBJ_LIST_HEAD* structure is declared as follows: ```c #define POBJ_LIST_HEAD(HEADNAME, TYPE) struct HEADNAME { TOID(TYPE) pe_first; PMEMmutex lock; }; ``` In the macro definitions, *TYPE* is the name of a user-defined structure, that must contain a field of type *POBJ_LIST_ENTRY*. The argument *HEADNAME* is the name of a user-defined structure that must be declared using the macro *POBJ_LIST_HEAD*. See the examples below for further explanation of how these macros are used. The macro *POBJ_LIST_ENTRY* declares a structure that connects the elements in the list. ```c #define POBJ_LIST_ENTRY(TYPE) struct { TOID(TYPE) pe_next; TOID(TYPE) pe_prev; }; ``` The macro **POBJ_LIST_FIRST**() returns the first element on the list referenced by *head*. If the list is empty **OID_NULL** is returned. The macro **POBJ_LIST_LAST**() returns the last element on the list referenced by *head*. If the list is empty **OID_NULL** is returned. The macro **POBJ_LIST_EMPTY**() evaluates to 1 if the list referenced by *head* is empty. Otherwise, 0 is returned. The macro **POBJ_LIST_NEXT**() returns the element next to the element *elm*. The macro **POBJ_LIST_PREV**() returns the element preceding the element *elm*. The macro **POBJ_LIST_FOREACH**() traverses the list referenced by *head* assigning a handle to each element in turn to *var* variable. The macro **POBJ_LIST_FOREACH_REVERSE**() traverses the list referenced by *head* in reverse order, assigning a handle to each element in turn to *var* variable. The *field* argument is the name of the field of type *POBJ_LIST_ENTRY* in the element structure. The macro **POBJ_LIST_INSERT_HEAD**() inserts the element *elm* at the head of the list referenced by *head*. The macro **POBJ_LIST_INSERT_TAIL**() inserts the element *elm* at the end of the list referenced by *head*. The macro **POBJ_LIST_INSERT_AFTER**() inserts the element *elm* into the list referenced by *head* after the element *listelm*. If *listelm* value is **TOID_NULL**, the object is inserted at the end of the list. The macro **POBJ_LIST_INSERT_BEFORE**() inserts the element *elm* into the list referenced by *head* before the element *listelm*. If *listelm* value is **TOID_NULL**, the object is inserted at the head of the list. The macro **POBJ_LIST_INSERT_NEW_HEAD**() atomically allocates a new object of size *size* and inserts it at the head of the list referenced by *head*. The newly allocated object is also added to the internal object container associated with a type number which is retrieved from the typed *OID* of the first element on list. The macro **POBJ_LIST_INSERT_NEW_TAIL**() atomically allocates a new object of size *size* and inserts it at the tail of the list referenced by *head*. The newly allocated object is also added to the internal object container associated with a type number which is retrieved from the typed *OID* of the first element on list. The macro **POBJ_LIST_INSERT_NEW_AFTER**() atomically allocates a new object of size *size* and inserts it into the list referenced by *head* after the element *listelm*. If *listelm* value is **TOID_NULL**, the object is inserted at the end of the list. The newly allocated object is also added to the internal object container associated with with a type number which is retrieved from the typed *OID* of the first element on list. The macro **POBJ_LIST_INSERT_NEW_BEFORE**() atomically allocates a new object of size *size* and inserts it into the list referenced by *head* before the element *listelm*. If *listelm* value is **TOID_NULL**, the object is inserted at the head of the list. The newly allocated object is also added to the internal object container associated with with a type number which is retrieved from the typed *OID* of the first element on list. The macro **POBJ_LIST_REMOVE**() removes the element *elm* from the list referenced by *head*. The macro **POBJ_LIST_REMOVE_FREE**() removes the element *elm* from the list referenced by *head* and frees the memory space represented by this element. The macro **POBJ_LIST_MOVE_ELEMENT_HEAD**() moves the element *elm* from the list referenced by *head* to the head of the list *head_new*. The *field* and *field_new* arguments are the names of the fields of type *POBJ_LIST_ENTRY* in the element structure that are used to connect the elements in both lists. The macro **POBJ_LIST_MOVE_ELEMENT_TAIL**() moves the element *elm* from the list referenced by *head* to the end of the list *head_new*. The *field* and *field_new* arguments are the names of the fields of type *POBJ_LIST_ENTRY* in the element structure that are used to connect the elements in both lists. The macro **POBJ_LIST_MOVE_ELEMENT_AFTER**() atomically removes the element *elm* from the list referenced by *head* and inserts it into the list referenced by *head_new* after the element *listelm*. If *listelm* value is *TOID_NULL*, the object is inserted at the end of the list. The *field* and *field_new* arguments are the names of the fields of type *POBJ_LIST_ENTRY* in the element structure that are used to connect the elements in both lists. The macro **POBJ_LIST_MOVE_ELEMENT_BEFORE**() atomically removes the element *elm* from the list referenced by *head* and inserts it into the list referenced by *head_new* before the element *listelm*. If *listelm* value is **TOID_NULL**, the object is inserted at the head of the list. The *field* and *field_new* arguments are the names of the fields of type *POBJ_LIST_ENTRY* in the element structure that are used to connect the elements in both lists. # SEE ALSO # **queue**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_tx_log_snapshots_max_size.30000664000000000000000000000002713615011243022767 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/tx_add_field.30000664000000000000000000000003313615011243016364 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_set_funcs.30000664000000000000000000000002613615011243017462 0ustar rootroot.so man7/libpmemobj.7 pmdk-1.8/doc/libpmemobj/pmemobj_cond_signal.30000664000000000000000000000003113615011243017745 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/tx_zalloc.30000664000000000000000000000002713615011243015760 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_memcpy_persist.3.md0000664000000000000000000002041313615011243021135 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_MEMCPY_PERSIST, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_memcpy_persist.3 -- man page for Low-level memory manipulation) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[EXAMPLES](#examples)
[SEE ALSO](#see-also)
# NAME # **pmemobj_persist**(), **pmemobj_xpersist**(), **pmemobj_flush**(), **pmemobj_xflush**(), **pmemobj_drain**(), **pmemobj_memcpy**(), **pmemobj_memmove**(), **pmemobj_memset**(), **pmemobj_memcpy_persist**(), **pmemobj_memset_persist**() - low-level memory manipulation functions # SYNOPSIS # ```c #include void pmemobj_persist(PMEMobjpool *pop, const void *addr, size_t len); void pmemobj_flush(PMEMobjpool *pop, const void *addr, size_t len); void pmemobj_drain(PMEMobjpool *pop); int pmemobj_xpersist(PMEMobjpool *pop, const void *addr, size_t len, unsigned flags); int pmemobj_xflush(PMEMobjpool *pop, const void *addr, size_t len, unsigned flags); void *pmemobj_memcpy(PMEMobjpool *pop, void *dest, const void *src, size_t len, unsigned flags); void *pmemobj_memmove(PMEMobjpool *pop, void *dest, const void *src, size_t len, unsigned flags); void *pmemobj_memset(PMEMobjpool *pop, void *dest, int c, size_t len, unsigned flags); void *pmemobj_memcpy_persist(PMEMobjpool *pop, void *dest, const void *src, size_t len); void *pmemobj_memset_persist(PMEMobjpool *pop, void *dest, int c, size_t len); ``` # DESCRIPTION # The **libpmemobj**-specific low-level memory manipulation functions described here leverage the knowledge of the additional configuration options available for **libpmemobj**(7) pools, such as replication. They also take advantage of the type of storage behind the pool and use appropriate flush/drain functions. It is advised to use these functions in conjunction with **libpmemobj**(7) objects rather than using low-level memory manipulation functions from **libpmem**. **pmemobj_persist**() forces any changes in the range \[*addr*, *addr*+*len*) to be stored durably in persistent memory. Internally this may call either **pmem_msync**(3) or **pmem_persist**(3). There are no alignment restrictions on the range described by *addr* and *len*, but **pmemobj_persist**() may expand the range as necessary to meet platform alignment requirements. >WARNING: Like **msync**(2), there is nothing atomic or transactional about this call. Any unwritten stores in the given range will be written, but some stores may have already been written by virtue of normal cache eviction/replacement policies. Correctly written code must not depend on stores waiting until **pmemobj_persist**() is called to become persistent - they can become persistent at any time before **pmemobj_persist**() is called. The **pmemobj_flush**() and **pmemobj_drain**() functions provide partial versions of the **pmemobj_persist**() function described above. These functions allow advanced programs to create their own variations of **pmemobj_persist**(). For example, a program that needs to flush several discontiguous ranges can call **pmemobj_flush**() for each range and then follow up by calling **pmemobj_drain**() once. For more information on partial flushing operations, see **pmem_flush**(3). **pmemobj_xpersist**() is a version of **pmemobj_persist**() function with additional *flags* argument. It supports only the **PMEMOBJ_F_RELAXED** flag. This flag indicates that memory transfer operation does not require 8-byte atomicity guarantees. **pmemobj_xflush**() is a version of **pmemobj_flush**() function with additional *flags* argument. It supports only the **PMEMOBJ_F_RELAXED** flag. The **pmemobj_memmove**(), **pmemobj_memcpy**() and **pmemobj_memset**() functions provide the same memory copying as their namesakes **memmove**(3), **memcpy**(3), and **memset**(3), and ensure that the result has been flushed to persistence before returning (unless **PMEMOBJ_MEM_NOFLUSH** flag was used). Valid flags for those functions: + **PMEMOBJ_F_RELAXED** - This flag indicates that memory transfer operation does not require 8-byte atomicity guarantees. + **PMEMOBJ_F_MEM_NOFLUSH** - Don't flush anything. This implies **PMEMOBJ_F_MEM_NODRAIN**. Using this flag only makes sense when it's followed by any function that flushes data. The remaining flags say *how* the operation should be done, and are merely hints. + **PMEMOBJ_F_MEM_NONTEMPORAL** - Use non-temporal instructions. This flag is mutually exclusive with **PMEMOBJ_F_MEM_TEMPORAL**. On x86\_64 this flag is mutually exclusive with **PMEMOBJ_F_MEM_NOFLUSH**. + **PMEMOBJ_F_MEM_TEMPORAL** - Use temporal instructions. This flag is mutually exclusive with **PMEMOBJ_F_MEM_NONTEMPORAL**. + **PMEMOBJ_F_MEM_WC** - Use write combining mode. This flag is mutually exclusive with **PMEMOBJ_F_MEM_WB**. On x86\_64 this is an alias for **PMEMOBJ_F_MEM_NONTEMPORAL**. On x86\_64 this flag is mutually exclusive with **PMEMOBJ_F_MEM_NOFLUSH**. + **PMEMOBJ_F_MEM_WB** - Use write back mode. This flag is mutually exclusive with **PMEMOBJ_F_MEM_WC**. On x86\_64 this is an alias for **PMEMOBJ_F_MEM_TEMPORAL**. **pmemobj_memcpy_persist**() is an alias for **pmemobj_memcpy**() with flags equal to 0. **pmemobj_memset_persist**() is an alias for **pmemobj_memset**() with flags equal to 0. # RETURN VALUE # **pmemobj_memmove**(), **pmemobj_memcpy**(), **pmemobj_memset**(), **pmemobj_memcpy_persist**() and **pmemobj_memset_persist**() return destination buffer. **pmemobj_persist**(), **pmemobj_flush**() and **pmemobj_drain**() do not return any value. **pmemobj_xpersist**() and **pmemobj_xflush**() returns non-zero value and sets errno to EINVAL only if not supported flags has been provided. # EXAMPLES # The following code is functionally equivalent to **pmemobj_memcpy_persist**(): ```c void * pmemobj_memcpy_persist(PMEMobjpool *pop, void *dest, const void *src, size_t len) { void *retval = memcpy(dest, src, len); pmemobj_persist(pop, dest, len); return retval; } ``` **pmemobj_persist**() can be thought of as this: ```c void pmemobj_persist(PMEMobjpool *pop, const void *addr, size_t len) { /* flush the processor caches */ pmemobj_flush(pop, addr, len); /* wait for any pmem stores to drain from HW buffers */ pmemobj_drain(pop); } ``` # SEE ALSO # **memcpy**(3), **memset**(3), **pmem_msync**(3), **pmem_persist**(3), **libpmem**(7) **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_alloc_usable_size.30000664000000000000000000000002413615011243021146 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_root_construct.30000664000000000000000000000002313615011243020555 0ustar rootroot.so pmemobj_root.3 pmdk-1.8/doc/libpmemobj/tx_xadd_direct.30000664000000000000000000000003313615011243016743 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/tx_xadd.30000664000000000000000000000003313615011243015411 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_xalloc.30000664000000000000000000000002713615011243017467 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_realloc.30000664000000000000000000000002713615011243017626 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_foreach_type.30000664000000000000000000000002413615011243017440 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_wcsdup.30000664000000000000000000000002713615011243017512 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_cancel.30000664000000000000000000000002513615011243016715 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/pmemobj_list_insert.30000644000000000000000000002303613615011417020037 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_LIST_INSERT" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_list_insert\f[](), \f[B]pmemobj_list_insert_new\f[](), \f[B]pmemobj_list_move\f[](), \f[B]pmemobj_list_remove\f[]() \- non\-transactional persistent atomic lists functions .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmemobj_list_insert(PMEMobjpool\ *pop,\ size_t\ pe_offset,\ void\ *head, \ \ \ \ PMEMoid\ dest,\ int\ before,\ PMEMoid\ oid); PMEMoid\ pmemobj_list_insert_new(PMEMobjpool\ *pop,\ size_t\ pe_offset, \ \ \ \ void\ *head,\ PMEMoid\ dest,\ int\ before,\ size_t\ size, \ \ \ \ uint64_t\ type_num,\ pmemobj_constr\ constructor,\ void\ arg); int\ pmemobj_list_move(PMEMobjpool\ *pop, \ \ \ \ size_t\ pe_old_offset,\ void\ *head_old, \ \ \ \ size_t\ pe_new_offset,\ void\ *head_new, \ \ \ \ PMEMoid\ dest,\ int\ before,\ PMEMoid\ oid); int\ pmemobj_list_remove(PMEMobjpool\ *pop,\ size_t\ pe_offset, \ \ \ \ void\ *head,\ PMEMoid\ oid,\ int\ free); \f[] .fi .SH DESCRIPTION .PP In addition to the container operations on internal object collections described in \f[B]pmemobj_first\f[](3), \f[B]libpmemobj\f[](7) provides a mechanism for organizing persistent objects in user\-defined, persistent, atomic, circular, doubly\-linked lists. All the routines and macros operating on the persistent lists provide atomicity with respect to any power\-fail interruptions. If any of those operations is torn by program failure or system crash, on recovery they are guaranteed to be entirely completed or discarded, leaving the lists, persistent memory heap and internal object containers in a consistent state. .PP The persistent atomic circular doubly linked lists support the following functionality: .IP \[bu] 2 Insertion of an object at the head of the list, or at the end of the list. .IP \[bu] 2 Insertion of an object before or after any element in the list. .IP \[bu] 2 Atomic allocation and insertion of a new object at the head of the list, or at the end of the list. .IP \[bu] 2 Atomic allocation and insertion of a new object before or after any element in the list. .IP \[bu] 2 Atomic moving of an element from one list to the specific location on another list. .IP \[bu] 2 Removal of any object in the list. .IP \[bu] 2 Atomic removal and freeing of any object in the list. .IP \[bu] 2 Forward or backward traversal through the list. .PP A list is headed by a \f[I]list_head\f[] structure containing the object handle of the first element on the list. The elements are doubly linked so that an arbitrary element can be removed without the need to traverse the list. New elements can be added to the list before or after an existing element, at the head of the list, or at the tail of the list. A list may be traversed in either direction. .PP The user\-defined structure of each element must contain a field of type \f[I]list_entry\f[] that holds the object handles to the previous and next element on the list. Both the \f[I]list_head\f[] and the \f[I]list_entry\f[] structures are declared in \f[B]\f[]. .PP The functions below are intended to be used outside transactions \- transactional variants are described in manpages to functions mentioned at \f[B]TRANSACTIONAL OBJECT MANIPULATION\f[] in \f[B]libpmemobj\f[](7). Note that operations performed using this non\-transactional API are independent from their transactional counterparts. If any non\-transactional allocations or list manipulations are performed within an open transaction, the changes will not be rolled back if such a transaction is aborted or interrupted. .PP The list insertion and move functions use a common set of arguments to define where an object will be inserted into the list. \f[I]dest\f[] identifies the element before or after which the object will be inserted, or, if \f[I]dest\f[] is \f[B]OID_NULL\f[], indicates that the object should be inserted at the head or tail of the list. \f[I]before\f[] determines where the object will be inserted: .IP \[bu] 2 \f[B]POBJ_LIST_DEST_BEFORE\f[] \- insert the element before the existing element \f[I]dest\f[] .IP \[bu] 2 \f[B]POBJ_LIST_DEST_AFTER\f[] \- insert the element after the existing element \f[I]dest\f[] .IP \[bu] 2 \f[B]POBJ_LIST_DEST_HEAD\f[] \- when \f[I]dest\f[] is \f[B]OID_NULL\f[], insert the element at the head of the list .IP \[bu] 2 \f[B]POBJ_LIST_DEST_TAIL\f[] \- when \f[I]dest\f[] is \f[B]OID_NULL\f[], insert the element at the tail of the list .RS .PP NOTE: Earlier versions of \f[B]libpmemobj\f[](7) do not define \f[B]POBJ_LIST_DEST_BEFORE\f[] and \f[B]POBJ_LIST_DEST_AFTER\f[]. Use 1 for before, and 0 for after. .RE .PP The \f[B]pmemobj_list_insert\f[]() function inserts the element represented by object handle \f[I]oid\f[] into the list referenced by \f[I]head\f[], at the location specified by \f[I]dest\f[] and \f[I]before\f[] as described above. \f[I]pe_offset\f[] specifies the offset of the structure that connects the elements in the list. All the handles \f[I]head\f[], \f[I]dest\f[] and \f[I]oid\f[] must point to objects allocated from memory pool \f[I]pop\f[]. \f[I]head\f[] and \f[I]oid\f[] cannot be \f[B]OID_NULL\f[]. .PP The \f[B]pmemobj_list_insert_new\f[]() function atomically allocates a new object of given \f[I]size\f[] and type \f[I]type_num\f[] and inserts it into the list referenced by \f[I]head\f[] at the location specified by \f[I]dest\f[] and \f[I]before\f[] as described above. \f[I]pe_offset\f[] specifies the offset of the structure that connects the elements in the list. The handles \f[I]head\f[] and \f[I]dest\f[] must point to objects allocated from memory pool \f[I]pop\f[]. Before returning, \f[B]pmemobj_list_insert_new\f[]() calls the \f[I]constructor\f[] function, passing the pool handle \f[I]pop\f[], the pointer to the newly allocated object \f[I]ptr\f[], and the \f[I]arg\f[] argument. It is guaranteed that the allocated object is either properly initialized or, if the allocation is interrupted before the constructor completes, the memory space reserved for the object is reclaimed. \f[I]head\f[] cannot be \f[B]OID_NULL\f[]. The allocated object is also added to the internal container associated with \f[I]type_num\f[], as described in \f[B]POBJ_FOREACH\f[](3). .PP The \f[B]pmemobj_list_move\f[]() function moves the object represented by object handle \f[I]oid\f[] from the list referenced by \f[I]head_old\f[] to the list referenced by \f[I]head_new\f[], inserting it at the location specified by \f[I]dest\f[] and \f[I]before\f[] as described above. \f[I]pe_old_offset\f[] and \f[I]pe_new_offset\f[] specify the offsets of the structures that connect the elements in the old and new lists, respectively. All the handles \f[I]head_old\f[], \f[I]head_new\f[], \f[I]dest\f[] and \f[I]oid\f[] must point to objects allocated from memory pool \f[I]pop\f[]. \f[I]head_old\f[], \f[I]head_new\f[] and \f[I]oid\f[] cannot be \f[B]OID_NULL\f[]. .PP The \f[B]pmemobj_list_remove\f[]() function removes the object represented by object handle \f[I]oid\f[] from the list referenced by \f[I]head\f[]. If \f[I]free\f[] is set, it also removes the object from the internal object container and frees the associated memory space. \f[I]pe_offset\f[] specifies the offset of the structure that connects the elements in the list. Both \f[I]head\f[] and \f[I]oid\f[] must point to objects allocated from memory pool \f[I]pop\f[] and cannot be \f[B]OID_NULL\f[]. .SH RETURN VALUE .PP On success, \f[B]pmemobj_list_insert\f[](), \f[B]pmemobj_list_remove\f[]() and \f[B]pmemobj_list_move\f[]() return 0. On error, they return \-1 and set \f[I]errno\f[] appropriately. .PP On success, \f[B]pmemobj_list_insert_new\f[]() returns a handle to the newly allocated object. If the constructor returns a non\-zero value, the allocation is canceled, \-1 is returned, and \f[I]errno\f[] is set to \f[B]ECANCELED\f[]. On other errors, \f[B]OID_NULL\f[] is returned and \f[I]errno\f[] is set appropriately. .SH SEE ALSO .PP \f[B]pmemobj_first\f[](3), \f[B]POBJ_FOREACH\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_memcpy.30000664000000000000000000000003513615011243016763 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pmemobj_rwlock_timedwrlock.30000664000000000000000000000003113615011243021372 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_xpublish.30000664000000000000000000000002513615011243020041 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/tx_xadd_field_direct.30000664000000000000000000000003313615011243020106 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_add_range_direct.30000664000000000000000000000003313615011243021440 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_cond_timedwait.30000664000000000000000000000003113615011243020457 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/libpmemobj.7.md0000664000000000000000000002606413615011243016515 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(LIBPMEMOBJ, 7) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2016-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (libpmemobj.7 -- man page for libpmemobj) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[LIBRARY API VERSIONING](#library-api-versioning-1)
[MANAGING LIBRARY BEHAVIOR](#managing-library-behavior)
[DEBUGGING AND ERROR HANDLING](#debugging-and-error-handling)
[EXAMPLE](#example)
[ACKNOWLEDGEMENTS](#acknowledgements)
[SEE ALSO](#see-also)
# NAME # **libpmemobj** - persistent memory transactional object store # SYNOPSIS # ```c #include cc _WINUX(,-std=gnu99) ... -lpmemobj -lpmem ``` _UNICODE() ##### Library API versioning: ##### ```c _UWFUNC(pmemobj_check_version, =q= unsigned major_required, unsigned minor_required=e=) ``` ##### Managing library behavior: ##### ```c void pmemobj_set_funcs( void *(*malloc_func)(size_t size), void (*free_func)(void *ptr), void *(*realloc_func)(void *ptr, size_t size), char *(*strdup_func)(const char *s)); ``` ##### Error handling: ##### ```c _UWFUNC(pmemobj_errormsg, void) ``` ##### Other library functions: ##### A description of other **libpmemobj** functions can be found on the following manual pages: + control and statistics: **pmemobj_ctl_get**(3) + create, open, close and validate: **pmemobj_open**(3) + low-level memory manipulation: **pmemobj_memcpy_persist**(3) + locking: **pmemobj_mutex_zero**(3) + persistent object identifier: **OID_IS_NULL**(3) + type-safety: **TOID_DECLARE**(3) + layout declaration: **POBJ_LAYOUT_BEGIN**(3) + non-transactional atomic allocations: **pmemobj_alloc**(3) + root object management: **pmemobj_root**(3) + object containers: **pmemobj_first**(3) + non-transactional persistent atomic circular doubly-linked list: **pmemobj_list_insert**(3), **POBJ_LIST_HEAD**(3) + transactional object manipulation: **pmemobj_tx_begin**(3), **pmemobj_tx_add_range**(3), **pmemobj_tx_alloc**(3) + delayed atomicity actions: **pmemobj_action**(3) (EXPERIMENTAL) # DESCRIPTION # **libpmemobj** provides a transactional object store in *persistent memory* (pmem) for applications that require transactions and persistent memory management using direct access storage (DAX), which is storage that supports load/store access without paging blocks from a block storage device. Some types of *non-volatile memory DIMMs* (NVDIMMs) provide this type of byte addressable access to storage. A *persistent memory aware file system* is typically used to expose the direct access to applications. Memory mapping a file from this type of file system results in load/store, non-paged access to pmem. **libpmemobj** builds on this type of memory mapped file using the low-level pmem support provided by **libpmem**(7), handling the transactional updates, flushing changes to persistence, and managing recovery for the application. _WINUX(,=q=**libpmemobj** requires the **-std=gnu99** compilation flag to build properly.=e=) **libpmemobj** is one of a collection of persistent memory libraries available. The others are: + **libpmemblk**(7), providing pmem-resident arrays of fixed-sized blocks with atomic updates. + **libpmemlog**(7), providing a pmem-resident log file. + **libpmem**(7), low-level persistent memory support. Under normal usage, **libpmemobj** will never print messages or intentionally cause the process to exit. The only exception to this is the debugging information, when enabled, as described under **DEBUGGING AND ERROR HANDLING**, below. # LIBRARY API VERSIONING # This section describes how the library API is versioned, allowing applications to work with an evolving API. The _UW(pmemobj_check_version) function is used to see if the installed **libpmemobj** supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile-time version information, supplied by defines in **\**, like this: ```c reason = _U(pmemobj_check_version)(PMEMOBJ_MAJOR_VERSION, PMEMOBJ_MINOR_VERSION); if (reason != NULL) { /* version check failed, reason string tells you why */ } ``` Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text *introduced in version x.y* in the section of this manual describing the feature. On success, _UW(pmemobj_check_version) returns NULL. Otherwise, the return value is a static string describing the reason the version check failed. The string returned by _UW(pmemobj_check_version) must not be modified or freed. # MANAGING LIBRARY BEHAVIOR # The **pmemobj_set_funcs**() function allows an application to override memory allocation calls used internally by **libpmemobj**. Passing in NULL for any of the handlers will cause the **libpmemobj** default function to be used. The library does not make heavy use of the system malloc functions, but it does allocate approximately 4-8 kilobytes for each memory pool in use. By default, **libpmemobj** supports up to 1024 parallel transactions/allocations. For debugging purposes it is possible to decrease this value by setting the **PMEMOBJ_NLANES** environment variable to the desired limit. # DEBUGGING AND ERROR HANDLING # If an error is detected during the call to a **libpmemobj** function, the application may retrieve an error message describing the reason for the failure from _UW(pmemobj_errormsg). This function returns a pointer to a static buffer containing the last error message logged for the current thread. If *errno* was set, the error message may include a description of the corresponding error code as returned by **strerror**(3). The error message buffer is thread-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a **libpmemobj** function indicated an error, or if *errno* was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. Two versions of **libpmemobj** are typically available on a development system. The normal version, accessed when a program is linked using the **-lpmemobj** option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run-time assertions. A second version of **libpmemobj**, accessed when a program uses the libraries under _DEBUGLIBPATH(), contains run-time assertions and trace points. The typical way to access the debug version is to set the environment variable **LD_LIBRARY_PATH** to _LDLIBPATH(). Debugging output is controlled using the following environment variables. These variables have no effect on the non-debug version of the library. + **PMEMOBJ_LOG_LEVEL** The value of **PMEMOBJ_LOG_LEVEL** enables trace points in the debug version of the library, as follows: + **0** - This is the default level when **PMEMOBJ_LOG_LEVEL** is not set. No log messages are emitted at this level. + **1** - Additional details on any errors detected are logged, in addition to returning the *errno*-based errors as usual. The same information may be retrieved using _UW(pmemobj_errormsg). + **2** - A trace of basic operations is logged. + **3** - Enables a very verbose amount of function call tracing in the library. + **4** - Enables voluminous and fairly obscure tracing information that is likely only useful to the **libpmemobj** developers. Unless **PMEMOBJ_LOG_FILE** is set, debugging output is written to *stderr*. + **PMEMOBJ_LOG_FILE** Specifies the name of a file where all logging information should be written. If the last character in the name is "-", the *PID* of the current process will be appended to the file name when the log file is created. If **PMEMOBJ_LOG_FILE** is not set, logging output is written to *stderr*. See also **libpmem**(7) to get information about other environment variables affecting **libpmemobj** behavior. # EXAMPLE # See for examples using the **libpmemobj** API. # ACKNOWLEDGEMENTS # **libpmemobj** builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: # SEE ALSO # **OID_IS_NULL**(3), **pmemobj_alloc**(3), **pmemobj_ctl_exec**(3), **pmemobj_ctl_get**(3), **pmemobj_ctl_set**(3), **pmemobj_first**(3), **pmemobj_list_insert**(3), **pmemobj_memcpy_persist**(3), **pmemobj_mutex_zero**(3), **pmemobj_open**(3), **pmemobj_root**(3), **pmemobj_tx_add_range**(3), **pmemobj_tx_alloc**(3), **pmemobj_tx_begin**(3), **POBJ_LAYOUT_BEGIN**(3), **POBJ_LIST_HEAD**(3), **strerror**(3), **TOID_DECLARE**(3), **libpmem**(7), **libpmemblk**(7), **libpmemlog**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_defer_free.30000664000000000000000000000002513615011243017556 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/oid_is_null.30000644000000000000000000002166213615011417016272 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "OID_IS_NULL" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]OID_IS_NULL\f[](), \f[B]OID_EQUALS\f[](), \f[B]pmemobj_direct\f[](), \f[B]pmemobj_oid\f[](), \f[B]pmemobj_type_num\f[](), \f[B]pmemobj_pool_by_oid\f[](), \f[B]pmemobj_pool_by_ptr\f[]() \- functions that allow mapping operations between object addresses, object handles, oids or type numbers .SH SYNOPSIS .IP .nf \f[C] #include\ OID_IS_NULL(PMEMoid\ oid) OID_EQUALS(PMEMoid\ lhs,\ PMEMoid\ rhs) void\ *pmemobj_direct(PMEMoid\ oid); PMEMoid\ pmemobj_oid(const\ void\ *addr); uint64_t\ pmemobj_type_num(PMEMoid\ oid); PMEMobjpool\ *pmemobj_pool_by_oid(PMEMoid\ oid); PMEMobjpool\ *pmemobj_pool_by_ptr(const\ void\ *addr); void\ *pmemobj_volatile(PMEMobjpool\ *pop,\ struct\ pmemvlt\ *vlt, \ \ \ \ size_t\ size,\ void\ *ptr, \ \ \ \ int\ (*constr)(void\ *ptr,\ void\ *arg),\ void\ *arg);\ (EXPERIMENTAL) \f[] .fi .SH DESCRIPTION .PP Each object stored in a persistent memory pool is represented by an object handle of type \f[I]PMEMoid\f[]. In practice, such a handle is a unique Object IDentifier (\f[I]OID\f[]) of global scope, which means that two objects from different pools will never have the same \f[I]OID\f[]. The special \f[B]OID_NULL\f[] macro defines a NULL\-like handle that does not represent any object. The size of a single object is limited by \f[B]PMEMOBJ_MAX_ALLOC_SIZE\f[]. Thus an allocation with a requested size greater than this value will fail. .PP An \f[I]OID\f[] cannot be used as a direct pointer to an object. Each time the program attempts to read or write object data, it must obtain the current memory address of the object by converting its \f[I]OID\f[] into a pointer. .PP In contrast to the memory address, the \f[I]OID\f[] value for given object does not change during the life of an object (except for \f[I]realloc\f[]), and remains valid after closing and reopening the pool. For this reason, if an object contains a reference to another persistent object, for example, to build some kind of a linked data structure, the reference must be an \f[I]OID\f[] and not a memory address. .PP \f[B]pmemobj_direct\f[]() returns a pointer to the \f[I]PMEMoid\f[] object with handle \f[I]oid\f[]. .PP \f[B]pmemobj_oid\f[]() returns a \f[I]PMEMoid\f[] handle to the object pointed to by \f[I]addr\f[]. .PP \f[B]pmemobj_type_num\f[]() returns the type number of the \f[I]PMEMoid\f[] object with handle \f[I]oid\f[]. .PP \f[B]pmemobj_pool_by_oid\f[]() returns a \f[I]PMEMobjpool\f[]* handle to the pool containing the \f[I]PMEMoid\f[] object with handle \f[I]oid\f[]. .PP \f[B]pmemobj_pool_by_ptr\f[]() returns a \f[I]PMEMobjpool\f[]* handle to the pool containing the address \f[I]addr\f[]. .PP At the time of allocation (or reallocation), each object may be assigned a number representing its type. Such a \f[I]type number\f[] may be used to arrange the persistent objects based on their actual user\-defined structure type, thus facilitating implementation of a simple run\-time type safety mechanism. This also allows iterating through all the objects of a given type that are stored in the persistent memory pool. See \f[B]pmemobj_first\f[](3) for more information. .PP The \f[B]OID_IS_NULL\f[]() macro checks if \f[I]PMEMoid\f[] represents a NULL object. .PP The \f[B]OID_EQUALS\f[]() macro compares two \f[I]PMEMoid\f[] objects. .PP For special cases where volatile (transient) variables need to be stored on persistent memory, there's a mechanism composed of \f[I]struct pmemvlt\f[] type and \f[B]pmemobj_volatile()\f[] function. To use it, the \f[I]struct pmemvlt\f[] needs to be placed in the neighborhood of transient data region. The \f[I]PMEMvlt\f[] macro can be used to construct such a region. The \f[I]struct pmemvlt\f[] must be zeroed prior to use. This can be easily done in object constructor or in a transaction directly after an allocation. When the \f[B]pmemobj_volatile()\f[] function is called on a \f[I]struct pmemvlt\f[], it will return the pointer to the data and it will ensure that the provided constructor function is called exactly once in the current instance of the pmemobj pool. The constructor is called with the \f[I]ptr\f[] pointer to the data, and this function will return the same pointer if the constructor returns \f[I]0\f[], otherwise NULL is returned. The \f[I]size\f[] argument must accurately describe the total size of the volatile memory region that will be accessed. Calling \f[B]pmemobj_volatile()\f[] on the same region with different sizes is undefined behavior. For this mechanism to be effective, all accesses to transient variables must go through it, otherwise there's a risk of the constructor not being called on the first load. Maintaining transient state on persistent memory is challenging due to difficulties with dynamic resources acquisition and subsequent resource release. For example, one needs to consider what happens with volatile state of an object which is being freed inside of a transaction, especially with regards to the possibility of an abort. It's generally recommended to entirely separate the persistent and transient states, and when it's not possible, to only store types which do not require lifecycle management (i.e., primitive types) inside of volatile regions. .SH RETURN VALUE .PP The \f[B]pmemobj_direct\f[]() function returns a pointer to the object represented by \f[I]oid\f[]. If \f[I]oid\f[] is \f[B]OID_NULL\f[], \f[B]pmemobj_direct\f[]() returns NULL. .PP The \f[B]pmemobj_oid\f[]() function returns a \f[I]PMEMoid\f[] handle to the object pointed to by \f[I]addr\f[]. If \f[I]addr\f[] is not from within a pmemobj pool, \f[B]OID_NULL\f[] is returned. If \f[I]addr\f[] is not the start of an object (does not point to the beginning of a valid allocation), the resulting \f[I]PMEMoid\f[] can be safely used only with: .IP \[bu] 2 \f[B]pmemobj_pool_by_oid\f[]() .IP \[bu] 2 \f[B]pmemobj_direct\f[]() .IP \[bu] 2 \f[B]pmemobj_tx_add_range\f[](3) .PP The \f[B]pmemobj_type_num\f[]() function returns the type number of the object represented by \f[I]oid\f[]. .PP The \f[B]pmemobj_pool_by_oid\f[]() function returns a handle to the pool that contains the object represented by \f[I]oid\f[]. If the pool is not open or \f[I]oid\f[] is \f[B]OID_NULL\f[], \f[B]pmemobj_pool_by_oid\f[]() returns NULL. .PP The \f[B]pmemobj_pool_by_ptr\f[]() function returns a handle to the pool that contains the address, or NULL if the address does not belong to any open pool. .SH NOTES .PP For performance reasons, on Linux and FreeBSD the \f[B]pmemobj_direct\f[]() function is inlined by default. To use the non\-inlined variant of \f[B]pmemobj_direct\f[](), define \f[B]PMEMOBJ_DIRECT_NON_INLINE\f[] prior to the \f[I]#include\f[] of \f[B]\f[], either with \f[I]#define\f[] or with the \f[I]\-D\f[] option to the compiler. .SH EXAMPLES .PP The following code shows how to store transient variables on persistent memory. .IP .nf \f[C] struct\ my_data\ { \ \ \ \ PMEMvlt(uint64_t)\ foo; \ \ \ \ uint64_t\ bar; }; int my_data_constructor(void\ *ptr,\ void\ *arg) { \ \ \ \ uint64_t\ *foo\ =\ ptr; \ \ \ \ *foo\ =\ 0; \ \ \ \ return\ 0; } PMEMobjpool\ *pop\ =\ ...; struct\ my_data\ *data\ =\ D_RW(...); uint64_t\ *foo\ =\ pmemobj_volatile(pop,\ &data\->foo.vlt,\ &data\->foo.value, \ \ \ \ my_data_constructor,\ NULL); assert(*foo\ ==\ 0); \f[] .fi .SH SEE ALSO .PP \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_tx_errno.30000664000000000000000000000002713615011243017332 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pobj_layout_end.30000664000000000000000000000003013615011243017130 0ustar rootroot.so pobj_layout_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_mutex_lock.30000664000000000000000000000003113615011243017637 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pmemobj_ctl_get.30000644000000000000000000004360213615011417017122 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_CTL_GET" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2020, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_ctl_get\f[](), \f[B]pmemobj_ctl_set\f[](), \f[B]pmemobj_ctl_exec\f[]() \- Query and modify libpmemobj internal behavior (EXPERIMENTAL) .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmemobj_ctl_get(PMEMobjpool\ *pop,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) int\ pmemobj_ctl_set(PMEMobjpool\ *pop,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) int\ pmemobj_ctl_exec(PMEMobjpool\ *pop,\ const\ char\ *name,\ void\ *arg);\ (EXPERIMENTAL) \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemobj_ctl_get\f[](), \f[B]pmemobj_ctl_set\f[]() and \f[B]pmemobj_ctl_exec\f[]() functions provide a uniform interface for querying and modifying the internal behavior of \f[B]libpmemobj\f[](7) through the control (CTL) namespace. .PP The \f[I]name\f[] argument specifies an entry point as defined in the CTL namespace specification. The entry point description specifies whether the extra \f[I]arg\f[] is required. Those two parameters together create a CTL query. The functions and the entry points are thread\-safe unless indicated otherwise below. If there are special conditions for calling an entry point, they are explicitly stated in its description. The functions propagate the return value of the entry point. If either \f[I]name\f[] or \f[I]arg\f[] is invalid, \-1 is returned. .PP If the provided ctl query is valid, the CTL functions will always return 0 on success and \-1 on failure, unless otherwise specified in the entry point description. .PP See more in \f[B]pmem_ctl\f[](5) man page. .SH CTL NAMESPACE .PP prefault.at_create | rw | global | int | int | \- | boolean .PP If set, every page of the pool will be touched and written to when the pool is created, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the \f[B]pmemobj_create\f[]() function. .PP prefault.at_open | rw | global | int | int | \- | boolean .PP If set, every page of the pool will be touched and written to when the pool is opened, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the \f[B]pmemobj_open\f[]() function. .PP sds.at_create | rw | global | int | int | \- | boolean .PP If set, force\-enables or force\-disables SDS feature during pool creation. Affects only the \f[B]pmemobj_create\f[]() function. See \f[B]pmempool_feature_query\f[](3) for information about SDS (SHUTDOWN_STATE) feature. .PP copy_on_write.at_open | rw | global | int | int | \- | boolean .PP If set, pool is mapped in such a way that modifications don't reach the underlying medium. From the user's perspective this means that when the pool is closed all changes are reverted. This feature is not supported for pools located on Device DAX. .PP tx.debug.skip_expensive_checks | rw | \- | int | int | \- | boolean .PP Turns off some expensive checks performed by the transaction module in \[lq]debug\[rq] builds. Ignored in \[lq]release\[rq] builds. .PP tx.debug.verify_user_buffers | rw | \- | int | int | \- | boolean .PP Enables verification of user buffers provided by \f[B]pmemobj_tx_log_append_buffer\f[](3) API. For now the only verified aspect is whether the same buffer is used simultaneously in 2 or more transactions or more than once in the same transaction. This value should not be modified at runtime if any transaction for the current pool is in progress. .PP tx.cache.size | rw | \- | long long | long long | \- | integer .PP Size in bytes of the transaction snapshot cache. In a larger cache the frequency of persistent allocations is lower, but with higher fixed cost. .PP This should be set to roughly the sum of sizes of the snapshotted regions in an average transaction in the pool. .PP This entry point is not thread safe and should not be modified if there are any transactions currently running. .PP This value must be a in a range between 0 and \f[B]PMEMOBJ_MAX_ALLOC_SIZE\f[], otherwise this entry point will fail. .PP tx.cache.threshold | rw | \- | long long | long long | \- | integer .PP This entry point is deprecated. All snapshots, regardless of the size, use the transactional cache. .PP tx.post_commit.queue_depth | rw | \- | int | int | \- | integer .PP This entry point is deprecated. .PP tx.post_commit.worker | r\- | \- | void * | \- | \- | \- .PP This entry point is deprecated. .PP tx.post_commit.stop | r\- | \- | void * | \- | \- | \- .PP This entry point is deprecated. .PP heap.narenas.automatic | r\- | \- | unsigned | \- | \- | \- .PP Reads the number of arenas used in automatic scheduling of memory operations for threads. By default, this value is equal to the number of available processors. An arena is a memory management structure which enables concurrency by taking exclusive ownership of parts of the heap and allowing associated threads to allocate without contention. .PP heap.narenas.total | r\- | \- | unsigned | \- | \- | \- .PP Reads the number of all created arenas. It includes automatic arenas created by default and arenas created using heap.arena.create CTL. .PP heap.narenas.max | rw\- | \- | unsigned | unsigned | \- | \- .PP Reads or writes the maximum number of arenas that can be created. This entry point is not thread\-safe with regards to heap operations (allocations, frees, reallocs). .PP heap.arena.[arena_id].size | r\- | \- | uint64_t | \- | \- | \- .PP Reads the total amount of memory in bytes which is currently exclusively owned by the arena. Large differences in this value between arenas might indicate an uneven scheduling of memory resources. The arena id cannot be 0. .PP heap.thread.arena_id | rw\- | \- | unsigned | unsigned | \- | \- .PP Reads the index of the arena assigned to the current thread or assigns arena with specific id to the current thread. The arena id cannot be 0. .PP heap.arena.create | \[en]x | \- | \- | \- | unsigned | \- .PP Creates and initializes one new arena in the heap. This entry point reads an id of the new created arena. .PP Newly created arenas by this CTL are inactive, which means that the arena will not be used in the automatic scheduling of memory requests. To activate the new arena, use heap.arena.[arena_id].automatic CTL. .PP Arena created using this CTL can be used for allocation by explicitly specifying the \f[I]arena_id\f[] for \f[B]POBJ_ARENA_ID(id)\f[] flag in \f[B]pmemobj_tx_xalloc\f[]()/\f[B]pmemobj_xalloc\f[]()/\f[B]pmemobj_xreserve()\f[] functions. .PP By default, the number of arenas is limited to 1024. .PP heap.arena.[arena_id].automatic | rw\- | \- | boolean | boolean | \- | \- .PP Reads or modifies the state of the arena. If set, the arena is used in automatic scheduling of memory operations for threads. This should be set to false if the application wants to manually manage allocator scalability through explicitly assigning arenas to threads by using heap.thread.arena_id. The arena id cannot be 0 and at least one automatic arena must exist. .PP heap.alloc_class.[class_id].desc | rw | \- | \f[C]struct\ pobj_alloc_class_desc\f[] | \f[C]struct\ pobj_alloc_class_desc\f[] | \- | integer, integer, integer, string .PP Describes an allocation class. Allows one to create or view the internal data structures of the allocator. .PP Creating custom allocation classes can be beneficial for both raw allocation throughput, scalability and, most importantly, fragmentation. By carefully constructing allocation classes that match the application workload, one can entirely eliminate external and internal fragmentation. For example, it is possible to easily construct a slab\-like allocation mechanism for any data structure. .PP The \f[C][class_id]\f[] is an index field. Only values between 0\-254 are valid. If setting an allocation class, but the \f[C]class_id\f[] is already taken, the function will return \-1. The values between 0\-127 are reserved for the default allocation classes of the library and can be used only for reading. .PP The recommended method for retrieving information about all allocation classes is to call this entry point for all class ids between 0 and 254 and discard those results for which the function returns an error. .PP This entry point takes a complex argument. .IP .nf \f[C] struct\ pobj_alloc_class_desc\ { \ \ \ \ size_t\ unit_size; \ \ \ \ size_t\ alignment; \ \ \ \ unsigned\ units_per_block; \ \ \ \ enum\ pobj_header_type\ header_type; \ \ \ \ unsigned\ class_id; }; \f[] .fi .PP The first field, \f[C]unit_size\f[], is an 8\-byte unsigned integer that defines the allocation class size. While theoretically limited only by \f[B]PMEMOBJ_MAX_ALLOC_SIZE\f[], for most workloads this value should be between 8 bytes and 2 megabytes. .PP The \f[C]alignment\f[] field specifies the user data alignment of objects allocated using the class. If set, must be a power of two and an even divisor of unit size. Alignment is limited to maximum of 2 megabytes. All objects have default alignment of 64 bytes, but the user data alignment is affected by the size of the chosen header. .PP The \f[C]units_per_block\f[] field defines how many units a single block of memory contains. This value will be adjusted to match the internal size of the block (256 kilobytes or a multiple thereof). For example, given a class with a \f[C]unit_size\f[] of 512 bytes and a \f[C]units_per_block\f[] of 1000, a single block of memory for that class will have 512 kilobytes. This is relevant because the bigger the block size, the less frequently blocks need to be fetched, resulting in lower contention on global heap state. If the CTL call is being done at runtime, the \f[C]units_per_block\f[] variable of the provided alloc class structure is modified to match the actual value. .PP The \f[C]header_type\f[] field defines the header of objects from the allocation class. There are three types: .IP \[bu] 2 \f[B]POBJ_HEADER_LEGACY\f[], string value: \f[C]legacy\f[]. Used for allocation classes prior to version 1.3 of the library. Not recommended for use. Incurs a 64 byte metadata overhead for every object. Fully supports all features. .IP \[bu] 2 \f[B]POBJ_HEADER_COMPACT\f[], string value: \f[C]compact\f[]. Used as default for all predefined allocation classes. Incurs a 16 byte metadata overhead for every object. Fully supports all features. .IP \[bu] 2 \f[B]POBJ_HEADER_NONE\f[], string value: \f[C]none\f[]. Header type that incurs no metadata overhead beyond a single bitmap entry. Can be used for very small allocation classes or when objects must be adjacent to each other. This header type does not support type numbers (type number is always .RS 2 .IP "0)" 3 or allocations that span more than one unit. .RE .PP The \f[C]class_id\f[] field is an optional, runtime\-only variable that allows the user to retrieve the identifier of the class. This will be equivalent to the provided \f[C][class_id]\f[]. This field cannot be set from a config file. .PP The allocation classes are a runtime state of the library and must be created after every open. It is highly recommended to use the configuration file to store the classes. .PP This structure is declared in the \f[C]libpmemobj/ctl.h\f[] header file. Please refer to this file for an in\-depth explanation of the allocation classes and relevant algorithms. .PP Allocation classes constructed in this way can be leveraged by explicitly specifying the class using \f[B]POBJ_CLASS_ID(id)\f[] flag in \f[B]pmemobj_tx_xalloc\f[]()/\f[B]pmemobj_xalloc\f[]() functions. .PP Example of a valid alloc class query string: .IP .nf \f[C] heap.alloc_class.128.desc=500,0,1000,compact \f[] .fi .PP This query, if executed, will create an allocation class with an id of 128 that has a unit size of 500 bytes, has at least 1000 units per block and uses a compact header. .PP For reading, function returns 0 if successful, if the allocation class does not exist it sets the errno to \f[B]ENOENT\f[] and returns \-1; .PP This entry point can fail if any of the parameters of the allocation class is invalid or if exactly the same class already exists. .PP heap.alloc_class.new.desc | \-w | \- | \- | \f[C]struct\ pobj_alloc_class_desc\f[] | \- | integer, integer, integer, string .PP Same as \f[C]heap.alloc_class.[class_id].desc\f[], but instead of requiring the user to provide the class_id, it automatically creates the allocation class with the first available identifier. .PP This should be used when it's impossible to guarantee unique allocation class naming in the application (e.g.\ when writing a library that uses libpmemobj). .PP The required class identifier will be stored in the \f[C]class_id\f[] field of the \f[C]struct\ pobj_alloc_class_desc\f[]. .PP stats.enabled | rw | \- | enum pobj_stats_enabled | enum pobj_stats_enabled | \- | string .PP Enables or disables runtime collection of statistics. There are two types of statistics: persistent and transient ones. Persistent statistics survive pool restarts, whereas transient ones don't. Statistics are not recalculated after enabling; any operations that occur between disabling and re\-enabling will not be reflected in subsequent values. .PP Only transient statistics are enabled by default. Enabling persistent statistics may have non\-trivial performance impact. .PP stats.heap.curr_allocated | r\- | \- | uint64_t | \- | \- | \- .PP Reads the number of bytes currently allocated in the heap. If statistics were disabled at any time in the lifetime of the heap, this value may be inaccurate. .PP This is a persistent statistic. .PP stats.heap.run_allocated | r\- | \- | uint64_t | \- | \- | \- .PP Reads the number of bytes currently allocated using run\-based allocation classes, i.e., huge allocations are not accounted for in this statistic. This is useful for comparison against stats.heap.run_active to estimate the ratio between active and allocated memory. .PP This is a transient statistic and is rebuilt every time the pool is opened. .PP stats.heap.run_active | r\- | \- | uint64_t | \- | \- | \- .PP Reads the number of bytes currently occupied by all run memory blocks, including both allocated and free space, i.e., this is all the all space that's not occupied by huge allocations. .PP This value is a sum of all allocated and free run memory. In systems where memory is efficiently used, \f[C]run_active\f[] should closely track \f[C]run_allocated\f[], and the amount of active, but free, memory should be minimal. .PP A large relative difference between active memory and allocated memory is indicative of heap fragmentation. This information can be used to make a decision to call \f[B]pmemobj_defrag()\f[](3) if the fragmentation looks to be high. .PP However, for small heaps \f[C]run_active\f[] might be disproportionately higher than \f[C]run_allocated\f[] because the allocator typically activates a significantly larger amount of memory than is required to satisfy a single request in the anticipation of future needs. For example, the first allocation of 100 bytes in a heap will trigger activation of 256 kilobytes of space. .PP This is a transient statistic and is rebuilt lazily every time the pool is opened. .PP heap.size.granularity | rw\- | \- | uint64_t | uint64_t | \- | long long .PP Reads or modifies the granularity with which the heap grows when OOM. Valid only if the poolset has been defined with directories. .PP A granularity of 0 specifies that the pool will not grow automatically. .PP This entry point can fail if the granularity value is non\-zero and smaller than \f[I]PMEMOBJ_MIN_PART\f[]. .PP heap.size.extend | \[en]x | \- | \- | \- | uint64_t | \- .PP Extends the heap by the given size. Must be larger than \f[I]PMEMOBJ_MIN_PART\f[]. .PP This entry point can fail if the pool does not support extend functionality or if there's not enough space left on the device. .PP debug.heap.alloc_pattern | rw | \- | int | int | \- | \- .PP Single byte pattern that is used to fill new uninitialized memory allocation. If the value is negative, no pattern is written. This is intended for debugging, and is disabled by default. .SH CTL EXTERNAL CONFIGURATION .PP In addition to direct function call, each write entry point can also be set using two alternative methods. .PP The first method is to load a configuration directly from the \f[B]PMEMOBJ_CONF\f[] environment variable. .PP The second method of loading an external configuration is to set the \f[B]PMEMOBJ_CONF_FILE\f[] environment variable to point to a file that contains a sequence of ctl queries. .PP See more in \f[B]pmem_ctl\f[](5) man page. .SH SEE ALSO .PP \f[B]libpmemobj\f[](7), \f[B]pmem_ctl\f[](5) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_check_version.30000664000000000000000000000002613615011243020313 0ustar rootroot.so man7/libpmemobj.7 pmdk-1.8/doc/libpmemobj/pobj_realloc.30000664000000000000000000000002413615011243016411 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_action.3.md0000664000000000000000000002243613615011243017356 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_ACTION, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_action.3 -- Delayed atomicity actions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[EXAMPLES](#examples)
[SEE ALSO](#see-also)
# NAME # **pmemobj_reserve**(), **pmemobj_xreserve**(), **pmemobj_defer_free**(), **pmemobj_set_value**(), **pmemobj_publish**(), **pmemobj_tx_publish**(), **pmemobj_tx_xpublish**(), **pmemobj_cancel**(), **POBJ_RESERVE_NEW**(), **POBJ_RESERVE_ALLOC**(), **POBJ_XRESERVE_NEW**(),**POBJ_XRESERVE_ALLOC**() - Delayed atomicity actions (EXPERIMENTAL) # SYNOPSIS # ```c #include PMEMoid pmemobj_reserve(PMEMobjpool *pop, struct pobj_action *act, size_t size, uint64_t type_num); (EXPERIMENTAL) PMEMoid pmemobj_xreserve(PMEMobjpool *pop, struct pobj_action *act, size_t size, uint64_t type_num, uint64_t flags); (EXPERIMENTAL) void pmemobj_defer_free(PMEMobjpool *pop, PMEMoid oid, struct pobj_action *act); void pmemobj_set_value(PMEMobjpool *pop, struct pobj_action *act, uint64_t *ptr, uint64_t value); (EXPERIMENTAL) int pmemobj_publish(PMEMobjpool *pop, struct pobj_action *actv, size_t actvcnt); (EXPERIMENTAL) int pmemobj_tx_publish(struct pobj_action *actv, size_t actvcnt); (EXPERIMENTAL) int pmemobj_tx_xpublish(struct pobj_action *actv, size_t actvcnt, uint64_t flags); (EXPERIMENTAL) void pmemobj_cancel(PMEMobjpool *pop, struct pobj_action *actv, size_t actvcnt); (EXPERIMENTAL) POBJ_RESERVE_NEW(pop, t, act) (EXPERIMENTAL) POBJ_RESERVE_ALLOC(pop, t, size, act) (EXPERIMENTAL) POBJ_XRESERVE_NEW(pop, t, act, flags) (EXPERIMENTAL) POBJ_XRESERVE_ALLOC(pop, t, size, act, flags) (EXPERIMENTAL) ``` # DESCRIPTION # All of the functions described so far have an immediate effect on the persistent state of the pool, and as such, the cost of maintaining fail-safety is paid outright and, most importantly, in the calling thread. This behavior makes implementing algorithms involving relaxed consistency guarantees difficult, if not outright impossible. The following set of functions introduce a mechanism that allows one to delay the persistent publication of a set of prepared actions to an arbitrary moment in time of the execution of a program. The publication is fail-safe atomic in the scope of the entire collection of actions. If a program exits without publishing the actions, or the actions are canceled, any resources reserved by those actions are released and placed back in the pool. A single action is represented by a single `struct pobj_action`. Functions that create actions take that structure by pointer, whereas functions that publish actions take array of actions and the size of the array. The actions can be created, and published, from different threads. When creating actions, the *act* argument must be non-NULL and point to a `struct pobj_action`, the structure will be populated by the function and must not be modified or deallocated until after publishing. The **pmemobj_reserve**() functions performs a transient reservation of an object. Behaves similarly to **pmemobj_alloc**(3), but performs no modification to the persistent state. The object returned by this function can be freely modified without worrying about fail-safe atomicity until the object has been published. Any modifications of the object must be manually persisted, just like in the case of the atomic API. **pmemobj_xreserve**() is equivalent to **pmemobj_reserve**(), but with an additional *flags* argument that is a bitmask of the following values: + **POBJ_XALLOC_ZERO** - zero the allocated object (and persist it) + **POBJ_CLASS_ID(class_id)** - allocate an object from the allocation class *class_id*. The class id cannot be 0. + **POBJ_ARENA_ID(arena_id)** - allocate an object from the arena specified by *arena_id*. The arena must exist, otherwise, the behavior is undefined. If *arena_id* is equal 0, then arena assigned to the current thread will be used. **pmemobj_defer_free**() function creates a deferred free action, meaning that the provided object will be freed when the action is published. Calling this function with a NULL OID is invalid and causes undefined behavior. The **pmemobj_set_value** function prepares an action that, once published, will modify the memory location pointed to by *ptr* to *value*. The **pmemobj_publish** function publishes the provided set of actions. The publication is fail-safe atomic. Once done, the persistent state will reflect the changes contained in the actions. The **pmemobj_tx_publish** function moves the provided actions to the scope of the transaction in which it is called. Only object reservations are supported in transactional publish. Once done, the reserved objects will follow normal transactional semantics. Can only be called during *TX_STAGE_WORK*. The **pmemobj_tx_xpublish**() function behaves exactly the same as **pmemobj_tx_publish**() when *flags* equals zero. *flags* is a bitmask of the following values: + **POBJ_XPUBLISH_NO_ABORT** - if the function does not end successfully, do not abort the transaction. The **pmemobj_cancel** function releases any resources held by the provided set of actions and invalidates all actions. The **POBJ_RESERVE_NEW** macro is a typed variant of **pmemobj_reserve**. The size of the reservation is determined from the provided type *t*. The **POBJ_RESERVE_ALLOC** macro is a typed variant of **pmemobj_reserve**. The *size* of the reservation is user-provided. The **POBJ_XRESERVE_NEW** and the **POBJ_XRESERVE_ALLOC** macros are equivalent to **POBJ_RESERVE_NEW** and the **POBJ_RESERVE_ALLOC**, but with an additional *flags* argument defined for **pmemobj_xreserve**(). # EXAMPLES # The following code shows atomic append of two objects into a singly linked list. ```c struct list_node { int value; PMEMoid next; }; /* statically allocate the array of actions */ struct pobj_action actv[4]; /* reserve, populate and persist the first object */ PMEMoid tail = pmemobj_reserve(pop, &actv[0], sizeof(struct list_node), 0); if (TOID_IS_NULL(tail)) return -1; D_RW(tail)->value = 1; D_RW(tail)->next = OID_NULL; pmemobj_persist(pop, D_RW(tail), sizeof(struct list_node)); /* reserve, populate and persist the second object */ PMEMoid head = pmemobj_reserve(pop, &actv[1], sizeof(struct list_node), 0); if (TOID_IS_NULL(head)) return -1; D_RW(head)->value = 2; D_RW(head)->next = tail; pmemobj_persist(pop, D_RW(head), sizeof(struct list_node)); /* create actions to set the PMEMoid to the new values */ pmemobj_set_value(pop, &actv[2], &D_RO(root)->head.pool_uuid_lo, head.pool_uuid_lo); pmemobj_set_value(pop, &actv[3], &D_RO(root)->head.off, head.off); /* atomically publish the above actions */ pmemobj_publish(pop, actv, 4); ``` # RETURN VALUE # On success, **pmemobj_reserve**() functions return a handle to the newly reserved object. Otherwise an *OID_NULL* is returned. On success, **pmemobj_tx_publish**() returns 0. Otherwise, the transaction is aborted, the stage is changed to *TX_STAGE_ONABORT* and *errno* is set appropriately. On success, **pmemobj_tx_xpublish**() returns 0. Otherwise, the error number is returned, **errno** is set and when flags do not contain **POBJ_XPUBLISH_NO_ABORT**, the transaction is aborted. On success, **pmemobj_publish**() returns 0. Otherwise, returns -1 and *errno* is set appropriately. # SEE ALSO # **pmemobj_alloc**(3), **pmemobj_tx_alloc**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pobj_next_type_num.30000664000000000000000000000002413615011243017666 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/pobj_list_remove.30000664000000000000000000000002513615011243017321 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_mutex_zero.30000644000000000000000000002703513615011417017704 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_MUTEX_ZERO" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_mutex_zero\f[](), \f[B]pmemobj_mutex_lock\f[](), \f[B]pmemobj_mutex_timedlock\f[](), \f[B]pmemobj_mutex_trylock\f[](), \f[B]pmemobj_mutex_unlock\f[](), .PP \f[B]pmemobj_rwlock_zero\f[](), \f[B]pmemobj_rwlock_rdlock\f[](), \f[B]pmemobj_rwlock_wrlock\f[](), \f[B]pmemobj_rwlock_timedrdlock\f[](), \f[B]pmemobj_rwlock_timedwrlock\f[](), \f[B]pmemobj_rwlock_tryrdlock\f[](), \f[B]pmemobj_rwlock_trywrlock\f[](), \f[B]pmemobj_rwlock_unlock\f[](), .PP \f[B]pmemobj_cond_zero\f[](), \f[B]pmemobj_cond_broadcast\f[](), \f[B]pmemobj_cond_signal\f[](), \f[B]pmemobj_cond_timedwait\f[](), \f[B]pmemobj_cond_wait\f[]() \- pmemobj synchronization primitives .SH SYNOPSIS .IP .nf \f[C] #include\ void\ pmemobj_mutex_zero(PMEMobjpool\ *pop,\ PMEMmutex\ *mutexp); int\ pmemobj_mutex_lock(PMEMobjpool\ *pop,\ PMEMmutex\ *mutexp); int\ pmemobj_mutex_timedlock(PMEMobjpool\ *pop,\ PMEMmutex\ *restrict\ mutexp, \ \ \ \ const\ struct\ timespec\ *restrict\ abs_timeout); int\ pmemobj_mutex_trylock(PMEMobjpool\ *pop,\ PMEMmutex\ *mutexp); int\ pmemobj_mutex_unlock(PMEMobjpool\ *pop,\ PMEMmutex\ *mutexp); void\ pmemobj_rwlock_zero(PMEMobjpool\ *pop,\ PMEMrwlock\ *rwlockp); int\ pmemobj_rwlock_rdlock(PMEMobjpool\ *pop,\ PMEMrwlock\ *rwlockp); int\ pmemobj_rwlock_wrlock(PMEMobjpool\ *pop,\ PMEMrwlock\ *rwlockp); int\ pmemobj_rwlock_timedrdlock(PMEMobjpool\ *pop,\ PMEMrwlock\ *restrict\ rwlockp, \ \ \ \ const\ struct\ timespec\ *restrict\ abs_timeout); int\ pmemobj_rwlock_timedwrlock(PMEMobjpool\ *pop,\ PMEMrwlock\ *restrict\ rwlockp, \ \ \ \ const\ struct\ timespec\ *restrict\ abs_timeout); int\ pmemobj_rwlock_tryrdlock(PMEMobjpool\ *pop,\ PMEMrwlock\ *rwlockp); int\ pmemobj_rwlock_trywrlock(PMEMobjpool\ *pop,\ PMEMrwlock\ *rwlockp); int\ pmemobj_rwlock_unlock(PMEMobjpool\ *pop,\ PMEMrwlock\ *rwlockp); void\ pmemobj_cond_zero(PMEMobjpool\ *pop,\ PMEMcond\ *condp); int\ pmemobj_cond_broadcast(PMEMobjpool\ *pop,\ PMEMcond\ *condp); int\ pmemobj_cond_signal(PMEMobjpool\ *pop,\ PMEMcond\ *condp); int\ pmemobj_cond_timedwait(PMEMobjpool\ *pop,\ PMEMcond\ *restrict\ condp, \ \ \ \ PMEMmutex\ *restrict\ mutexp,\ const\ struct\ timespec\ *restrict\ abs_timeout); int\ pmemobj_cond_wait(PMEMobjpool\ *pop,\ PMEMcond\ *restrict\ condp, \ \ \ \ PMEMmutex\ *restrict\ mutexp); \f[] .fi .SH DESCRIPTION .PP \f[B]libpmemobj\f[](7) provides several types of synchronization primitives designed to be used with persistent memory. The pmem\-aware lock implementation is based on the standard POSIX Threads Library, as described in \f[B]pthread_mutex_init\f[](3), \f[B]pthread_rwlock_init\f[](3) and \f[B]pthread_cond_init\f[](3). Pmem\-aware locks provide semantics similar to standard \f[B]pthread\f[] locks, except that they are embedded in pmem\-resident objects and are considered initialized by zeroing them. Therefore, locks allocated with \f[B]pmemobj_zalloc\f[](3) or \f[B]pmemobj_tx_zalloc\f[](3) do not require another initialization step. For performance reasons, they are also padded up to 64 bytes (cache line size). .PP On FreeBSD, since all \f[B]pthread\f[] locks are dynamically allocated, while the lock object is still padded up to 64 bytes for consistency with Linux, only the pointer to the lock is embedded in the pmem\-resident object. \f[B]libpmemobj\f[](7) transparently manages freeing of the locks when the pool is closed. .PP The fundamental property of pmem\-aware locks is their automatic reinitialization every time the persistent object store pool is opened. Thus, all the pmem\-aware locks may be considered initialized (unlocked) immediately after the pool is opened, regardless of their state at the time the pool was closed for the last time. .PP Pmem\-aware mutexes, read/write locks and condition variables must be declared with the \f[I]PMEMmutex\f[], \f[I]PMEMrwlock\f[], or \f[I]PMEMcond\f[] type, respectively. .PP The \f[B]pmemobj_mutex_zero\f[]() function explicitly initializes the pmem\-aware mutex \f[I]mutexp\f[] by zeroing it. Initialization is not necessary if the object containing the mutex has been allocated using \f[B]pmemobj_zalloc\f[](3) or \f[B]pmemobj_tx_zalloc\f[](3). .PP The \f[B]pmemobj_mutex_lock\f[]() function locks the pmem\-aware mutex \f[I]mutexp\f[]. If the mutex is already locked, the calling thread will block until the mutex becomes available. If this is the first use of the mutex since the opening of the pool \f[I]pop\f[], the mutex is automatically reinitialized and then locked. .PP \f[B]pmemobj_mutex_timedlock\f[]() performs the same action as \f[B]pmemobj_mutex_lock\f[](), but will not wait beyond \f[I]abs_timeout\f[] to obtain the lock before returning. .PP The \f[B]pmemobj_mutex_trylock\f[]() function locks pmem\-aware mutex \f[I]mutexp\f[]. If the mutex is already locked, \f[B]pthread_mutex_trylock\f[]() will not block waiting for the mutex, but will return an error. If this is the first use of the mutex since the opening of the pool \f[I]pop\f[], the mutex is automatically reinitialized and then locked. .PP The \f[B]pmemobj_mutex_unlock\f[]() function unlocks the pmem\-aware mutex \f[I]mutexp\f[]. Undefined behavior follows if a thread tries to unlock a mutex that has not been locked by it, or if a thread tries to release a mutex that is already unlocked or has not been initialized. .PP The \f[B]pmemobj_rwlock_zero\f[]() function is used to explicitly initialize the pmem\-aware read/write lock \f[I]rwlockp\f[] by zeroing it. Initialization is not necessary if the object containing the lock has been allocated using \f[B]pmemobj_zalloc\f[](3) or \f[B]pmemobj_tx_zalloc\f[](3). .PP The \f[B]pmemobj_rwlock_rdlock\f[]() function acquires a read lock on \f[I]rwlockp\f[], provided that the lock is not presently held for writing and no writer threads are presently blocked on the lock. If the read lock cannot be acquired immediately, the calling thread blocks until it can acquire the lock. If this is the first use of the lock since the opening of the pool \f[I]pop\f[], the lock is automatically reinitialized and then acquired. .PP \f[B]pmemobj_rwlock_timedrdlock\f[]() performs the same action as \f[B]pmemobj_rwlock_rdlock\f[](), but will not wait beyond \f[I]abs_timeout\f[] to obtain the lock before returning. A thread may hold multiple concurrent read locks. If so, \f[B]pmemobj_rwlock_unlock\f[]() must be called once for each lock obtained. The results of acquiring a read lock while the calling thread holds a write lock are undefined. .PP The \f[B]pmemobj_rwlock_wrlock\f[]() function blocks until a write lock can be acquired against read/write lock \f[I]rwlockp\f[]. If this is the first use of the lock since the opening of the pool \f[I]pop\f[], the lock is automatically reinitialized and then acquired. .PP \f[B]pmemobj_rwlock_timedwrlock\f[]() performs the same action, but will not wait beyond \f[I]abs_timeout\f[] to obtain the lock before returning. .PP The \f[B]pmemobj_rwlock_tryrdlock\f[]() function performs the same action as \f[B]pmemobj_rwlock_rdlock\f[](), but does not block if the lock cannot be immediately obtained. The results are undefined if the calling thread already holds the lock at the time the call is made. .PP The \f[B]pmemobj_rwlock_trywrlock\f[]() function performs the same action as \f[B]pmemobj_rwlock_wrlock\f[](), but does not block if the lock cannot be immediately obtained. The results are undefined if the calling thread already holds the lock at the time the call is made. .PP The \f[B]pmemobj_rwlock_unlock\f[]() function is used to release the read/write lock previously obtained by \f[B]pmemobj_rwlock_rdlock\f[](), \f[B]pmemobj_rwlock_wrlock\f[](), \f[B]pthread_rwlock_tryrdlock\f[](), or \f[B]pmemobj_rwlock_trywrlock\f[](). .PP The \f[B]pmemobj_cond_zero\f[]() function explicitly initializes the pmem\-aware condition variable \f[I]condp\f[] by zeroing it. Initialization is not necessary if the object containing the condition variable has been allocated using \f[B]pmemobj_zalloc\f[](3) or \f[B]pmemobj_tx_zalloc\f[](3). .PP The difference between \f[B]pmemobj_cond_broadcast\f[]() and \f[B]pmemobj_cond_signal\f[]() is that the former unblocks all threads waiting for the condition variable, whereas the latter blocks only one waiting thread. If no threads are waiting on \f[I]condp\f[], neither function has any effect. If more than one thread is blocked on a condition variable, the used scheduling policy determines the order in which threads are unblocked. The same mutex used for waiting must be held while calling either function. Although neither function strictly enforces this requirement, undefined behavior may follow if the mutex is not held. .PP The \f[B]pmemobj_cond_timedwait\f[]() and \f[B]pmemobj_cond_wait\f[]() functions block on a condition variable. They must be called with mutex \f[I]mutexp\f[] locked by the calling thread, or undefined behavior results. These functions atomically release mutex \f[I]mutexp\f[] and cause the calling thread to block on the condition variable \f[I]condp\f[]; atomically here means \[lq]atomically with respect to access by another thread to the mutex and then the condition variable\[rq]. That is, if another thread is able to acquire the mutex after the about\-to\-block thread has released it, then a subsequent call to \f[B]pmemobj_cond_broadcast\f[]() or \f[B]pmemobj_cond_signal\f[]() in that thread will behave as if it were issued after the about\-to\-block thread has blocked. Upon successful return, the mutex will be locked and owned by the calling thread. .SH RETURN VALUE .PP The \f[B]pmemobj_mutex_zero\f[](), \f[B]pmemobj_rwlock_zero\f[]() and \f[B]pmemobj_cond_zero\f[]() functions return no value. .PP Other locking functions return 0 on success. Otherwise, an error number will be returned to indicate the error. .SH SEE ALSO .PP \f[B]pmemobj_tx_zalloc\f[](3), \f[B]pmemobj_zalloc\f[](3), \f[B]pthread_cond_init\f[](3), \f[B]pthread_mutex_init\f[](3), \f[B]pthread_rwlock_init\f[](3), \f[B]libpmem\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_first.3.md0000664000000000000000000001243213615011243017223 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_FIRST, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_first.3 -- man page for pmemobj container operations) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[SEE ALSO](#see-also)
# NAME # **pmemobj_first**(), **pmemobj_next**(), **POBJ_FIRST**(), **POBJ_FIRST_TYPE_NUM**(), **POBJ_NEXT**(), **POBJ_NEXT_TYPE_NUM**(), **POBJ_FOREACH**(), **POBJ_FOREACH_SAFE**(), **POBJ_FOREACH_TYPE**(), **POBJ_FOREACH_SAFE_TYPE**() - pmemobj container operations # SYNOPSIS # ```c #include PMEMoid pmemobj_first(PMEMobjpool *pop); PMEMoid pmemobj_next(PMEMoid oid); POBJ_FIRST(PMEMobjpool *pop, TYPE) POBJ_FIRST_TYPE_NUM(PMEMobjpool *pop, uint64_t type_num) POBJ_NEXT(TOID oid) POBJ_NEXT_TYPE_NUM(PMEMoid oid) POBJ_FOREACH(PMEMobjpool *pop, PMEMoid varoid) POBJ_FOREACH_SAFE(PMEMobjpool *pop, PMEMoid varoid, PMEMoid nvaroid) POBJ_FOREACH_TYPE(PMEMobjpool *pop, TOID var) POBJ_FOREACH_SAFE_TYPE(PMEMobjpool *pop, TOID var, TOID nvar) ``` # DESCRIPTION # The **libpmemobj**(7) container operations provide a mechanism that allows iteration through the internal object collection, either looking for a specific object, or performing a specific operation on each object of a given type. Software should not make any assumptions about the order of the objects in the internal object containers. The **pmemobj_first**() function returns the first object from the pool. The **POBJ_FIRST**() macro returns the first object from the pool of the type specified by *TYPE*. The **POBJ_FIRST_TYPE_NUM**() macro returns the first object from the pool of the type specified by *type_num*. The **pmemobj_next**() function returns the next object from the pool. The **POBJ_NEXT**() macro returns the next object of the same type as the object referenced by *oid*. The **POBJ_NEXT_TYPE_NUM**() macro returns the next object of the same type number as the object referenced by *oid*. The following four macros provide a more convenient way to iterate through the internal collections, performing a specific operation on each object. The **POBJ_FOREACH**() macro performs a specific operation on each allocated object stored in the persistent memory pool *pop*. It traverses the internal collection of all the objects, assigning a handle to each element in turn to *varoid*. The **POBJ_FOREACH_TYPE**() macro performs a specific operation on each allocated object stored in the persistent memory pool *pop* that has the same type as *var*. It traverses the internal collection of all the objects of the specified type, assigning a handle to each element in turn to *var*. The macros **POBJ_FOREACH_SAFE**() and **POBJ_FOREACH_SAFE_TYPE**() work in a similar fashion as **POBJ_FOREACH**() and **POBJ_FOREACH_TYPE**(), except that prior to performing the operation on the object, they preserve a handle to the next object in the collection by assigning it to *nvaroid* or *nvar*, respectively. This allows safe deletion of selected objects while iterating through the collection. # RETURN VALUE # **pmemobj_first**() returns the first object from the pool, or, if the pool is empty, **OID_NULL**. **pmemobj_next**() returns the next object from the pool. If the object referenced by *oid* is the last object in the collection, or if *oid* is *OID_NULL*, **pmemobj_next**() returns **OID_NULL**. # SEE ALSO # **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_list_insert.3.md0000664000000000000000000002220713615011243020434 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_LIST_INSERT, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_list_insert.3 -- man page for non-transactional persistent atomic lists) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemobj_list_insert**(), **pmemobj_list_insert_new**(), **pmemobj_list_move**(), **pmemobj_list_remove**() - non-transactional persistent atomic lists functions # SYNOPSIS # ```c #include int pmemobj_list_insert(PMEMobjpool *pop, size_t pe_offset, void *head, PMEMoid dest, int before, PMEMoid oid); PMEMoid pmemobj_list_insert_new(PMEMobjpool *pop, size_t pe_offset, void *head, PMEMoid dest, int before, size_t size, uint64_t type_num, pmemobj_constr constructor, void arg); int pmemobj_list_move(PMEMobjpool *pop, size_t pe_old_offset, void *head_old, size_t pe_new_offset, void *head_new, PMEMoid dest, int before, PMEMoid oid); int pmemobj_list_remove(PMEMobjpool *pop, size_t pe_offset, void *head, PMEMoid oid, int free); ``` # DESCRIPTION # In addition to the container operations on internal object collections described in **pmemobj_first**(3), **libpmemobj**(7) provides a mechanism for organizing persistent objects in user-defined, persistent, atomic, circular, doubly-linked lists. All the routines and macros operating on the persistent lists provide atomicity with respect to any power-fail interruptions. If any of those operations is torn by program failure or system crash, on recovery they are guaranteed to be entirely completed or discarded, leaving the lists, persistent memory heap and internal object containers in a consistent state. The persistent atomic circular doubly linked lists support the following functionality: + Insertion of an object at the head of the list, or at the end of the list. + Insertion of an object before or after any element in the list. + Atomic allocation and insertion of a new object at the head of the list, or at the end of the list. + Atomic allocation and insertion of a new object before or after any element in the list. + Atomic moving of an element from one list to the specific location on another list. + Removal of any object in the list. + Atomic removal and freeing of any object in the list. + Forward or backward traversal through the list. A list is headed by a *list_head* structure containing the object handle of the first element on the list. The elements are doubly linked so that an arbitrary element can be removed without the need to traverse the list. New elements can be added to the list before or after an existing element, at the head of the list, or at the tail of the list. A list may be traversed in either direction. The user-defined structure of each element must contain a field of type *list_entry* that holds the object handles to the previous and next element on the list. Both the *list_head* and the *list_entry* structures are declared in **\**. The functions below are intended to be used outside transactions - transactional variants are described in manpages to functions mentioned at **TRANSACTIONAL OBJECT MANIPULATION** in **libpmemobj**(7). Note that operations performed using this non-transactional API are independent from their transactional counterparts. If any non-transactional allocations or list manipulations are performed within an open transaction, the changes will not be rolled back if such a transaction is aborted or interrupted. The list insertion and move functions use a common set of arguments to define where an object will be inserted into the list. *dest* identifies the element before or after which the object will be inserted, or, if *dest* is **OID_NULL**, indicates that the object should be inserted at the head or tail of the list. *before* determines where the object will be inserted: + **POBJ_LIST_DEST_BEFORE** - insert the element before the existing element *dest* + **POBJ_LIST_DEST_AFTER** - insert the element after the existing element *dest* + **POBJ_LIST_DEST_HEAD** - when *dest* is **OID_NULL**, insert the element at the head of the list + **POBJ_LIST_DEST_TAIL** - when *dest* is **OID_NULL**, insert the element at the tail of the list >NOTE: Earlier versions of **libpmemobj**(7) do not define **POBJ_LIST_DEST_BEFORE** and **POBJ_LIST_DEST_AFTER**. Use 1 for before, and 0 for after. The **pmemobj_list_insert**() function inserts the element represented by object handle *oid* into the list referenced by *head*, at the location specified by *dest* and *before* as described above. *pe_offset* specifies the offset of the structure that connects the elements in the list. All the handles *head*, *dest* and *oid* must point to objects allocated from memory pool *pop*. *head* and *oid* cannot be **OID_NULL**. The **pmemobj_list_insert_new**() function atomically allocates a new object of given *size* and type *type_num* and inserts it into the list referenced by *head* at the location specified by *dest* and *before* as described above. *pe_offset* specifies the offset of the structure that connects the elements in the list. The handles *head* and *dest* must point to objects allocated from memory pool *pop*. Before returning, **pmemobj_list_insert_new**() calls the *constructor* function, passing the pool handle *pop*, the pointer to the newly allocated object *ptr*, and the *arg* argument. It is guaranteed that the allocated object is either properly initialized or, if the allocation is interrupted before the constructor completes, the memory space reserved for the object is reclaimed. *head* cannot be **OID_NULL**. The allocated object is also added to the internal container associated with *type_num*, as described in **POBJ_FOREACH**(3). The **pmemobj_list_move**() function moves the object represented by object handle *oid* from the list referenced by *head_old* to the list referenced by *head_new*, inserting it at the location specified by *dest* and *before* as described above. *pe_old_offset* and *pe_new_offset* specify the offsets of the structures that connect the elements in the old and new lists, respectively. All the handles *head_old*, *head_new*, *dest* and *oid* must point to objects allocated from memory pool *pop*. *head_old*, *head_new* and *oid* cannot be **OID_NULL**. The **pmemobj_list_remove**() function removes the object represented by object handle *oid* from the list referenced by *head*. If *free* is set, it also removes the object from the internal object container and frees the associated memory space. *pe_offset* specifies the offset of the structure that connects the elements in the list. Both *head* and *oid* must point to objects allocated from memory pool *pop* and cannot be **OID_NULL**. # RETURN VALUE # On success, **pmemobj_list_insert**(), **pmemobj_list_remove**() and **pmemobj_list_move**() return 0. On error, they return -1 and set *errno* appropriately. On success, **pmemobj_list_insert_new**() returns a handle to the newly allocated object. If the constructor returns a non-zero value, the allocation is canceled, -1 is returned, and *errno* is set to **ECANCELED**. On other errors, **OID_NULL** is returned and *errno* is set appropriately. # SEE ALSO # **pmemobj_first**(3), **POBJ_FOREACH**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_realloc.30000664000000000000000000000002413615011243017110 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_defrag.30000664000000000000000000000002413615011243016717 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_f_mem_nodrain.30000664000000000000000000000003513615011243020266 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/toid_valid.30000664000000000000000000000002313615011243016073 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pmemobj_root.3.md0000664000000000000000000001320313615011243017054 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_ROOT, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_root.3 -- man page for root object management) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemobj_root**(), **pmemobj_root_construct**() **POBJ_ROOT**(), **pmemobj_root_size**() - root object management # SYNOPSIS # ```c #include PMEMoid pmemobj_root(PMEMobjpool *pop, size_t size); PMEMoid pmemobj_root_construct(PMEMobjpool *pop, size_t size, pmemobj_constr constructor, void *arg); POBJ_ROOT(PMEMobjpool *pop, TYPE) size_t pmemobj_root_size(PMEMobjpool *pop); ``` # DESCRIPTION # The root object of a persistent memory pool is an entry point for all other persistent objects allocated using the **libpmemobj** API. In other words, every object stored in the persistent memory pool has the root object at the end of its reference path. It may be assumed that for each persistent memory pool the root object always exists, and there is exactly one root object in each pool. The **pmemobj_root**() function creates or resizes the root object for the persistent memory pool *pop*. If this is the first call to **pmemobj_root**(), the requested *size* is greater than zero and the root object does not exist, it is implicitly allocated in a thread-safe manner, so the function may be called by more than one thread simultaneously (as long as all threads use the identical *size* value). The size of the root object is guaranteed to be not less than the requested *size*. If the requested size is larger than the current size, the root object is automatically resized. In such case, the old data is preserved and the extra space is zeroed. If the requested size is equal to or smaller than the current size, the root object remains unchanged. If the requested *size* is equal to zero, the root object is not allocated. **pmemobj_root_construct**() performs the same actions as **pmemobj_root**(), but instead of zeroing the newly allocated object a *constructor* function is called to initialize the object. The constructor is also called on reallocations. The **POBJ_ROOT**() macro works the same way as the **pmemobj_root**() function except it returns a typed *OID* value. The **pmemobj_root_size**() function returns the current size of the root object associated with the persistent memory pool *pop*. # RETURN VALUE # Upon success, **pmemobj_root**() returns a handle to the root object associated with the persistent memory pool *pop*. The same root object handle is returned in all the threads. If the requested object size is larger than the maximum allocation size supported for the pool, or if there is not enough free space in the pool to satisfy a reallocation request, **pmemobj_root**() returns **OID_NULL** and sets *errno* to ENOMEM. If the *size* was equal to zero and the root object has not been allocated, **pmemobj_root**() returns **OID_NULL** and sets *errno* to EINVAL. If the **pmemobj_root_construct**() constructor fails, the allocation is canceled, **pmemobj_root_construct**() returns *OID_NULL*, and *errno* is set to **ECANCELED**. **pmemobj_root_size**() can be used in the constructor to check whether this is the first call to the constructor. **POBJ_ROOT**() returns a typed *OID* of type *TYPE* instead of the *PMEMoid* returned by **pmemobj_root**(). The **pmemobj_root_size**() function returns the current size of the root object associated with the persistent memory pool *pop*. The returned size is the largest value requested by any of the earlier **pmemobj_root**() calls. If the root object has not been allocated yet, **pmemobj_root_size**() returns 0. # SEE ALSO # **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_memmove.30000664000000000000000000000003513615011243017136 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/tx_begin_cb.30000664000000000000000000000002713615011243016224 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_mutex_trylock.30000664000000000000000000000003113615011243020376 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/tx_strdup.30000664000000000000000000000002713615011243016015 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_f_mem_temporal.30000664000000000000000000000003513615011243020457 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/d_ro.30000664000000000000000000000002313615011243014700 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/d_rw.30000664000000000000000000000002313615011243014710 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/tx_realloc.30000664000000000000000000000002713615011243016115 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_list_remove.30000664000000000000000000000003213615011243020016 0ustar rootroot.so pmemobj_list_insert.3 pmdk-1.8/doc/libpmemobj/pobj_list_insert_new_head.30000664000000000000000000000002513615011243021162 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_f_mem_wc.30000664000000000000000000000003513615011243017245 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_xlog_append_buffer.30000664000000000000000000000002713615011243022036 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/toid_declare.3.md0000664000000000000000000001257313615011243017007 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(TOID_DECLARE, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (toid_declare.3 -- man page for obj type safety mechanism) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[SEE ALSO](#see-also)
# NAME # **TOID_DECLARE**(), **TOID_DECLARE_ROOT**(), **TOID**(), **TOID_TYPE_NUM**(), **TOID_TYPE_NUM_OF**(), **TOID_VALID**(), **OID_INSTANCEOF**(), **TOID_ASSIGN**(), **TOID_IS_NULL**(), **TOID_EQUALS**(), **TOID_TYPEOF**(), **TOID_OFFSETOF**(), **DIRECT_RW**(), **D_RW**(), **DIRECT_RO**(), **D_RO**() - libpmemobj type safety mechanism # SYNOPSIS # ```c #include TOID_DECLARE(TYPE, uint64_t type_num) TOID_DECLARE_ROOT(ROOT_TYPE) TOID(TYPE) TOID_TYPE_NUM(TYPE) TOID_TYPE_NUM_OF(TOID oid) TOID_VALID(TOID oid) OID_INSTANCEOF(PMEMoid oid, TYPE) TOID_ASSIGN(TOID o, VALUE) TOID_IS_NULL(TOID o) TOID_EQUALS(TOID lhs, TOID rhs) TOID_TYPEOF(TOID o) TOID_OFFSETOF(TOID o, FILED) DIRECT_RW(TOID oid) D_RW(TOID oid) DIRECT_RO(TOID oid) D_RO(TOID oid) ``` # DESCRIPTION # Operating on untyped object handles, as well as on direct untyped object pointers (*void\**), may be confusing and error-prone. To facilitate type safety, **libpmemobj**(7) defines a set of macros that provide static type enforcement, catching potential errors at compile time. For example, a compile-time error is generated when an attempt is made to assign a handle to an object of one type to the object handle variable of another type of object. The **TOID_DECLARE**() macro declares a typed *OID* of user-defined type *TYPE* and type number *type_num*. The **TOID_DECLARE_ROOT**() macro declares a typed *OID* of user-defined type *ROOT_TYPE* and root object type number **POBJ_ROOT_TYPE_NUM**. The **TOID**() macro declares a handle to an object of type *TYPE*, where *TYPE* is the name of a user-defined structure. The typed *OID* must be declared first using the **TOID_DECLARE**(), **TOID_DECLARE_ROOT**(), **POBJ_LAYOUT_TOID**(3) or **POBJ_LAYOUT_ROOT**(3) macros. The **TOID_TYPE_NUM**() macro returns the type number of the type specified by *TYPE*. The **TOID_TYPE_NUM_OF**() macro returns the type number of the object specified by *oid*. The type number is read from the typed *OID*. The **TOID_VALID**() macro validates whether the type number stored in the object's metadata is equal to the type number read from the typed *OID*. The **OID_INSTANCEOF**() macro checks whether the *oid* is of type *TYPE*. The **TOID_ASSIGN**() macro assigns the object handle *VALUE* to typed *OID* *o*. The **TOID_IS_NULL**() macro evaluates to true if the object handle represented by *o* is **OID_NULL**. The **TOID_EQUALS**() macro evaluates to true if both the *lhs* and *rhs* object handles reference the same persistent object. The **TOID_TYPEOF**() macro returns the type of the object handle represented by typed *OID* *o*. The **TOID_OFFSETOF**() macro returns the offset of the *FIELD* member from the start of the object represented by *o*. The **DIRECT_RW**() macro and its shortened form **D_RW**() return a typed write pointer (*TYPE\**) to an object represented by *oid*. If *oid* is **OID_NULL**, the macro evaluates to NULL. The **DIRECT_RO**() macro and its shortened form **D_RO**() return a typed read-only (const) pointer (*TYPE\**) to an object represented by *oid*. If *oid* is **OID_NULL**, the macro evaluates to NULL. # SEE ALSO # **OID_IS_NULL**(3), **POBJ_LAYOUT_ROOT**(3), **POBJ_LAYOUT_TOID**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_tx_lock.30000664000000000000000000000002713615011243017135 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_process.30000664000000000000000000000002713615011243017663 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_list_insert_new.30000664000000000000000000000003213615011243020676 0ustar rootroot.so pmemobj_list_insert.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_add_range.30000644000000000000000000002416113615011417020117 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_TX_ADD_RANGE" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_tx_add_range\f[](), \f[B]pmemobj_tx_add_range_direct\f[](), \f[B]pmemobj_tx_xadd_range\f[](), \f[B]pmemobj_tx_xadd_range_direct\f[]() .PP \f[B]TX_ADD\f[](), \f[B]TX_ADD_FIELD\f[](), \f[B]TX_ADD_DIRECT\f[](), \f[B]TX_ADD_FIELD_DIRECT\f[](), .PP \f[B]TX_XADD\f[](), \f[B]TX_XADD_FIELD\f[](), \f[B]TX_XADD_DIRECT\f[](), \f[B]TX_XADD_FIELD_DIRECT\f[](), .PP \f[B]TX_SET\f[](), \f[B]TX_SET_DIRECT\f[](), \f[B]TX_MEMCPY\f[](), \f[B]TX_MEMSET\f[]() \- transactional object manipulation .SH SYNOPSIS .IP .nf \f[C] #include\ int\ pmemobj_tx_add_range(PMEMoid\ oid,\ uint64_t\ off,\ size_t\ size); int\ pmemobj_tx_add_range_direct(const\ void\ *ptr,\ size_t\ size); int\ pmemobj_tx_xadd_range(PMEMoid\ oid,\ uint64_t\ off,\ size_t\ size,\ uint64_t\ flags); int\ pmemobj_tx_xadd_range_direct(const\ void\ *ptr,\ size_t\ size,\ uint64_t\ flags); TX_ADD(TOID\ o) TX_ADD_FIELD(TOID\ o,\ FIELD) TX_ADD_DIRECT(TYPE\ *p) TX_ADD_FIELD_DIRECT(TYPE\ *p,\ FIELD) TX_XADD(TOID\ o,\ uint64_t\ flags) TX_XADD_FIELD(TOID\ o,\ FIELD,\ uint64_t\ flags) TX_XADD_DIRECT(TYPE\ *p,\ uint64_t\ flags) TX_XADD_FIELD_DIRECT(TYPE\ *p,\ FIELD,\ uint64_t\ flags) TX_SET(TOID\ o,\ FIELD,\ VALUE) TX_SET_DIRECT(TYPE\ *p,\ FIELD,\ VALUE) TX_MEMCPY(void\ *dest,\ const\ void\ *src,\ size_t\ num) TX_MEMSET(void\ *dest,\ int\ c,\ size_t\ num) \f[] .fi .SH DESCRIPTION .PP \f[B]pmemobj_tx_add_range\f[]() takes a \[lq]snapshot\[rq] of the memory block of given \f[I]size\f[], located at given offset \f[I]off\f[] in the object specified by \f[I]oid\f[], and saves it to the undo log. The application is then free to directly modify the object in that memory range. In case of a failure or abort, all the changes within this range will be rolled back. The supplied block of memory has to be within the pool registered in the transaction. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_xadd_range\f[]() function behaves exactly the same as \f[B]pmemobj_tx_add_range\f[]() when \f[I]flags\f[] equals zero. \f[I]flags\f[] is a bitmask of the following values: .IP \[bu] 2 \f[B]POBJ_XADD_NO_FLUSH\f[] \- skip flush on commit (when application deals with flushing or uses pmemobj_memcpy_persist) .IP \[bu] 2 \f[B]POBJ_XADD_NO_SNAPSHOT\f[] \- added range will not be \[lq]snapshotted\[rq], i.e.\ any changes made within it during the transaction will not be rolled backed after abort .IP \[bu] 2 \f[B]POBJ_XADD_ASSUME_INITIALIZED\f[] \- added range is assumed to be initialized. If this flag is not specified, passing uninitialized memory will result in an error when run under Valgrind memcheck. .IP \[bu] 2 \f[B]POBJ_XADD_NO_ABORT\f[] \- if the function does not end successfully, do not abort the transaction. .PP \f[B]pmemobj_tx_add_range_direct\f[]() behaves the same as \f[B]pmemobj_tx_add_range\f[]() with the exception that it operates on virtual memory addresses and not persistent memory objects. It takes a \[lq]snapshot\[rq] of a persistent memory block of given \f[I]size\f[], located at the given address \f[I]ptr\f[] in the virtual memory space and saves it to the undo log. The application is then free to directly modify the object in that memory range. In case of a failure or abort, all the changes within this range will be rolled back. The supplied block of memory has to be within the pool registered in the transaction. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_xadd_range_direct\f[]() function behaves exactly the same as \f[B]pmemobj_tx_add_range_direct\f[]() when \f[I]flags\f[] equals zero. \f[I]flags\f[] is a bitmask of the following values: .IP \[bu] 2 \f[B]POBJ_XADD_NO_FLUSH\f[] \- skip flush on commit (when application deals with flushing or uses pmemobj_memcpy_persist) .IP \[bu] 2 \f[B]POBJ_XADD_NO_SNAPSHOT\f[] \- added range will not be \[lq]snapshotted\[rq], i.e.\ any changes made within it during the transaction will not be rolled backed after abort .IP \[bu] 2 \f[B]POBJ_XADD_ASSUME_INITIALIZED\f[] \- added range is assumed to be initialized. If this flag is not specified, passing uninitialized memory will result in an error when run under Valgrind memcheck. .IP \[bu] 2 \f[B]POBJ_XADD_NO_ABORT\f[] \- if the function does not end successfully, do not abort the transaction. .PP Similarly to the macros controlling the transaction flow, \f[B]libpmemobj\f[] defines a set of macros that simplify the transactional operations on persistent objects. Note that those macros operate on typed object handles, thus eliminating the need to specify the size of the object, or the size and offset of the field in the user\-defined structure that is to be modified. .PP The \f[B]TX_ADD_FIELD\f[]() macro saves the current value of given \f[I]FIELD\f[] of the object referenced by a handle \f[I]o\f[] in the undo log. The application is then free to directly modify the specified \f[I]FIELD\f[]. In case of a failure or abort, the saved value will be restored. .PP The \f[B]TX_XADD_FIELD\f[]() macro works exactly like \f[B]TX_ADD_FIELD\f[] when \f[I]flags\f[] equals 0. The \f[I]flags\f[] argument is a bitmask of values described in \f[B]pmemobj_tx_xadd_range\f[], above. .PP The \f[B]TX_ADD\f[]() macro takes a \[lq]snapshot\[rq] of the entire object referenced by object handle \f[I]o\f[] and saves it in the undo log. The object size is determined from its \f[I]TYPE\f[]. The application is then free to directly modify the object. In case of a failure or abort, all the changes within the object will be rolled back. .PP The \f[B]TX_XADD\f[]() macro works exactly like \f[B]TX_ADD\f[] when \f[I]flags\f[] equals 0. The \f[I]flags\f[] argument is a bitmask of values as described in \f[B]pmemobj_tx_xadd_range\f[], above. .PP The \f[B]TX_ADD_FIELD_DIRECT\f[]() macro saves the current value of the given \f[I]FIELD\f[] of the object referenced by (direct) pointer \f[I]p\f[] in the undo log. The application is then free to directly modify the specified \f[I]FIELD\f[]. In case of a failure or abort, the saved value will be restored. .PP The \f[B]TX_XADD_FIELD_DIRECT\f[]() macro works exactly like \f[B]TX_ADD_FIELD_DIRECT\f[] when \f[I]flags\f[] equals 0. The \f[I]flags\f[] argument is a bitmask of values as described in \f[B]pmemobj_tx_xadd_range_direct\f[], above. .PP The \f[B]TX_ADD_DIRECT\f[]() macro takes a \[lq]snapshot\[rq] of the entire object referenced by (direct) pointer \f[I]p\f[] and saves it in the undo log. The object size is determined from its \f[I]TYPE\f[]. The application is then free to directly modify the object. In case of a failure or abort, all the changes within the object will be rolled back. .PP The \f[B]TX_XADD_DIRECT\f[]() macro works exactly like \f[B]TX_ADD_DIRECT\f[] when \f[I]flags\f[] equals 0. The \f[I]flags\f[] argument is a bitmask of values as described in \f[B]pmemobj_tx_xadd_range_direct\f[], above. .PP The \f[B]TX_SET\f[]() macro saves the current value of the given \f[I]FIELD\f[] of the object referenced by handle \f[I]o\f[] in the undo log, and then sets its new \f[I]VALUE\f[]. In case of a failure or abort, the saved value will be restored. .PP The \f[B]TX_SET_DIRECT\f[]() macro saves in the undo log the current value of given \f[I]FIELD\f[] of the object referenced by (direct) pointer \f[I]p\f[], and then set its new \f[I]VALUE\f[]. In case of a failure or abort, the saved value will be restored. .PP The \f[B]TX_MEMCPY\f[]() macro saves in the undo log the current content of \f[I]dest\f[] buffer and then overwrites the first \f[I]num\f[] bytes of its memory area with the data copied from the buffer pointed by \f[I]src\f[]. In case of a failure or abort, the saved value will be restored. .PP The \f[B]TX_MEMSET\f[]() macro saves the current content of the \f[I]dest\f[] buffer in the undo log and then fills the first \f[I]num\f[] bytes of its memory area with the constant byte \f[I]c\f[]. In case of a failure or abort, the saved value will be restored. .SH RETURN VALUE .PP On success, \f[B]pmemobj_tx_add_range\f[]() and \f[B]pmemobj_tx_add_range_direct\f[]() return 0. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]errno\f[] is set appropriately and transaction is aborted. .PP On success, \f[B]pmemobj_tx_xadd_range\f[]() and \f[B]pmemobj_tx_xadd_range_direct\f[]() returns 0. Otherwise, the error number is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XADD_NO_ABORT\f[], the transaction is aborted. .SH SEE ALSO .PP \f[B]pmemobj_tx_alloc\f[](3), \f[B]pmemobj_tx_begin\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_tx_xadd_range_direct.30000664000000000000000000000003313615011243021630 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pobj_free.30000664000000000000000000000002413615011243015711 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_root.30000644000000000000000000001275313615011417016467 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_ROOT" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_root\f[](), \f[B]pmemobj_root_construct\f[]() \f[B]POBJ_ROOT\f[](), \f[B]pmemobj_root_size\f[]() \- root object management .SH SYNOPSIS .IP .nf \f[C] #include\ PMEMoid\ pmemobj_root(PMEMobjpool\ *pop,\ size_t\ size); PMEMoid\ pmemobj_root_construct(PMEMobjpool\ *pop,\ size_t\ size, \ \ \ \ pmemobj_constr\ constructor,\ void\ *arg); POBJ_ROOT(PMEMobjpool\ *pop,\ TYPE) size_t\ pmemobj_root_size(PMEMobjpool\ *pop); \f[] .fi .SH DESCRIPTION .PP The root object of a persistent memory pool is an entry point for all other persistent objects allocated using the \f[B]libpmemobj\f[] API. In other words, every object stored in the persistent memory pool has the root object at the end of its reference path. It may be assumed that for each persistent memory pool the root object always exists, and there is exactly one root object in each pool. .PP The \f[B]pmemobj_root\f[]() function creates or resizes the root object for the persistent memory pool \f[I]pop\f[]. If this is the first call to \f[B]pmemobj_root\f[](), the requested \f[I]size\f[] is greater than zero and the root object does not exist, it is implicitly allocated in a thread\-safe manner, so the function may be called by more than one thread simultaneously (as long as all threads use the identical \f[I]size\f[] value). The size of the root object is guaranteed to be not less than the requested \f[I]size\f[]. If the requested size is larger than the current size, the root object is automatically resized. In such case, the old data is preserved and the extra space is zeroed. If the requested size is equal to or smaller than the current size, the root object remains unchanged. If the requested \f[I]size\f[] is equal to zero, the root object is not allocated. .PP \f[B]pmemobj_root_construct\f[]() performs the same actions as \f[B]pmemobj_root\f[](), but instead of zeroing the newly allocated object a \f[I]constructor\f[] function is called to initialize the object. The constructor is also called on reallocations. .PP The \f[B]POBJ_ROOT\f[]() macro works the same way as the \f[B]pmemobj_root\f[]() function except it returns a typed \f[I]OID\f[] value. .PP The \f[B]pmemobj_root_size\f[]() function returns the current size of the root object associated with the persistent memory pool \f[I]pop\f[]. .SH RETURN VALUE .PP Upon success, \f[B]pmemobj_root\f[]() returns a handle to the root object associated with the persistent memory pool \f[I]pop\f[]. The same root object handle is returned in all the threads. If the requested object size is larger than the maximum allocation size supported for the pool, or if there is not enough free space in the pool to satisfy a reallocation request, \f[B]pmemobj_root\f[]() returns \f[B]OID_NULL\f[] and sets \f[I]errno\f[] to ENOMEM. If the \f[I]size\f[] was equal to zero and the root object has not been allocated, \f[B]pmemobj_root\f[]() returns \f[B]OID_NULL\f[] and sets \f[I]errno\f[] to EINVAL. .PP If the \f[B]pmemobj_root_construct\f[]() constructor fails, the allocation is canceled, \f[B]pmemobj_root_construct\f[]() returns \f[I]OID_NULL\f[], and \f[I]errno\f[] is set to \f[B]ECANCELED\f[]. \f[B]pmemobj_root_size\f[]() can be used in the constructor to check whether this is the first call to the constructor. .PP \f[B]POBJ_ROOT\f[]() returns a typed \f[I]OID\f[] of type \f[I]TYPE\f[] instead of the \f[I]PMEMoid\f[] returned by \f[B]pmemobj_root\f[](). .PP The \f[B]pmemobj_root_size\f[]() function returns the current size of the root object associated with the persistent memory pool \f[I]pop\f[]. The returned size is the largest value requested by any of the earlier \f[B]pmemobj_root\f[]() calls. If the root object has not been allocated yet, \f[B]pmemobj_root_size\f[]() returns 0. .SH SEE ALSO .PP \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_memset.30000664000000000000000000000003513615011243016763 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pobj_znew.30000664000000000000000000000002413615011243015753 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_root.30000664000000000000000000000002313615011243015752 0ustar rootroot.so pmemobj_root.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_set_user_data.30000664000000000000000000000002713615011243021027 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pobj_list_head.30000644000000000000000000003135313615011417016736 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "POBJ_LIST_HEAD" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]POBJ_LIST_HEAD\f[](), \f[B]POBJ_LIST_ENTRY\f[](), \f[B]POBJ_LIST_FIRST\f[](), \f[B]POBJ_LIST_LAST\f[](), \f[B]POBJ_LIST_EMPTY\f[](), \f[B]POBJ_LIST_NEXT\f[](), \f[B]POBJ_LIST_PREV\f[](), .PP \f[B]POBJ_LIST_FOREACH\f[](), \f[B]POBJ_LIST_FOREACH_REVERSE\f[](), .PP \f[B]POBJ_LIST_INSERT_HEAD\f[](), \f[B]POBJ_LIST_INSERT_TAIL\f[](), \f[B]POBJ_LIST_INSERT_AFTER\f[](), \f[B]POBJ_LIST_INSERT_BEFORE\f[](), \f[B]POBJ_LIST_INSERT_NEW_HEAD\f[](), \f[B]POBJ_LIST_INSERT_NEW_TAIL\f[](), \f[B]POBJ_LIST_INSERT_NEW_AFTER\f[](), \f[B]POBJ_LIST_INSERT_NEW_BEFORE\f[](), .PP \f[B]POBJ_LIST_REMOVE\f[](), \f[B]POBJ_LIST_REMOVE_FREE\f[](), .PP \f[B]POBJ_LIST_MOVE_ELEMENT_HEAD\f[](), \f[B]POBJ_LIST_MOVE_ELEMENT_TAIL\f[](), \f[B]POBJ_LIST_MOVE_ELEMENT_AFTER\f[](), \f[B]POBJ_LIST_MOVE_ELEMENT_BEFORE\f[]() \- type\-safe non\-transactional persistent atomic lists .SH SYNOPSIS .IP .nf \f[C] #include\ POBJ_LIST_HEAD(HEADNAME,\ TYPE) POBJ_LIST_ENTRY(TYPE) POBJ_LIST_FIRST(POBJ_LIST_HEAD\ *head) POBJ_LIST_LAST(POBJ_LIST_HEAD\ *head,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_EMPTY(POBJ_LIST_HEAD\ *head) POBJ_LIST_NEXT(TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_PREV(TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_FOREACH(TOID\ var,\ POBJ_LIST_HEAD\ *head,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_FOREACH_REVERSE(TOID\ var,\ POBJ_LIST_HEAD\ *head,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_INSERT_HEAD(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_INSERT_TAIL(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_INSERT_AFTER(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ TOID\ listelm,\ TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_INSERT_BEFORE(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ TOID\ listelm,\ TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_INSERT_NEW_HEAD(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ POBJ_LIST_ENTRY\ FIELD,\ size_t\ size, \ \ \ \ pmemobj_constr\ constructor,\ void\ *arg) POBJ_LIST_INSERT_NEW_TAIL(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ POBJ_LIST_ENTRY\ FIELD,\ size_t\ size, \ \ \ \ pmemobj_constr\ constructor,\ void\ *arg) POBJ_LIST_INSERT_NEW_AFTER(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ TOID\ listelm,\ POBJ_LIST_ENTRY\ FIELD,\ size_t\ size, \ \ \ \ pmemobj_constr\ constructor,\ void\ *arg) POBJ_LIST_INSERT_NEW_BEFORE(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ TOID\ listelm,\ POBJ_LIST_ENTRY\ FIELD,\ size_t\ size, \ \ \ \ pmemobj_constr\ constructor,\ void\ *arg) POBJ_LIST_REMOVE(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_REMOVE_FREE(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD) POBJ_LIST_MOVE_ELEMENT_HEAD(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ POBJ_LIST_HEAD\ *head_new,\ TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD, \ \ \ \ POBJ_LIST_ENTRY\ field_new) POBJ_LIST_MOVE_ELEMENT_TAIL(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ POBJ_LIST_HEAD\ *head_new,\ TOID\ elm,\ POBJ_LIST_ENTRY\ FIELD, \ \ \ \ POBJ_LIST_ENTRY\ field_new) POBJ_LIST_MOVE_ELEMENT_AFTER(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ POBJ_LIST_HEAD\ *head_new,\ TOID\ listelm,\ TOID\ elm, \ \ \ \ POBJ_LIST_ENTRY\ FIELD,\ POBJ_LIST_ENTRY\ field_new) POBJ_LIST_MOVE_ELEMENT_BEFORE(PMEMobjpool\ *pop,\ POBJ_LIST_HEAD\ *head, \ \ \ \ POBJ_LIST_HEAD\ *head_new,\ TOID\ listelm,\ TOID\ elm, \ \ \ \ POBJ_LIST_ENTRY\ FIELD,\ POBJ_LIST_ENTRY\ field_new) \f[] .fi .SH DESCRIPTION .PP The following macros define and operate on a type\-safe persistent atomic circular doubly linked list data structure that consist of a set of persistent objects of a well\-known type. Unlike the functions described in the previous section, these macros provide type enforcement by requiring declaration of type of the objects stored in given list, and not allowing mixing objects of different types in a single list. .PP The functionality and semantics of those macros is similar to circular queues defined in \f[B]queue\f[](3). .PP The majority of the macros must specify the handle to the memory pool \f[I]pop\f[] and the name of the \f[I]field\f[] in the user\-defined structure, which must be of type \f[I]POBJ_LIST_ENTRY\f[] and is used to connect the elements in the list. .PP A list is headed by a structure defined by the \f[B]POBJ_LIST_HEAD\f[]() macro. This structure contains an object handle of the first element on the list. The elements are doubly linked so that an arbitrary element can be removed without a need to traverse the list. New elements can be added to the list before or after an existing element, at the head of the list, or at the end of the list. A list may be traversed in either direction. A \f[I]POBJ_LIST_HEAD\f[] structure is declared as follows: .IP .nf \f[C] #define\ POBJ_LIST_HEAD(HEADNAME,\ TYPE) struct\ HEADNAME { \ \ \ \ TOID(TYPE)\ pe_first; \ \ \ \ PMEMmutex\ lock; }; \f[] .fi .PP In the macro definitions, \f[I]TYPE\f[] is the name of a user\-defined structure, that must contain a field of type \f[I]POBJ_LIST_ENTRY\f[]. The argument \f[I]HEADNAME\f[] is the name of a user\-defined structure that must be declared using the macro \f[I]POBJ_LIST_HEAD\f[]. See the examples below for further explanation of how these macros are used. .PP The macro \f[I]POBJ_LIST_ENTRY\f[] declares a structure that connects the elements in the list. .IP .nf \f[C] #define\ POBJ_LIST_ENTRY(TYPE) struct { \ \ \ \ TOID(TYPE)\ pe_next; \ \ \ \ TOID(TYPE)\ pe_prev; }; \f[] .fi .PP The macro \f[B]POBJ_LIST_FIRST\f[]() returns the first element on the list referenced by \f[I]head\f[]. If the list is empty \f[B]OID_NULL\f[] is returned. .PP The macro \f[B]POBJ_LIST_LAST\f[]() returns the last element on the list referenced by \f[I]head\f[]. If the list is empty \f[B]OID_NULL\f[] is returned. .PP The macro \f[B]POBJ_LIST_EMPTY\f[]() evaluates to 1 if the list referenced by \f[I]head\f[] is empty. Otherwise, 0 is returned. .PP The macro \f[B]POBJ_LIST_NEXT\f[]() returns the element next to the element \f[I]elm\f[]. .PP The macro \f[B]POBJ_LIST_PREV\f[]() returns the element preceding the element \f[I]elm\f[]. .PP The macro \f[B]POBJ_LIST_FOREACH\f[]() traverses the list referenced by \f[I]head\f[] assigning a handle to each element in turn to \f[I]var\f[] variable. .PP The macro \f[B]POBJ_LIST_FOREACH_REVERSE\f[]() traverses the list referenced by \f[I]head\f[] in reverse order, assigning a handle to each element in turn to \f[I]var\f[] variable. The \f[I]field\f[] argument is the name of the field of type \f[I]POBJ_LIST_ENTRY\f[] in the element structure. .PP The macro \f[B]POBJ_LIST_INSERT_HEAD\f[]() inserts the element \f[I]elm\f[] at the head of the list referenced by \f[I]head\f[]. .PP The macro \f[B]POBJ_LIST_INSERT_TAIL\f[]() inserts the element \f[I]elm\f[] at the end of the list referenced by \f[I]head\f[]. .PP The macro \f[B]POBJ_LIST_INSERT_AFTER\f[]() inserts the element \f[I]elm\f[] into the list referenced by \f[I]head\f[] after the element \f[I]listelm\f[]. If \f[I]listelm\f[] value is \f[B]TOID_NULL\f[], the object is inserted at the end of the list. .PP The macro \f[B]POBJ_LIST_INSERT_BEFORE\f[]() inserts the element \f[I]elm\f[] into the list referenced by \f[I]head\f[] before the element \f[I]listelm\f[]. If \f[I]listelm\f[] value is \f[B]TOID_NULL\f[], the object is inserted at the head of the list. .PP The macro \f[B]POBJ_LIST_INSERT_NEW_HEAD\f[]() atomically allocates a new object of size \f[I]size\f[] and inserts it at the head of the list referenced by \f[I]head\f[]. The newly allocated object is also added to the internal object container associated with a type number which is retrieved from the typed \f[I]OID\f[] of the first element on list. .PP The macro \f[B]POBJ_LIST_INSERT_NEW_TAIL\f[]() atomically allocates a new object of size \f[I]size\f[] and inserts it at the tail of the list referenced by \f[I]head\f[]. The newly allocated object is also added to the internal object container associated with a type number which is retrieved from the typed \f[I]OID\f[] of the first element on list. .PP The macro \f[B]POBJ_LIST_INSERT_NEW_AFTER\f[]() atomically allocates a new object of size \f[I]size\f[] and inserts it into the list referenced by \f[I]head\f[] after the element \f[I]listelm\f[]. If \f[I]listelm\f[] value is \f[B]TOID_NULL\f[], the object is inserted at the end of the list. The newly allocated object is also added to the internal object container associated with with a type number which is retrieved from the typed \f[I]OID\f[] of the first element on list. .PP The macro \f[B]POBJ_LIST_INSERT_NEW_BEFORE\f[]() atomically allocates a new object of size \f[I]size\f[] and inserts it into the list referenced by \f[I]head\f[] before the element \f[I]listelm\f[]. If \f[I]listelm\f[] value is \f[B]TOID_NULL\f[], the object is inserted at the head of the list. The newly allocated object is also added to the internal object container associated with with a type number which is retrieved from the typed \f[I]OID\f[] of the first element on list. .PP The macro \f[B]POBJ_LIST_REMOVE\f[]() removes the element \f[I]elm\f[] from the list referenced by \f[I]head\f[]. .PP The macro \f[B]POBJ_LIST_REMOVE_FREE\f[]() removes the element \f[I]elm\f[] from the list referenced by \f[I]head\f[] and frees the memory space represented by this element. .PP The macro \f[B]POBJ_LIST_MOVE_ELEMENT_HEAD\f[]() moves the element \f[I]elm\f[] from the list referenced by \f[I]head\f[] to the head of the list \f[I]head_new\f[]. The \f[I]field\f[] and \f[I]field_new\f[] arguments are the names of the fields of type \f[I]POBJ_LIST_ENTRY\f[] in the element structure that are used to connect the elements in both lists. .PP The macro \f[B]POBJ_LIST_MOVE_ELEMENT_TAIL\f[]() moves the element \f[I]elm\f[] from the list referenced by \f[I]head\f[] to the end of the list \f[I]head_new\f[]. The \f[I]field\f[] and \f[I]field_new\f[] arguments are the names of the fields of type \f[I]POBJ_LIST_ENTRY\f[] in the element structure that are used to connect the elements in both lists. .PP The macro \f[B]POBJ_LIST_MOVE_ELEMENT_AFTER\f[]() atomically removes the element \f[I]elm\f[] from the list referenced by \f[I]head\f[] and inserts it into the list referenced by \f[I]head_new\f[] after the element \f[I]listelm\f[]. If \f[I]listelm\f[] value is \f[I]TOID_NULL\f[], the object is inserted at the end of the list. The \f[I]field\f[] and \f[I]field_new\f[] arguments are the names of the fields of type \f[I]POBJ_LIST_ENTRY\f[] in the element structure that are used to connect the elements in both lists. .PP The macro \f[B]POBJ_LIST_MOVE_ELEMENT_BEFORE\f[]() atomically removes the element \f[I]elm\f[] from the list referenced by \f[I]head\f[] and inserts it into the list referenced by \f[I]head_new\f[] before the element \f[I]listelm\f[]. If \f[I]listelm\f[] value is \f[B]TOID_NULL\f[], the object is inserted at the head of the list. The \f[I]field\f[] and \f[I]field_new\f[] arguments are the names of the fields of type \f[I]POBJ_LIST_ENTRY\f[] in the element structure that are used to connect the elements in both lists. .SH SEE ALSO .PP \f[B]queue\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pobj_list_move_element_after.30000664000000000000000000000002513615011243021664 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_ctl_set.30000664000000000000000000000002613615011243017126 0ustar rootroot.so pmemobj_ctl_get.3 pmdk-1.8/doc/libpmemobj/pobj_list_insert_after.30000664000000000000000000000002513615011243020511 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/tx_alloc.30000664000000000000000000000002713615011243015566 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/tx_add_field_direct.30000664000000000000000000000003313615011243017716 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_direct.30000664000000000000000000000002213615011243016737 0ustar rootroot.so oid_is_null.3 pmdk-1.8/doc/libpmemobj/direct_ro.30000664000000000000000000000002313615011243015727 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pobj_list_insert_tail.30000664000000000000000000000002513615011243020341 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pobj_first.30000664000000000000000000000002413615011243016117 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/pobj_foreach.30000664000000000000000000000002413615011243016377 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/pobj_list_next.30000664000000000000000000000002513615011243017002 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_set_user_data.30000664000000000000000000000002313615011243020310 0ustar rootroot.so pmemobj_open.3 pmdk-1.8/doc/libpmemobj/pmemobj_free.30000664000000000000000000000002413615011243016410 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_alloc.30000644000000000000000000003551613615011417017313 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_TX_ALLOC" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_tx_alloc\f[](), \f[B]pmemobj_tx_zalloc\f[](), \f[B]pmemobj_tx_xalloc\f[](), \f[B]pmemobj_tx_realloc\f[](), \f[B]pmemobj_tx_zrealloc\f[](), \f[B]pmemobj_tx_strdup\f[](), \f[B]pmemobj_tx_xstrdup\f[](), \f[B]pmemobj_tx_wcsdup\f[](), \f[B]pmemobj_tx_xwcsdup\f[](), \f[B]pmemobj_tx_free\f[](), \f[B]pmemobj_tx_xfree\f[]() .PP \f[B]TX_NEW\f[](), \f[B]TX_ALLOC\f[](), \f[B]TX_ZNEW\f[](), \f[B]TX_ZALLOC\f[](), \f[B]TX_XALLOC\f[](), \f[B]TX_REALLOC\f[](), \f[B]TX_ZREALLOC\f[](), \f[B]TX_STRDUP\f[](), \f[B]TX_XSTRDUP\f[](), \f[B]TX_WCSDUP\f[](), \f[B]TX_XWCSDUP\f[](), \f[B]TX_FREE\f[](), \f[B]TX_XFREE\f[]() \- transactional object manipulation .SH SYNOPSIS .IP .nf \f[C] #include\ PMEMoid\ pmemobj_tx_alloc(size_t\ size,\ uint64_t\ type_num); PMEMoid\ pmemobj_tx_zalloc(size_t\ size,\ uint64_t\ type_num); PMEMoid\ pmemobj_tx_xalloc(size_t\ size,\ uint64_t\ type_num,\ uint64_t\ flags); PMEMoid\ pmemobj_tx_realloc(PMEMoid\ oid,\ size_t\ size,\ uint64_t\ type_num); PMEMoid\ pmemobj_tx_zrealloc(PMEMoid\ oid,\ size_t\ size,\ uint64_t\ type_num); PMEMoid\ pmemobj_tx_strdup(const\ char\ *s,\ uint64_t\ type_num); PMEMoid\ pmemobj_tx_wcsdup(const\ wchar_t\ *s,\ uint64_t\ type_num); int\ pmemobj_tx_free(PMEMoid\ oid); int\ pmemobj_tx_xfree(PMEMoid\ oid,\ uint64_t\ flags); TX_NEW(TYPE) TX_ALLOC(TYPE,\ size_t\ size) TX_ZNEW(TYPE) TX_ZALLOC(TYPE,\ size_t\ size) TX_XALLOC(TYPE,\ size_t\ size,\ uint64_t\ flags) TX_REALLOC(TOID\ o,\ size_t\ size) TX_ZREALLOC(TOID\ o,\ size_t\ size) TX_STRDUP(const\ char\ *s,\ uint64_t\ type_num) TX_WCSDUP(const\ wchar_t\ *s,\ uint64_t\ type_num) TX_FREE(TOID\ o) TX_XFREE(TOID\ o,\ uint64_t\ flags) \f[] .fi .SH DESCRIPTION .PP The \f[B]pmemobj_tx_alloc\f[]() function transactionally allocates a new object of given \f[I]size\f[] and \f[I]type_num\f[]. In contrast to the non\-transactional allocations, the objects are added to the internal object containers of given \f[I]type_num\f[] only after the transaction is committed, making the objects visible to the \f[B]POBJ_FOREACH_*\f[]() macros. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_zalloc\f[]() function transactionally allocates a new zeroed object of given \f[I]size\f[] and \f[I]type_num\f[]. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_xalloc\f[]() function transactionally allocates a new object of given \f[I]size\f[] and \f[I]type_num\f[]. The \f[I]flags\f[] argument is a bitmask of the following values: .IP \[bu] 2 \f[B]POBJ_XALLOC_ZERO\f[] \- zero the allocated object (equivalent of pmemobj_tx_zalloc) .IP \[bu] 2 \f[B]POBJ_XALLOC_NO_FLUSH\f[] \- skip flush on commit (when application deals with flushing or uses pmemobj_memcpy_persist) .IP \[bu] 2 \f[B]POBJ_CLASS_ID(class_id)\f[] \- allocate an object from the allocation class with id equal to \f[I]class_id\f[] .IP \[bu] 2 \f[B]POBJ_ARENA_ID(arena_id)\f[] \- allocate an object from the arena specified by \f[I]arena_id\f[]. The arena must exist, otherwise, the behavior is undefined. If \f[I]arena_id\f[] is equal 0, then arena assigned to the current thread will be used. .IP \[bu] 2 \f[B]POBJ_XALLOC_NO_ABORT\f[] \- if the function does not end successfully, do not abort the transaction. .PP This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_realloc\f[]() function transactionally resizes an existing object to the given \f[I]size\f[] and changes its type to \f[I]type_num\f[]. If \f[I]oid\f[] is \f[B]OID_NULL\f[], then the call is equivalent to \f[I]pmemobj_tx_alloc(pop, size, type_num)\f[]. If \f[I]size\f[] is equal to zero and \f[I]oid\f[] is not \f[B]OID_NULL\f[], then the call is equivalent to \f[I]pmemobj_tx_free(oid)\f[]. If the new size is larger than the old size, the added memory will \f[I]not\f[] be initialized. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_zrealloc\f[]() function transactionally resizes an existing object to the given \f[I]size\f[] and changes its type to \f[I]type_num\f[]. If the new size is larger than the old size, the extended new space is zeroed. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_strdup\f[]() function transactionally allocates a new object containing a duplicate of the string \f[I]s\f[] and assigns it a type \f[I]type_num\f[]. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_xstrdup\f[]() function behaves exactly the same as \f[B]pmemobj_tx_strdup\f[]() when \f[I]flags\f[] equals zero. The \f[I]flags\f[] argument is a bitmask of values described in \f[B]pmemobj_tx_xalloc\f[] section. .PP The \f[B]pmemobj_tx_wcsdup\f[]() function transactionally allocates a new object containing a duplicate of the wide character string \f[I]s\f[] and assigns it a type \f[I]type_num\f[]. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_xwcsdup\f[]() function behaves exactly the same as \f[B]pmemobj_tx_wcsdup\f[]() when \f[I]flags\f[] equals zero. The \f[I]flags\f[] argument is a bitmask of values described in \f[B]pmemobj_tx_xalloc\f[] section. .PP The \f[B]pmemobj_tx_free\f[]() function transactionally frees an existing object referenced by \f[I]oid\f[]. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_xfree\f[]() function behaves exactly the same as \f[B]pmemobj_tx_free\f[]() when \f[I]flags\f[] equals zero. \f[I]flags\f[] is a bitmask of the following value: .IP \[bu] 2 \f[B]POBJ_XFREE_NO_ABORT\f[] \- if the function does not end successfully, do not abort the transaction. .PP This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]TX_NEW\f[]() macro transactionally allocates a new object of given \f[I]TYPE\f[] and assigns it a type number read from the typed \f[I]OID\f[]. The allocation size is determined from the size of the user\-defined structure \f[I]TYPE\f[]. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the newly allocated object. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_ALLOC\f[]() macro transactionally allocates a new object of given \f[I]TYPE\f[] and assigns it a type number read from the typed \f[I]OID\f[]. The allocation size is passed by \f[I]size\f[] parameter. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the newly allocated object. Otherwise, the stage is set to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_ZNEW\f[]() macro transactionally allocates a new zeroed object of given \f[I]TYPE\f[] and assigns it a type number read from the typed \f[I]OID\f[]. The allocation size is determined from the size of the user\-defined structure \f[I]TYPE\f[]. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the newly allocated object. Otherwise, stage changes to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_ZALLOC\f[]() macro transactionally allocates a new zeroed object of given \f[I]TYPE\f[] and assigns it a type number read from the typed \f[I]OID\f[]. The allocation size is passed by \f[I]size\f[] argument. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the newly allocated object. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_XALLOC\f[]() macro transactionally allocates a new object of given \f[I]TYPE\f[] and assigns it a type number read from the typed \f[I]OID\f[]. The allocation size is passed by \f[I]size\f[] argument. The \f[I]flags\f[] argument is a bitmask of values described in \f[B]pmemobj_tx_xalloc\f[] section. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the newly allocated object. Otherwise, the \f[B]OID_NULL\f[] is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XALLOC_NO_ABORT\f[], the transaction is aborted. .PP The \f[B]TX_REALLOC\f[]() macro transactionally resizes an existing object referenced by a handle \f[I]o\f[] to the given \f[I]size\f[]. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the reallocated object. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_ZREALLOC\f[]() macro transactionally resizes an existing object referenced by a handle \f[I]o\f[] to the given \f[I]size\f[]. If the new size is larger than the old size, the extended new space is zeroed. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the reallocated object. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_STRDUP\f[]() macro transactionally allocates a new object containing a duplicate of the string \f[I]s\f[] and assigns it type \f[I]type_num\f[]. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the newly allocated object. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_XSTRDUP\f[]() macro transactionally allocates a new object containing a duplicate of the string \f[I]s\f[] and assigns it type \f[I]type_num\f[]. The \f[I]flags\f[] argument is a bitmask of values described in \f[B]pmemobj_tx_xalloc\f[] section. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the newly allocated object. Otherwise, the \f[B]OID_NULL\f[] is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XALLOC_NO_ABORT\f[], the transaction is aborted. .PP The \f[B]TX_WCSDUP\f[]() macro transactionally allocates a new object containing a duplicate of the wide character string \f[I]s\f[] and assigns it a type \f[I]type_num\f[]. If successful and called during \f[B]TX_STAGE_WORK\f[], it returns a handle to the newly allocated object. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_XWCSDUP\f[]() macro transactionally allocates a new object containing a duplicate of the wide character string \f[I]s\f[] and assigns it a type \f[I]type_num\f[]. The \f[I]flags\f[] argument is a bitmask of values described in \f[B]pmemobj_tx_xalloc\f[] section. If successful and called during \f[B]TX_STAGE_WORK\f[] it returns a handle to the newly allocated object. Otherwise, the \f[B]OID_NULL\f[] is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XALLOC_NO_ABORT\f[], the transaction is aborted. .PP The \f[B]TX_FREE\f[]() macro transactionally frees the memory space represented by an object handle \f[I]o\f[]. If \f[I]o\f[] is \f[B]OID_NULL\f[], no operation is performed. If successful and called during \f[B]TX_STAGE_WORK\f[], \f[B]TX_FREE\f[]() returns 0. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[] and \f[I]errno\f[] is set appropriately. .PP The \f[B]TX_XFREE\f[]() macro transactionally frees the memory space represented by an object handle \f[I]o\f[]. If \f[I]o\f[] is \f[B]OID_NULL\f[], no operation is performed. The \f[I]flags\f[] argument is a bitmask of values described in \f[B]pmemobj_tx_xfree\f[] section. If successful and called during \f[B]TX_STAGE_WORK\f[], \f[B]TX_FREE\f[]() returns 0. Otherwise, the error number is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XFREE_NO_ABORT\f[], the transaction is aborted. .SH RETURN VALUE .PP On success, the \f[B]pmemobj_tx_alloc\f[](), \f[B]pmemobj_tx_zalloc\f[](), \f[B]pmemobj_tx_strdup\f[]() and \f[B]pmemobj_tx_wcsdup\f[]() functions return a handle to the newly allocated object. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. If \f[I]size\f[] equals 0, \f[B]OID_NULL\f[] is returned and \f[I]errno\f[] is set appropriately. .PP On success, the \f[B]pmemobj_tx_xalloc\f[](), \f[B]pmemobj_tx_xstrdup\f[]() and \f[B]pmemobj_tx_xwcsdup\f[]() functions return a handle to the newly allocated object. Otherwise, the \f[B]OID_NULL\f[] is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XALLOC_NO_ABORT\f[], the transaction is aborted. .PP On success, \f[B]pmemobj_tx_realloc\f[]() and \f[B]pmemobj_tx_zrealloc\f[]() return a handle to the resized object. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]OID_NULL\f[] is returned, and \f[I]errno\f[] is set appropriately. Note that the object handle value may change as a result of reallocation. .PP On success, \f[B]pmemobj_tx_free\f[]() returns 0. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]errno\f[] is set appropriately and transaction is aborted .PP On success \f[B]pmemobj_tx_xfree\f[]() returns 0. Otherwise, the error number is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XFREE_NO_ABORT\f[], the transaction is aborted. .SH SEE ALSO .PP \f[B]pmemobj_tx_add_range\f[](3), \f[B]pmemobj_tx_begin\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_rwlock_zero.30000664000000000000000000000003113615011243020025 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pobj_foreach_safe.30000664000000000000000000000002413615011243017375 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_zalloc.30000664000000000000000000000002713615011243017471 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_pool_by_oid.30000664000000000000000000000002213615011243017763 0ustar rootroot.so oid_is_null.3 pmdk-1.8/doc/libpmemobj/pmemobj_rwlock_timedrdlock.30000664000000000000000000000003113615011243021347 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pobj_reserve_alloc.30000664000000000000000000000002513615011243017616 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/pmemobj_oid.30000664000000000000000000000002213615011243016240 0ustar rootroot.so oid_is_null.3 pmdk-1.8/doc/libpmemobj/pobj_list_insert_new_after.30000664000000000000000000000002513615011243021362 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_add_range.3.md0000664000000000000000000002337113615011243020517 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_TX_ADD_RANGE, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_tx_add_range.3 -- man page for transactional object manipulation) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemobj_tx_add_range**(), **pmemobj_tx_add_range_direct**(), **pmemobj_tx_xadd_range**(), **pmemobj_tx_xadd_range_direct**() **TX_ADD**(), **TX_ADD_FIELD**(), **TX_ADD_DIRECT**(), **TX_ADD_FIELD_DIRECT**(), **TX_XADD**(), **TX_XADD_FIELD**(), **TX_XADD_DIRECT**(), **TX_XADD_FIELD_DIRECT**(), **TX_SET**(), **TX_SET_DIRECT**(), **TX_MEMCPY**(), **TX_MEMSET**() - transactional object manipulation # SYNOPSIS # ```c #include int pmemobj_tx_add_range(PMEMoid oid, uint64_t off, size_t size); int pmemobj_tx_add_range_direct(const void *ptr, size_t size); int pmemobj_tx_xadd_range(PMEMoid oid, uint64_t off, size_t size, uint64_t flags); int pmemobj_tx_xadd_range_direct(const void *ptr, size_t size, uint64_t flags); TX_ADD(TOID o) TX_ADD_FIELD(TOID o, FIELD) TX_ADD_DIRECT(TYPE *p) TX_ADD_FIELD_DIRECT(TYPE *p, FIELD) TX_XADD(TOID o, uint64_t flags) TX_XADD_FIELD(TOID o, FIELD, uint64_t flags) TX_XADD_DIRECT(TYPE *p, uint64_t flags) TX_XADD_FIELD_DIRECT(TYPE *p, FIELD, uint64_t flags) TX_SET(TOID o, FIELD, VALUE) TX_SET_DIRECT(TYPE *p, FIELD, VALUE) TX_MEMCPY(void *dest, const void *src, size_t num) TX_MEMSET(void *dest, int c, size_t num) ``` # DESCRIPTION # **pmemobj_tx_add_range**() takes a "snapshot" of the memory block of given *size*, located at given offset *off* in the object specified by *oid*, and saves it to the undo log. The application is then free to directly modify the object in that memory range. In case of a failure or abort, all the changes within this range will be rolled back. The supplied block of memory has to be within the pool registered in the transaction. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_xadd_range**() function behaves exactly the same as **pmemobj_tx_add_range**() when *flags* equals zero. *flags* is a bitmask of the following values: + **POBJ_XADD_NO_FLUSH** - skip flush on commit (when application deals with flushing or uses pmemobj_memcpy_persist) + **POBJ_XADD_NO_SNAPSHOT** - added range will not be "snapshotted", i.e. any changes made within it during the transaction will not be rolled backed after abort + **POBJ_XADD_ASSUME_INITIALIZED** - added range is assumed to be initialized. If this flag is not specified, passing uninitialized memory will result in an error when run under Valgrind memcheck. + **POBJ_XADD_NO_ABORT** - if the function does not end successfully, do not abort the transaction. **pmemobj_tx_add_range_direct**() behaves the same as **pmemobj_tx_add_range**() with the exception that it operates on virtual memory addresses and not persistent memory objects. It takes a "snapshot" of a persistent memory block of given *size*, located at the given address *ptr* in the virtual memory space and saves it to the undo log. The application is then free to directly modify the object in that memory range. In case of a failure or abort, all the changes within this range will be rolled back. The supplied block of memory has to be within the pool registered in the transaction. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_xadd_range_direct**() function behaves exactly the same as **pmemobj_tx_add_range_direct**() when *flags* equals zero. *flags* is a bitmask of the following values: + **POBJ_XADD_NO_FLUSH** - skip flush on commit (when application deals with flushing or uses pmemobj_memcpy_persist) + **POBJ_XADD_NO_SNAPSHOT** - added range will not be "snapshotted", i.e. any changes made within it during the transaction will not be rolled backed after abort + **POBJ_XADD_ASSUME_INITIALIZED** - added range is assumed to be initialized. If this flag is not specified, passing uninitialized memory will result in an error when run under Valgrind memcheck. + **POBJ_XADD_NO_ABORT** - if the function does not end successfully, do not abort the transaction. Similarly to the macros controlling the transaction flow, **libpmemobj** defines a set of macros that simplify the transactional operations on persistent objects. Note that those macros operate on typed object handles, thus eliminating the need to specify the size of the object, or the size and offset of the field in the user-defined structure that is to be modified. The **TX_ADD_FIELD**() macro saves the current value of given *FIELD* of the object referenced by a handle *o* in the undo log. The application is then free to directly modify the specified *FIELD*. In case of a failure or abort, the saved value will be restored. The **TX_XADD_FIELD**() macro works exactly like **TX_ADD_FIELD** when *flags* equals 0. The *flags* argument is a bitmask of values described in **pmemobj_tx_xadd_range**, above. The **TX_ADD**() macro takes a "snapshot" of the entire object referenced by object handle *o* and saves it in the undo log. The object size is determined from its *TYPE*. The application is then free to directly modify the object. In case of a failure or abort, all the changes within the object will be rolled back. The **TX_XADD**() macro works exactly like **TX_ADD** when *flags* equals 0. The *flags* argument is a bitmask of values as described in **pmemobj_tx_xadd_range**, above. The **TX_ADD_FIELD_DIRECT**() macro saves the current value of the given *FIELD* of the object referenced by (direct) pointer *p* in the undo log. The application is then free to directly modify the specified *FIELD*. In case of a failure or abort, the saved value will be restored. The **TX_XADD_FIELD_DIRECT**() macro works exactly like **TX_ADD_FIELD_DIRECT** when *flags* equals 0. The *flags* argument is a bitmask of values as described in **pmemobj_tx_xadd_range_direct**, above. The **TX_ADD_DIRECT**() macro takes a "snapshot" of the entire object referenced by (direct) pointer *p* and saves it in the undo log. The object size is determined from its *TYPE*. The application is then free to directly modify the object. In case of a failure or abort, all the changes within the object will be rolled back. The **TX_XADD_DIRECT**() macro works exactly like **TX_ADD_DIRECT** when *flags* equals 0. The *flags* argument is a bitmask of values as described in **pmemobj_tx_xadd_range_direct**, above. The **TX_SET**() macro saves the current value of the given *FIELD* of the object referenced by handle *o* in the undo log, and then sets its new *VALUE*. In case of a failure or abort, the saved value will be restored. The **TX_SET_DIRECT**() macro saves in the undo log the current value of given *FIELD* of the object referenced by (direct) pointer *p*, and then set its new *VALUE*. In case of a failure or abort, the saved value will be restored. The **TX_MEMCPY**() macro saves in the undo log the current content of *dest* buffer and then overwrites the first *num* bytes of its memory area with the data copied from the buffer pointed by *src*. In case of a failure or abort, the saved value will be restored. The **TX_MEMSET**() macro saves the current content of the *dest* buffer in the undo log and then fills the first *num* bytes of its memory area with the constant byte *c*. In case of a failure or abort, the saved value will be restored. # RETURN VALUE # On success, **pmemobj_tx_add_range**() and **pmemobj_tx_add_range_direct**() return 0. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **errno** is set appropriately and transaction is aborted. On success, **pmemobj_tx_xadd_range**() and **pmemobj_tx_xadd_range_direct**() returns 0. Otherwise, the error number is returned, **errno** is set and when flags do not contain **POBJ_XADD_NO_ABORT**, the transaction is aborted. # SEE ALSO # **pmemobj_tx_alloc**(3), **pmemobj_tx_begin**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_reserve.30000664000000000000000000000002513615011243017143 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/toid_declare.30000644000000000000000000001271313615011420016377 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "TOID_DECLARE" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]TOID_DECLARE\f[](), \f[B]TOID_DECLARE_ROOT\f[](), \f[B]TOID\f[](), \f[B]TOID_TYPE_NUM\f[](), \f[B]TOID_TYPE_NUM_OF\f[](), \f[B]TOID_VALID\f[](), \f[B]OID_INSTANCEOF\f[](), \f[B]TOID_ASSIGN\f[](), \f[B]TOID_IS_NULL\f[](), \f[B]TOID_EQUALS\f[](), \f[B]TOID_TYPEOF\f[](), \f[B]TOID_OFFSETOF\f[](), \f[B]DIRECT_RW\f[](), \f[B]D_RW\f[](), \f[B]DIRECT_RO\f[](), \f[B]D_RO\f[]() \- libpmemobj type safety mechanism .SH SYNOPSIS .IP .nf \f[C] #include\ TOID_DECLARE(TYPE,\ uint64_t\ type_num) TOID_DECLARE_ROOT(ROOT_TYPE) TOID(TYPE) TOID_TYPE_NUM(TYPE) TOID_TYPE_NUM_OF(TOID\ oid) TOID_VALID(TOID\ oid) OID_INSTANCEOF(PMEMoid\ oid,\ TYPE) TOID_ASSIGN(TOID\ o,\ VALUE) TOID_IS_NULL(TOID\ o) TOID_EQUALS(TOID\ lhs,\ TOID\ rhs) TOID_TYPEOF(TOID\ o) TOID_OFFSETOF(TOID\ o,\ FILED) DIRECT_RW(TOID\ oid) D_RW(TOID\ oid) DIRECT_RO(TOID\ oid) D_RO(TOID\ oid) \f[] .fi .SH DESCRIPTION .PP Operating on untyped object handles, as well as on direct untyped object pointers (\f[I]void*\f[]), may be confusing and error\-prone. To facilitate type safety, \f[B]libpmemobj\f[](7) defines a set of macros that provide static type enforcement, catching potential errors at compile time. For example, a compile\-time error is generated when an attempt is made to assign a handle to an object of one type to the object handle variable of another type of object. .PP The \f[B]TOID_DECLARE\f[]() macro declares a typed \f[I]OID\f[] of user\-defined type \f[I]TYPE\f[] and type number \f[I]type_num\f[]. .PP The \f[B]TOID_DECLARE_ROOT\f[]() macro declares a typed \f[I]OID\f[] of user\-defined type \f[I]ROOT_TYPE\f[] and root object type number \f[B]POBJ_ROOT_TYPE_NUM\f[]. .PP The \f[B]TOID\f[]() macro declares a handle to an object of type \f[I]TYPE\f[], where \f[I]TYPE\f[] is the name of a user\-defined structure. The typed \f[I]OID\f[] must be declared first using the \f[B]TOID_DECLARE\f[](), \f[B]TOID_DECLARE_ROOT\f[](), \f[B]POBJ_LAYOUT_TOID\f[](3) or \f[B]POBJ_LAYOUT_ROOT\f[](3) macros. .PP The \f[B]TOID_TYPE_NUM\f[]() macro returns the type number of the type specified by \f[I]TYPE\f[]. .PP The \f[B]TOID_TYPE_NUM_OF\f[]() macro returns the type number of the object specified by \f[I]oid\f[]. The type number is read from the typed \f[I]OID\f[]. .PP The \f[B]TOID_VALID\f[]() macro validates whether the type number stored in the object's metadata is equal to the type number read from the typed \f[I]OID\f[]. .PP The \f[B]OID_INSTANCEOF\f[]() macro checks whether the \f[I]oid\f[] is of type \f[I]TYPE\f[]. .PP The \f[B]TOID_ASSIGN\f[]() macro assigns the object handle \f[I]VALUE\f[] to typed \f[I]OID\f[] \f[I]o\f[]. .PP The \f[B]TOID_IS_NULL\f[]() macro evaluates to true if the object handle represented by \f[I]o\f[] is \f[B]OID_NULL\f[]. .PP The \f[B]TOID_EQUALS\f[]() macro evaluates to true if both the \f[I]lhs\f[] and \f[I]rhs\f[] object handles reference the same persistent object. .PP The \f[B]TOID_TYPEOF\f[]() macro returns the type of the object handle represented by typed \f[I]OID\f[] \f[I]o\f[]. .PP The \f[B]TOID_OFFSETOF\f[]() macro returns the offset of the \f[I]FIELD\f[] member from the start of the object represented by \f[I]o\f[]. .PP The \f[B]DIRECT_RW\f[]() macro and its shortened form \f[B]D_RW\f[]() return a typed write pointer (\f[I]TYPE*\f[]) to an object represented by \f[I]oid\f[]. If \f[I]oid\f[] is \f[B]OID_NULL\f[], the macro evaluates to NULL. .PP The \f[B]DIRECT_RO\f[]() macro and its shortened form \f[B]D_RO\f[]() return a typed read\-only (const) pointer (\f[I]TYPE*\f[]) to an object represented by \f[I]oid\f[]. If \f[I]oid\f[] is \f[B]OID_NULL\f[], the macro evaluates to NULL. .SH SEE ALSO .PP \f[B]OID_IS_NULL\f[](3), \f[B]POBJ_LAYOUT_ROOT\f[](3), \f[B]POBJ_LAYOUT_TOID\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pobj_reserve_new.30000664000000000000000000000002513615011243017315 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/pmemobj_mutex_zero.3.md0000664000000000000000000002643113615011243020301 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_MUTEX_ZERO, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_mutex_zero.3 -- man page for locking functions from libpmemobj library) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemobj_mutex_zero**(), **pmemobj_mutex_lock**(), **pmemobj_mutex_timedlock**(), **pmemobj_mutex_trylock**(), **pmemobj_mutex_unlock**(), **pmemobj_rwlock_zero**(), **pmemobj_rwlock_rdlock**(), **pmemobj_rwlock_wrlock**(), **pmemobj_rwlock_timedrdlock**(), **pmemobj_rwlock_timedwrlock**(), **pmemobj_rwlock_tryrdlock**(), **pmemobj_rwlock_trywrlock**(), **pmemobj_rwlock_unlock**(), **pmemobj_cond_zero**(), **pmemobj_cond_broadcast**(), **pmemobj_cond_signal**(), **pmemobj_cond_timedwait**(), **pmemobj_cond_wait**() - pmemobj synchronization primitives # SYNOPSIS # ```c #include void pmemobj_mutex_zero(PMEMobjpool *pop, PMEMmutex *mutexp); int pmemobj_mutex_lock(PMEMobjpool *pop, PMEMmutex *mutexp); int pmemobj_mutex_timedlock(PMEMobjpool *pop, PMEMmutex *restrict mutexp, const struct timespec *restrict abs_timeout); int pmemobj_mutex_trylock(PMEMobjpool *pop, PMEMmutex *mutexp); int pmemobj_mutex_unlock(PMEMobjpool *pop, PMEMmutex *mutexp); void pmemobj_rwlock_zero(PMEMobjpool *pop, PMEMrwlock *rwlockp); int pmemobj_rwlock_rdlock(PMEMobjpool *pop, PMEMrwlock *rwlockp); int pmemobj_rwlock_wrlock(PMEMobjpool *pop, PMEMrwlock *rwlockp); int pmemobj_rwlock_timedrdlock(PMEMobjpool *pop, PMEMrwlock *restrict rwlockp, const struct timespec *restrict abs_timeout); int pmemobj_rwlock_timedwrlock(PMEMobjpool *pop, PMEMrwlock *restrict rwlockp, const struct timespec *restrict abs_timeout); int pmemobj_rwlock_tryrdlock(PMEMobjpool *pop, PMEMrwlock *rwlockp); int pmemobj_rwlock_trywrlock(PMEMobjpool *pop, PMEMrwlock *rwlockp); int pmemobj_rwlock_unlock(PMEMobjpool *pop, PMEMrwlock *rwlockp); void pmemobj_cond_zero(PMEMobjpool *pop, PMEMcond *condp); int pmemobj_cond_broadcast(PMEMobjpool *pop, PMEMcond *condp); int pmemobj_cond_signal(PMEMobjpool *pop, PMEMcond *condp); int pmemobj_cond_timedwait(PMEMobjpool *pop, PMEMcond *restrict condp, PMEMmutex *restrict mutexp, const struct timespec *restrict abs_timeout); int pmemobj_cond_wait(PMEMobjpool *pop, PMEMcond *restrict condp, PMEMmutex *restrict mutexp); ``` # DESCRIPTION # **libpmemobj**(7) provides several types of synchronization primitives designed to be used with persistent memory. The pmem-aware lock implementation is based on the standard POSIX Threads Library, as described in **pthread_mutex_init**(3), **pthread_rwlock_init**(3) and **pthread_cond_init**(3). Pmem-aware locks provide semantics similar to standard **pthread** locks, except that they are embedded in pmem-resident objects and are considered initialized by zeroing them. Therefore, locks allocated with **pmemobj_zalloc**(3) or **pmemobj_tx_zalloc**(3) do not require another initialization step. For performance reasons, they are also padded up to 64 bytes (cache line size). On FreeBSD, since all **pthread** locks are dynamically allocated, while the lock object is still padded up to 64 bytes for consistency with Linux, only the pointer to the lock is embedded in the pmem-resident object. **libpmemobj**(7) transparently manages freeing of the locks when the pool is closed. The fundamental property of pmem-aware locks is their automatic reinitialization every time the persistent object store pool is opened. Thus, all the pmem-aware locks may be considered initialized (unlocked) immediately after the pool is opened, regardless of their state at the time the pool was closed for the last time. Pmem-aware mutexes, read/write locks and condition variables must be declared with the *PMEMmutex*, *PMEMrwlock*, or *PMEMcond* type, respectively. The **pmemobj_mutex_zero**() function explicitly initializes the pmem-aware mutex *mutexp* by zeroing it. Initialization is not necessary if the object containing the mutex has been allocated using **pmemobj_zalloc**(3) or **pmemobj_tx_zalloc**(3). The **pmemobj_mutex_lock**() function locks the pmem-aware mutex *mutexp*. If the mutex is already locked, the calling thread will block until the mutex becomes available. If this is the first use of the mutex since the opening of the pool *pop*, the mutex is automatically reinitialized and then locked. **pmemobj_mutex_timedlock**() performs the same action as **pmemobj_mutex_lock**(), but will not wait beyond *abs_timeout* to obtain the lock before returning. The **pmemobj_mutex_trylock**() function locks pmem-aware mutex *mutexp*. If the mutex is already locked, **pthread_mutex_trylock**() will not block waiting for the mutex, but will return an error. If this is the first use of the mutex since the opening of the pool *pop*, the mutex is automatically reinitialized and then locked. The **pmemobj_mutex_unlock**() function unlocks the pmem-aware mutex *mutexp*. Undefined behavior follows if a thread tries to unlock a mutex that has not been locked by it, or if a thread tries to release a mutex that is already unlocked or has not been initialized. The **pmemobj_rwlock_zero**() function is used to explicitly initialize the pmem-aware read/write lock *rwlockp* by zeroing it. Initialization is not necessary if the object containing the lock has been allocated using **pmemobj_zalloc**(3) or **pmemobj_tx_zalloc**(3). The **pmemobj_rwlock_rdlock**() function acquires a read lock on *rwlockp*, provided that the lock is not presently held for writing and no writer threads are presently blocked on the lock. If the read lock cannot be acquired immediately, the calling thread blocks until it can acquire the lock. If this is the first use of the lock since the opening of the pool *pop*, the lock is automatically reinitialized and then acquired. **pmemobj_rwlock_timedrdlock**() performs the same action as **pmemobj_rwlock_rdlock**(), but will not wait beyond *abs_timeout* to obtain the lock before returning. A thread may hold multiple concurrent read locks. If so, **pmemobj_rwlock_unlock**() must be called once for each lock obtained. The results of acquiring a read lock while the calling thread holds a write lock are undefined. The **pmemobj_rwlock_wrlock**() function blocks until a write lock can be acquired against read/write lock *rwlockp*. If this is the first use of the lock since the opening of the pool *pop*, the lock is automatically reinitialized and then acquired. **pmemobj_rwlock_timedwrlock**() performs the same action, but will not wait beyond *abs_timeout* to obtain the lock before returning. The **pmemobj_rwlock_tryrdlock**() function performs the same action as **pmemobj_rwlock_rdlock**(), but does not block if the lock cannot be immediately obtained. The results are undefined if the calling thread already holds the lock at the time the call is made. The **pmemobj_rwlock_trywrlock**() function performs the same action as **pmemobj_rwlock_wrlock**(), but does not block if the lock cannot be immediately obtained. The results are undefined if the calling thread already holds the lock at the time the call is made. The **pmemobj_rwlock_unlock**() function is used to release the read/write lock previously obtained by **pmemobj_rwlock_rdlock**(), **pmemobj_rwlock_wrlock**(), **pthread_rwlock_tryrdlock**(), or **pmemobj_rwlock_trywrlock**(). The **pmemobj_cond_zero**() function explicitly initializes the pmem-aware condition variable *condp* by zeroing it. Initialization is not necessary if the object containing the condition variable has been allocated using **pmemobj_zalloc**(3) or **pmemobj_tx_zalloc**(3). The difference between **pmemobj_cond_broadcast**() and **pmemobj_cond_signal**() is that the former unblocks all threads waiting for the condition variable, whereas the latter blocks only one waiting thread. If no threads are waiting on *condp*, neither function has any effect. If more than one thread is blocked on a condition variable, the used scheduling policy determines the order in which threads are unblocked. The same mutex used for waiting must be held while calling either function. Although neither function strictly enforces this requirement, undefined behavior may follow if the mutex is not held. The **pmemobj_cond_timedwait**() and **pmemobj_cond_wait**() functions block on a condition variable. They must be called with mutex *mutexp* locked by the calling thread, or undefined behavior results. These functions atomically release mutex *mutexp* and cause the calling thread to block on the condition variable *condp*; atomically here means "atomically with respect to access by another thread to the mutex and then the condition variable". That is, if another thread is able to acquire the mutex after the about-to-block thread has released it, then a subsequent call to **pmemobj_cond_broadcast**() or **pmemobj_cond_signal**() in that thread will behave as if it were issued after the about-to-block thread has blocked. Upon successful return, the mutex will be locked and owned by the calling thread. # RETURN VALUE # The **pmemobj_mutex_zero**(), **pmemobj_rwlock_zero**() and **pmemobj_cond_zero**() functions return no value. Other locking functions return 0 on success. Otherwise, an error number will be returned to indicate the error. # SEE ALSO # **pmemobj_tx_zalloc**(3), **pmemobj_zalloc**(3), **pthread_cond_init**(3), **pthread_mutex_init**(3), **pthread_rwlock_init**(3), **libpmem**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_publish.30000664000000000000000000000002513615011243017136 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/toid_assign.30000664000000000000000000000002313615011243016260 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_log_intents_max_size.30000664000000000000000000000002713615011243022431 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pobj_list_foreach.30000664000000000000000000000002513615011243017433 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/libpmemobj.70000644000000000000000000002654613615011416016123 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "LIBPMEMOBJ" "7" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2016-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]libpmemobj\f[] \- persistent memory transactional object store .SH SYNOPSIS .IP .nf \f[C] #include\ cc\ \-std=gnu99\ ...\ \-lpmemobj\ \-lpmem \f[] .fi .SS Library API versioning: .IP .nf \f[C] const\ char\ *pmemobj_check_version( \ \ \ \ unsigned\ major_required, \ \ \ \ unsigned\ minor_required); \f[] .fi .SS Managing library behavior: .IP .nf \f[C] void\ pmemobj_set_funcs( \ \ \ \ void\ *(*malloc_func)(size_t\ size), \ \ \ \ void\ (*free_func)(void\ *ptr), \ \ \ \ void\ *(*realloc_func)(void\ *ptr,\ size_t\ size), \ \ \ \ char\ *(*strdup_func)(const\ char\ *s)); \f[] .fi .SS Error handling: .IP .nf \f[C] const\ char\ *pmemobj_errormsg(void); \f[] .fi .SS Other library functions: .PP A description of other \f[B]libpmemobj\f[] functions can be found on the following manual pages: .IP \[bu] 2 control and statistics: \f[B]pmemobj_ctl_get\f[](3) .IP \[bu] 2 create, open, close and validate: \f[B]pmemobj_open\f[](3) .IP \[bu] 2 low\-level memory manipulation: \f[B]pmemobj_memcpy_persist\f[](3) .IP \[bu] 2 locking: \f[B]pmemobj_mutex_zero\f[](3) .IP \[bu] 2 persistent object identifier: \f[B]OID_IS_NULL\f[](3) .IP \[bu] 2 type\-safety: \f[B]TOID_DECLARE\f[](3) .IP \[bu] 2 layout declaration: \f[B]POBJ_LAYOUT_BEGIN\f[](3) .IP \[bu] 2 non\-transactional atomic allocations: \f[B]pmemobj_alloc\f[](3) .IP \[bu] 2 root object management: \f[B]pmemobj_root\f[](3) .IP \[bu] 2 object containers: \f[B]pmemobj_first\f[](3) .IP \[bu] 2 non\-transactional persistent atomic circular doubly\-linked list: \f[B]pmemobj_list_insert\f[](3), \f[B]POBJ_LIST_HEAD\f[](3) .IP \[bu] 2 transactional object manipulation: \f[B]pmemobj_tx_begin\f[](3), \f[B]pmemobj_tx_add_range\f[](3), \f[B]pmemobj_tx_alloc\f[](3) .IP \[bu] 2 delayed atomicity actions: \f[B]pmemobj_action\f[](3) (EXPERIMENTAL) .SH DESCRIPTION .PP \f[B]libpmemobj\f[] provides a transactional object store in \f[I]persistent memory\f[] (pmem) for applications that require transactions and persistent memory management using direct access storage (DAX), which is storage that supports load/store access without paging blocks from a block storage device. Some types of \f[I]non\-volatile memory DIMMs\f[] (NVDIMMs) provide this type of byte addressable access to storage. A \f[I]persistent memory aware file system\f[] is typically used to expose the direct access to applications. Memory mapping a file from this type of file system results in load/store, non\-paged access to pmem. \f[B]libpmemobj\f[] builds on this type of memory mapped file using the low\-level pmem support provided by \f[B]libpmem\f[](7), handling the transactional updates, flushing changes to persistence, and managing recovery for the application. .PP \f[B]libpmemobj\f[] requires the \f[B]\-std=gnu99\f[] compilation flag to build properly. .PP \f[B]libpmemobj\f[] is one of a collection of persistent memory libraries available. The others are: .IP \[bu] 2 \f[B]libpmemblk\f[](7), providing pmem\-resident arrays of fixed\-sized blocks with atomic updates. .IP \[bu] 2 \f[B]libpmemlog\f[](7), providing a pmem\-resident log file. .IP \[bu] 2 \f[B]libpmem\f[](7), low\-level persistent memory support. .PP Under normal usage, \f[B]libpmemobj\f[] will never print messages or intentionally cause the process to exit. The only exception to this is the debugging information, when enabled, as described under \f[B]DEBUGGING AND ERROR HANDLING\f[], below. .SH LIBRARY API VERSIONING .PP This section describes how the library API is versioned, allowing applications to work with an evolving API. .PP The \f[B]pmemobj_check_version\f[]() function is used to see if the installed \f[B]libpmemobj\f[] supports the version of the library API required by an application. The easiest way to do this is for the application to supply the compile\-time version information, supplied by defines in \f[B]\f[], like this: .IP .nf \f[C] reason\ =\ pmemobj_check_version(PMEMOBJ_MAJOR_VERSION, \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ PMEMOBJ_MINOR_VERSION); if\ (reason\ !=\ NULL)\ { \ \ \ \ /*\ version\ check\ failed,\ reason\ string\ tells\ you\ why\ */ } \f[] .fi .PP Any mismatch in the major version number is considered a failure, but a library with a newer minor version number will pass this check since increasing minor versions imply backwards compatibility. .PP An application can also check specifically for the existence of an interface by checking for the version where that interface was introduced. These versions are documented in this man page as follows: unless otherwise specified, all interfaces described here are available in version 1.0 of the library. Interfaces added after version 1.0 will contain the text \f[I]introduced in version x.y\f[] in the section of this manual describing the feature. .PP On success, \f[B]pmemobj_check_version\f[]() returns NULL. Otherwise, the return value is a static string describing the reason the version check failed. The string returned by \f[B]pmemobj_check_version\f[]() must not be modified or freed. .SH MANAGING LIBRARY BEHAVIOR .PP The \f[B]pmemobj_set_funcs\f[]() function allows an application to override memory allocation calls used internally by \f[B]libpmemobj\f[]. Passing in NULL for any of the handlers will cause the \f[B]libpmemobj\f[] default function to be used. The library does not make heavy use of the system malloc functions, but it does allocate approximately 4\-8 kilobytes for each memory pool in use. .PP By default, \f[B]libpmemobj\f[] supports up to 1024 parallel transactions/allocations. For debugging purposes it is possible to decrease this value by setting the \f[B]PMEMOBJ_NLANES\f[] environment variable to the desired limit. .SH DEBUGGING AND ERROR HANDLING .PP If an error is detected during the call to a \f[B]libpmemobj\f[] function, the application may retrieve an error message describing the reason for the failure from \f[B]pmemobj_errormsg\f[](). This function returns a pointer to a static buffer containing the last error message logged for the current thread. If \f[I]errno\f[] was set, the error message may include a description of the corresponding error code as returned by \f[B]strerror\f[](3). The error message buffer is thread\-local; errors encountered in one thread do not affect its value in other threads. The buffer is never cleared by any library function; its content is significant only when the return value of the immediately preceding call to a \f[B]libpmemobj\f[] function indicated an error, or if \f[I]errno\f[] was set. The application must not modify or free the error message string, but it may be modified by subsequent calls to other library functions. .PP Two versions of \f[B]libpmemobj\f[] are typically available on a development system. The normal version, accessed when a program is linked using the \f[B]\-lpmemobj\f[] option, is optimized for performance. That version skips checks that impact performance and never logs any trace information or performs any run\-time assertions. .PP A second version of \f[B]libpmemobj\f[], accessed when a program uses the libraries under \f[B]/usr/lib/pmdk_debug\f[], contains run\-time assertions and trace points. The typical way to access the debug version is to set the environment variable \f[B]LD_LIBRARY_PATH\f[] to \f[B]/usr/lib/pmdk_debug\f[] or \f[B]/usr/lib64/pmdk_debug\f[], as appropriate. Debugging output is controlled using the following environment variables. These variables have no effect on the non\-debug version of the library. .IP \[bu] 2 \f[B]PMEMOBJ_LOG_LEVEL\f[] .PP The value of \f[B]PMEMOBJ_LOG_LEVEL\f[] enables trace points in the debug version of the library, as follows: .IP \[bu] 2 \f[B]0\f[] \- This is the default level when \f[B]PMEMOBJ_LOG_LEVEL\f[] is not set. No log messages are emitted at this level. .IP \[bu] 2 \f[B]1\f[] \- Additional details on any errors detected are logged, in addition to returning the \f[I]errno\f[]\-based errors as usual. The same information may be retrieved using \f[B]pmemobj_errormsg\f[](). .IP \[bu] 2 \f[B]2\f[] \- A trace of basic operations is logged. .IP \[bu] 2 \f[B]3\f[] \- Enables a very verbose amount of function call tracing in the library. .IP \[bu] 2 \f[B]4\f[] \- Enables voluminous and fairly obscure tracing information that is likely only useful to the \f[B]libpmemobj\f[] developers. .PP Unless \f[B]PMEMOBJ_LOG_FILE\f[] is set, debugging output is written to \f[I]stderr\f[]. .IP \[bu] 2 \f[B]PMEMOBJ_LOG_FILE\f[] .PP Specifies the name of a file where all logging information should be written. If the last character in the name is \[lq]\-\[rq], the \f[I]PID\f[] of the current process will be appended to the file name when the log file is created. If \f[B]PMEMOBJ_LOG_FILE\f[] is not set, logging output is written to \f[I]stderr\f[]. .PP See also \f[B]libpmem\f[](7) to get information about other environment variables affecting \f[B]libpmemobj\f[] behavior. .SH EXAMPLE .PP See for examples using the \f[B]libpmemobj\f[] API. .SH ACKNOWLEDGEMENTS .PP \f[B]libpmemobj\f[] builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: .SH SEE ALSO .PP \f[B]OID_IS_NULL\f[](3), \f[B]pmemobj_alloc\f[](3), \f[B]pmemobj_ctl_exec\f[](3), \f[B]pmemobj_ctl_get\f[](3), \f[B]pmemobj_ctl_set\f[](3), \f[B]pmemobj_first\f[](3), \f[B]pmemobj_list_insert\f[](3), \f[B]pmemobj_memcpy_persist\f[](3), \f[B]pmemobj_mutex_zero\f[](3), \f[B]pmemobj_open\f[](3), \f[B]pmemobj_root\f[](3), \f[B]pmemobj_tx_add_range\f[](3), \f[B]pmemobj_tx_alloc\f[](3), \f[B]pmemobj_tx_begin\f[](3), \f[B]POBJ_LAYOUT_BEGIN\f[](3), \f[B]POBJ_LIST_HEAD\f[](3), \f[B]strerror\f[](3), \f[B]TOID_DECLARE\f[](3), \f[B]libpmem\f[](7), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_errormsg.30000664000000000000000000000002613615011243017331 0ustar rootroot.so man7/libpmemobj.7 pmdk-1.8/doc/libpmemobj/pmemobj_persist.30000664000000000000000000000003513615011243017162 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_begin.30000644000000000000000000006500513615011417017301 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_TX_BEGIN" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_tx_stage\f[](), .PP \f[B]pmemobj_tx_begin\f[](), \f[B]pmemobj_tx_lock\f[](), \f[B]pmemobj_tx_xlock\f[](), \f[B]pmemobj_tx_abort\f[](), \f[B]pmemobj_tx_commit\f[](), \f[B]pmemobj_tx_end\f[](), \f[B]pmemobj_tx_errno\f[](), \f[B]pmemobj_tx_process\f[](), .PP \f[B]TX_BEGIN_PARAM\f[](), \f[B]TX_BEGIN_CB\f[](), \f[B]TX_BEGIN\f[](), \f[B]TX_ONABORT\f[], \f[B]TX_ONCOMMIT\f[], \f[B]TX_FINALLY\f[], \f[B]TX_END\f[], .PP \f[B]pmemobj_tx_log_append_buffer\f[](), \f[B]pmemobj_tx_xlog_append_buffer\f[](), \f[B]pmemobj_tx_log_auto_alloc\f[](), \f[B]pmemobj_tx_log_snapshots_max_size\f[](), \f[B]pmemobj_tx_log_intents_max_size\f[](), .PP \f[B]pmemobj_tx_set_user_data\f[](), \f[B]pmemobj_tx_get_user_data\f[]() \- transactional object manipulation .SH SYNOPSIS .IP .nf \f[C] #include\ enum\ tx_stage\ pmemobj_tx_stage(void); int\ pmemobj_tx_begin(PMEMobjpool\ *pop,\ jmp_buf\ *env,\ enum\ pobj_tx_param,\ ...); int\ pmemobj_tx_lock(enum\ tx_lock\ lock_type,\ void\ *lockp); int\ pmemobj_tx_xlock(enum\ tx_lock\ lock_type,\ void\ *lockp,\ uint64_t\ flags); void\ pmemobj_tx_abort(int\ errnum); void\ pmemobj_tx_commit(void); int\ pmemobj_tx_end(void); int\ pmemobj_tx_errno(void); void\ pmemobj_tx_process(void); TX_BEGIN_PARAM(PMEMobjpool\ *pop,\ ...) TX_BEGIN_CB(PMEMobjpool\ *pop,\ cb,\ arg,\ ...) TX_BEGIN(PMEMobjpool\ *pop) TX_ONABORT TX_ONCOMMIT TX_FINALLY TX_END int\ pmemobj_tx_log_append_buffer(enum\ pobj_log_type\ type,\ void\ *addr,\ size_t\ size); int\ pmemobj_tx_xlog_append_buffer(enum\ pobj_log_type\ type,\ void\ *addr,\ size_t\ size,\ uint64_t\ flags); int\ pmemobj_tx_log_auto_alloc(enum\ pobj_log_type\ type,\ int\ on_off); size_t\ pmemobj_tx_log_snapshots_max_size(size_t\ *sizes,\ size_t\ nsizes); size_t\ pmemobj_tx_log_intents_max_size(size_t\ nintents); void\ pmemobj_tx_set_user_data(void\ *data); void\ *pmemobj_tx_get_user_data(void); \f[] .fi .SH DESCRIPTION .PP The non\-transactional functions and macros described in \f[B]pmemobj_alloc\f[](3), \f[B]pmemobj_list_insert\f[](3) and \f[B]POBJ_LIST_HEAD\f[](3) only guarantee the atomicity of a single operation on an object. In case of more complex changes involving multiple operations on an object, or allocation and modification of multiple objects, data consistency and fail\-safety may be provided only by using \f[I]atomic transactions\f[]. .PP A transaction is defined as series of operations on persistent memory objects that either all occur, or nothing occurs. In particular, if the execution of a transaction is interrupted by a power failure or a system crash, it is guaranteed that after system restart, all the changes made as a part of the uncompleted transaction will be rolled back, restoring the consistent state of the memory pool from the moment when the transaction was started. .PP Note that transactions do not provide atomicity with respect to other threads. All the modifications performed within the transactions are immediately visible to other threads. Therefore it is the responsibility of the application to implement a proper thread synchronization mechanism. .PP Each thread may have only one transaction open at a time, but that transaction may be nested. Nested transactions are flattened. Committing the nested transaction does not commit the outer transaction; however, errors in the nested transaction are propagated up to the outermost level, resulting in the interruption of the entire transaction. .PP Each transaction is visible only for the thread that started it. No other threads can add operations, commit or abort the transaction initiated by another thread. Multiple threads may have transactions open on a given memory pool at the same time. .PP Please see the \f[B]CAVEATS\f[] section below for known limitations of the transactional API. .PP The \f[B]pmemobj_tx_stage\f[]() function returns the current \f[I]transaction stage\f[] for a thread. Stages are changed only by the \f[B]pmemobj_tx_*\f[]() functions. Transaction stages are defined as follows: .IP \[bu] 2 \f[B]TX_STAGE_NONE\f[] \- no open transaction in this thread .IP \[bu] 2 \f[B]TX_STAGE_WORK\f[] \- transaction in progress .IP \[bu] 2 \f[B]TX_STAGE_ONCOMMIT\f[] \- successfully committed .IP \[bu] 2 \f[B]TX_STAGE_ONABORT\f[] \- starting the transaction failed or transaction aborted .IP \[bu] 2 \f[B]TX_STAGE_FINALLY\f[] \- ready for clean up .PP The \f[B]pmemobj_tx_begin\f[]() function starts a new transaction in the current thread. If called within an open transaction, it starts a nested transaction. The caller may use the \f[I]env\f[] argument to provide a pointer to a calling environment to be restored in case of transaction abort. This information must be provided by the caller using the \f[B]setjmp\f[](3) macro. .PP A new transaction may be started only if the current stage is \f[B]TX_STAGE_NONE\f[] or \f[B]TX_STAGE_WORK\f[]. If successful, the \f[I]transaction stage\f[] changes to \f[B]TX_STAGE_WORK\f[]. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[]. .PP Optionally, a list of parameters for the transaction may be provided. Each parameter consists of a type followed by a type\-specific number of values. Currently there are 4 types: .IP \[bu] 2 \f[B]TX_PARAM_NONE\f[], used as a termination marker. No following value. .IP \[bu] 2 \f[B]TX_PARAM_MUTEX\f[], followed by one value, a pmem\-resident PMEMmutex .IP \[bu] 2 \f[B]TX_PARAM_RWLOCK\f[], followed by one value, a pmem\-resident PMEMrwlock .IP \[bu] 2 \f[B]TX_PARAM_CB\f[], followed by two values: a callback function of type \f[I]pmemobj_tx_callback\f[], and a void pointer .PP Using \f[B]TX_PARAM_MUTEX\f[] or \f[B]TX_PARAM_RWLOCK\f[] causes the specified lock to be acquired at the beginning of the transaction. \f[B]TX_PARAM_RWLOCK\f[] acquires the lock for writing. It is guaranteed that \f[B]pmemobj_tx_begin\f[]() will acquire all locks prior to successful completion, and they will be held by the current thread until the outermost transaction is finished. Locks are taken in order from left to right. To avoid deadlocks, the user is responsible for proper lock ordering. .PP \f[B]TX_PARAM_CB\f[] registers the specified callback function to be executed at each transaction stage. For \f[B]TX_STAGE_WORK\f[], the callback is executed prior to commit. For all other stages, the callback is executed as the first operation after a stage change. It will also be called after each transaction; in this case the \f[I]stage\f[] parameter will be set to \f[B]TX_STAGE_NONE\f[]. \f[I]pmemobj_tx_callback\f[] must be compatible with: .IP .nf \f[C] void\ func(PMEMobjpool\ *pop,\ enum\ pobj_tx_stage\ stage,\ void\ *arg) \f[] .fi .PP \f[I]pop\f[] is a pool identifier used in \f[B]pmemobj_tx_begin\f[](), \f[I]stage\f[] is a current transaction stage and \f[I]arg\f[] is the second parameter of \f[B]TX_PARAM_CB\f[]. Without considering transaction nesting, this mechanism can be considered an alternative method for executing code between stages (instead of \f[B]TX_ONCOMMIT\f[], \f[B]TX_ONABORT\f[], etc). However, there are 2 significant differences when nested transactions are used: .IP \[bu] 2 The registered function is executed only in the outermost transaction, even if registered in an inner transaction. .IP \[bu] 2 There can be only one callback in the entire transaction, that is, the callback cannot be changed in an inner transaction. .PP Note that \f[B]TX_PARAM_CB\f[] does not replace the \f[B]TX_ONCOMMIT\f[], \f[B]TX_ONABORT\f[], etc. macros. They can be used together: the callback will be executed \f[I]before\f[] a \f[B]TX_ONCOMMIT\f[], \f[B]TX_ONABORT\f[], etc. section. .PP \f[B]TX_PARAM_CB\f[] can be used when the code dealing with transaction stage changes is shared between multiple users or when it must be executed only in the outer transaction. For example it can be very useful when the application must synchronize persistent and transient state. .PP The \f[B]pmemobj_tx_lock\f[]() function acquires the lock \f[I]lockp\f[] of type \f[I]lock_type\f[] and adds it to the current transaction. \f[I]lock_type\f[] may be \f[B]TX_LOCK_MUTEX\f[] or \f[B]TX_LOCK_RWLOCK\f[]; \f[I]lockp\f[] must be of type \f[I]PMEMmutex\f[] or \f[I]PMEMrwlock\f[], respectively. If \f[I]lock_type\f[] is \f[B]TX_LOCK_RWLOCK\f[] the lock is acquired for writing. If the lock is not successfully acquired, the function returns an error number. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_xlock\f[]() function behaves exactly the same as \f[B]pmemobj_tx_lock\f[]() when \f[I]flags\f[] equals \f[B]POBJ_XLOCK_NO_ABORT\f[]. When \f[I]flags\f[] equals 0 and if the lock is not successfully acquired,the transaction is aborted. \f[I]flags\f[] is a bitmask of the following values: .IP \[bu] 2 \f[B]POBJ_XLOCK_NO_ABORT\f[] \- if the function does not end successfully, do not abort the transaction. .PP \f[B]pmemobj_tx_abort\f[]() aborts the current transaction and causes a transition to \f[B]TX_STAGE_ONABORT\f[]. If \f[I]errnum\f[] is equal to 0, the transaction error code is set to \f[B]ECANCELED\f[]; otherwise, it is set to \f[I]errnum\f[]. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_commit\f[]() function commits the current open transaction and causes a transition to \f[B]TX_STAGE_ONCOMMIT\f[]. If called in the context of the outermost transaction, all the changes may be considered as durably written upon successful completion. This function must be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_end\f[]() function performs a cleanup of the current transaction. If called in the context of the outermost transaction, it releases all the locks acquired by \f[B]pmemobj_tx_begin\f[]() for outer and nested transactions. If called in the context of a nested transaction, it returns to the context of the outer transaction in \f[B]TX_STAGE_WORK\f[], without releasing any locks. The \f[B]pmemobj_tx_end\f[]() function can be called during \f[B]TX_STAGE_NONE\f[] if transitioned to this stage using \f[B]pmemobj_tx_process\f[](). If not already in \f[B]TX_STAGE_NONE\f[], it causes the transition to \f[B]TX_STAGE_NONE\f[]. \f[B]pmemobj_tx_end\f[] must always be called for each \f[B]pmemobj_tx_begin\f[](), even if starting the transaction failed. This function must \f[I]not\f[] be called during \f[B]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_errno\f[]() function returns the error code of the last transaction. .PP The \f[B]pmemobj_tx_process\f[]() function performs the actions associated with the current stage of the transaction, and makes the transition to the next stage. It must be called in a transaction. The current stage must always be obtained by a call to \f[B]pmemobj_tx_stage\f[](). \f[B]pmemobj_tx_process\f[]() performs the following transitions in the transaction stage flow: .IP \[bu] 2 \f[B]TX_STAGE_WORK\f[] \-> \f[B]TX_STAGE_ONCOMMIT\f[] .IP \[bu] 2 \f[B]TX_STAGE_ONABORT\f[] \-> \f[B]TX_STAGE_FINALLY\f[] .IP \[bu] 2 \f[B]TX_STAGE_ONCOMMIT\f[] \-> \f[B]TX_STAGE_FINALLY\f[] .IP \[bu] 2 \f[B]TX_STAGE_FINALLY\f[] \-> \f[B]TX_STAGE_NONE\f[] .IP \[bu] 2 \f[B]TX_STAGE_NONE\f[] \-> \f[B]TX_STAGE_NONE\f[] .PP \f[B]pmemobj_tx_process\f[]() must not be called after calling \f[B]pmemobj_tx_end\f[]() for the outermost transaction. .PP In addition to the above API, \f[B]libpmemobj\f[](7) offers a more intuitive method of building transactions using the set of macros described below. When using these macros, the complete transaction flow looks like this: .IP .nf \f[C] TX_BEGIN(Pop)\ { \ \ \ \ /*\ the\ actual\ transaction\ code\ goes\ here...\ */ }\ TX_ONCOMMIT\ { \ \ \ \ /* \ \ \ \ \ *\ optional\ \-\ executed\ only\ if\ the\ above\ block \ \ \ \ \ *\ successfully\ completes \ \ \ \ \ */ }\ TX_ONABORT\ { \ \ \ \ /* \ \ \ \ \ *\ optional\ \-\ executed\ only\ if\ starting\ the\ transaction\ fails, \ \ \ \ \ *\ or\ if\ transaction\ is\ aborted\ by\ an\ error\ or\ a\ call\ to \ \ \ \ \ *\ pmemobj_tx_abort() \ \ \ \ \ */ }\ TX_FINALLY\ { \ \ \ \ /* \ \ \ \ \ *\ optional\ \-\ if\ exists,\ it\ is\ executed\ after \ \ \ \ \ *\ TX_ONCOMMIT\ or\ TX_ONABORT\ block \ \ \ \ \ */ }\ TX_END\ /*\ mandatory\ */ \f[] .fi .IP .nf \f[C] TX_BEGIN_PARAM(PMEMobjpool\ *pop,\ ...) TX_BEGIN_CB(PMEMobjpool\ *pop,\ cb,\ arg,\ ...) TX_BEGIN(PMEMobjpool\ *pop) \f[] .fi .PP The \f[B]TX_BEGIN_PARAM\f[](), \f[B]TX_BEGIN_CB\f[]() and \f[B]TX_BEGIN\f[]() macros start a new transaction in the same way as \f[B]pmemobj_tx_begin\f[](), except that instead of the environment buffer provided by a caller, they set up the local \f[I]jmp_buf\f[] buffer and use it to catch the transaction abort. The \f[B]TX_BEGIN\f[]() macro starts a transaction without any options. \f[B]TX_BEGIN_PARAM\f[] may be used when there is a need to acquire locks prior to starting a transaction (such as for a multi\-threaded program) or set up a transaction stage callback. \f[B]TX_BEGIN_CB\f[] is just a wrapper around \f[B]TX_BEGIN_PARAM\f[] that validates the callback signature. (For compatibility there is also a \f[B]TX_BEGIN_LOCK\f[] macro, which is an alias for \f[B]TX_BEGIN_PARAM\f[]). Each of these macros must be followed by a block of code with all the operations that are to be performed atomically. .PP The \f[B]TX_ONABORT\f[] macro starts a block of code that will be executed only if starting the transaction fails due to an error in \f[B]pmemobj_tx_begin\f[](), or if the transaction is aborted. This block is optional, but in practice it should not be omitted. If it is desirable to crash the application when a transaction aborts and there is no \f[B]TX_ONABORT\f[] section, the application can define the \f[B]POBJ_TX_CRASH_ON_NO_ONABORT\f[] macro before inclusion of \f[B]\f[]. This provides a default \f[B]TX_ONABORT\f[] section which just calls \f[B]abort\f[](3). .PP The \f[B]TX_ONCOMMIT\f[] macro starts a block of code that will be executed only if the transaction is successfully committed, which means that the execution of code in the \f[B]TX_BEGIN\f[]() block has not been interrupted by an error or by a call to \f[B]pmemobj_tx_abort\f[](). This block is optional. .PP The \f[B]TX_FINALLY\f[] macro starts a block of code that will be executed regardless of whether the transaction is committed or aborted. This block is optional. .PP The \f[B]TX_END\f[] macro cleans up and closes the transaction started by the \f[B]TX_BEGIN\f[]() / \f[B]TX_BEGIN_PARAM\f[]() / \f[B]TX_BEGIN_CB\f[]() macros. It is mandatory to terminate each transaction with this macro. If the transaction was aborted, \f[I]errno\f[] is set appropriately. .SS TRANSACTION LOG TUNING .PP From libpmemobj implementation perspective there are two types of operations in a transaction: .IP \[bu] 2 \f[B]snapshots\f[], where action must be persisted immediately, .IP \[bu] 2 \f[B]intents\f[], where action can be persisted at the transaction commit phase .PP \f[B]pmemobj_tx_add_range\f[](3) and all its variants belong to the \f[B]snapshots\f[] group. .PP \f[B]pmemobj_tx_alloc\f[](3) (with its variants), \f[B]pmemobj_tx_free\f[](3), \f[B]pmemobj_tx_realloc\f[](3) (with its variants) and \f[B]pmemobj_tx_publish\f[](3) belong to the \f[B]intents\f[] group. Even though \f[B]pmemobj_tx_alloc\f[]() allocates memory immediately, it modifies only the runtime state and postpones persistent memory modifications to the commit phase. \f[B]pmemobj_tx_free\f[](3) cannot free the object immediately, because of possible transaction rollback, so it postpones both the action and persistent memory modifications to the commit phase. \f[B]pmemobj_tx_realloc\f[](3) is just a combination of those two. \f[B]pmemobj_tx_publish\f[](3) postpones reservations and deferred frees to the commit phase. .PP Those two types of operations (snapshots and intents) require that libpmemobj builds a persistent log of operations. Intent log (also known as a \[lq]redo log\[rq]) is applied on commit and snapshot log (also known as an \[lq]undo log\[rq]) is applied on abort. .PP When libpmemobj transaction starts, it's not possible to predict how much persistent memory space will be needed for those logs. This means that libpmemobj must internally allocate this space whenever it's needed. This has two downsides: .IP \[bu] 2 when transaction snapshots a lot of memory or does a lot of allocations, libpmemobj may need to do many internal allocations, which must be freed when transaction ends, adding time overhead when big transactions are frequent, .IP \[bu] 2 transactions can start to fail due to not enough space for logs \- this can be especially problematic for transactions that want to \f[B]deallocate\f[] objects, as those might also fail .PP To solve both of these problems libpmemobj exposes the following functions: .IP \[bu] 2 \f[B]pmemobj_tx_log_append_buffer\f[](), .IP \[bu] 2 \f[B]pmemobj_tx_xlog_append_buffer\f[](), .IP \[bu] 2 \f[B]pmemobj_tx_log_auto_alloc\f[]() .PP \f[B]pmemobj_tx_log_append_buffer\f[]() appends a given range of memory [\f[I]addr\f[], \f[I]addr\f[] + \f[I]size\f[]) to the log \f[I]type\f[] of the current transaction. \f[I]type\f[] can be one of the two values (with meanings described above): .IP \[bu] 2 \f[B]TX_LOG_TYPE_SNAPSHOT\f[], .IP \[bu] 2 \f[B]TX_LOG_TYPE_INTENT\f[] .PP The range of memory \f[B]must\f[] belong to the same pool the transaction is on and \f[B]must not\f[] be used by more than one thread at the same time. The latter condition can be verified with tx.debug.verify_user_buffers ctl (see \f[B]pmemobj_ctl_get\f[](3)). .PP The \f[B]pmemobj_tx_xlog_append_buffer\f[]() function behaves exactly the same as \f[B]pmemobj_tx_log_append_buffer\f[]() when \f[I]flags\f[] equals zero. \f[I]flags\f[] is a bitmask of the following values: .IP \[bu] 2 \f[B]POBJ_XLOG_APPEND_BUFFER_NO_ABORT\f[] \- if the function does not end successfully, do not abort the transaction. .PP \f[B]pmemobj_tx_log_snapshots_max_size\f[] calculates the \f[B]maximum\f[] size of a buffer which will be able to hold \f[I]nsizes\f[] snapshots, each of size \f[I]sizes[i]\f[]. Application should not expect this function to return the same value between restarts. In future versions of libpmemobj this function can return smaller (because of better accuracy or space optimizations) or higher (because of higher alignment required for better performance) value. This function is independent of transaction stage and can be called both inside and outside of transaction. If the returned value S is greater than \f[B]PMEMOBJ_MAX_ALLOC_SIZE\f[], the buffer should be split into N chunks of size \f[B]PMEMOBJ_MAX_ALLOC_SIZE\f[], where N is equal to (S / \f[B]PMEMOBJ_MAX_ALLOC_SIZE\f[]) (rounded down) and the last chunk of size (S \- (N * \f[B]PMEMOBJ_MAX_ALLOC_SIZE\f[])). .PP \f[B]pmemobj_tx_log_intents_max_size\f[] calculates the \f[B]maximum\f[] size of a buffer which will be able to hold \f[I]nintents\f[] intents. Just like with \f[B]pmemobj_tx_log_snapshots_max_size\f[], application should not expect this function to return the same value between restarts, for the same reasons. This function is independent of transaction stage and can be called both inside and outside of transaction. .PP \f[B]pmemobj_tx_log_auto_alloc\f[]() disables (\f[I]on_off\f[] set to 0) or enables (\f[I]on_off\f[] set to 1) automatic allocation of internal logs of given \f[I]type\f[]. It can be used to verify that the buffer set with \f[B]pmemobj_tx_log_append_buffer\f[]() is big enough to hold the log, without reaching out\-of\-space scenario. .PP The \f[B]pmemobj_tx_set_user_data\f[]() function associates custom volatile state, represented by pointer \f[I]data\f[], with the current transaction. This state can later be retrieved using \f[B]pmemobj_tx_get_user_data\f[]() function. If \f[B]pmemobj_tx_set_user_data\f[]() was not called for a current transaction, \f[B]pmemobj_tx_get_user_data\f[]() will return NULL. These functions must be called during \f[B]TX_STAGE_WORK\f[] or \f[B]TX_STAGE_ONABORT\f[] or \f[B]TX_STAGE_ONCOMMIT\f[] or \f[B]TX_STAGE_FINALLY\f[]. .SH RETURN VALUE .PP The \f[B]pmemobj_tx_stage\f[]() function returns the stage of the current transaction stage for a thread. .PP On success, \f[B]pmemobj_tx_begin\f[]() returns 0. Otherwise, an error number is returned. .PP The \f[B]pmemobj_tx_begin\f[]() and \f[B]pmemobj_tx_lock\f[]() functions return zero if \f[I]lockp\f[] is successfully added to the transaction. Otherwise, an error number is returned. .PP The \f[B]pmemobj_tx_xlock\f[]() function return zero if \f[I]lockp\f[] is successfully added to the transaction. Otherwise, the error number is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XLOCK_NO_ABORT\f[], the transaction is aborted. .PP The \f[B]pmemobj_tx_abort\f[]() and \f[B]pmemobj_tx_commit\f[]() functions return no value. .PP The \f[B]pmemobj_tx_end\f[]() function returns 0 if the transaction was successful. Otherwise it returns the error code set by \f[B]pmemobj_tx_abort\f[](). Note that \f[B]pmemobj_tx_abort\f[]() can be called internally by the library. .PP The \f[B]pmemobj_tx_errno\f[]() function returns the error code of the last transaction. .PP The \f[B]pmemobj_tx_process\f[]() function returns no value. .PP On success, \f[B]pmemobj_tx_log_append_buffer\f[]() returns 0. Otherwise, the stage is changed to \f[B]TX_STAGE_ONABORT\f[], \f[B]errno\f[] is set appropriately and transaction is aborted. .PP On success, \f[B]pmemobj_tx_xlog_append_buffer\f[]() returns 0. Otherwise, the error number is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XLOG_NO_ABORT\f[], the transaction is aborted. .PP On success, \f[B]pmemobj_tx_log_auto_alloc\f[]() returns 0. Otherwise, the transaction is aborted and an error number is returned. .PP On success, \f[B]pmemobj_tx_log_snapshots_max_size\f[]() returns size of the buffer. On failure it returns \f[I]SIZE_MAX\f[] and sets \f[I]errno\f[] appropriately. .PP On success, \f[B]pmemobj_tx_log_intents_max_size\f[]() returns size of the buffer. On failure it returns \f[I]SIZE_MAX\f[] and sets \f[I]errno\f[] appropriately. .SH CAVEATS .PP Transaction flow control is governed by the \f[B]setjmp\f[](3) and \f[B]longjmp\f[](3) macros, and they are used in both the macro and function flavors of the API. The transaction will longjmp on transaction abort. This has one major drawback, which is described in the ISO C standard subsection 7.13.2.1. It says that \f[B]the values of objects of automatic storage duration that are local to the function containing the setjmp invocation that do not have volatile\-qualified type and have been changed between the setjmp invocation and longjmp call are indeterminate.\f[] .PP The following example illustrates the issue described above. .IP .nf \f[C] int\ *bad_example_1\ =\ (int\ *)0xBAADF00D; int\ *bad_example_2\ =\ (int\ *)0xBAADF00D; int\ *bad_example_3\ =\ (int\ *)0xBAADF00D; int\ *\ volatile\ good_example\ =\ (int\ *)0xBAADF00D; TX_BEGIN(pop)\ { \ \ \ \ bad_example_1\ =\ malloc(sizeof(int)); \ \ \ \ bad_example_2\ =\ malloc(sizeof(int)); \ \ \ \ bad_example_3\ =\ malloc(sizeof(int)); \ \ \ \ good_example\ =\ malloc(sizeof(int)); \ \ \ \ /*\ manual\ or\ library\ abort\ called\ here\ */ \ \ \ \ pmemobj_tx_abort(EINVAL); }\ TX_ONCOMMIT\ { \ \ \ \ /* \ \ \ \ \ *\ This\ section\ is\ longjmp\-safe \ \ \ \ \ */ }\ TX_ONABORT\ { \ \ \ \ /* \ \ \ \ \ *\ This\ section\ is\ not\ longjmp\-safe \ \ \ \ \ */ \ \ \ \ free(good_example);\ /*\ OK\ */ \ \ \ \ free(bad_example_1);\ /*\ undefined\ behavior\ */ }\ TX_FINALLY\ { \ \ \ \ /* \ \ \ \ \ *\ This\ section\ is\ not\ longjmp\-safe\ on\ transaction\ abort\ only \ \ \ \ \ */ \ \ \ \ free(bad_example_2);\ /*\ undefined\ behavior\ */ }\ TX_END free(bad_example_3);\ /*\ undefined\ behavior\ */ \f[] .fi .PP Objects which are not volatile\-qualified, are of automatic storage duration and have been changed between the invocations of \f[B]setjmp\f[](3) and \f[B]longjmp\f[](3) (that also means within the work section of the transaction after \f[B]TX_BEGIN\f[]()) should not be used after a transaction abort, or should be used with utmost care. This also includes code after the \f[B]TX_END\f[] macro. .PP \f[B]libpmemobj\f[](7) is not cancellation\-safe. The pool will never be corrupted because of a canceled thread, but other threads may stall waiting on locks taken by that thread. If the application wants to use \f[B]pthread_cancel\f[](3), it must disable cancellation before calling any \f[B]libpmemobj\f[](7) APIs (see \f[B]pthread_setcancelstate\f[](3) with \f[B]PTHREAD_CANCEL_DISABLE\f[]), and re\-enable it afterwards. Deferring cancellation (\f[B]pthread_setcanceltype\f[](3) with \f[B]PTHREAD_CANCEL_DEFERRED\f[]) is not safe enough, because \f[B]libpmemobj\f[](7) internally may call functions that are specified as cancellation points in POSIX. .PP \f[B]libpmemobj\f[](7) relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. \f[B]dlclose\f[](3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. .SH SEE ALSO .PP \f[B]dlclose\f[](3), \f[B]longjmp\f[](3), \f[B]pmemobj_tx_add_range\f[](3), \f[B]pmemobj_tx_alloc\f[](3), \f[B]pthread_setcancelstate\f[](3), \f[B]pthread_setcanceltype\f[](3), \f[B]setjmp\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pobj_list_move_element_before.30000664000000000000000000000002513615011243022025 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_alloc.3.md0000664000000000000000000003777513615011243017207 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_ALLOC, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2020, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_alloc.3 -- man page for non-transactional atomic allocations) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemobj_alloc**(), **pmemobj_xalloc**(), **pmemobj_zalloc**(), **pmemobj_realloc**(), **pmemobj_zrealloc**(), **pmemobj_strdup**(), **pmemobj_wcsdup**(), **pmemobj_alloc_usable_size**(), **pmemobj_defrag**(), **POBJ_NEW**(), **POBJ_ALLOC**(), **POBJ_ZNEW**(), **POBJ_ZALLOC**(), **POBJ_REALLOC**(), **POBJ_ZREALLOC**(), **POBJ_FREE**() - non-transactional atomic allocations # SYNOPSIS # ```c #include typedef int (*pmemobj_constr)(**PMEMobjpool *pop, void *ptr, void *arg); int pmemobj_alloc(PMEMobjpool *pop, PMEMoid *oidp, size_t size, uint64_t type_num, pmemobj_constr constructor, void *arg); int pmemobj_xalloc(PMEMobjpool *pop, PMEMoid *oidp, size_t size, uint64_t type_num, uint64_t flags, pmemobj_constr constructor, void *arg); (EXPERIMENTAL) int pmemobj_zalloc(PMEMobjpool *pop, PMEMoid *oidp, size_t size, uint64_t type_num); void pmemobj_free(PMEMoid *oidp); int pmemobj_realloc(PMEMobjpool *pop, PMEMoid *oidp, size_t size, uint64_t type_num); int pmemobj_zrealloc(PMEMobjpool *pop, PMEMoid *oidp, size_t size, uint64_t type_num); int pmemobj_strdup(PMEMobjpool *pop, PMEMoid *oidp, const char *s, uint64_t type_num); int pmemobj_wcsdup(PMEMobjpool *pop, PMEMoid *oidp, const wchar_t *s, uint64_t type_num); size_t pmemobj_alloc_usable_size(PMEMoid oid); int pmemobj_defrag(PMEMobjpool *pop, PMEMoid **oidv, size_t oidcnt, struct pobj_defrag_result *result); POBJ_NEW(PMEMobjpool *pop, TOID *oidp, TYPE, pmemobj_constr constructor, void *arg) POBJ_ALLOC(PMEMobjpool *pop, TOID *oidp, TYPE, size_t size, pmemobj_constr constructor, void *arg) POBJ_ZNEW(PMEMobjpool *pop, TOID *oidp, TYPE) POBJ_ZALLOC(PMEMobjpool *pop, TOID *oidp, TYPE, size_t size) POBJ_REALLOC(PMEMobjpool *pop, TOID *oidp, TYPE, size_t size) POBJ_ZREALLOC(PMEMobjpool *pop, TOID *oidp, TYPE, size_t size) POBJ_FREE(TOID *oidp) ``` # DESCRIPTION # Functions described in this document provide the mechanism to allocate, resize and free objects from the persistent memory pool in a thread-safe and fail-safe manner. All the routines are atomic with respect to other threads and any power-fail interruptions. If any of these operations is torn by program failure or system crash, on recovery they are guaranteed to be entirely completed or discarded, leaving the persistent memory heap and internal object containers in a consistent state. All these functions should be used outside transactions. If executed within an open transaction they are considered durable immediately after completion. Changes made with these functions will not be rolled back if the transaction is aborted or interrupted. They have no information about other changes made by transactional API, so if the same data is modified in a single transaction using transactional and then non-transactional API, transaction abort will likely corrupt the data. The allocations are always aligned to a cache-line boundary. The *pmemobj_constr* type represents a constructor for atomic allocation from the persistent memory heap associated with memory pool *pop*. *ptr* is a pointer to the allocated memory area and *arg* is a user-defined argument passed to the constructor. The **pmemobj_alloc**() function allocates a new object from the persistent memory heap associated with memory pool *pop*. The *PMEMoid* of the allocated object is stored in *oidp*. If *oidp* is NULL, then the newly allocated object may be accessed only by iterating objects in the object container associated with the type number *type_num*, as described in **POBJ_FOREACH**(3). If *oidp* points to a memory location from the **pmemobj** heap, *oidp* is modified atomically. Before returning, **pmemobj_alloc**() calls the *constructor* function, passing the pool handle *pop*, the pointer to the newly allocated object in *ptr*, and the *arg* argument. It is guaranteed that the allocated object is either properly initialized, or if the allocation is interrupted before the constructor completes, the memory space reserved for the object is reclaimed. *size* can be any non-zero value; however, due to internal padding and object metadata, the actual size of the allocation will differ from the requested size by at least 64 bytes. For this reason, making allocations of a size less than 64 bytes is extremely inefficient and discouraged. The allocated object is added to the internal container associated with *type_num*. **pmemobj_xalloc**() is equivalent to **pmemobj_alloc**(), but with an additional *flags* argument that is a bitmask of the following values: + **POBJ_XALLOC_ZERO** - zero the allocated object (equivalent of **pmemobj_zalloc**()) + **POBJ_CLASS_ID(class_id)** - allocate an object from the allocation class *class_id*. The class id cannot be 0. + **POBJ_ARENA_ID(arena_id)** - allocate an object from the arena specified by *arena_id*. The arena must exist, otherwise, the behavior is undefined. If *arena_id* is equal 0, then arena assigned to the current thread will be used. The **pmemobj_zalloc**() function allocates a new zeroed object from the persistent memory heap associated with memory pool *pop*. The *PMEMoid* of the allocated object is stored in *oidp*. If *oidp* is NULL, then the newly allocated object may be accessed only by iterating objects in the object container associated with the type number *type_num*, as described in **POBJ_FOREACH**(3). If *oidp* points to a memory location from the **pmemobj** heap, *oidp* is modified atomically. *size* can be any non-zero value; however, due to internal padding and object metadata, the actual size of the allocation will differ from the requested one by at least 64 bytes. For this reason, making allocations of a size less than 64 bytes is extremely inefficient and discouraged. The allocated object is added to the internal container associated with *type_num*. The **pmemobj_free**() function frees the memory space represented by *oidp*, which must have been allocated by a previous call to **pmemobj_alloc**(), **pmemobj_xalloc**(), **pmemobj_zalloc**(), **pmemobj_realloc**(), or **pmemobj_zrealloc**(). **pmemobj_free**() provides the same semantics as **free**(3), but instead of operating on the process heap supplied by the system, it operates on the persistent memory heap. If *oidp* is **OID_NULL**, no operation is performed. If *oidp* is NULL or if it points to the root object's *OID*, the behavior of **pmemobj_free**() is undefined. *oidp* is set to **OID_NULL** after the memory is freed. If *oidp* points to a memory location from the **pmemobj** heap, *oidp* is modified atomically. The **pmemobj_realloc**() function changes the size of the object represented by *oidp* to *size* bytes. **pmemobj_realloc**() provides similar semantics to **realloc**(3), but operates on the persistent memory heap associated with memory pool *pop*. The resized object is also added or moved to the internal container associated with type number *type_num*. The contents will be unchanged in the range from the start of the region up to the minimum of the old and new sizes. If the new size is larger than the old size, the added memory will *not* be initialized. If *oidp* is *OID_NULL*, then the call is equivalent to *pmemobj_alloc(pop, size, type_num)*. If *size* is equal to zero, and *oidp* is not **OID_NULL**, then the call is equivalent to *pmemobj_free(oid)*. Unless *oidp* is **OID_NULL**, it must have been allocated by an earlier call to **pmemobj_alloc**(), **pmemobj_xalloc**(), **pmemobj_zalloc**(), **pmemobj_realloc**(), or **pmemobj_zrealloc**(). Note that the object handle value may change as a result of reallocation. If the object was moved, the memory space represented by *oid* is reclaimed. If *oidp* points to a memory location from the **pmemobj** heap, *oidp* is modified atomically. If *oidp* is NULL or if it points to the root object's *OID*, the behavior of **pmemobj_realloc**() is undefined. **pmemobj_zrealloc**() is equivalent to **pmemobj_realloc**(), except that if the new size is larger than the old size, the added memory will be zeroed. The **pmemobj_strdup**() function stores a handle to a new object in *oidp* which is a duplicate of the string *s*. **pmemobj_strdup**() provides the same semantics as **strdup**(3), but operates on the persistent memory heap associated with memory pool *pop*. If *oidp* is NULL, then the newly allocated object may be accessed only by iterating objects in the object container associated with type number *type_num*, as described in **POBJ_FOREACH**(3). If *oidp* points to a memory location from the **pmemobj** heap, *oidp* is modified atomically. The allocated string object is also added to the internal container associated with type number *type_num*. Memory for the new string is obtained with **pmemobj_alloc**(), on the given memory pool, and can be freed with **pmemobj_free**() on the same memory pool. **pmemobj_wcsdup**() is equivalent to **pmemobj_strdup**(), but operates on a wide character string (wchar_t) rather than a standard character string. The **pmemobj_alloc_usable_size**() function provides the same semantics as **malloc_usable_size**(3), but instead of the process heap supplied by the system, it operates on the persistent memory heap. The **POBJ_NEW**() macro is a wrapper around the **pmemobj_alloc**() function. Instead of taking a pointer to *PMEMoid*, it takes a pointer to the typed *OID* of type name *TYPE*, and passes the size and type number from the typed *OID* to **pmemobj_alloc**(). The **POBJ_ALLOC**() macro is equivalent to **POBJ_NEW**, except that instead of using the size of the typed *OID*, passes *size* to **pmemobj_alloc**(). The **POBJ_ZNEW**() macro is a wrapper around the **pmemobj_zalloc**() function. Instead of taking a pointer to *PMEMoid*, it takes a pointer to the typed *OID* of type name *TYPE*, and passes the size and type number from the typed *OID* to **pmemobj_zalloc**(). The **POBJ_ZALLOC**() macro is equivalent to **POBJ_ZNEW**, except that instead of using the size of the typed *OID*, passes *size* to **pmemobj_zalloc**(). The **POBJ_REALLOC**() macro is a wrapper around the **pmemobj_realloc**() function. Instead of taking a pointer to *PMEMoid*, it takes a pointer to the typed *OID* of type name *TYPE*, and passes the type number from the typed *OID* to **pmemobj_realloc**(). The **POBJ_ZREALLOC**() macro is a wrapper around the **pmemobj_zrealloc**() function. Instead of taking a pointer to *PMEMoid*, it takes a pointer to the typed *OID* of type name *TYPE*, and passes the type number from the typed *OID* to **pmemobj_zrealloc**(). The **POBJ_FREE**() macro is a wrapper around the **pmemobj_free**() function which takes a pointer to the typed *OID* instead of to *PMEMoid*. The **pmemobj_defrag**() function performs defragmentation on the objects provided through the array of pointers to PMEMoids *oidv* with size *oidcnt*. If an object from the provided array is selected to be moved to a new location in the heap, it is reallocated and all provided pointers to that object are atomically updated. To maintain data structure consistency, applications should always provide all pointers for an object to **pmemobj_defrag** method. This ensures that, even in the presence of failures, all pointers to the object will either point to the old or a new location. All objects and pointers to objects should belong to the pool *pop* or, in case of pointers, can also reside in volatile memory. Defragmentation across pools is not supported. Objects in the array that are *OID_NULL* are skipped over and no operation is performed on them. All other objects must have been allocated by an earlier call to **pmemobj_alloc**(), **pmemobj_xalloc**(), **pmemobj_zalloc**(), **pmemobj_realloc**(), **pmemobj_zrealloc**(), **pmemobj_strdup**() or **pmemobj_wcsdup**(). The *result* variable is an instance of *struct pobj_defrag_result* and, if not NULL, can be used to read *total*, the number of objects found that were processed, and *relocated*, the number of objects that were relocated during defragmentation. These variables are always initialized and can be non-zero, even if the return value of **pmemobj_defrag**() indicated a failure. This is because the failure might have occurred after some objects were already processed. # RETURN VALUE # On success, **pmemobj_alloc**() and **pmemobj_xalloc** return 0. If *oidp* is not NULL, the *PMEMoid* of the newly allocated object is stored in *oidp*. If the allocation fails, -1 is returned and *errno* is set appropriately. If the constructor returns a non-zero value, the allocation is canceled, -1 is returned, and *errno* is set to **ECANCELED**. If *size* equals 0, or the *flags* for **pmemobj_xalloc** are invalid, -1 is returned, *errno* is set to **EINVAL**, and *oidp* is left untouched. On success, **pmemobj_zalloc**() returns 0. If *oidp* is not NULL, the *PMEMoid* of the newly allocated object is stored in *oidp*. If the allocation fails, it returns -1 and sets *errno* appropriately. If *size* equals 0, it returns -1, sets *errno* to **EINVAL**, and leaves *oidp* untouched. The **pmemobj_free**() function returns no value. On success, **pmemobj_realloc**() and **pmemobj_zrealloc**() return 0 and update *oidp* if necessary. On error, they return -1 and set *errno* appropriately. On success, **pmemobj_strdup**() and **pmemobj_wcsdup**() return 0. If *oidp* is not NULL, the *PMEMoid* of the duplicated string object is stored in *oidp*. If *s* is NULL, they return -1, set *errno* to **EINVAL**, and leave *oidp* untouched. On other errors, they return -1 and set *errno* appropriately. The **pmemobj_alloc_usable_size**() function returns the number of usable bytes in the object represented by *oid*. If *oid* is **OID_NULL**, it returns 0. On success, **pmemobj_defrag**() returns 0. If defragmentation was unsuccessful or only partially successful (i.e. if it was aborted halfway through due to lack of resources), -1 is returned. # SEE ALSO # **free**(3), **POBJ_FOREACH**(3), **realloc**(3), **strdup**(3), **wcsdup**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pobj_list_move_element_tail.30000664000000000000000000000002513615011243021514 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pobj_list_insert_new_before.30000664000000000000000000000002513615011243021523 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pobj_first_type_num.30000664000000000000000000000002413615011243020037 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/toid_typeof.30000664000000000000000000000002313615011243016302 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/toid_type_num_of.30000664000000000000000000000002313615011243017320 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_xfree.30000664000000000000000000000002713615011243017316 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/tx_new.30000664000000000000000000000002713615011243015265 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/tx_onabort.30000664000000000000000000000002713615011243016140 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_abort.30000664000000000000000000000002713615011243017314 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_rwlock_tryrdlock.30000664000000000000000000000003113615011243021063 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pmemobj_rwlock_rdlock.30000664000000000000000000000003113615011243020324 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pobj_list_last.30000664000000000000000000000002513615011243016767 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_open.30000644000000000000000000002337613615011417016450 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_OPEN" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_open\f[](), \f[B]pmemobj_create\f[](), \f[B]pmemobj_close\f[](), \f[B]pmemobj_check\f[]() \f[B]pmemobj_set_user_data\f[](), \f[B]pmemobj_get_user_data\f[]() \- create, open, close and validate persistent memory transactional object store .SH SYNOPSIS .IP .nf \f[C] #include\ PMEMobjpool\ *pmemobj_open(const\ char\ *path,\ const\ char\ *layout); PMEMobjpool\ *pmemobj_create(const\ char\ *path,\ const\ char\ *layout, \ \ \ \ size_t\ poolsize,\ mode_t\ mode); void\ pmemobj_close(PMEMobjpool\ *pop); int\ pmemobj_check(const\ char\ *path,\ const\ char\ *layout); void\ pmemobj_set_user_data(PMEMobjpool\ *pop,\ void\ *data); void\ *pmemobj_get_user_data(PMEMobjpool\ *pop); \f[] .fi .SH DESCRIPTION .PP To use the pmem\-resident transactional object store provided by \f[B]libpmemobj\f[](7), a \f[I]memory pool\f[] must first be created with the \f[B]pmemobj_create\f[]() function described below. Existing pools may be opened with the \f[B]pmemobj_open\f[]() function. .PP None of the three functions described below are thread\-safe with respect to any other \f[B]libpmemobj\f[](7) function. In other words, when creating, opening or deleting a pool, nothing else in the library can happen in parallel, and therefore these functions should be called from the main thread. .PP Once created, the memory pool is represented by an opaque handle, of type \f[I]PMEMobjpool*\f[], which is passed to most of the other \f[B]libpmemobj\f[](7) functions. Internally, \f[B]libpmemobj\f[](7) will use either \f[B]pmem_persist\f[](3) or \f[B]msync\f[](2) when it needs to flush changes, depending on whether the memory pool appears to be persistent memory or a regular file (see the \f[B]pmem_is_pmem\f[](3) function in \f[B]libpmem\f[](7) for more information). There is no need for applications to flush changes directly when using the object memory API provided by \f[B]libpmemobj\f[](7). .PP The \f[B]pmemobj_create\f[]() function creates a transactional object store with the given total \f[I]poolsize\f[]. \f[I]path\f[] specifies the name of the memory pool file to be created. \f[I]layout\f[] specifies the application's layout type in the form of a string. The layout name is not interpreted by \f[B]libpmemobj\f[](7), but may be used as a check when \f[B]pmemobj_open\f[]() is called. The layout name, including the terminating null byte (`\\0'), cannot be longer than \f[B]PMEMOBJ_MAX_LAYOUT\f[] as defined in \f[B]\f[]. A NULL \f[I]layout\f[] is equivalent to using an empty string as a layout name. \f[I]mode\f[] specifies the permissions to use when creating the file, as described by \f[B]creat\f[](2). The memory pool file is fully allocated to the size \f[I]poolsize\f[] using \f[B]posix_fallocate\f[](3). The caller may choose to take responsibility for creating the memory pool file by creating it before calling \f[B]pmemobj_create\f[](), and then specifying \f[I]poolsize\f[] as zero. In this case \f[B]pmemobj_create\f[]() will take the pool size from the size of the existing file and will verify that the file appears to be empty by searching for any non\-zero data in the pool header at the beginning of the file. The minimum net pool size allowed by the library for a local transactional object store is defined in \f[B]\f[] as \f[B]PMEMOBJ_MIN_POOL\f[]. For remote replicas the minimum file size is defined in \f[B]\f[] as \f[B]RPMEM_MIN_PART\f[]. .PP Depending on the configuration of the system, the available non\-volatile memory space may be divided into multiple memory devices. In such case, the maximum size of the pmemobj memory pool could be limited by the capacity of a single memory device. \f[B]libpmemobj\f[](7) allows building persistent memory resident object store spanning multiple memory devices by creation of persistent memory pools consisting of multiple files, where each part of such a \f[I]pool set\f[] may be stored on a different memory device or pmem\-aware filesystem. .PP Creation of all the parts of the pool set can be done with \f[B]pmemobj_create\f[](); however, the recommended method for creating pool sets is with the \f[B]pmempool\f[](1) utility. .PP When creating a pool set consisting of multiple files, the \f[I]path\f[] argument passed to \f[B]pmemobj_create\f[]() must point to the special \f[I]set\f[] file that defines the pool layout and the location of all the parts of the pool set. The \f[I]poolsize\f[] argument must be 0. The meaning of the \f[I]layout\f[] and \f[I]mode\f[] arguments does not change, except that the same \f[I]mode\f[] is used for creation of all the parts of the pool set. .PP The \f[I]set\f[] file is a plain text file, the structure of which is described in \f[B]poolset\f[](5). .PP The \f[B]pmemobj_open\f[]() function opens an existing object store memory pool. Similar to \f[B]pmemobj_create\f[](), \f[I]path\f[] must identify either an existing obj memory pool file, or the \f[I]set\f[] file used to create a pool set. If \f[I]layout\f[] is non\-NULL, it is compared to the layout name provided to \f[B]pmemobj_create\f[]() when the pool was first created. This can be used to verify that the layout of the pool matches what was expected. The application must have permission to open the file and memory map it with read/write permissions. .PP Be aware that if the pool contains bad blocks inside, opening can be aborted by the SIGBUS signal, because currently the pool is not checked against bad blocks during opening. It can be turned on by setting the CHECK_BAD_BLOCKS compat feature. For details see description of this feature in \f[B]pmempool\-feature\f[](1). .PP The \f[B]pmemobj_close\f[]() function closes the memory pool indicated by \f[I]pop\f[] and deletes the memory pool handle. The object store itself lives on in the file that contains it and may be re\-opened at a later time using \f[B]pmemobj_open\f[]() as described above. .PP The \f[B]pmemobj_check\f[]() function performs a consistency check of the file indicated by \f[I]path\f[]. \f[B]pmemobj_check\f[]() opens the given \f[I]path\f[] read\-only so it never makes any changes to the file. This function is not supported on Device DAX. .PP The \f[B]pmemobj_set_user_data\f[]() function associates custom volatile state, represented by pointer \f[I]data\f[], with the given pool \f[I]pop\f[]. This state can later be retrieved using \f[B]pmemobj_get_user_data\f[]() function. This state does not survive pool close. If \f[B]pmemobj_set_user_data\f[]() was not called for a given pool, \f[B]pmemobj_get_user_data\f[]() will return NULL. .SH RETURN VALUE .PP The \f[B]pmemobj_create\f[]() function returns a memory pool handle to be used with most of the functions in \f[B]libpmemobj\f[](7). On error it returns NULL and sets \f[I]errno\f[] appropriately. .PP The \f[B]pmemobj_open\f[]() function returns a memory pool handle to be used with most of the functions in \f[B]libpmemobj\f[](7). If an error prevents the pool from being opened, or if the given \f[I]layout\f[] does not match the pool's layout, \f[B]pmemobj_open\f[]() returns NULL and sets \f[I]errno\f[] appropriately. .PP The \f[B]pmemobj_close\f[]() function returns no value. .PP The \f[B]pmemobj_check\f[]() function returns 1 if the memory pool is found to be consistent. Any inconsistencies found will cause \f[B]pmemobj_check\f[]() to return 0, in which case the use of the file with \f[B]libpmemobj\f[](7) will result in undefined behavior. The debug version of \f[B]libpmemobj\f[](7) will provide additional details on inconsistencies when \f[B]PMEMOBJ_LOG_LEVEL\f[] is at least 1, as described in the \f[B]DEBUGGING AND ERROR HANDLING\f[] section in \f[B]libpmemobj\f[](7). \f[B]pmemobj_check\f[]() returns \-1 and sets \f[I]errno\f[] if it cannot perform the consistency check due to other errors. .SH CAVEATS .PP Not all file systems support \f[B]posix_fallocate\f[](3). \f[B]pmemobj_create\f[]() will fail if the underlying file system does not support \f[B]posix_fallocate\f[](3). .SH SEE ALSO .PP \f[B]creat\f[](2), \f[B]msync\f[](2), \f[B]pmem_is_pmem\f[](3), \f[B]pmem_persist\f[](3), \f[B]posix_fallocate\f[](3), \f[B]libpmem\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/tx_memset.30000664000000000000000000000003313615011243015763 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_flush.30000664000000000000000000000003513615011243016612 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pobj_xreserve_alloc.30000664000000000000000000000002513615011243020006 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/pmemobj_memset_persist.30000664000000000000000000000003513615011243020534 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/.gitignore0000664000000000000000000000046113615011243015667 0ustar rootrootlibpmemobj.7 oid_is_null.3 pmemobj_action.3 pmemobj_alloc.3 pmemobj_ctl_get.3 pmemobj_first.3 pmemobj_list_insert.3 pmemobj_memcpy_persist.3 pmemobj_mutex_zero.3 pmemobj_open.3 pmemobj_root.3 pmemobj_tx_add_range.3 pmemobj_tx_alloc.3 pmemobj_tx_begin.3 pobj_layout_begin.3 pobj_list_head.3 toid_declare.3 pmdk-1.8/doc/libpmemobj/pmemobj_action.30000644000000000000000000002300713615011417016753 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_ACTION" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_reserve\f[](), \f[B]pmemobj_xreserve\f[](), \f[B]pmemobj_defer_free\f[](), \f[B]pmemobj_set_value\f[](), \f[B]pmemobj_publish\f[](), \f[B]pmemobj_tx_publish\f[](), \f[B]pmemobj_tx_xpublish\f[](), \f[B]pmemobj_cancel\f[](), \f[B]POBJ_RESERVE_NEW\f[](), \f[B]POBJ_RESERVE_ALLOC\f[](), \f[B]POBJ_XRESERVE_NEW\f[](),\f[B]POBJ_XRESERVE_ALLOC\f[]() \- Delayed atomicity actions (EXPERIMENTAL) .SH SYNOPSIS .IP .nf \f[C] #include\ PMEMoid\ pmemobj_reserve(PMEMobjpool\ *pop,\ struct\ pobj_action\ *act, \ \ \ \ size_t\ size,\ uint64_t\ type_num);\ (EXPERIMENTAL) PMEMoid\ pmemobj_xreserve(PMEMobjpool\ *pop,\ struct\ pobj_action\ *act, \ \ \ \ size_t\ size,\ uint64_t\ type_num,\ uint64_t\ flags);\ (EXPERIMENTAL) void\ pmemobj_defer_free(PMEMobjpool\ *pop,\ PMEMoid\ oid,\ struct\ pobj_action\ *act); void\ pmemobj_set_value(PMEMobjpool\ *pop,\ struct\ pobj_action\ *act, \ \ \ \ uint64_t\ *ptr,\ uint64_t\ value);\ (EXPERIMENTAL) int\ pmemobj_publish(PMEMobjpool\ *pop,\ struct\ pobj_action\ *actv, \ \ \ \ size_t\ actvcnt);\ (EXPERIMENTAL) int\ pmemobj_tx_publish(struct\ pobj_action\ *actv,\ size_t\ actvcnt);\ (EXPERIMENTAL) int\ pmemobj_tx_xpublish(struct\ pobj_action\ *actv,\ size_t\ actvcnt,\ uint64_t\ flags);\ (EXPERIMENTAL) void\ pmemobj_cancel(PMEMobjpool\ *pop,\ struct\ pobj_action\ *actv, \ \ \ \ size_t\ actvcnt);\ (EXPERIMENTAL) POBJ_RESERVE_NEW(pop,\ t,\ act)\ (EXPERIMENTAL) POBJ_RESERVE_ALLOC(pop,\ t,\ size,\ act)\ (EXPERIMENTAL) POBJ_XRESERVE_NEW(pop,\ t,\ act,\ flags)\ (EXPERIMENTAL) POBJ_XRESERVE_ALLOC(pop,\ t,\ size,\ act,\ flags)\ (EXPERIMENTAL) \f[] .fi .SH DESCRIPTION .PP All of the functions described so far have an immediate effect on the persistent state of the pool, and as such, the cost of maintaining fail\-safety is paid outright and, most importantly, in the calling thread. This behavior makes implementing algorithms involving relaxed consistency guarantees difficult, if not outright impossible. .PP The following set of functions introduce a mechanism that allows one to delay the persistent publication of a set of prepared actions to an arbitrary moment in time of the execution of a program. .PP The publication is fail\-safe atomic in the scope of the entire collection of actions. If a program exits without publishing the actions, or the actions are canceled, any resources reserved by those actions are released and placed back in the pool. .PP A single action is represented by a single \f[C]struct\ pobj_action\f[]. Functions that create actions take that structure by pointer, whereas functions that publish actions take array of actions and the size of the array. The actions can be created, and published, from different threads. When creating actions, the \f[I]act\f[] argument must be non\-NULL and point to a \f[C]struct\ pobj_action\f[], the structure will be populated by the function and must not be modified or deallocated until after publishing. .PP The \f[B]pmemobj_reserve\f[]() functions performs a transient reservation of an object. Behaves similarly to \f[B]pmemobj_alloc\f[](3), but performs no modification to the persistent state. The object returned by this function can be freely modified without worrying about fail\-safe atomicity until the object has been published. Any modifications of the object must be manually persisted, just like in the case of the atomic API. .PP \f[B]pmemobj_xreserve\f[]() is equivalent to \f[B]pmemobj_reserve\f[](), but with an additional \f[I]flags\f[] argument that is a bitmask of the following values: .IP \[bu] 2 \f[B]POBJ_XALLOC_ZERO\f[] \- zero the allocated object (and persist it) .IP \[bu] 2 \f[B]POBJ_CLASS_ID(class_id)\f[] \- allocate an object from the allocation class \f[I]class_id\f[]. The class id cannot be 0. .IP \[bu] 2 \f[B]POBJ_ARENA_ID(arena_id)\f[] \- allocate an object from the arena specified by \f[I]arena_id\f[]. The arena must exist, otherwise, the behavior is undefined. If \f[I]arena_id\f[] is equal 0, then arena assigned to the current thread will be used. .PP \f[B]pmemobj_defer_free\f[]() function creates a deferred free action, meaning that the provided object will be freed when the action is published. Calling this function with a NULL OID is invalid and causes undefined behavior. .PP The \f[B]pmemobj_set_value\f[] function prepares an action that, once published, will modify the memory location pointed to by \f[I]ptr\f[] to \f[I]value\f[]. .PP The \f[B]pmemobj_publish\f[] function publishes the provided set of actions. The publication is fail\-safe atomic. Once done, the persistent state will reflect the changes contained in the actions. .PP The \f[B]pmemobj_tx_publish\f[] function moves the provided actions to the scope of the transaction in which it is called. Only object reservations are supported in transactional publish. Once done, the reserved objects will follow normal transactional semantics. Can only be called during \f[I]TX_STAGE_WORK\f[]. .PP The \f[B]pmemobj_tx_xpublish\f[]() function behaves exactly the same as \f[B]pmemobj_tx_publish\f[]() when \f[I]flags\f[] equals zero. \f[I]flags\f[] is a bitmask of the following values: .IP \[bu] 2 \f[B]POBJ_XPUBLISH_NO_ABORT\f[] \- if the function does not end successfully, do not abort the transaction. .PP The \f[B]pmemobj_cancel\f[] function releases any resources held by the provided set of actions and invalidates all actions. .PP The \f[B]POBJ_RESERVE_NEW\f[] macro is a typed variant of \f[B]pmemobj_reserve\f[]. The size of the reservation is determined from the provided type \f[I]t\f[]. .PP The \f[B]POBJ_RESERVE_ALLOC\f[] macro is a typed variant of \f[B]pmemobj_reserve\f[]. The \f[I]size\f[] of the reservation is user\-provided. .PP The \f[B]POBJ_XRESERVE_NEW\f[] and the \f[B]POBJ_XRESERVE_ALLOC\f[] macros are equivalent to \f[B]POBJ_RESERVE_NEW\f[] and the \f[B]POBJ_RESERVE_ALLOC\f[], but with an additional \f[I]flags\f[] argument defined for \f[B]pmemobj_xreserve\f[](). .SH EXAMPLES .PP The following code shows atomic append of two objects into a singly linked list. .IP .nf \f[C] struct\ list_node\ { \ \ \ \ int\ value; \ \ \ \ PMEMoid\ next; }; /*\ statically\ allocate\ the\ array\ of\ actions\ */ struct\ pobj_action\ actv[4]; /*\ reserve,\ populate\ and\ persist\ the\ first\ object\ */ PMEMoid\ tail\ =\ pmemobj_reserve(pop,\ &actv[0],\ sizeof(struct\ list_node),\ 0); if\ (TOID_IS_NULL(tail)) \ \ \ \ return\ \-1; D_RW(tail)\->value\ =\ 1; D_RW(tail)\->next\ =\ OID_NULL; pmemobj_persist(pop,\ D_RW(tail),\ sizeof(struct\ list_node)); /*\ reserve,\ populate\ and\ persist\ the\ second\ object\ */ PMEMoid\ head\ =\ pmemobj_reserve(pop,\ &actv[1],\ sizeof(struct\ list_node),\ 0); if\ (TOID_IS_NULL(head)) \ \ \ \ return\ \-1; D_RW(head)\->value\ =\ 2; D_RW(head)\->next\ =\ tail; pmemobj_persist(pop,\ D_RW(head),\ sizeof(struct\ list_node)); /*\ create\ actions\ to\ set\ the\ PMEMoid\ to\ the\ new\ values\ */ pmemobj_set_value(pop,\ &actv[2],\ &D_RO(root)\->head.pool_uuid_lo,\ head.pool_uuid_lo); pmemobj_set_value(pop,\ &actv[3],\ &D_RO(root)\->head.off,\ head.off); /*\ atomically\ publish\ the\ above\ actions\ */ pmemobj_publish(pop,\ actv,\ 4); \f[] .fi .SH RETURN VALUE .PP On success, \f[B]pmemobj_reserve\f[]() functions return a handle to the newly reserved object. Otherwise an \f[I]OID_NULL\f[] is returned. .PP On success, \f[B]pmemobj_tx_publish\f[]() returns 0. Otherwise, the transaction is aborted, the stage is changed to \f[I]TX_STAGE_ONABORT\f[] and \f[I]errno\f[] is set appropriately. .PP On success, \f[B]pmemobj_tx_xpublish\f[]() returns 0. Otherwise, the error number is returned, \f[B]errno\f[] is set and when flags do not contain \f[B]POBJ_XPUBLISH_NO_ABORT\f[], the transaction is aborted. .PP On success, \f[B]pmemobj_publish\f[]() returns 0. Otherwise, returns \-1 and \f[I]errno\f[] is set appropriately. .SH SEE ALSO .PP \f[B]pmemobj_alloc\f[](3), \f[B]pmemobj_tx_alloc\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_tx_alloc.3.md0000664000000000000000000003350013615011243017700 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_TX_ALLOC, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_tx_alloc.3 -- man page for transactional object manipulation) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[SEE ALSO](#see-also)
# NAME # **pmemobj_tx_alloc**(), **pmemobj_tx_zalloc**(), **pmemobj_tx_xalloc**(), **pmemobj_tx_realloc**(), **pmemobj_tx_zrealloc**(), **pmemobj_tx_strdup**(), **pmemobj_tx_xstrdup**(), **pmemobj_tx_wcsdup**(), **pmemobj_tx_xwcsdup**(), **pmemobj_tx_free**(), **pmemobj_tx_xfree**() **TX_NEW**(), **TX_ALLOC**(), **TX_ZNEW**(), **TX_ZALLOC**(), **TX_XALLOC**(), **TX_REALLOC**(), **TX_ZREALLOC**(), **TX_STRDUP**(), **TX_XSTRDUP**(), **TX_WCSDUP**(), **TX_XWCSDUP**(), **TX_FREE**(), **TX_XFREE**() - transactional object manipulation # SYNOPSIS # ```c #include PMEMoid pmemobj_tx_alloc(size_t size, uint64_t type_num); PMEMoid pmemobj_tx_zalloc(size_t size, uint64_t type_num); PMEMoid pmemobj_tx_xalloc(size_t size, uint64_t type_num, uint64_t flags); PMEMoid pmemobj_tx_realloc(PMEMoid oid, size_t size, uint64_t type_num); PMEMoid pmemobj_tx_zrealloc(PMEMoid oid, size_t size, uint64_t type_num); PMEMoid pmemobj_tx_strdup(const char *s, uint64_t type_num); PMEMoid pmemobj_tx_wcsdup(const wchar_t *s, uint64_t type_num); int pmemobj_tx_free(PMEMoid oid); int pmemobj_tx_xfree(PMEMoid oid, uint64_t flags); TX_NEW(TYPE) TX_ALLOC(TYPE, size_t size) TX_ZNEW(TYPE) TX_ZALLOC(TYPE, size_t size) TX_XALLOC(TYPE, size_t size, uint64_t flags) TX_REALLOC(TOID o, size_t size) TX_ZREALLOC(TOID o, size_t size) TX_STRDUP(const char *s, uint64_t type_num) TX_WCSDUP(const wchar_t *s, uint64_t type_num) TX_FREE(TOID o) TX_XFREE(TOID o, uint64_t flags) ``` # DESCRIPTION # The **pmemobj_tx_alloc**() function transactionally allocates a new object of given *size* and *type_num*. In contrast to the non-transactional allocations, the objects are added to the internal object containers of given *type_num* only after the transaction is committed, making the objects visible to the **POBJ_FOREACH_\***() macros. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_zalloc**() function transactionally allocates a new zeroed object of given *size* and *type_num*. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_xalloc**() function transactionally allocates a new object of given *size* and *type_num*. The *flags* argument is a bitmask of the following values: + **POBJ_XALLOC_ZERO** - zero the allocated object (equivalent of pmemobj_tx_zalloc) + **POBJ_XALLOC_NO_FLUSH** - skip flush on commit (when application deals with flushing or uses pmemobj_memcpy_persist) + **POBJ_CLASS_ID(class_id)** - allocate an object from the allocation class with id equal to *class_id* + **POBJ_ARENA_ID(arena_id)** - allocate an object from the arena specified by *arena_id*. The arena must exist, otherwise, the behavior is undefined. If *arena_id* is equal 0, then arena assigned to the current thread will be used. + **POBJ_XALLOC_NO_ABORT** - if the function does not end successfully, do not abort the transaction. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_realloc**() function transactionally resizes an existing object to the given *size* and changes its type to *type_num*. If *oid* is **OID_NULL**, then the call is equivalent to *pmemobj_tx_alloc(pop, size, type_num)*. If *size* is equal to zero and *oid* is not **OID_NULL**, then the call is equivalent to *pmemobj_tx_free(oid)*. If the new size is larger than the old size, the added memory will *not* be initialized. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_zrealloc**() function transactionally resizes an existing object to the given *size* and changes its type to *type_num*. If the new size is larger than the old size, the extended new space is zeroed. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_strdup**() function transactionally allocates a new object containing a duplicate of the string *s* and assigns it a type *type_num*. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_xstrdup**() function behaves exactly the same as **pmemobj_tx_strdup**() when *flags* equals zero. The *flags* argument is a bitmask of values described in **pmemobj_tx_xalloc** section. The **pmemobj_tx_wcsdup**() function transactionally allocates a new object containing a duplicate of the wide character string *s* and assigns it a type *type_num*. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_xwcsdup**() function behaves exactly the same as **pmemobj_tx_wcsdup**() when *flags* equals zero. The *flags* argument is a bitmask of values described in **pmemobj_tx_xalloc** section. The **pmemobj_tx_free**() function transactionally frees an existing object referenced by *oid*. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_xfree**() function behaves exactly the same as **pmemobj_tx_free**() when *flags* equals zero. *flags* is a bitmask of the following value: + **POBJ_XFREE_NO_ABORT** - if the function does not end successfully, do not abort the transaction. This function must be called during **TX_STAGE_WORK**. The **TX_NEW**() macro transactionally allocates a new object of given *TYPE* and assigns it a type number read from the typed *OID*. The allocation size is determined from the size of the user-defined structure *TYPE*. If successful and called during **TX_STAGE_WORK** it returns a handle to the newly allocated object. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. The **TX_ALLOC**() macro transactionally allocates a new object of given *TYPE* and assigns it a type number read from the typed *OID*. The allocation size is passed by *size* parameter. If successful and called during **TX_STAGE_WORK** it returns a handle to the newly allocated object. Otherwise, the stage is set to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. The **TX_ZNEW**() macro transactionally allocates a new zeroed object of given *TYPE* and assigns it a type number read from the typed *OID*. The allocation size is determined from the size of the user-defined structure *TYPE*. If successful and called during **TX_STAGE_WORK** it returns a handle to the newly allocated object. Otherwise, stage changes to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. The **TX_ZALLOC**() macro transactionally allocates a new zeroed object of given *TYPE* and assigns it a type number read from the typed *OID*. The allocation size is passed by *size* argument. If successful and called during **TX_STAGE_WORK** it returns a handle to the newly allocated object. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. The **TX_XALLOC**() macro transactionally allocates a new object of given *TYPE* and assigns it a type number read from the typed *OID*. The allocation size is passed by *size* argument. The *flags* argument is a bitmask of values described in **pmemobj_tx_xalloc** section. If successful and called during **TX_STAGE_WORK** it returns a handle to the newly allocated object. Otherwise, the **OID_NULL** is returned, **errno** is set and when flags do not contain **POBJ_XALLOC_NO_ABORT**, the transaction is aborted. The **TX_REALLOC**() macro transactionally resizes an existing object referenced by a handle *o* to the given *size*. If successful and called during **TX_STAGE_WORK** it returns a handle to the reallocated object. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. The **TX_ZREALLOC**() macro transactionally resizes an existing object referenced by a handle *o* to the given *size*. If the new size is larger than the old size, the extended new space is zeroed. If successful and called during **TX_STAGE_WORK** it returns a handle to the reallocated object. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. The **TX_STRDUP**() macro transactionally allocates a new object containing a duplicate of the string *s* and assigns it type *type_num*. If successful and called during **TX_STAGE_WORK** it returns a handle to the newly allocated object. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. The **TX_XSTRDUP**() macro transactionally allocates a new object containing a duplicate of the string *s* and assigns it type *type_num*. The *flags* argument is a bitmask of values described in **pmemobj_tx_xalloc** section. If successful and called during **TX_STAGE_WORK** it returns a handle to the newly allocated object. Otherwise, the **OID_NULL** is returned, **errno** is set and when flags do not contain **POBJ_XALLOC_NO_ABORT**, the transaction is aborted. The **TX_WCSDUP**() macro transactionally allocates a new object containing a duplicate of the wide character string *s* and assigns it a type *type_num*. If successful and called during **TX_STAGE_WORK**, it returns a handle to the newly allocated object. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. The **TX_XWCSDUP**() macro transactionally allocates a new object containing a duplicate of the wide character string *s* and assigns it a type *type_num*. The *flags* argument is a bitmask of values described in **pmemobj_tx_xalloc** section. If successful and called during **TX_STAGE_WORK** it returns a handle to the newly allocated object. Otherwise, the **OID_NULL** is returned, **errno** is set and when flags do not contain **POBJ_XALLOC_NO_ABORT**, the transaction is aborted. The **TX_FREE**() macro transactionally frees the memory space represented by an object handle *o*. If *o* is **OID_NULL**, no operation is performed. If successful and called during **TX_STAGE_WORK**, **TX_FREE**() returns 0. Otherwise, the stage is changed to **TX_STAGE_ONABORT** and *errno* is set appropriately. The **TX_XFREE**() macro transactionally frees the memory space represented by an object handle *o*. If *o* is **OID_NULL**, no operation is performed. The *flags* argument is a bitmask of values described in **pmemobj_tx_xfree** section. If successful and called during **TX_STAGE_WORK**, **TX_FREE**() returns 0. Otherwise, the error number is returned, **errno** is set and when flags do not contain **POBJ_XFREE_NO_ABORT**, the transaction is aborted. # RETURN VALUE # On success, the **pmemobj_tx_alloc**(), **pmemobj_tx_zalloc**(), **pmemobj_tx_strdup**() and **pmemobj_tx_wcsdup**() functions return a handle to the newly allocated object. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. If *size* equals 0, **OID_NULL** is returned and *errno* is set appropriately. On success, the **pmemobj_tx_xalloc**(), **pmemobj_tx_xstrdup**() and **pmemobj_tx_xwcsdup**() functions return a handle to the newly allocated object. Otherwise, the **OID_NULL** is returned, **errno** is set and when flags do not contain **POBJ_XALLOC_NO_ABORT**, the transaction is aborted. On success, **pmemobj_tx_realloc**() and **pmemobj_tx_zrealloc**() return a handle to the resized object. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **OID_NULL** is returned, and *errno* is set appropriately. Note that the object handle value may change as a result of reallocation. On success, **pmemobj_tx_free**() returns 0. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **errno** is set appropriately and transaction is aborted On success **pmemobj_tx_xfree**() returns 0. Otherwise, the error number is returned, **errno** is set and when flags do not contain **POBJ_XFREE_NO_ABORT**, the transaction is aborted. # SEE ALSO # **pmemobj_tx_add_range**(3), **pmemobj_tx_begin**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/tx_oncommit.30000664000000000000000000000002713615011243016321 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_xalloc.30000664000000000000000000000002413615011243016751 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/tx_xstrdup.30000664000000000000000000000002713615011243016205 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/oid_instanceof.30000664000000000000000000000002313615011243016741 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pmemobj_zrealloc.30000664000000000000000000000002413615011243017302 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_zrealloc.30000664000000000000000000000002413615011243016603 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_foreach_safe_type.30000664000000000000000000000002413615011243020436 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/pmemobj_rwlock_unlock.30000664000000000000000000000003113615011243020341 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pobj_list_move_element_head.30000664000000000000000000000002513615011243021464 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_next.30000664000000000000000000000002413615011243016445 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/pmemobj_mutex_timedlock.30000664000000000000000000000003113615011243020662 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pobj_list_empty.30000664000000000000000000000002513615011243017162 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_xpersist.30000664000000000000000000000003513615011243017352 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pmemobj_ctl_get.3.md0000664000000000000000000004300613615011243017516 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_CTL_GET, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2020, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_ctl_get.3 -- man page for libpmemobj CTL) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[CTL NAMESPACE](#ctl-namespace)
[CTL EXTERNAL CONFIGURATION](#ctl-external-configuration)
[SEE ALSO](#see-also)
# NAME # _UW(pmemobj_ctl_get), _UW(pmemobj_ctl_set), _UW(pmemobj_ctl_exec) - Query and modify libpmemobj internal behavior (EXPERIMENTAL) # SYNOPSIS # ```c #include _UWFUNCR2(int, pmemobj_ctl_get, PMEMobjpool *pop, *name, void *arg, =q= (EXPERIMENTAL)=e=) _UWFUNCR2(int, pmemobj_ctl_set, PMEMobjpool *pop, *name, void *arg, =q= (EXPERIMENTAL)=e=) _UWFUNCR2(int, pmemobj_ctl_exec, PMEMobjpool *pop, *name, void *arg, =q= (EXPERIMENTAL)=e=) ``` _UNICODE() # DESCRIPTION # The _UW(pmemobj_ctl_get), _UW(pmemobj_ctl_set) and _UW(pmemobj_ctl_exec) functions provide a uniform interface for querying and modifying the internal behavior of **libpmemobj**(7) through the control (CTL) namespace. The *name* argument specifies an entry point as defined in the CTL namespace specification. The entry point description specifies whether the extra *arg* is required. Those two parameters together create a CTL query. The functions and the entry points are thread-safe unless indicated otherwise below. If there are special conditions for calling an entry point, they are explicitly stated in its description. The functions propagate the return value of the entry point. If either *name* or *arg* is invalid, -1 is returned. If the provided ctl query is valid, the CTL functions will always return 0 on success and -1 on failure, unless otherwise specified in the entry point description. See more in **pmem_ctl**(5) man page. # CTL NAMESPACE # prefault.at_create | rw | global | int | int | - | boolean If set, every page of the pool will be touched and written to when the pool is created, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the _UW(pmemobj_create) function. prefault.at_open | rw | global | int | int | - | boolean If set, every page of the pool will be touched and written to when the pool is opened, in order to trigger page allocation and minimize the performance impact of pagefaults. Affects only the _UW(pmemobj_open) function. sds.at_create | rw | global | int | int | - | boolean If set, force-enables or force-disables SDS feature during pool creation. Affects only the _UW(pmemobj_create) function. See **pmempool_feature_query**(3) for information about SDS (SHUTDOWN_STATE) feature. copy_on_write.at_open | rw | global | int | int | - | boolean If set, pool is mapped in such a way that modifications don't reach the underlying medium. From the user's perspective this means that when the pool is closed all changes are reverted. This feature is not supported for pools located on Device DAX. tx.debug.skip_expensive_checks | rw | - | int | int | - | boolean Turns off some expensive checks performed by the transaction module in "debug" builds. Ignored in "release" builds. tx.debug.verify_user_buffers | rw | - | int | int | - | boolean Enables verification of user buffers provided by **pmemobj_tx_log_append_buffer**(3) API. For now the only verified aspect is whether the same buffer is used simultaneously in 2 or more transactions or more than once in the same transaction. This value should not be modified at runtime if any transaction for the current pool is in progress. tx.cache.size | rw | - | long long | long long | - | integer Size in bytes of the transaction snapshot cache. In a larger cache the frequency of persistent allocations is lower, but with higher fixed cost. This should be set to roughly the sum of sizes of the snapshotted regions in an average transaction in the pool. This entry point is not thread safe and should not be modified if there are any transactions currently running. This value must be a in a range between 0 and **PMEMOBJ_MAX_ALLOC_SIZE**, otherwise this entry point will fail. tx.cache.threshold | rw | - | long long | long long | - | integer This entry point is deprecated. All snapshots, regardless of the size, use the transactional cache. tx.post_commit.queue_depth | rw | - | int | int | - | integer This entry point is deprecated. tx.post_commit.worker | r- | - | void * | - | - | - This entry point is deprecated. tx.post_commit.stop | r- | - | void * | - | - | - This entry point is deprecated. heap.narenas.automatic | r- | - | unsigned | - | - | - Reads the number of arenas used in automatic scheduling of memory operations for threads. By default, this value is equal to the number of available processors. An arena is a memory management structure which enables concurrency by taking exclusive ownership of parts of the heap and allowing associated threads to allocate without contention. heap.narenas.total | r- | - | unsigned | - | - | - Reads the number of all created arenas. It includes automatic arenas created by default and arenas created using heap.arena.create CTL. heap.narenas.max | rw- | - | unsigned | unsigned | - | - Reads or writes the maximum number of arenas that can be created. This entry point is not thread-safe with regards to heap operations (allocations, frees, reallocs). heap.arena.[arena_id].size | r- | - | uint64_t | - | - | - Reads the total amount of memory in bytes which is currently exclusively owned by the arena. Large differences in this value between arenas might indicate an uneven scheduling of memory resources. The arena id cannot be 0. heap.thread.arena_id | rw- | - | unsigned | unsigned | - | - Reads the index of the arena assigned to the current thread or assigns arena with specific id to the current thread. The arena id cannot be 0. heap.arena.create | --x | - | - | - | unsigned | - Creates and initializes one new arena in the heap. This entry point reads an id of the new created arena. Newly created arenas by this CTL are inactive, which means that the arena will not be used in the automatic scheduling of memory requests. To activate the new arena, use heap.arena.[arena_id].automatic CTL. Arena created using this CTL can be used for allocation by explicitly specifying the *arena_id* for **POBJ_ARENA_ID(id)** flag in **pmemobj_tx_xalloc**()/**pmemobj_xalloc**()/**pmemobj_xreserve()** functions. By default, the number of arenas is limited to 1024. heap.arena.[arena_id].automatic | rw- | - | boolean | boolean | - | - Reads or modifies the state of the arena. If set, the arena is used in automatic scheduling of memory operations for threads. This should be set to false if the application wants to manually manage allocator scalability through explicitly assigning arenas to threads by using heap.thread.arena_id. The arena id cannot be 0 and at least one automatic arena must exist. heap.alloc_class.[class_id].desc | rw | - | `struct pobj_alloc_class_desc` | `struct pobj_alloc_class_desc` | - | integer, integer, integer, string Describes an allocation class. Allows one to create or view the internal data structures of the allocator. Creating custom allocation classes can be beneficial for both raw allocation throughput, scalability and, most importantly, fragmentation. By carefully constructing allocation classes that match the application workload, one can entirely eliminate external and internal fragmentation. For example, it is possible to easily construct a slab-like allocation mechanism for any data structure. The `[class_id]` is an index field. Only values between 0-254 are valid. If setting an allocation class, but the `class_id` is already taken, the function will return -1. The values between 0-127 are reserved for the default allocation classes of the library and can be used only for reading. The recommended method for retrieving information about all allocation classes is to call this entry point for all class ids between 0 and 254 and discard those results for which the function returns an error. This entry point takes a complex argument. ``` struct pobj_alloc_class_desc { size_t unit_size; size_t alignment; unsigned units_per_block; enum pobj_header_type header_type; unsigned class_id; }; ``` The first field, `unit_size`, is an 8-byte unsigned integer that defines the allocation class size. While theoretically limited only by **PMEMOBJ_MAX_ALLOC_SIZE**, for most workloads this value should be between 8 bytes and 2 megabytes. The `alignment` field specifies the user data alignment of objects allocated using the class. If set, must be a power of two and an even divisor of unit size. Alignment is limited to maximum of 2 megabytes. All objects have default alignment of 64 bytes, but the user data alignment is affected by the size of the chosen header. The `units_per_block` field defines how many units a single block of memory contains. This value will be adjusted to match the internal size of the block (256 kilobytes or a multiple thereof). For example, given a class with a `unit_size` of 512 bytes and a `units_per_block` of 1000, a single block of memory for that class will have 512 kilobytes. This is relevant because the bigger the block size, the less frequently blocks need to be fetched, resulting in lower contention on global heap state. If the CTL call is being done at runtime, the `units_per_block` variable of the provided alloc class structure is modified to match the actual value. The `header_type` field defines the header of objects from the allocation class. There are three types: - **POBJ_HEADER_LEGACY**, string value: `legacy`. Used for allocation classes prior to version 1.3 of the library. Not recommended for use. Incurs a 64 byte metadata overhead for every object. Fully supports all features. - **POBJ_HEADER_COMPACT**, string value: `compact`. Used as default for all predefined allocation classes. Incurs a 16 byte metadata overhead for every object. Fully supports all features. - **POBJ_HEADER_NONE**, string value: `none`. Header type that incurs no metadata overhead beyond a single bitmap entry. Can be used for very small allocation classes or when objects must be adjacent to each other. This header type does not support type numbers (type number is always 0) or allocations that span more than one unit. The `class_id` field is an optional, runtime-only variable that allows the user to retrieve the identifier of the class. This will be equivalent to the provided `[class_id]`. This field cannot be set from a config file. The allocation classes are a runtime state of the library and must be created after every open. It is highly recommended to use the configuration file to store the classes. This structure is declared in the `libpmemobj/ctl.h` header file. Please refer to this file for an in-depth explanation of the allocation classes and relevant algorithms. Allocation classes constructed in this way can be leveraged by explicitly specifying the class using **POBJ_CLASS_ID(id)** flag in **pmemobj_tx_xalloc**()/**pmemobj_xalloc**() functions. Example of a valid alloc class query string: ``` heap.alloc_class.128.desc=500,0,1000,compact ``` This query, if executed, will create an allocation class with an id of 128 that has a unit size of 500 bytes, has at least 1000 units per block and uses a compact header. For reading, function returns 0 if successful, if the allocation class does not exist it sets the errno to **ENOENT** and returns -1; This entry point can fail if any of the parameters of the allocation class is invalid or if exactly the same class already exists. heap.alloc_class.new.desc | -w | - | - | `struct pobj_alloc_class_desc` | - | integer, integer, integer, string Same as `heap.alloc_class.[class_id].desc`, but instead of requiring the user to provide the class_id, it automatically creates the allocation class with the first available identifier. This should be used when it's impossible to guarantee unique allocation class naming in the application (e.g. when writing a library that uses libpmemobj). The required class identifier will be stored in the `class_id` field of the `struct pobj_alloc_class_desc`. stats.enabled | rw | - | enum pobj_stats_enabled | enum pobj_stats_enabled | - | string Enables or disables runtime collection of statistics. There are two types of statistics: persistent and transient ones. Persistent statistics survive pool restarts, whereas transient ones don't. Statistics are not recalculated after enabling; any operations that occur between disabling and re-enabling will not be reflected in subsequent values. Only transient statistics are enabled by default. Enabling persistent statistics may have non-trivial performance impact. stats.heap.curr_allocated | r- | - | uint64_t | - | - | - Reads the number of bytes currently allocated in the heap. If statistics were disabled at any time in the lifetime of the heap, this value may be inaccurate. This is a persistent statistic. stats.heap.run_allocated | r- | - | uint64_t | - | - | - Reads the number of bytes currently allocated using run-based allocation classes, i.e., huge allocations are not accounted for in this statistic. This is useful for comparison against stats.heap.run_active to estimate the ratio between active and allocated memory. This is a transient statistic and is rebuilt every time the pool is opened. stats.heap.run_active | r- | - | uint64_t | - | - | - Reads the number of bytes currently occupied by all run memory blocks, including both allocated and free space, i.e., this is all the all space that's not occupied by huge allocations. This value is a sum of all allocated and free run memory. In systems where memory is efficiently used, `run_active` should closely track `run_allocated`, and the amount of active, but free, memory should be minimal. A large relative difference between active memory and allocated memory is indicative of heap fragmentation. This information can be used to make a decision to call **pmemobj_defrag()**(3) if the fragmentation looks to be high. However, for small heaps `run_active` might be disproportionately higher than `run_allocated` because the allocator typically activates a significantly larger amount of memory than is required to satisfy a single request in the anticipation of future needs. For example, the first allocation of 100 bytes in a heap will trigger activation of 256 kilobytes of space. This is a transient statistic and is rebuilt lazily every time the pool is opened. heap.size.granularity | rw- | - | uint64_t | uint64_t | - | long long Reads or modifies the granularity with which the heap grows when OOM. Valid only if the poolset has been defined with directories. A granularity of 0 specifies that the pool will not grow automatically. This entry point can fail if the granularity value is non-zero and smaller than *PMEMOBJ_MIN_PART*. heap.size.extend | --x | - | - | - | uint64_t | - Extends the heap by the given size. Must be larger than *PMEMOBJ_MIN_PART*. This entry point can fail if the pool does not support extend functionality or if there's not enough space left on the device. debug.heap.alloc_pattern | rw | - | int | int | - | - Single byte pattern that is used to fill new uninitialized memory allocation. If the value is negative, no pattern is written. This is intended for debugging, and is disabled by default. # CTL EXTERNAL CONFIGURATION # In addition to direct function call, each write entry point can also be set using two alternative methods. The first method is to load a configuration directly from the **PMEMOBJ_CONF** environment variable. The second method of loading an external configuration is to set the **PMEMOBJ_CONF_FILE** environment variable to point to a file that contains a sequence of ctl queries. See more in **pmem_ctl**(5) man page. # SEE ALSO # **libpmemobj**(7), **pmem_ctl**(5) and **** pmdk-1.8/doc/libpmemobj/pobj_layout_toid.30000664000000000000000000000003013615011243017321 0ustar rootroot.so pobj_layout_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_log_append_buffer.30000664000000000000000000000002713615011243021646 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/tx_set.30000664000000000000000000000003313615011243015264 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_rwlock_wrlock.30000664000000000000000000000003113615011243020347 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/toid_declare_root.30000664000000000000000000000002313615011243017436 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/tx_znew.30000664000000000000000000000002713615011243015457 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/tx_free.30000664000000000000000000000002713615011243015415 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_list_entry.30000664000000000000000000000002513615011243017165 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/tx_xwcsdup.30000664000000000000000000000002713615011243016171 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_get_user_data.30000664000000000000000000000002313615011243020274 0ustar rootroot.so pmemobj_open.3 pmdk-1.8/doc/libpmemobj/pmemobj_f_mem_nontemporal.30000664000000000000000000000003513615011243021172 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pmemobj_cond_wait.30000664000000000000000000000003113615011243017434 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/toid_offsetof.30000664000000000000000000000002313615011243016607 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pmemobj_memcpy_persist.30000644000000000000000000002105313615011417020540 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_MEMCPY_PERSIST" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_persist\f[](), \f[B]pmemobj_xpersist\f[](), \f[B]pmemobj_flush\f[](), \f[B]pmemobj_xflush\f[](), \f[B]pmemobj_drain\f[](), \f[B]pmemobj_memcpy\f[](), \f[B]pmemobj_memmove\f[](), \f[B]pmemobj_memset\f[](), \f[B]pmemobj_memcpy_persist\f[](), \f[B]pmemobj_memset_persist\f[]() \- low\-level memory manipulation functions .SH SYNOPSIS .IP .nf \f[C] #include\ void\ pmemobj_persist(PMEMobjpool\ *pop,\ const\ void\ *addr, \ \ \ \ size_t\ len); void\ pmemobj_flush(PMEMobjpool\ *pop,\ const\ void\ *addr, \ \ \ \ size_t\ len); void\ pmemobj_drain(PMEMobjpool\ *pop); int\ pmemobj_xpersist(PMEMobjpool\ *pop,\ const\ void\ *addr, \ \ \ \ size_t\ len,\ unsigned\ flags); int\ pmemobj_xflush(PMEMobjpool\ *pop,\ const\ void\ *addr, \ \ \ \ size_t\ len,\ unsigned\ flags); void\ *pmemobj_memcpy(PMEMobjpool\ *pop,\ void\ *dest, \ \ \ \ const\ void\ *src,\ size_t\ len,\ unsigned\ flags); void\ *pmemobj_memmove(PMEMobjpool\ *pop,\ void\ *dest, \ \ \ \ const\ void\ *src,\ size_t\ len,\ unsigned\ flags); void\ *pmemobj_memset(PMEMobjpool\ *pop,\ void\ *dest, \ \ \ \ int\ c,\ size_t\ len,\ unsigned\ flags); void\ *pmemobj_memcpy_persist(PMEMobjpool\ *pop,\ void\ *dest, \ \ \ \ const\ void\ *src,\ size_t\ len); void\ *pmemobj_memset_persist(PMEMobjpool\ *pop,\ void\ *dest, \ \ \ \ int\ c,\ size_t\ len); \f[] .fi .SH DESCRIPTION .PP The \f[B]libpmemobj\f[]\-specific low\-level memory manipulation functions described here leverage the knowledge of the additional configuration options available for \f[B]libpmemobj\f[](7) pools, such as replication. They also take advantage of the type of storage behind the pool and use appropriate flush/drain functions. It is advised to use these functions in conjunction with \f[B]libpmemobj\f[](7) objects rather than using low\-level memory manipulation functions from \f[B]libpmem\f[]. .PP \f[B]pmemobj_persist\f[]() forces any changes in the range [\f[I]addr\f[], \f[I]addr\f[]+\f[I]len\f[]) to be stored durably in persistent memory. Internally this may call either \f[B]pmem_msync\f[](3) or \f[B]pmem_persist\f[](3). There are no alignment restrictions on the range described by \f[I]addr\f[] and \f[I]len\f[], but \f[B]pmemobj_persist\f[]() may expand the range as necessary to meet platform alignment requirements. .RS .PP WARNING: Like \f[B]msync\f[](2), there is nothing atomic or transactional about this call. Any unwritten stores in the given range will be written, but some stores may have already been written by virtue of normal cache eviction/replacement policies. Correctly written code must not depend on stores waiting until \f[B]pmemobj_persist\f[]() is called to become persistent \- they can become persistent at any time before \f[B]pmemobj_persist\f[]() is called. .RE .PP The \f[B]pmemobj_flush\f[]() and \f[B]pmemobj_drain\f[]() functions provide partial versions of the \f[B]pmemobj_persist\f[]() function described above. These functions allow advanced programs to create their own variations of \f[B]pmemobj_persist\f[](). For example, a program that needs to flush several discontiguous ranges can call \f[B]pmemobj_flush\f[]() for each range and then follow up by calling \f[B]pmemobj_drain\f[]() once. For more information on partial flushing operations, see \f[B]pmem_flush\f[](3). .PP \f[B]pmemobj_xpersist\f[]() is a version of \f[B]pmemobj_persist\f[]() function with additional \f[I]flags\f[] argument. It supports only the \f[B]PMEMOBJ_F_RELAXED\f[] flag. This flag indicates that memory transfer operation does not require 8\-byte atomicity guarantees. .PP \f[B]pmemobj_xflush\f[]() is a version of \f[B]pmemobj_flush\f[]() function with additional \f[I]flags\f[] argument. It supports only the \f[B]PMEMOBJ_F_RELAXED\f[] flag. .PP The \f[B]pmemobj_memmove\f[](), \f[B]pmemobj_memcpy\f[]() and \f[B]pmemobj_memset\f[]() functions provide the same memory copying as their namesakes \f[B]memmove\f[](3), \f[B]memcpy\f[](3), and \f[B]memset\f[](3), and ensure that the result has been flushed to persistence before returning (unless \f[B]PMEMOBJ_MEM_NOFLUSH\f[] flag was used). Valid flags for those functions: .IP \[bu] 2 \f[B]PMEMOBJ_F_RELAXED\f[] \- This flag indicates that memory transfer operation does not require 8\-byte atomicity guarantees. .IP \[bu] 2 \f[B]PMEMOBJ_F_MEM_NOFLUSH\f[] \- Don't flush anything. This implies \f[B]PMEMOBJ_F_MEM_NODRAIN\f[]. Using this flag only makes sense when it's followed by any function that flushes data. .PP The remaining flags say \f[I]how\f[] the operation should be done, and are merely hints. .IP \[bu] 2 \f[B]PMEMOBJ_F_MEM_NONTEMPORAL\f[] \- Use non\-temporal instructions. This flag is mutually exclusive with \f[B]PMEMOBJ_F_MEM_TEMPORAL\f[]. On x86_64 this flag is mutually exclusive with \f[B]PMEMOBJ_F_MEM_NOFLUSH\f[]. .IP \[bu] 2 \f[B]PMEMOBJ_F_MEM_TEMPORAL\f[] \- Use temporal instructions. This flag is mutually exclusive with \f[B]PMEMOBJ_F_MEM_NONTEMPORAL\f[]. .IP \[bu] 2 \f[B]PMEMOBJ_F_MEM_WC\f[] \- Use write combining mode. This flag is mutually exclusive with \f[B]PMEMOBJ_F_MEM_WB\f[]. On x86_64 this is an alias for \f[B]PMEMOBJ_F_MEM_NONTEMPORAL\f[]. On x86_64 this flag is mutually exclusive with \f[B]PMEMOBJ_F_MEM_NOFLUSH\f[]. .IP \[bu] 2 \f[B]PMEMOBJ_F_MEM_WB\f[] \- Use write back mode. This flag is mutually exclusive with \f[B]PMEMOBJ_F_MEM_WC\f[]. On x86_64 this is an alias for \f[B]PMEMOBJ_F_MEM_TEMPORAL\f[]. .PP \f[B]pmemobj_memcpy_persist\f[]() is an alias for \f[B]pmemobj_memcpy\f[]() with flags equal to 0. .PP \f[B]pmemobj_memset_persist\f[]() is an alias for \f[B]pmemobj_memset\f[]() with flags equal to 0. .SH RETURN VALUE .PP \f[B]pmemobj_memmove\f[](), \f[B]pmemobj_memcpy\f[](), \f[B]pmemobj_memset\f[](), \f[B]pmemobj_memcpy_persist\f[]() and \f[B]pmemobj_memset_persist\f[]() return destination buffer. .PP \f[B]pmemobj_persist\f[](), \f[B]pmemobj_flush\f[]() and \f[B]pmemobj_drain\f[]() do not return any value. .PP \f[B]pmemobj_xpersist\f[]() and \f[B]pmemobj_xflush\f[]() returns non\-zero value and sets errno to EINVAL only if not supported flags has been provided. .SH EXAMPLES .PP The following code is functionally equivalent to \f[B]pmemobj_memcpy_persist\f[](): .IP .nf \f[C] void\ * pmemobj_memcpy_persist(PMEMobjpool\ *pop,\ void\ *dest, \ \ \ \ const\ void\ *src,\ size_t\ len) { \ \ \ \ void\ *retval\ =\ memcpy(dest,\ src,\ len); \ \ \ \ pmemobj_persist(pop,\ dest,\ len); \ \ \ \ return\ retval; } \f[] .fi .PP \f[B]pmemobj_persist\f[]() can be thought of as this: .IP .nf \f[C] void pmemobj_persist(PMEMobjpool\ *pop,\ const\ void\ *addr,\ size_t\ len) { \ \ \ \ /*\ flush\ the\ processor\ caches\ */ \ \ \ \ pmemobj_flush(pop,\ addr,\ len); \ \ \ \ /*\ wait\ for\ any\ pmem\ stores\ to\ drain\ from\ HW\ buffers\ */ \ \ \ \ pmemobj_drain(pop); } \f[] .fi .SH SEE ALSO .PP \f[B]memcpy\f[](3), \f[B]memset\f[](3), \f[B]pmem_msync\f[](3), \f[B]pmem_persist\f[](3), \f[B]libpmem\f[](7) \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pobj_next.30000664000000000000000000000002413615011243015746 0ustar rootroot.so pmemobj_first.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_zrealloc.30000664000000000000000000000002713615011243020020 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/toid_type_num.30000664000000000000000000000002313615011243016634 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pobj_zalloc.30000664000000000000000000000002413615011243016254 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/tx_xalloc.30000664000000000000000000000002713615011243015756 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_root_size.30000664000000000000000000000002313615011243017503 0ustar rootroot.so pmemobj_root.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_commit.30000664000000000000000000000002713615011243017475 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_xflush.30000664000000000000000000000003513615011243017002 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pmemobj_open.3.md0000664000000000000000000002320313615011243017033 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_OPEN, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_open.3 -- man page for most commonly used functions from libpmemobj library) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[CAVEATS](#caveats)
[SEE ALSO](#see-also)
# NAME # _UW(pmemobj_open), _UW(pmemobj_create), **pmemobj_close**(), _UW(pmemobj_check) **pmemobj_set_user_data**(), **pmemobj_get_user_data**() - create, open, close and validate persistent memory transactional object store # SYNOPSIS # ```c #include _UWFUNCR1(PMEMobjpool, *pmemobj_open, *path, const char *layout) _UWFUNCR1(PMEMobjpool, *pmemobj_create, *path, =q=const char *layout, size_t poolsize, mode_t mode=e=) void pmemobj_close(PMEMobjpool *pop); _UWFUNCR1(int, pmemobj_check, *path, const char *layout) void pmemobj_set_user_data(PMEMobjpool *pop, void *data); void *pmemobj_get_user_data(PMEMobjpool *pop); ``` _UNICODE() # DESCRIPTION # To use the pmem-resident transactional object store provided by **libpmemobj**(7), a *memory pool* must first be created with the _UW(pmemobj_create) function described below. Existing pools may be opened with the _UW(pmemobj_open) function. None of the three functions described below are thread-safe with respect to any other **libpmemobj**(7) function. In other words, when creating, opening or deleting a pool, nothing else in the library can happen in parallel, and therefore these functions should be called from the main thread. Once created, the memory pool is represented by an opaque handle, of type *PMEMobjpool\**, which is passed to most of the other **libpmemobj**(7) functions. Internally, **libpmemobj**(7) will use either **pmem_persist**(3) or **msync**(2) when it needs to flush changes, depending on whether the memory pool appears to be persistent memory or a regular file (see the **pmem_is_pmem**(3) function in **libpmem**(7) for more information). There is no need for applications to flush changes directly when using the object memory API provided by **libpmemobj**(7). The _UW(pmemobj_create) function creates a transactional object store with the given total *poolsize*. *path* specifies the name of the memory pool file to be created. *layout* specifies the application's layout type in the form of a string. The layout name is not interpreted by **libpmemobj**(7), but may be used as a check when _UW(pmemobj_open) is called. The layout name, including the terminating null byte ('\0'), cannot be longer than **PMEMOBJ_MAX_LAYOUT** as defined in **\**. A NULL *layout* is equivalent to using an empty string as a layout name. *mode* specifies the permissions to use when creating the file, as described by **creat**(2). The memory pool file is fully allocated to the size *poolsize* using **posix_fallocate**(3). The caller may choose to take responsibility for creating the memory pool file by creating it before calling _UW(pmemobj_create), and then specifying *poolsize* as zero. In this case _UW(pmemobj_create) will take the pool size from the size of the existing file and will verify that the file appears to be empty by searching for any non-zero data in the pool header at the beginning of the file. The minimum net pool size allowed by the library for a local transactional object store is defined in **\** as **PMEMOBJ_MIN_POOL**. _WINUX(,=q=For remote replicas the minimum file size is defined in **\** as **RPMEM_MIN_PART**.=e=) Depending on the configuration of the system, the available non-volatile memory space may be divided into multiple memory devices. In such case, the maximum size of the pmemobj memory pool could be limited by the capacity of a single memory device. **libpmemobj**(7) allows building persistent memory resident object store spanning multiple memory devices by creation of persistent memory pools consisting of multiple files, where each part of such a *pool set* may be stored on a different memory device or pmem-aware filesystem. Creation of all the parts of the pool set can be done with _UW(pmemobj_create); however, the recommended method for creating pool sets is with the **pmempool**(1) utility. When creating a pool set consisting of multiple files, the *path* argument passed to _UW(pmemobj_create) must point to the special *set* file that defines the pool layout and the location of all the parts of the pool set. The *poolsize* argument must be 0. The meaning of the *layout* and *mode* arguments does not change, except that the same *mode* is used for creation of all the parts of the pool set. The *set* file is a plain text file, the structure of which is described in **poolset**(5). The _UW(pmemobj_open) function opens an existing object store memory pool. Similar to _UW(pmemobj_create), *path* must identify either an existing obj memory pool file, or the *set* file used to create a pool set. If *layout* is non-NULL, it is compared to the layout name provided to _UW(pmemobj_create) when the pool was first created. This can be used to verify that the layout of the pool matches what was expected. The application must have permission to open the file and memory map it with read/write permissions. Be aware that if the pool contains bad blocks inside, opening can be aborted by the SIGBUS signal, because currently the pool is not checked against bad blocks during opening. It can be turned on by setting the CHECK_BAD_BLOCKS compat feature. For details see description of this feature in **pmempool-feature**(1). The **pmemobj_close**() function closes the memory pool indicated by *pop* and deletes the memory pool handle. The object store itself lives on in the file that contains it and may be re-opened at a later time using _UW(pmemobj_open) as described above. The _UW(pmemobj_check) function performs a consistency check of the file indicated by *path*. _UW(pmemobj_check) opens the given *path* read-only so it never makes any changes to the file. This function is not supported on Device DAX. The **pmemobj_set_user_data**() function associates custom volatile state, represented by pointer *data*, with the given pool *pop*. This state can later be retrieved using **pmemobj_get_user_data**() function. This state does not survive pool close. If **pmemobj_set_user_data**() was not called for a given pool, **pmemobj_get_user_data**() will return NULL. # RETURN VALUE # The _UW(pmemobj_create) function returns a memory pool handle to be used with most of the functions in **libpmemobj**(7). On error it returns NULL and sets *errno* appropriately. The _UW(pmemobj_open) function returns a memory pool handle to be used with most of the functions in **libpmemobj**(7). If an error prevents the pool from being opened, or if the given *layout* does not match the pool's layout, _UW(pmemobj_open) returns NULL and sets *errno* appropriately. The **pmemobj_close**() function returns no value. The _UW(pmemobj_check) function returns 1 if the memory pool is found to be consistent. Any inconsistencies found will cause _UW(pmemobj_check) to return 0, in which case the use of the file with **libpmemobj**(7) will result in undefined behavior. The debug version of **libpmemobj**(7) will provide additional details on inconsistencies when **PMEMOBJ_LOG_LEVEL** is at least 1, as described in the **DEBUGGING AND ERROR HANDLING** section in **libpmemobj**(7). _UW(pmemobj_check) returns -1 and sets *errno* if it cannot perform the consistency check due to other errors. # CAVEATS # Not all file systems support **posix_fallocate**(3). _UW(pmemobj_create) will fail if the underlying file system does not support **posix_fallocate**(3). # SEE ALSO # **creat**(2), **msync**(2), **pmem_is_pmem**(3), **pmem_persist**(3), **posix_fallocate**(3), **libpmem**(7), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_type_num.30000664000000000000000000000002213615011243017325 0ustar rootroot.so oid_is_null.3 pmdk-1.8/doc/libpmemobj/pmemobj_xreserve.30000664000000000000000000000002513615011243017333 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/oid_equals.30000664000000000000000000000002213615011243016101 0ustar rootroot.so oid_is_null.3 pmdk-1.8/doc/libpmemobj/toid_equals.30000664000000000000000000000002313615011243016266 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pobj_layout_begin.30000644000000000000000000001130613615011417017457 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "POBJ_LAYOUT_BEGIN" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]POBJ_LAYOUT_BEGIN\f[](), \f[B]POBJ_LAYOUT_TOID\f[](), \f[B]POBJ_LAYOUT_ROOT\f[](), \f[B]POBJ_LAYOUT_NAME\f[](), \f[B]POBJ_LAYOUT_END\f[](), \f[B]POBJ_LAYOUT_TYPES_NUM\f[]() \- persistent memory transactional object store layout .SH SYNOPSIS .IP .nf \f[C] #include\ POBJ_LAYOUT_BEGIN(layout) POBJ_LAYOUT_TOID(layout,\ TYPE) POBJ_LAYOUT_ROOT(layout,\ ROOT_TYPE) POBJ_LAYOUT_NAME(layout) POBJ_LAYOUT_END(layout) POBJ_LAYOUT_TYPES_NUM(layout) \f[] .fi .SH DESCRIPTION .PP \f[B]libpmemobj\f[](7) defines a set of macros for convenient declaration of a pool's layout. The layout declaration consists of declarations of a number of used types. The declared types will be assigned consecutive type numbers. Declared types may be used in conjunction with type safety macros (see \f[B]TOID_DECLARE\f[](3)). Once created, the layout declaration must not be changed unless any new types are added at the end of the existing layout declaration. Modifying any existing declaration may lead to changes in the type numbers of declared types, which in consequence may cause data corruption. .PP The \f[B]POBJ_LAYOUT_BEGIN\f[]() macro indicates a begin of declaration of layout. The \f[I]LAYOUT\f[] argument is a name of layout. This argument must be passed to all macros related to the declaration of layout. .PP The \f[B]POBJ_LAYOUT_TOID\f[]() macro declares a typed \f[I]OID\f[] for type passed as \f[I]TYPE\f[] argument inside the declaration of layout. All types declared using this macro are assigned with consecutive type numbers. This macro must be used between the \f[B]POBJ_LAYOUT_BEGIN\f[]() and \f[B]POBJ_LAYOUT_END\f[]() macros, with the same name passed as \f[I]LAYOUT\f[] argument. .PP The \f[B]POBJ_LAYOUT_ROOT\f[]() macro declares a typed \f[I]OID\f[] for type passed as \f[I]ROOT_TYPE\f[] argument inside the declaration of layout. The typed \f[I]OID\f[] will be assigned with type number for root object \f[B]POBJ_ROOT_TYPE_NUM\f[]. .PP The \f[B]POBJ_LAYOUT_END\f[]() macro ends the declaration of layout. .PP The \f[B]POBJ_LAYOUT_NAME\f[]() macro returns the name of layout as a null\-terminated string. .PP The \f[B]POBJ_LAYOUT_TYPES_NUM\f[]() macro returns number of types declared using the \f[B]POBJ_LAYOUT_TOID\f[]() macro within the layout declaration. .SH EXAMPLE .PP This is an example of layout declaration: .IP .nf \f[C] POBJ_LAYOUT_BEGIN(mylayout); POBJ_LAYOUT_ROOT(mylayout,\ struct\ root); POBJ_LAYOUT_TOID(mylayout,\ struct\ node); POBJ_LAYOUT_TOID(mylayout,\ struct\ foo); POBJ_LAYOUT_END(mylayout); struct\ root { \ \ \ \ TOID(struct\ node)\ node; }; struct\ node { \ \ \ \ TOID(struct\ node)\ next; \ \ \ \ TOID(struct\ foo)\ foo; }; \f[] .fi .PP The name of layout and the number of declared types can be retrieved using the following code: .IP .nf \f[C] const\ char\ *layout_name\ =\ POBJ_LAYOUT_NAME(mylayout); int\ num_of_types\ =\ POBJ_LAYOUT_TYPES_NUM(mylayout); \f[] .fi .SH SEE ALSO .PP \f[B]TOID_DECLARE\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_tx_strdup.30000664000000000000000000000002713615011243017526 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/tx_finally.30000664000000000000000000000002713615011243016132 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/tx_end.30000664000000000000000000000002713615011243015242 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_xadd_range.30000664000000000000000000000003313615011243020276 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pobj_list_insert_head.30000664000000000000000000000002513615011243020311 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_xwcsdup.30000664000000000000000000000002713615011243017702 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_close.30000664000000000000000000000002313615011243016573 0ustar rootroot.so pmemobj_open.3 pmdk-1.8/doc/libpmemobj/pobj_list_insert_new_tail.30000664000000000000000000000002513615011243021212 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pobj_xreserve_new.30000664000000000000000000000002513615011243017505 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/toid_is_null.30000664000000000000000000000002313615011243016441 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pobj_new.30000664000000000000000000000002413615011243015561 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/tx_set_direct.30000664000000000000000000000003313615011243016616 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_list_move.30000664000000000000000000000003213615011243017467 0ustar rootroot.so pmemobj_list_insert.3 pmdk-1.8/doc/libpmemobj/tx_add_direct.30000664000000000000000000000003313615011243016553 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_log_auto_alloc.30000664000000000000000000000002713615011243021170 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/tx_begin.30000664000000000000000000000002713615011243015560 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pobj_list_prev.30000664000000000000000000000002513615011243017000 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/tx_zrealloc.30000664000000000000000000000002713615011243016307 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_cond_broadcast.30000664000000000000000000000003113615011243020432 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pmemobj_alloc.30000644000000000000000000004237113615011417016575 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_ALLOC" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2020, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_alloc\f[](), \f[B]pmemobj_xalloc\f[](), \f[B]pmemobj_zalloc\f[](), \f[B]pmemobj_realloc\f[](), \f[B]pmemobj_zrealloc\f[](), \f[B]pmemobj_strdup\f[](), \f[B]pmemobj_wcsdup\f[](), \f[B]pmemobj_alloc_usable_size\f[](), \f[B]pmemobj_defrag\f[](), \f[B]POBJ_NEW\f[](), \f[B]POBJ_ALLOC\f[](), \f[B]POBJ_ZNEW\f[](), \f[B]POBJ_ZALLOC\f[](), \f[B]POBJ_REALLOC\f[](), \f[B]POBJ_ZREALLOC\f[](), \f[B]POBJ_FREE\f[]() \- non\-transactional atomic allocations .SH SYNOPSIS .IP .nf \f[C] #include\ typedef\ int\ (*pmemobj_constr)(**PMEMobjpool\ *pop,\ void\ *ptr,\ void\ *arg); int\ pmemobj_alloc(PMEMobjpool\ *pop,\ PMEMoid\ *oidp,\ size_t\ size, \ \ \ \ uint64_t\ type_num,\ pmemobj_constr\ constructor,\ void\ *arg); int\ pmemobj_xalloc(PMEMobjpool\ *pop,\ PMEMoid\ *oidp,\ size_t\ size, \ \ \ \ uint64_t\ type_num,\ uint64_t\ flags,\ pmemobj_constr\ constructor, \ \ \ \ void\ *arg);\ (EXPERIMENTAL) int\ pmemobj_zalloc(PMEMobjpool\ *pop,\ PMEMoid\ *oidp,\ size_t\ size, \ \ \ \ uint64_t\ type_num); void\ pmemobj_free(PMEMoid\ *oidp); int\ pmemobj_realloc(PMEMobjpool\ *pop,\ PMEMoid\ *oidp,\ size_t\ size, \ \ \ \ uint64_t\ type_num); int\ pmemobj_zrealloc(PMEMobjpool\ *pop,\ PMEMoid\ *oidp,\ size_t\ size, \ \ \ \ uint64_t\ type_num); int\ pmemobj_strdup(PMEMobjpool\ *pop,\ PMEMoid\ *oidp,\ const\ char\ *s, \ \ \ \ uint64_t\ type_num); int\ pmemobj_wcsdup(PMEMobjpool\ *pop,\ PMEMoid\ *oidp,\ const\ wchar_t\ *s, \ \ \ \ uint64_t\ type_num); size_t\ pmemobj_alloc_usable_size(PMEMoid\ oid); int\ pmemobj_defrag(PMEMobjpool\ *pop,\ PMEMoid\ **oidv,\ size_t\ oidcnt, \ \ \ \ struct\ pobj_defrag_result\ *result); POBJ_NEW(PMEMobjpool\ *pop,\ TOID\ *oidp,\ TYPE,\ pmemobj_constr\ constructor, \ \ \ \ void\ *arg) POBJ_ALLOC(PMEMobjpool\ *pop,\ TOID\ *oidp,\ TYPE,\ size_t\ size, \ \ \ \ pmemobj_constr\ constructor,\ void\ *arg) POBJ_ZNEW(PMEMobjpool\ *pop,\ TOID\ *oidp,\ TYPE) POBJ_ZALLOC(PMEMobjpool\ *pop,\ TOID\ *oidp,\ TYPE,\ size_t\ size) POBJ_REALLOC(PMEMobjpool\ *pop,\ TOID\ *oidp,\ TYPE,\ size_t\ size) POBJ_ZREALLOC(PMEMobjpool\ *pop,\ TOID\ *oidp,\ TYPE,\ size_t\ size) POBJ_FREE(TOID\ *oidp) \f[] .fi .SH DESCRIPTION .PP Functions described in this document provide the mechanism to allocate, resize and free objects from the persistent memory pool in a thread\-safe and fail\-safe manner. All the routines are atomic with respect to other threads and any power\-fail interruptions. If any of these operations is torn by program failure or system crash, on recovery they are guaranteed to be entirely completed or discarded, leaving the persistent memory heap and internal object containers in a consistent state. .PP All these functions should be used outside transactions. If executed within an open transaction they are considered durable immediately after completion. Changes made with these functions will not be rolled back if the transaction is aborted or interrupted. They have no information about other changes made by transactional API, so if the same data is modified in a single transaction using transactional and then non\-transactional API, transaction abort will likely corrupt the data. .PP The allocations are always aligned to a cache\-line boundary. .PP The \f[I]pmemobj_constr\f[] type represents a constructor for atomic allocation from the persistent memory heap associated with memory pool \f[I]pop\f[]. \f[I]ptr\f[] is a pointer to the allocated memory area and \f[I]arg\f[] is a user\-defined argument passed to the constructor. .PP The \f[B]pmemobj_alloc\f[]() function allocates a new object from the persistent memory heap associated with memory pool \f[I]pop\f[]. The \f[I]PMEMoid\f[] of the allocated object is stored in \f[I]oidp\f[]. If \f[I]oidp\f[] is NULL, then the newly allocated object may be accessed only by iterating objects in the object container associated with the type number \f[I]type_num\f[], as described in \f[B]POBJ_FOREACH\f[](3). If \f[I]oidp\f[] points to a memory location from the \f[B]pmemobj\f[] heap, \f[I]oidp\f[] is modified atomically. Before returning, \f[B]pmemobj_alloc\f[]() calls the \f[I]constructor\f[] function, passing the pool handle \f[I]pop\f[], the pointer to the newly allocated object in \f[I]ptr\f[], and the \f[I]arg\f[] argument. It is guaranteed that the allocated object is either properly initialized, or if the allocation is interrupted before the constructor completes, the memory space reserved for the object is reclaimed. \f[I]size\f[] can be any non\-zero value; however, due to internal padding and object metadata, the actual size of the allocation will differ from the requested size by at least 64 bytes. For this reason, making allocations of a size less than 64 bytes is extremely inefficient and discouraged. The allocated object is added to the internal container associated with \f[I]type_num\f[]. .PP \f[B]pmemobj_xalloc\f[]() is equivalent to \f[B]pmemobj_alloc\f[](), but with an additional \f[I]flags\f[] argument that is a bitmask of the following values: .IP \[bu] 2 \f[B]POBJ_XALLOC_ZERO\f[] \- zero the allocated object (equivalent of \f[B]pmemobj_zalloc\f[]()) .IP \[bu] 2 \f[B]POBJ_CLASS_ID(class_id)\f[] \- allocate an object from the allocation class \f[I]class_id\f[]. The class id cannot be 0. .IP \[bu] 2 \f[B]POBJ_ARENA_ID(arena_id)\f[] \- allocate an object from the arena specified by \f[I]arena_id\f[]. The arena must exist, otherwise, the behavior is undefined. If \f[I]arena_id\f[] is equal 0, then arena assigned to the current thread will be used. .PP The \f[B]pmemobj_zalloc\f[]() function allocates a new zeroed object from the persistent memory heap associated with memory pool \f[I]pop\f[]. The \f[I]PMEMoid\f[] of the allocated object is stored in \f[I]oidp\f[]. If \f[I]oidp\f[] is NULL, then the newly allocated object may be accessed only by iterating objects in the object container associated with the type number \f[I]type_num\f[], as described in \f[B]POBJ_FOREACH\f[](3). If \f[I]oidp\f[] points to a memory location from the \f[B]pmemobj\f[] heap, \f[I]oidp\f[] is modified atomically. \f[I]size\f[] can be any non\-zero value; however, due to internal padding and object metadata, the actual size of the allocation will differ from the requested one by at least 64 bytes. For this reason, making allocations of a size less than 64 bytes is extremely inefficient and discouraged. The allocated object is added to the internal container associated with \f[I]type_num\f[]. .PP The \f[B]pmemobj_free\f[]() function frees the memory space represented by \f[I]oidp\f[], which must have been allocated by a previous call to \f[B]pmemobj_alloc\f[](), \f[B]pmemobj_xalloc\f[](), \f[B]pmemobj_zalloc\f[](), \f[B]pmemobj_realloc\f[](), or \f[B]pmemobj_zrealloc\f[](). \f[B]pmemobj_free\f[]() provides the same semantics as \f[B]free\f[](3), but instead of operating on the process heap supplied by the system, it operates on the persistent memory heap. If \f[I]oidp\f[] is \f[B]OID_NULL\f[], no operation is performed. If \f[I]oidp\f[] is NULL or if it points to the root object's \f[I]OID\f[], the behavior of \f[B]pmemobj_free\f[]() is undefined. \f[I]oidp\f[] is set to \f[B]OID_NULL\f[] after the memory is freed. If \f[I]oidp\f[] points to a memory location from the \f[B]pmemobj\f[] heap, \f[I]oidp\f[] is modified atomically. .PP The \f[B]pmemobj_realloc\f[]() function changes the size of the object represented by \f[I]oidp\f[] to \f[I]size\f[] bytes. \f[B]pmemobj_realloc\f[]() provides similar semantics to \f[B]realloc\f[](3), but operates on the persistent memory heap associated with memory pool \f[I]pop\f[]. The resized object is also added or moved to the internal container associated with type number \f[I]type_num\f[]. The contents will be unchanged in the range from the start of the region up to the minimum of the old and new sizes. If the new size is larger than the old size, the added memory will \f[I]not\f[] be initialized. If \f[I]oidp\f[] is \f[I]OID_NULL\f[], then the call is equivalent to \f[I]pmemobj_alloc(pop, size, type_num)\f[]. If \f[I]size\f[] is equal to zero, and \f[I]oidp\f[] is not \f[B]OID_NULL\f[], then the call is equivalent to \f[I]pmemobj_free(oid)\f[]. Unless \f[I]oidp\f[] is \f[B]OID_NULL\f[], it must have been allocated by an earlier call to \f[B]pmemobj_alloc\f[](), \f[B]pmemobj_xalloc\f[](), \f[B]pmemobj_zalloc\f[](), \f[B]pmemobj_realloc\f[](), or \f[B]pmemobj_zrealloc\f[](). Note that the object handle value may change as a result of reallocation. If the object was moved, the memory space represented by \f[I]oid\f[] is reclaimed. If \f[I]oidp\f[] points to a memory location from the \f[B]pmemobj\f[] heap, \f[I]oidp\f[] is modified atomically. If \f[I]oidp\f[] is NULL or if it points to the root object's \f[I]OID\f[], the behavior of \f[B]pmemobj_realloc\f[]() is undefined. .PP \f[B]pmemobj_zrealloc\f[]() is equivalent to \f[B]pmemobj_realloc\f[](), except that if the new size is larger than the old size, the added memory will be zeroed. .PP The \f[B]pmemobj_strdup\f[]() function stores a handle to a new object in \f[I]oidp\f[] which is a duplicate of the string \f[I]s\f[]. \f[B]pmemobj_strdup\f[]() provides the same semantics as \f[B]strdup\f[](3), but operates on the persistent memory heap associated with memory pool \f[I]pop\f[]. If \f[I]oidp\f[] is NULL, then the newly allocated object may be accessed only by iterating objects in the object container associated with type number \f[I]type_num\f[], as described in \f[B]POBJ_FOREACH\f[](3). If \f[I]oidp\f[] points to a memory location from the \f[B]pmemobj\f[] heap, \f[I]oidp\f[] is modified atomically. The allocated string object is also added to the internal container associated with type number \f[I]type_num\f[]. Memory for the new string is obtained with \f[B]pmemobj_alloc\f[](), on the given memory pool, and can be freed with \f[B]pmemobj_free\f[]() on the same memory pool. .PP \f[B]pmemobj_wcsdup\f[]() is equivalent to \f[B]pmemobj_strdup\f[](), but operates on a wide character string (wchar_t) rather than a standard character string. .PP The \f[B]pmemobj_alloc_usable_size\f[]() function provides the same semantics as \f[B]malloc_usable_size\f[](3), but instead of the process heap supplied by the system, it operates on the persistent memory heap. .PP The \f[B]POBJ_NEW\f[]() macro is a wrapper around the \f[B]pmemobj_alloc\f[]() function. Instead of taking a pointer to \f[I]PMEMoid\f[], it takes a pointer to the typed \f[I]OID\f[] of type name \f[I]TYPE\f[], and passes the size and type number from the typed \f[I]OID\f[] to \f[B]pmemobj_alloc\f[](). .PP The \f[B]POBJ_ALLOC\f[]() macro is equivalent to \f[B]POBJ_NEW\f[], except that instead of using the size of the typed \f[I]OID\f[], passes \f[I]size\f[] to \f[B]pmemobj_alloc\f[](). .PP The \f[B]POBJ_ZNEW\f[]() macro is a wrapper around the \f[B]pmemobj_zalloc\f[]() function. Instead of taking a pointer to \f[I]PMEMoid\f[], it takes a pointer to the typed \f[I]OID\f[] of type name \f[I]TYPE\f[], and passes the size and type number from the typed \f[I]OID\f[] to \f[B]pmemobj_zalloc\f[](). .PP The \f[B]POBJ_ZALLOC\f[]() macro is equivalent to \f[B]POBJ_ZNEW\f[], except that instead of using the size of the typed \f[I]OID\f[], passes \f[I]size\f[] to \f[B]pmemobj_zalloc\f[](). .PP The \f[B]POBJ_REALLOC\f[]() macro is a wrapper around the \f[B]pmemobj_realloc\f[]() function. Instead of taking a pointer to \f[I]PMEMoid\f[], it takes a pointer to the typed \f[I]OID\f[] of type name \f[I]TYPE\f[], and passes the type number from the typed \f[I]OID\f[] to \f[B]pmemobj_realloc\f[](). .PP The \f[B]POBJ_ZREALLOC\f[]() macro is a wrapper around the \f[B]pmemobj_zrealloc\f[]() function. Instead of taking a pointer to \f[I]PMEMoid\f[], it takes a pointer to the typed \f[I]OID\f[] of type name \f[I]TYPE\f[], and passes the type number from the typed \f[I]OID\f[] to \f[B]pmemobj_zrealloc\f[](). .PP The \f[B]POBJ_FREE\f[]() macro is a wrapper around the \f[B]pmemobj_free\f[]() function which takes a pointer to the typed \f[I]OID\f[] instead of to \f[I]PMEMoid\f[]. .PP The \f[B]pmemobj_defrag\f[]() function performs defragmentation on the objects provided through the array of pointers to PMEMoids \f[I]oidv\f[] with size \f[I]oidcnt\f[]. If an object from the provided array is selected to be moved to a new location in the heap, it is reallocated and all provided pointers to that object are atomically updated. To maintain data structure consistency, applications should always provide all pointers for an object to \f[B]pmemobj_defrag\f[] method. This ensures that, even in the presence of failures, all pointers to the object will either point to the old or a new location. All objects and pointers to objects should belong to the pool \f[I]pop\f[] or, in case of pointers, can also reside in volatile memory. Defragmentation across pools is not supported. Objects in the array that are \f[I]OID_NULL\f[] are skipped over and no operation is performed on them. All other objects must have been allocated by an earlier call to \f[B]pmemobj_alloc\f[](), \f[B]pmemobj_xalloc\f[](), \f[B]pmemobj_zalloc\f[](), \f[B]pmemobj_realloc\f[](), \f[B]pmemobj_zrealloc\f[](), \f[B]pmemobj_strdup\f[]() or \f[B]pmemobj_wcsdup\f[](). The \f[I]result\f[] variable is an instance of \f[I]struct pobj_defrag_result\f[] and, if not NULL, can be used to read \f[I]total\f[], the number of objects found that were processed, and \f[I]relocated\f[], the number of objects that were relocated during defragmentation. These variables are always initialized and can be non\-zero, even if the return value of \f[B]pmemobj_defrag\f[]() indicated a failure. This is because the failure might have occurred after some objects were already processed. .SH RETURN VALUE .PP On success, \f[B]pmemobj_alloc\f[]() and \f[B]pmemobj_xalloc\f[] return 0. If \f[I]oidp\f[] is not NULL, the \f[I]PMEMoid\f[] of the newly allocated object is stored in \f[I]oidp\f[]. If the allocation fails, \-1 is returned and \f[I]errno\f[] is set appropriately. If the constructor returns a non\-zero value, the allocation is canceled, \-1 is returned, and \f[I]errno\f[] is set to \f[B]ECANCELED\f[]. If \f[I]size\f[] equals 0, or the \f[I]flags\f[] for \f[B]pmemobj_xalloc\f[] are invalid, \-1 is returned, \f[I]errno\f[] is set to \f[B]EINVAL\f[], and \f[I]oidp\f[] is left untouched. .PP On success, \f[B]pmemobj_zalloc\f[]() returns 0. If \f[I]oidp\f[] is not NULL, the \f[I]PMEMoid\f[] of the newly allocated object is stored in \f[I]oidp\f[]. If the allocation fails, it returns \-1 and sets \f[I]errno\f[] appropriately. If \f[I]size\f[] equals 0, it returns \-1, sets \f[I]errno\f[] to \f[B]EINVAL\f[], and leaves \f[I]oidp\f[] untouched. .PP The \f[B]pmemobj_free\f[]() function returns no value. .PP On success, \f[B]pmemobj_realloc\f[]() and \f[B]pmemobj_zrealloc\f[]() return 0 and update \f[I]oidp\f[] if necessary. On error, they return \-1 and set \f[I]errno\f[] appropriately. .PP On success, \f[B]pmemobj_strdup\f[]() and \f[B]pmemobj_wcsdup\f[]() return 0. If \f[I]oidp\f[] is not NULL, the \f[I]PMEMoid\f[] of the duplicated string object is stored in \f[I]oidp\f[]. If \f[I]s\f[] is NULL, they return \-1, set \f[I]errno\f[] to \f[B]EINVAL\f[], and leave \f[I]oidp\f[] untouched. On other errors, they return \-1 and set \f[I]errno\f[] appropriately. .PP The \f[B]pmemobj_alloc_usable_size\f[]() function returns the number of usable bytes in the object represented by \f[I]oid\f[]. If \f[I]oid\f[] is \f[B]OID_NULL\f[], it returns 0. .PP On success, \f[B]pmemobj_defrag\f[]() returns 0. If defragmentation was unsuccessful or only partially successful (i.e.\ if it was aborted halfway through due to lack of resources), \-1 is returned. .SH SEE ALSO .PP \f[B]free\f[](3), \f[B]POBJ_FOREACH\f[](3), \f[B]realloc\f[](3), \f[B]strdup\f[](3), \f[B]wcsdup\f[](3), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pmemobj_tx_begin.3.md0000664000000000000000000006121513615011243017676 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMEMOBJ_TX_BEGIN, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmemobj_tx_begin.3 -- man page for transactional object manipulation) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
[CAVEATS](#caveats)
[SEE ALSO](#see-also)
# NAME # **pmemobj_tx_stage**(), **pmemobj_tx_begin**(), **pmemobj_tx_lock**(), **pmemobj_tx_xlock**(), **pmemobj_tx_abort**(), **pmemobj_tx_commit**(), **pmemobj_tx_end**(), **pmemobj_tx_errno**(), **pmemobj_tx_process**(), **TX_BEGIN_PARAM**(), **TX_BEGIN_CB**(), **TX_BEGIN**(), **TX_ONABORT**, **TX_ONCOMMIT**, **TX_FINALLY**, **TX_END**, **pmemobj_tx_log_append_buffer**(), **pmemobj_tx_xlog_append_buffer**(), **pmemobj_tx_log_auto_alloc**(), **pmemobj_tx_log_snapshots_max_size**(), **pmemobj_tx_log_intents_max_size**(), **pmemobj_tx_set_user_data**(), **pmemobj_tx_get_user_data**() - transactional object manipulation # SYNOPSIS # ```c #include enum tx_stage pmemobj_tx_stage(void); int pmemobj_tx_begin(PMEMobjpool *pop, jmp_buf *env, enum pobj_tx_param, ...); int pmemobj_tx_lock(enum tx_lock lock_type, void *lockp); int pmemobj_tx_xlock(enum tx_lock lock_type, void *lockp, uint64_t flags); void pmemobj_tx_abort(int errnum); void pmemobj_tx_commit(void); int pmemobj_tx_end(void); int pmemobj_tx_errno(void); void pmemobj_tx_process(void); TX_BEGIN_PARAM(PMEMobjpool *pop, ...) TX_BEGIN_CB(PMEMobjpool *pop, cb, arg, ...) TX_BEGIN(PMEMobjpool *pop) TX_ONABORT TX_ONCOMMIT TX_FINALLY TX_END int pmemobj_tx_log_append_buffer(enum pobj_log_type type, void *addr, size_t size); int pmemobj_tx_xlog_append_buffer(enum pobj_log_type type, void *addr, size_t size, uint64_t flags); int pmemobj_tx_log_auto_alloc(enum pobj_log_type type, int on_off); size_t pmemobj_tx_log_snapshots_max_size(size_t *sizes, size_t nsizes); size_t pmemobj_tx_log_intents_max_size(size_t nintents); void pmemobj_tx_set_user_data(void *data); void *pmemobj_tx_get_user_data(void); ``` # DESCRIPTION # The non-transactional functions and macros described in **pmemobj_alloc**(3), **pmemobj_list_insert**(3) and **POBJ_LIST_HEAD**(3) only guarantee the atomicity of a single operation on an object. In case of more complex changes involving multiple operations on an object, or allocation and modification of multiple objects, data consistency and fail-safety may be provided only by using *atomic transactions*. A transaction is defined as series of operations on persistent memory objects that either all occur, or nothing occurs. In particular, if the execution of a transaction is interrupted by a power failure or a system crash, it is guaranteed that after system restart, all the changes made as a part of the uncompleted transaction will be rolled back, restoring the consistent state of the memory pool from the moment when the transaction was started. Note that transactions do not provide atomicity with respect to other threads. All the modifications performed within the transactions are immediately visible to other threads. Therefore it is the responsibility of the application to implement a proper thread synchronization mechanism. Each thread may have only one transaction open at a time, but that transaction may be nested. Nested transactions are flattened. Committing the nested transaction does not commit the outer transaction; however, errors in the nested transaction are propagated up to the outermost level, resulting in the interruption of the entire transaction. Each transaction is visible only for the thread that started it. No other threads can add operations, commit or abort the transaction initiated by another thread. Multiple threads may have transactions open on a given memory pool at the same time. Please see the **CAVEATS** section below for known limitations of the transactional API. The **pmemobj_tx_stage**() function returns the current *transaction stage* for a thread. Stages are changed only by the **pmemobj_tx_\***() functions. Transaction stages are defined as follows: + **TX_STAGE_NONE** - no open transaction in this thread + **TX_STAGE_WORK** - transaction in progress + **TX_STAGE_ONCOMMIT** - successfully committed + **TX_STAGE_ONABORT** - starting the transaction failed or transaction aborted + **TX_STAGE_FINALLY** - ready for clean up The **pmemobj_tx_begin**() function starts a new transaction in the current thread. If called within an open transaction, it starts a nested transaction. The caller may use the *env* argument to provide a pointer to a calling environment to be restored in case of transaction abort. This information must be provided by the caller using the **setjmp**(3) macro. A new transaction may be started only if the current stage is **TX_STAGE_NONE** or **TX_STAGE_WORK**. If successful, the *transaction stage* changes to **TX_STAGE_WORK**. Otherwise, the stage is changed to **TX_STAGE_ONABORT**. Optionally, a list of parameters for the transaction may be provided. Each parameter consists of a type followed by a type-specific number of values. Currently there are 4 types: + **TX_PARAM_NONE**, used as a termination marker. No following value. + **TX_PARAM_MUTEX**, followed by one value, a pmem-resident PMEMmutex + **TX_PARAM_RWLOCK**, followed by one value, a pmem-resident PMEMrwlock + **TX_PARAM_CB**, followed by two values: a callback function of type *pmemobj_tx_callback*, and a void pointer Using **TX_PARAM_MUTEX** or **TX_PARAM_RWLOCK** causes the specified lock to be acquired at the beginning of the transaction. **TX_PARAM_RWLOCK** acquires the lock for writing. It is guaranteed that **pmemobj_tx_begin**() will acquire all locks prior to successful completion, and they will be held by the current thread until the outermost transaction is finished. Locks are taken in order from left to right. To avoid deadlocks, the user is responsible for proper lock ordering. **TX_PARAM_CB** registers the specified callback function to be executed at each transaction stage. For **TX_STAGE_WORK**, the callback is executed prior to commit. For all other stages, the callback is executed as the first operation after a stage change. It will also be called after each transaction; in this case the *stage* parameter will be set to **TX_STAGE_NONE**. *pmemobj_tx_callback* must be compatible with: ``` void func(PMEMobjpool *pop, enum pobj_tx_stage stage, void *arg) ``` *pop* is a pool identifier used in **pmemobj_tx_begin**(), *stage* is a current transaction stage and *arg* is the second parameter of **TX_PARAM_CB**. Without considering transaction nesting, this mechanism can be considered an alternative method for executing code between stages (instead of **TX_ONCOMMIT**, **TX_ONABORT**, etc). However, there are 2 significant differences when nested transactions are used: + The registered function is executed only in the outermost transaction, even if registered in an inner transaction. + There can be only one callback in the entire transaction, that is, the callback cannot be changed in an inner transaction. Note that **TX_PARAM_CB** does not replace the **TX_ONCOMMIT**, **TX_ONABORT**, etc. macros. They can be used together: the callback will be executed *before* a **TX_ONCOMMIT**, **TX_ONABORT**, etc. section. **TX_PARAM_CB** can be used when the code dealing with transaction stage changes is shared between multiple users or when it must be executed only in the outer transaction. For example it can be very useful when the application must synchronize persistent and transient state. The **pmemobj_tx_lock**() function acquires the lock *lockp* of type *lock_type* and adds it to the current transaction. *lock_type* may be **TX_LOCK_MUTEX** or **TX_LOCK_RWLOCK**; *lockp* must be of type *PMEMmutex* or *PMEMrwlock*, respectively. If *lock_type* is **TX_LOCK_RWLOCK** the lock is acquired for writing. If the lock is not successfully acquired, the function returns an error number. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_xlock**() function behaves exactly the same as **pmemobj_tx_lock**() when *flags* equals **POBJ_XLOCK_NO_ABORT**. When *flags* equals 0 and if the lock is not successfully acquired,the transaction is aborted. *flags* is a bitmask of the following values: + **POBJ_XLOCK_NO_ABORT** - if the function does not end successfully, do not abort the transaction. **pmemobj_tx_abort**() aborts the current transaction and causes a transition to **TX_STAGE_ONABORT**. If *errnum* is equal to 0, the transaction error code is set to **ECANCELED**; otherwise, it is set to *errnum*. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_commit**() function commits the current open transaction and causes a transition to **TX_STAGE_ONCOMMIT**. If called in the context of the outermost transaction, all the changes may be considered as durably written upon successful completion. This function must be called during **TX_STAGE_WORK**. The **pmemobj_tx_end**() function performs a cleanup of the current transaction. If called in the context of the outermost transaction, it releases all the locks acquired by **pmemobj_tx_begin**() for outer and nested transactions. If called in the context of a nested transaction, it returns to the context of the outer transaction in **TX_STAGE_WORK**, without releasing any locks. The **pmemobj_tx_end**() function can be called during **TX_STAGE_NONE** if transitioned to this stage using **pmemobj_tx_process**(). If not already in **TX_STAGE_NONE**, it causes the transition to **TX_STAGE_NONE**. **pmemobj_tx_end** must always be called for each **pmemobj_tx_begin**(), even if starting the transaction failed. This function must *not* be called during **TX_STAGE_WORK**. The **pmemobj_tx_errno**() function returns the error code of the last transaction. The **pmemobj_tx_process**() function performs the actions associated with the current stage of the transaction, and makes the transition to the next stage. It must be called in a transaction. The current stage must always be obtained by a call to **pmemobj_tx_stage**(). **pmemobj_tx_process**() performs the following transitions in the transaction stage flow: + **TX_STAGE_WORK** -> **TX_STAGE_ONCOMMIT** + **TX_STAGE_ONABORT** -> **TX_STAGE_FINALLY** + **TX_STAGE_ONCOMMIT** -> **TX_STAGE_FINALLY** + **TX_STAGE_FINALLY** -> **TX_STAGE_NONE** + **TX_STAGE_NONE** -> **TX_STAGE_NONE** **pmemobj_tx_process**() must not be called after calling **pmemobj_tx_end**() for the outermost transaction. In addition to the above API, **libpmemobj**(7) offers a more intuitive method of building transactions using the set of macros described below. When using these macros, the complete transaction flow looks like this: ```c TX_BEGIN(Pop) { /* the actual transaction code goes here... */ } TX_ONCOMMIT { /* * optional - executed only if the above block * successfully completes */ } TX_ONABORT { /* * optional - executed only if starting the transaction fails, * or if transaction is aborted by an error or a call to * pmemobj_tx_abort() */ } TX_FINALLY { /* * optional - if exists, it is executed after * TX_ONCOMMIT or TX_ONABORT block */ } TX_END /* mandatory */ ``` ```c TX_BEGIN_PARAM(PMEMobjpool *pop, ...) TX_BEGIN_CB(PMEMobjpool *pop, cb, arg, ...) TX_BEGIN(PMEMobjpool *pop) ``` The **TX_BEGIN_PARAM**(), **TX_BEGIN_CB**() and **TX_BEGIN**() macros start a new transaction in the same way as **pmemobj_tx_begin**(), except that instead of the environment buffer provided by a caller, they set up the local *jmp_buf* buffer and use it to catch the transaction abort. The **TX_BEGIN**() macro starts a transaction without any options. **TX_BEGIN_PARAM** may be used when there is a need to acquire locks prior to starting a transaction (such as for a multi-threaded program) or set up a transaction stage callback. **TX_BEGIN_CB** is just a wrapper around **TX_BEGIN_PARAM** that validates the callback signature. (For compatibility there is also a **TX_BEGIN_LOCK** macro, which is an alias for **TX_BEGIN_PARAM**). Each of these macros must be followed by a block of code with all the operations that are to be performed atomically. The **TX_ONABORT** macro starts a block of code that will be executed only if starting the transaction fails due to an error in **pmemobj_tx_begin**(), or if the transaction is aborted. This block is optional, but in practice it should not be omitted. If it is desirable to crash the application when a transaction aborts and there is no **TX_ONABORT** section, the application can define the **POBJ_TX_CRASH_ON_NO_ONABORT** macro before inclusion of **\**. This provides a default **TX_ONABORT** section which just calls **abort**(3). The **TX_ONCOMMIT** macro starts a block of code that will be executed only if the transaction is successfully committed, which means that the execution of code in the **TX_BEGIN**() block has not been interrupted by an error or by a call to **pmemobj_tx_abort**(). This block is optional. The **TX_FINALLY** macro starts a block of code that will be executed regardless of whether the transaction is committed or aborted. This block is optional. The **TX_END** macro cleans up and closes the transaction started by the **TX_BEGIN**() / **TX_BEGIN_PARAM**() / **TX_BEGIN_CB**() macros. It is mandatory to terminate each transaction with this macro. If the transaction was aborted, *errno* is set appropriately. ## TRANSACTION LOG TUNING ## From libpmemobj implementation perspective there are two types of operations in a transaction: + **snapshots**, where action must be persisted immediately, + **intents**, where action can be persisted at the transaction commit phase **pmemobj_tx_add_range**(3) and all its variants belong to the **snapshots** group. **pmemobj_tx_alloc**(3) (with its variants), **pmemobj_tx_free**(3), **pmemobj_tx_realloc**(3) (with its variants) and **pmemobj_tx_publish**(3) belong to the **intents** group. Even though **pmemobj_tx_alloc**() allocates memory immediately, it modifies only the runtime state and postpones persistent memory modifications to the commit phase. **pmemobj_tx_free**(3) cannot free the object immediately, because of possible transaction rollback, so it postpones both the action and persistent memory modifications to the commit phase. **pmemobj_tx_realloc**(3) is just a combination of those two. **pmemobj_tx_publish**(3) postpones reservations and deferred frees to the commit phase. Those two types of operations (snapshots and intents) require that libpmemobj builds a persistent log of operations. Intent log (also known as a "redo log") is applied on commit and snapshot log (also known as an "undo log") is applied on abort. When libpmemobj transaction starts, it's not possible to predict how much persistent memory space will be needed for those logs. This means that libpmemobj must internally allocate this space whenever it's needed. This has two downsides: + when transaction snapshots a lot of memory or does a lot of allocations, libpmemobj may need to do many internal allocations, which must be freed when transaction ends, adding time overhead when big transactions are frequent, + transactions can start to fail due to not enough space for logs - this can be especially problematic for transactions that want to **deallocate** objects, as those might also fail To solve both of these problems libpmemobj exposes the following functions: + **pmemobj_tx_log_append_buffer**(), + **pmemobj_tx_xlog_append_buffer**(), + **pmemobj_tx_log_auto_alloc**() **pmemobj_tx_log_append_buffer**() appends a given range of memory [*addr*, *addr* + *size*) to the log *type* of the current transaction. *type* can be one of the two values (with meanings described above): + **TX_LOG_TYPE_SNAPSHOT**, + **TX_LOG_TYPE_INTENT** The range of memory **must** belong to the same pool the transaction is on and **must not** be used by more than one thread at the same time. The latter condition can be verified with tx.debug.verify_user_buffers ctl (see **pmemobj_ctl_get**(3)). The **pmemobj_tx_xlog_append_buffer**() function behaves exactly the same as **pmemobj_tx_log_append_buffer**() when *flags* equals zero. *flags* is a bitmask of the following values: + **POBJ_XLOG_APPEND_BUFFER_NO_ABORT** - if the function does not end successfully, do not abort the transaction. **pmemobj_tx_log_snapshots_max_size** calculates the **maximum** size of a buffer which will be able to hold *nsizes* snapshots, each of size *sizes[i]*. Application should not expect this function to return the same value between restarts. In future versions of libpmemobj this function can return smaller (because of better accuracy or space optimizations) or higher (because of higher alignment required for better performance) value. This function is independent of transaction stage and can be called both inside and outside of transaction. If the returned value S is greater than **PMEMOBJ_MAX_ALLOC_SIZE**, the buffer should be split into N chunks of size **PMEMOBJ_MAX_ALLOC_SIZE**, where N is equal to (S / **PMEMOBJ_MAX_ALLOC_SIZE**) (rounded down) and the last chunk of size (S - (N * **PMEMOBJ_MAX_ALLOC_SIZE**)). **pmemobj_tx_log_intents_max_size** calculates the **maximum** size of a buffer which will be able to hold *nintents* intents. Just like with **pmemobj_tx_log_snapshots_max_size**, application should not expect this function to return the same value between restarts, for the same reasons. This function is independent of transaction stage and can be called both inside and outside of transaction. **pmemobj_tx_log_auto_alloc**() disables (*on_off* set to 0) or enables (*on_off* set to 1) automatic allocation of internal logs of given *type*. It can be used to verify that the buffer set with **pmemobj_tx_log_append_buffer**() is big enough to hold the log, without reaching out-of-space scenario. The **pmemobj_tx_set_user_data**() function associates custom volatile state, represented by pointer *data*, with the current transaction. This state can later be retrieved using **pmemobj_tx_get_user_data**() function. If **pmemobj_tx_set_user_data**() was not called for a current transaction, **pmemobj_tx_get_user_data**() will return NULL. These functions must be called during **TX_STAGE_WORK** or **TX_STAGE_ONABORT** or **TX_STAGE_ONCOMMIT** or **TX_STAGE_FINALLY**. # RETURN VALUE # The **pmemobj_tx_stage**() function returns the stage of the current transaction stage for a thread. On success, **pmemobj_tx_begin**() returns 0. Otherwise, an error number is returned. The **pmemobj_tx_begin**() and **pmemobj_tx_lock**() functions return zero if *lockp* is successfully added to the transaction. Otherwise, an error number is returned. The **pmemobj_tx_xlock**() function return zero if *lockp* is successfully added to the transaction. Otherwise, the error number is returned, **errno** is set and when flags do not contain **POBJ_XLOCK_NO_ABORT**, the transaction is aborted. The **pmemobj_tx_abort**() and **pmemobj_tx_commit**() functions return no value. The **pmemobj_tx_end**() function returns 0 if the transaction was successful. Otherwise it returns the error code set by **pmemobj_tx_abort**(). Note that **pmemobj_tx_abort**() can be called internally by the library. The **pmemobj_tx_errno**() function returns the error code of the last transaction. The **pmemobj_tx_process**() function returns no value. On success, **pmemobj_tx_log_append_buffer**() returns 0. Otherwise, the stage is changed to **TX_STAGE_ONABORT**, **errno** is set appropriately and transaction is aborted. On success, **pmemobj_tx_xlog_append_buffer**() returns 0. Otherwise, the error number is returned, **errno** is set and when flags do not contain **POBJ_XLOG_NO_ABORT**, the transaction is aborted. On success, **pmemobj_tx_log_auto_alloc**() returns 0. Otherwise, the transaction is aborted and an error number is returned. On success, **pmemobj_tx_log_snapshots_max_size**() returns size of the buffer. On failure it returns *SIZE_MAX* and sets *errno* appropriately. On success, **pmemobj_tx_log_intents_max_size**() returns size of the buffer. On failure it returns *SIZE_MAX* and sets *errno* appropriately. # CAVEATS # Transaction flow control is governed by the **setjmp**(3) and **longjmp**(3) macros, and they are used in both the macro and function flavors of the API. The transaction will longjmp on transaction abort. This has one major drawback, which is described in the ISO C standard subsection 7.13.2.1. It says that **the values of objects of automatic storage duration that are local to the function containing the setjmp invocation that do not have volatile-qualified type and have been changed between the setjmp invocation and longjmp call are indeterminate.** The following example illustrates the issue described above. ```c int *bad_example_1 = (int *)0xBAADF00D; int *bad_example_2 = (int *)0xBAADF00D; int *bad_example_3 = (int *)0xBAADF00D; int * volatile good_example = (int *)0xBAADF00D; TX_BEGIN(pop) { bad_example_1 = malloc(sizeof(int)); bad_example_2 = malloc(sizeof(int)); bad_example_3 = malloc(sizeof(int)); good_example = malloc(sizeof(int)); /* manual or library abort called here */ pmemobj_tx_abort(EINVAL); } TX_ONCOMMIT { /* * This section is longjmp-safe */ } TX_ONABORT { /* * This section is not longjmp-safe */ free(good_example); /* OK */ free(bad_example_1); /* undefined behavior */ } TX_FINALLY { /* * This section is not longjmp-safe on transaction abort only */ free(bad_example_2); /* undefined behavior */ } TX_END free(bad_example_3); /* undefined behavior */ ``` Objects which are not volatile-qualified, are of automatic storage duration and have been changed between the invocations of **setjmp**(3) and **longjmp**(3) (that also means within the work section of the transaction after **TX_BEGIN**()) should not be used after a transaction abort, or should be used with utmost care. This also includes code after the **TX_END** macro. **libpmemobj**(7) is not cancellation-safe. The pool will never be corrupted because of a canceled thread, but other threads may stall waiting on locks taken by that thread. If the application wants to use **pthread_cancel**(3), it must disable cancellation before calling any **libpmemobj**(7) APIs (see **pthread_setcancelstate**(3) with **PTHREAD_CANCEL_DISABLE**), and re-enable it afterwards. Deferring cancellation (**pthread_setcanceltype**(3) with **PTHREAD_CANCEL_DEFERRED**) is not safe enough, because **libpmemobj**(7) internally may call functions that are specified as cancellation points in POSIX. **libpmemobj**(7) relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. **dlclose**(3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. # SEE ALSO # **dlclose**(3), **longjmp**(3), **pmemobj_tx_add_range**(3), **pmemobj_tx_alloc**(3), **pthread_setcancelstate**(3), **pthread_setcanceltype**(3), **setjmp**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_first.30000644000000000000000000001223513615011417016626 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMEMOBJ_FIRST" "3" "2020-01-31" "PMDK - pmemobj API version 2.3" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmemobj_first\f[](), \f[B]pmemobj_next\f[](), \f[B]POBJ_FIRST\f[](), \f[B]POBJ_FIRST_TYPE_NUM\f[](), \f[B]POBJ_NEXT\f[](), \f[B]POBJ_NEXT_TYPE_NUM\f[](), \f[B]POBJ_FOREACH\f[](), \f[B]POBJ_FOREACH_SAFE\f[](), \f[B]POBJ_FOREACH_TYPE\f[](), \f[B]POBJ_FOREACH_SAFE_TYPE\f[]() \- pmemobj container operations .SH SYNOPSIS .IP .nf \f[C] #include\ PMEMoid\ pmemobj_first(PMEMobjpool\ *pop); PMEMoid\ pmemobj_next(PMEMoid\ oid); POBJ_FIRST(PMEMobjpool\ *pop,\ TYPE) POBJ_FIRST_TYPE_NUM(PMEMobjpool\ *pop,\ uint64_t\ type_num) POBJ_NEXT(TOID\ oid) POBJ_NEXT_TYPE_NUM(PMEMoid\ oid) POBJ_FOREACH(PMEMobjpool\ *pop,\ PMEMoid\ varoid) POBJ_FOREACH_SAFE(PMEMobjpool\ *pop,\ PMEMoid\ varoid,\ PMEMoid\ nvaroid) POBJ_FOREACH_TYPE(PMEMobjpool\ *pop,\ TOID\ var) POBJ_FOREACH_SAFE_TYPE(PMEMobjpool\ *pop,\ TOID\ var,\ TOID\ nvar) \f[] .fi .SH DESCRIPTION .PP The \f[B]libpmemobj\f[](7) container operations provide a mechanism that allows iteration through the internal object collection, either looking for a specific object, or performing a specific operation on each object of a given type. Software should not make any assumptions about the order of the objects in the internal object containers. .PP The \f[B]pmemobj_first\f[]() function returns the first object from the pool. .PP The \f[B]POBJ_FIRST\f[]() macro returns the first object from the pool of the type specified by \f[I]TYPE\f[]. .PP The \f[B]POBJ_FIRST_TYPE_NUM\f[]() macro returns the first object from the pool of the type specified by \f[I]type_num\f[]. .PP The \f[B]pmemobj_next\f[]() function returns the next object from the pool. .PP The \f[B]POBJ_NEXT\f[]() macro returns the next object of the same type as the object referenced by \f[I]oid\f[]. .PP The \f[B]POBJ_NEXT_TYPE_NUM\f[]() macro returns the next object of the same type number as the object referenced by \f[I]oid\f[]. .PP The following four macros provide a more convenient way to iterate through the internal collections, performing a specific operation on each object. .PP The \f[B]POBJ_FOREACH\f[]() macro performs a specific operation on each allocated object stored in the persistent memory pool \f[I]pop\f[]. It traverses the internal collection of all the objects, assigning a handle to each element in turn to \f[I]varoid\f[]. .PP The \f[B]POBJ_FOREACH_TYPE\f[]() macro performs a specific operation on each allocated object stored in the persistent memory pool \f[I]pop\f[] that has the same type as \f[I]var\f[]. It traverses the internal collection of all the objects of the specified type, assigning a handle to each element in turn to \f[I]var\f[]. .PP The macros \f[B]POBJ_FOREACH_SAFE\f[]() and \f[B]POBJ_FOREACH_SAFE_TYPE\f[]() work in a similar fashion as \f[B]POBJ_FOREACH\f[]() and \f[B]POBJ_FOREACH_TYPE\f[](), except that prior to performing the operation on the object, they preserve a handle to the next object in the collection by assigning it to \f[I]nvaroid\f[] or \f[I]nvar\f[], respectively. This allows safe deletion of selected objects while iterating through the collection. .SH RETURN VALUE .PP \f[B]pmemobj_first\f[]() returns the first object from the pool, or, if the pool is empty, \f[B]OID_NULL\f[]. .PP \f[B]pmemobj_next\f[]() returns the next object from the pool. If the object referenced by \f[I]oid\f[] is the last object in the collection, or if \f[I]oid\f[] is \f[I]OID_NULL\f[], \f[B]pmemobj_next\f[]() returns \f[B]OID_NULL\f[]. .SH SEE ALSO .PP \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/libpmemobj/pobj_list_first.30000664000000000000000000000002513615011243017153 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/toid.30000664000000000000000000000002313615011243014714 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/oid_is_null.3.md0000664000000000000000000002140713615011243016665 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(OID_IS_NULL, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (oid_is_null.3 -- man page for persistent object identifier and functions) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[RETURN VALUE](#return-value)
_WINUX(,[NOTES](#notes)
) [SEE ALSO](#see-also)
# NAME # **OID_IS_NULL**(), **OID_EQUALS**(), **pmemobj_direct**(), **pmemobj_oid**(), **pmemobj_type_num**(), **pmemobj_pool_by_oid**(), **pmemobj_pool_by_ptr**() - functions that allow mapping operations between object addresses, object handles, oids or type numbers # SYNOPSIS # ```c #include OID_IS_NULL(PMEMoid oid) OID_EQUALS(PMEMoid lhs, PMEMoid rhs) void *pmemobj_direct(PMEMoid oid); PMEMoid pmemobj_oid(const void *addr); uint64_t pmemobj_type_num(PMEMoid oid); PMEMobjpool *pmemobj_pool_by_oid(PMEMoid oid); PMEMobjpool *pmemobj_pool_by_ptr(const void *addr); void *pmemobj_volatile(PMEMobjpool *pop, struct pmemvlt *vlt, size_t size, void *ptr, int (*constr)(void *ptr, void *arg), void *arg); (EXPERIMENTAL) ``` # DESCRIPTION # Each object stored in a persistent memory pool is represented by an object handle of type *PMEMoid*. In practice, such a handle is a unique Object IDentifier (*OID*) of global scope, which means that two objects from different pools will never have the same *OID*. The special **OID_NULL** macro defines a NULL-like handle that does not represent any object. The size of a single object is limited by **PMEMOBJ_MAX_ALLOC_SIZE**. Thus an allocation with a requested size greater than this value will fail. An *OID* cannot be used as a direct pointer to an object. Each time the program attempts to read or write object data, it must obtain the current memory address of the object by converting its *OID* into a pointer. In contrast to the memory address, the *OID* value for given object does not change during the life of an object (except for *realloc*), and remains valid after closing and reopening the pool. For this reason, if an object contains a reference to another persistent object, for example, to build some kind of a linked data structure, the reference must be an *OID* and not a memory address. **pmemobj_direct**() returns a pointer to the *PMEMoid* object with handle *oid*. **pmemobj_oid**() returns a *PMEMoid* handle to the object pointed to by *addr*. **pmemobj_type_num**() returns the type number of the *PMEMoid* object with handle *oid*. **pmemobj_pool_by_oid**() returns a *PMEMobjpool*\* handle to the pool containing the *PMEMoid* object with handle *oid*. **pmemobj_pool_by_ptr**() returns a *PMEMobjpool*\* handle to the pool containing the address *addr*. At the time of allocation (or reallocation), each object may be assigned a number representing its type. Such a *type number* may be used to arrange the persistent objects based on their actual user-defined structure type, thus facilitating implementation of a simple run-time type safety mechanism. This also allows iterating through all the objects of a given type that are stored in the persistent memory pool. See **pmemobj_first**(3) for more information. The **OID_IS_NULL**() macro checks if *PMEMoid* represents a NULL object. The **OID_EQUALS**() macro compares two *PMEMoid* objects. For special cases where volatile (transient) variables need to be stored on persistent memory, there's a mechanism composed of *struct pmemvlt* type and **pmemobj_volatile()** function. To use it, the *struct pmemvlt* needs to be placed in the neighborhood of transient data region. The *PMEMvlt* macro can be used to construct such a region. The *struct pmemvlt* must be zeroed prior to use. This can be easily done in object constructor or in a transaction directly after an allocation. When the **pmemobj_volatile()** function is called on a *struct pmemvlt*, it will return the pointer to the data and it will ensure that the provided constructor function is called exactly once in the current instance of the pmemobj pool. The constructor is called with the *ptr* pointer to the data, and this function will return the same pointer if the constructor returns *0*, otherwise NULL is returned. The *size* argument must accurately describe the total size of the volatile memory region that will be accessed. Calling **pmemobj_volatile()** on the same region with different sizes is undefined behavior. For this mechanism to be effective, all accesses to transient variables must go through it, otherwise there's a risk of the constructor not being called on the first load. Maintaining transient state on persistent memory is challenging due to difficulties with dynamic resources acquisition and subsequent resource release. For example, one needs to consider what happens with volatile state of an object which is being freed inside of a transaction, especially with regards to the possibility of an abort. It's generally recommended to entirely separate the persistent and transient states, and when it's not possible, to only store types which do not require lifecycle management (i.e., primitive types) inside of volatile regions. # RETURN VALUE # The **pmemobj_direct**() function returns a pointer to the object represented by *oid*. If *oid* is **OID_NULL**, **pmemobj_direct**() returns NULL. The **pmemobj_oid**() function returns a *PMEMoid* handle to the object pointed to by *addr*. If *addr* is not from within a pmemobj pool, **OID_NULL** is returned. If *addr* is not the start of an object (does not point to the beginning of a valid allocation), the resulting *PMEMoid* can be safely used only with: + **pmemobj_pool_by_oid**() + **pmemobj_direct**() + **pmemobj_tx_add_range**(3) The **pmemobj_type_num**() function returns the type number of the object represented by *oid*. The **pmemobj_pool_by_oid**() function returns a handle to the pool that contains the object represented by *oid*. If the pool is not open or *oid* is **OID_NULL**, **pmemobj_pool_by_oid**() returns NULL. The **pmemobj_pool_by_ptr**() function returns a handle to the pool that contains the address, or NULL if the address does not belong to any open pool. _WINUX(,=q= # NOTES # For performance reasons, on Linux and FreeBSD the **pmemobj_direct**() function is inlined by default. To use the non-inlined variant of **pmemobj_direct**(), define **PMEMOBJ_DIRECT_NON_INLINE** prior to the *\#include* of **\**, either with *\#define* or with the *\-D* option to the compiler.=e=) # EXAMPLES # The following code shows how to store transient variables on persistent memory. ```c struct my_data { PMEMvlt(uint64_t) foo; uint64_t bar; }; int my_data_constructor(void *ptr, void *arg) { uint64_t *foo = ptr; *foo = 0; return 0; } PMEMobjpool *pop = ...; struct my_data *data = D_RW(...); uint64_t *foo = pmemobj_volatile(pop, &data->foo.vlt, &data->foo.value, my_data_constructor, NULL); assert(*foo == 0); ``` # SEE ALSO # **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_ctl_exec.30000664000000000000000000000002613615011243017257 0ustar rootroot.so pmemobj_ctl_get.3 pmdk-1.8/doc/libpmemobj/pmemobj_create.30000664000000000000000000000002313615011243016731 0ustar rootroot.so pmemobj_open.3 pmdk-1.8/doc/libpmemobj/pmemobj_pool_by_ptr.30000664000000000000000000000002213615011243020015 0ustar rootroot.so oid_is_null.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_xlock.30000664000000000000000000000002713615011243017325 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pobj_list_foreach_reverse.30000664000000000000000000000002513615011243021166 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_strdup.30000664000000000000000000000002413615011243017010 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_layout_name.30000664000000000000000000000003013615011243017302 0ustar rootroot.so pobj_layout_begin.3 pmdk-1.8/doc/libpmemobj/tx_xadd_field.30000664000000000000000000000003313615011243016554 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_drain.30000664000000000000000000000003513615011243016566 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/direct_rw.30000664000000000000000000000002313615011243015737 0ustar rootroot.so toid_declare.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_get_user_data.30000664000000000000000000000002713615011243021013 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_check.30000664000000000000000000000002313615011243016543 0ustar rootroot.so pmemobj_open.3 pmdk-1.8/doc/libpmemobj/pmemobj_wcsdup.30000664000000000000000000000002413615011243016774 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/tx_xfree.30000664000000000000000000000002713615011243015605 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/tx_add.30000664000000000000000000000003313615011243015221 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_f_mem_noflush.30000664000000000000000000000003513615011243020312 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pobj_alloc.30000664000000000000000000000002413615011243016062 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_layout_root.30000664000000000000000000000003013615011243017345 0ustar rootroot.so pobj_layout_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_end.30000664000000000000000000000002713615011243016753 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_rwlock_trywrlock.30000664000000000000000000000003113615011243021106 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pmemobj_mutex_unlock.30000664000000000000000000000003113615011243020202 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_publish.30000664000000000000000000000002513615011243017651 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/tx_memcpy.30000664000000000000000000000003313615011243015763 0ustar rootroot.so pmemobj_tx_add_range.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_free.30000664000000000000000000000002713615011243017126 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_layout_types_num.30000664000000000000000000000003013615011243020405 0ustar rootroot.so pobj_layout_begin.3 pmdk-1.8/doc/libpmemobj/pmemobj_f_mem_wb.30000664000000000000000000000003513615011243017244 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pmemobj_cond_zero.30000664000000000000000000000003113615011243017447 0ustar rootroot.so pmemobj_mutex_zero.3 pmdk-1.8/doc/libpmemobj/pmemobj_zalloc.30000664000000000000000000000002413615011243016753 0ustar rootroot.so pmemobj_alloc.3 pmdk-1.8/doc/libpmemobj/pmemobj_f_relaxed.30000664000000000000000000000003513615011243017422 0ustar rootroot.so pmemobj_memcpy_persist.3 pmdk-1.8/doc/libpmemobj/pmemobj_volatile.30000664000000000000000000000002213615011243017304 0ustar rootroot.so oid_is_null.3 pmdk-1.8/doc/libpmemobj/pmemobj_tx_xstrdup.30000664000000000000000000000002713615011243017716 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_layout_begin.3.md0000664000000000000000000001163713615011243020064 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(POBJ_LAYOUT_BEGIN, 3) collection: libpmemobj header: PMDK date: pmemobj API version 2.3 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pobj_layout_begin.3 -- man page for declaration of pool's layout) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **POBJ_LAYOUT_BEGIN**(), **POBJ_LAYOUT_TOID**(), **POBJ_LAYOUT_ROOT**(), **POBJ_LAYOUT_NAME**(), **POBJ_LAYOUT_END**(), **POBJ_LAYOUT_TYPES_NUM**() - persistent memory transactional object store layout # SYNOPSIS # ```c #include POBJ_LAYOUT_BEGIN(layout) POBJ_LAYOUT_TOID(layout, TYPE) POBJ_LAYOUT_ROOT(layout, ROOT_TYPE) POBJ_LAYOUT_NAME(layout) POBJ_LAYOUT_END(layout) POBJ_LAYOUT_TYPES_NUM(layout) ``` # DESCRIPTION # **libpmemobj**(7) defines a set of macros for convenient declaration of a pool's layout. The layout declaration consists of declarations of a number of used types. The declared types will be assigned consecutive type numbers. Declared types may be used in conjunction with type safety macros (see **TOID_DECLARE**(3)). Once created, the layout declaration must not be changed unless any new types are added at the end of the existing layout declaration. Modifying any existing declaration may lead to changes in the type numbers of declared types, which in consequence may cause data corruption. The **POBJ_LAYOUT_BEGIN**() macro indicates a begin of declaration of layout. The *LAYOUT* argument is a name of layout. This argument must be passed to all macros related to the declaration of layout. The **POBJ_LAYOUT_TOID**() macro declares a typed *OID* for type passed as *TYPE* argument inside the declaration of layout. All types declared using this macro are assigned with consecutive type numbers. This macro must be used between the **POBJ_LAYOUT_BEGIN**() and **POBJ_LAYOUT_END**() macros, with the same name passed as *LAYOUT* argument. The **POBJ_LAYOUT_ROOT**() macro declares a typed *OID* for type passed as *ROOT_TYPE* argument inside the declaration of layout. The typed *OID* will be assigned with type number for root object **POBJ_ROOT_TYPE_NUM**. The **POBJ_LAYOUT_END**() macro ends the declaration of layout. The **POBJ_LAYOUT_NAME**() macro returns the name of layout as a null-terminated string. The **POBJ_LAYOUT_TYPES_NUM**() macro returns number of types declared using the **POBJ_LAYOUT_TOID**() macro within the layout declaration. # EXAMPLE # This is an example of layout declaration: ```c POBJ_LAYOUT_BEGIN(mylayout); POBJ_LAYOUT_ROOT(mylayout, struct root); POBJ_LAYOUT_TOID(mylayout, struct node); POBJ_LAYOUT_TOID(mylayout, struct foo); POBJ_LAYOUT_END(mylayout); struct root { TOID(struct node) node; }; struct node { TOID(struct node) next; TOID(struct foo) foo; }; ``` The name of layout and the number of declared types can be retrieved using the following code: ```c const char *layout_name = POBJ_LAYOUT_NAME(mylayout); int num_of_types = POBJ_LAYOUT_TYPES_NUM(mylayout); ``` # SEE ALSO # **TOID_DECLARE**(3), **libpmemobj**(7) and **** pmdk-1.8/doc/libpmemobj/pmemobj_tx_stage.30000664000000000000000000000002713615011243017310 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/tx_wcsdup.30000664000000000000000000000002713615011243016001 0ustar rootroot.so pmemobj_tx_alloc.3 pmdk-1.8/doc/libpmemobj/pobj_list_remove_free.30000664000000000000000000000002513615011243020322 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/libpmemobj/pmemobj_set_value.30000664000000000000000000000002513615011243017457 0ustar rootroot.so pmemobj_action.3 pmdk-1.8/doc/libpmemobj/tx_begin_param.30000664000000000000000000000002713615011243016740 0ustar rootroot.so pmemobj_tx_begin.3 pmdk-1.8/doc/libpmemobj/pobj_list_insert_before.30000664000000000000000000000002513615011243020652 0ustar rootroot.so pobj_list_head.3 pmdk-1.8/doc/pmreorder/0000775000000000000000000000000013615011420013552 5ustar rootrootpmdk-1.8/doc/pmreorder/.gitignore0000664000000000000000000000001413615011243015540 0ustar rootrootpmreorder.1 pmdk-1.8/doc/pmreorder/pmreorder.10000644000000000000000000002700213615011420015632 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "PMREORDER" "1" "2020-01-31" "PMDK - pmreorder version 1.5" "PMDK Programmer's Manual" .hy .\" Copyright 2018-2019, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP \f[B]pmreorder\f[] \- performs a persistent consistency check using a store reordering mechanism .SH SYNOPSIS .IP .nf \f[C] $\ python\ pmreorder\ \f[] .fi .SH DESCRIPTION .PP The pmreorder tool is a collection of python scripts designed to parse and replay operations logged by pmemcheck \- a persistent memory checking tool. .PP Pmreorder performs the store reordering between persistent memory barriers \- a sequence of flush\-fence operations. It uses a consistency checking routine provided in the command line options to check whether files are in a consistent state. .PP Considering that logging, replaying and reordering of operations are very time consuming, it is recommended to use as few stores as possible in test workloads. .SH OPTIONS .PP \f[C]\-h,\ \-\-help\f[] .PP Prints synopsis and list of options. .PP \f[C]\-l\ ,\ \-\-logfile\ \f[] .PP The pmemcheck log file to process. .PP \f[C]\-c\ ,\ \-\-checker\ \f[] .PP Consistency checker type. .PP \f[C]\-p\ ,\ \-\-path\ \f[] .PP Path to the consistency checker. Checker function has to return 0 for consistent cases and 1 otherwise. .PP \f[C]\-n\ ,\ \-\-name\ \f[] .PP The symbol name of the consistency checking function in the library. Valid only if the checker type is \f[C]lib\f[]. .PP \f[C]\-o\ ,\ \-\-output\ \f[] .PP Set the logger output file. .PP \f[C]\-e\ ,\f[] .PP \f[C]\-\-output\-level\ \f[] .PP Set the output log level. .PP \f[C]\-r\ \ ,\f[] .PP \f[C]\-\-default\-engine\ \ \f[] .PP Set the initial reorder engine. Default value is \f[C]NoReorderNoCheck\f[]. .PP \f[C]\-x\ ,\ \-\-extended\-macros\ \f[] .PP Assign an engine types to the defined marker. .PP \f[C]\-v,\ \-\-version\f[] .PP Prints current version of pmreorder. .SH ENGINES .PP By default, the \f[B]NoReorderNoCheck\f[] engine is used, which means that for each set of stores, the tool will pass\-through all sequences of stores not reordered and will not run consistency checker on them. .PP To enable different types of the reorder engine and begin proper reordering tests, a number of other engines exist: .IP \[bu] 2 \f[B]NoReorderDoCheck\f[] \- pass\-through of unchanged operations. Checks correctness of the stores as they were logged. Useful for operations that do not require fail safety. .IP .nf \f[C] Example: \ \ \ \ \ \ \ \ input:\ (a,\ b,\ c) \ \ \ \ \ \ \ \ output:\ (a,\ b,\ c) \f[] .fi .IP \[bu] 2 \f[B]ReorderAccumulative\f[] \- checks correctness on a growing subset of the original sequence. .IP .nf \f[C] Example: \ \ \ \ \ \ \ \ input:\ (a,\ b,\ c) \ \ \ \ \ \ \ \ output: \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ () \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b,\ c) \f[] .fi .IP \[bu] 2 \f[B]ReorderReverseAccumulative\f[] \- checks correctness on a reverted growing subset of the original sequence. .IP .nf \f[C] Example: \ \ \ \ \ \ \ \ input:\ (a,\ b,\ c) \ \ \ \ \ \ \ \ output: \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ () \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ b) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ b,\ a) \f[] .fi .IP \[bu] 2 \f[B]ReorderPartial\f[] \- checks consistency on 3 randomly selected sequences from a set of 1000 combinations of the original log, without repetitions. .IP .nf \f[C] \ Example: \ \ \ \ \ \ \ \ \ input:\ (a,\ b,\ c) \ \ \ \ \ \ \ \ \ output: \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ c) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b,\ c) \f[] .fi .IP \[bu] 2 \f[B]ReorderFull\f[] \- for each set of stores generates and checks consistency of all possible store permutations. This might prove to be very computationally expensive for most workloads. It can be useful for critical sections of code with limited number of stores. .IP .nf \f[C] \ Example: \ \ \ \ \ \ \ \ input:\ (a,\ b,\ c) \ \ \ \ \ \ \ \ output: \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ () \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ c) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ a) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ c) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ a) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ b) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ b,\ c) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (a,\ c,\ b) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ a,\ c) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (b,\ c,\ a) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ a,\ b) \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ (c,\ b,\ a) \f[] .fi .PP When the engine is passed with an \f[C]\-r\f[] option, it will be used for each logged set of stores. Additionally, the \f[C]\-x\f[] parameter can be used to switch engines separately for any marked code sections. For more details about \f[C]\-x\f[] extended macros functionality see section INSTRUMENTATION below. .SH INSTRUMENTATION .PP The core of \f[B]pmreorder\f[] is based on user\-provided named markers. Sections of code can be `marked' depending on their importance, and the degree of reordering can be customized by the use of various provided engines. .PP For this purpose, Valgrind's pmemcheck tool exposes a generic marker macro: .IP \[bu] 2 \f[B]VALGRIND_PMC_EMIT_LOG(value)\f[] .PP It emits log to \f[I]store_log\f[] during pmemcheck processing. \f[I]value\f[] is a user\-defined marker name. For more details about pmemcheck execution see PMEMCHECK STORE LOG section below. .PP Example: .IP .nf \f[C] main.c \&. \&. \&. VALGRIND_PMC_EMIT_LOG("PMREORDER_MEMSET_PERSIST.BEGIN"); pmem_memset_persist(...); VALGRIND_PMC_EMIT_LOG("PMREORDER_MEMSET_PERSIST.END"); \&. \&. \&. \f[] .fi .PP There are a few rules for macros creation: .IP \[bu] 2 Valid macro can have any name, but begin and end section have to match \- they are case sensitive. .IP \[bu] 2 Macro must have \f[C]\&.BEGIN\f[] or \f[C]\&.END\f[] suffix. .IP \[bu] 2 Macros can't be crossed. .PP Defined markers can be assigned engines types and configured through the \f[B]pmreorder\f[] tool using the \f[C]\-x\f[] parameter. .PP There are two ways to set macro options: .IP \[bu] 2 Using command line interface in format: .IP .nf \f[C] PMREORDER_MARKER_NAME1=ReorderName1,PMREORDER_MARKER_NAME2=ReorderName2 \f[] .fi .IP \[bu] 2 Using configuration file in .json format: .IP .nf \f[C] { \ \ \ \ "PMREORDER_MARKER_NAME1":"ReorderName1", \ \ \ \ "PMREORDER_MARKER_NAME2":"ReorderName2" } \f[] .fi .PP For more details about available engines types, see ENGINES section above. .PP \f[B]libpmemobj\f[](7) and \f[B]libpmem\f[](7) also provide set of macros that allow to change reordering engine on library or function level: .PP \f[C]\f[] .PP Example of configuration on function level: .IP .nf \f[C] { \ \ \ \ "pmemobj_open":"NoReorderNoCheck", \ \ \ \ "pmemobj_memcpy_persist":"ReorderPartial" } \f[] .fi .PP Example of configuration on library level (affecting all library functions): .IP .nf \f[C] { \ \ \ \ "libpmemobj":"NoReorderNoCheck" } \f[] .fi .PP List of marked \f[B]libpmemobj\f[](7) API functions: .IP .nf \f[C] pmemobj_alloc pmemobj_cancel pmemobj_check pmemobj_close pmemobj_create pmemobj_ctl_exec pmemobj_ctl_set pmemobj_free pmemobj_list_insert pmemobj_list_insert_new pmemobj_list_move pmemobj_list_remove pmemobj_memcpy pmemobj_memmove pmemobj_memset pmemobj_memcpy_persist pmemobj_memset_persist pmemobj_open pmemobj_publish pmemobj_realloc pmemobj_reserve pmemobj_root pmemobj_root_construct pmemobj_strdup pmemobj_tx_abort pmemobj_tx_add_range pmemobj_tx_add_range_direct pmemobj_tx_alloc pmemobj_tx_commit pmemobj_tx_free pmemobj_tx_publish pmemobj_tx_realloc pmemobj_tx_strdup pmemobj_tx_wcsdup pmemobj_tx_xadd_range pmemobj_tx_xadd_range_direct pmemobj_tx_xalloc pmemobj_tx_zalloc pmemobj_tx_zrealloc pmemobj_wcsdup pmemobj_xalloc pmemobj_xreserve pmemobj_zalloc pmemobj_zrealloc \f[] .fi .PP List of marked \f[B]libpmem\f[](7) API functions: .IP .nf \f[C] pmem_memmove pmem_memcpy pmem_memset pmem_memmove_nodrain pmem_memcpy_nodrain pmem_memset_nodrain pmem_memmove_persist pmem_memcpy_persist pmem_memset_persist \f[] .fi .SH PMEMCHECK STORE LOG .PP To generate \f[I]store_log\f[] for \f[B]pmreorder\f[] run pmemcheck with additional parameters: .IP .nf \f[C] valgrind\ \\ \ \ \ \ \-\-tool=pmemcheck\ \\ \ \ \ \ \-q\ \\ \ \ \ \ \-\-log\-stores=yes\ \\ \ \ \ \ \-\-print\-summary=no\ \\ \ \ \ \ \-\-log\-file=store_log.log\ \\ \ \ \ \ \-\-log\-stores\-stacktraces=yes\ \\ \ \ \ \ \-\-log\-stores\-stacktraces\-depth=2\ \\ \ \ \ \ \-\-expect\-fence\-after\-clflush=yes\ \\ \ \ \ \ test_binary\ writer_parameter \f[] .fi .PP For further details of pmemcheck parameters see pmemcheck documentation (https://github.com/pmem/valgrind/blob/pmem-3.13/pmemcheck/docs/pmc-manual.xml) .SH ENVIRONMENT .PP By default all logging from PMDK libraries is disabled. To enable API macros logging set environment variable: .IP \[bu] 2 \f[B]PMREORDER_EMIT_LOG\f[]=1 .SH EXAMPLE .IP .nf \f[C] python\ pmreorder.py\ \\ \ \ \ \ \ \ \ \ \-l\ store_log.log\ \\ \ \ \ \ \ \ \ \ \-r\ NoReorderDoCheck\ \\ \ \ \ \ \ \ \ \ \-o\ pmreorder_out.log\ \\ \ \ \ \ \ \ \ \ \-c\ prog\ \\ \ \ \ \ \ \ \ \ \-x\ PMREORDER_MARKER_NAME=ReorderPartial\ \\ \ \ \ \ \ \ \ \ \-p\ checker_binary\ checker_parameter \f[] .fi .PP Checker binary will be used to run consistency checks on \[lq]store_log.log\[rq], output of pmemcheck tool. Any inconsistent stores found during \f[B]pmreorder\f[] analysis will be logged to \f[C]pmreorder_out.log\f[]. .SH SEE ALSO .PP \f[B]\f[] pmdk-1.8/doc/pmreorder/pmreorder.1.md0000664000000000000000000002502713615011243016243 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(PMREORDER, 1) collection: pmreorder header: PMDK date: pmreorder version 1.5 ... [comment]: <> (Copyright 2018-2019, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (pmreorder.1 -- man page for pmreorder) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[OPTIONS](#options)
[ENGINES](#engines)
[INSTRUMENTATION](#instrumentation)
[PMEMCHECK STORE LOG](#pmemcheck-store-log)
[ENVIRONMENT](#environment)
[EXAMPLE](#example)
[SEE ALSO](#see-also)
# NAME # **pmreorder** - performs a persistent consistency check using a store reordering mechanism # SYNOPSIS # ``` $ python pmreorder ``` # DESCRIPTION # The pmreorder tool is a collection of python scripts designed to parse and replay operations logged by pmemcheck - a persistent memory checking tool. Pmreorder performs the store reordering between persistent memory barriers - a sequence of flush-fence operations. It uses a consistency checking routine provided in the command line options to check whether files are in a consistent state. Considering that logging, replaying and reordering of operations are very time consuming, it is recommended to use as few stores as possible in test workloads. # OPTIONS # `-h, --help` Prints synopsis and list of options. `-l , --logfile ` The pmemcheck log file to process. `-c , --checker ` Consistency checker type. `-p , --path ` Path to the consistency checker. Checker function has to return 0 for consistent cases and 1 otherwise. `-n , --name ` The symbol name of the consistency checking function in the library. Valid only if the checker type is `lib`. `-o , --output ` Set the logger output file. `-e ,` ` --output-level ` Set the output log level. `-r ,` `--default-engine ` Set the initial reorder engine. Default value is `NoReorderNoCheck`. `-x , --extended-macros ` Assign an engine types to the defined marker. `-v, --version` Prints current version of pmreorder. # ENGINES # By default, the **NoReorderNoCheck** engine is used, which means that for each set of stores, the tool will pass-through all sequences of stores not reordered and will not run consistency checker on them. To enable different types of the reorder engine and begin proper reordering tests, a number of other engines exist: + **NoReorderDoCheck** - pass-through of unchanged operations. Checks correctness of the stores as they were logged. Useful for operations that do not require fail safety. ``` Example: input: (a, b, c) output: (a, b, c) ``` + **ReorderAccumulative** - checks correctness on a growing subset of the original sequence. ``` Example: input: (a, b, c) output: () (a) (a, b) (a, b, c) ``` + **ReorderReverseAccumulative** - checks correctness on a reverted growing subset of the original sequence. ``` Example: input: (a, b, c) output: () (c) (c, b) (c, b, a) ``` + **ReorderPartial** - checks consistency on 3 randomly selected sequences from a set of 1000 combinations of the original log, without repetitions. ``` Example: input: (a, b, c) output: (b, c) (b) (a, b, c) ``` + **ReorderFull** - for each set of stores generates and checks consistency of all possible store permutations. This might prove to be very computationally expensive for most workloads. It can be useful for critical sections of code with limited number of stores. ``` Example: input: (a, b, c) output: () (a) (b) (c) (a, b) (a, c) (b, a) (b, c) (c, a) (c, b) (a, b, c) (a, c, b) (b, a, c) (b, c, a) (c, a, b) (c, b, a) ``` When the engine is passed with an `-r` option, it will be used for each logged set of stores. Additionally, the `-x` parameter can be used to switch engines separately for any marked code sections. For more details about `-x` extended macros functionality see section INSTRUMENTATION below. # INSTRUMENTATION # The core of **pmreorder** is based on user-provided named markers. Sections of code can be 'marked' depending on their importance, and the degree of reordering can be customized by the use of various provided engines. For this purpose, Valgrind's pmemcheck tool exposes a generic marker macro: + **VALGRIND_PMC_EMIT_LOG(value)** It emits log to *store_log* during pmemcheck processing. *value* is a user-defined marker name. For more details about pmemcheck execution see PMEMCHECK STORE LOG section below. Example: ``` main.c . . . VALGRIND_PMC_EMIT_LOG("PMREORDER_MEMSET_PERSIST.BEGIN"); pmem_memset_persist(...); VALGRIND_PMC_EMIT_LOG("PMREORDER_MEMSET_PERSIST.END"); . . . ``` There are a few rules for macros creation: + Valid macro can have any name, but begin and end section have to match - they are case sensitive. + Macro must have `.BEGIN` or `.END` suffix. + Macros can't be crossed. Defined markers can be assigned engines types and configured through the **pmreorder** tool using the `-x` parameter. There are two ways to set macro options: + Using command line interface in format: ``` PMREORDER_MARKER_NAME1=ReorderName1,PMREORDER_MARKER_NAME2=ReorderName2 ``` + Using configuration file in .json format: ``` { "PMREORDER_MARKER_NAME1":"ReorderName1", "PMREORDER_MARKER_NAME2":"ReorderName2" } ``` For more details about available engines types, see ENGINES section above. **libpmemobj**(7) and **libpmem**(7) also provide set of macros that allow to change reordering engine on library or function level: `` Example of configuration on function level: ``` { "pmemobj_open":"NoReorderNoCheck", "pmemobj_memcpy_persist":"ReorderPartial" } ``` Example of configuration on library level (affecting all library functions): ``` { "libpmemobj":"NoReorderNoCheck" } ``` List of marked **libpmemobj**(7) API functions: ``` pmemobj_alloc pmemobj_cancel pmemobj_check pmemobj_close pmemobj_create pmemobj_ctl_exec pmemobj_ctl_set pmemobj_free pmemobj_list_insert pmemobj_list_insert_new pmemobj_list_move pmemobj_list_remove pmemobj_memcpy pmemobj_memmove pmemobj_memset pmemobj_memcpy_persist pmemobj_memset_persist pmemobj_open pmemobj_publish pmemobj_realloc pmemobj_reserve pmemobj_root pmemobj_root_construct pmemobj_strdup pmemobj_tx_abort pmemobj_tx_add_range pmemobj_tx_add_range_direct pmemobj_tx_alloc pmemobj_tx_commit pmemobj_tx_free pmemobj_tx_publish pmemobj_tx_realloc pmemobj_tx_strdup pmemobj_tx_wcsdup pmemobj_tx_xadd_range pmemobj_tx_xadd_range_direct pmemobj_tx_xalloc pmemobj_tx_zalloc pmemobj_tx_zrealloc pmemobj_wcsdup pmemobj_xalloc pmemobj_xreserve pmemobj_zalloc pmemobj_zrealloc ``` List of marked **libpmem**(7) API functions: ``` pmem_memmove pmem_memcpy pmem_memset pmem_memmove_nodrain pmem_memcpy_nodrain pmem_memset_nodrain pmem_memmove_persist pmem_memcpy_persist pmem_memset_persist ``` # PMEMCHECK STORE LOG # To generate *store_log* for **pmreorder** run pmemcheck with additional parameters: ``` valgrind \ --tool=pmemcheck \ -q \ --log-stores=yes \ --print-summary=no \ --log-file=store_log.log \ --log-stores-stacktraces=yes \ --log-stores-stacktraces-depth=2 \ --expect-fence-after-clflush=yes \ test_binary writer_parameter ``` For further details of pmemcheck parameters see [pmemcheck documentation](https://github.com/pmem/valgrind/blob/pmem-3.13/pmemcheck/docs/pmc-manual.xml) # ENVIRONMENT # By default all logging from PMDK libraries is disabled. To enable API macros logging set environment variable: + **PMREORDER_EMIT_LOG**=1 # EXAMPLE # ``` python pmreorder.py \ -l store_log.log \ -r NoReorderDoCheck \ -o pmreorder_out.log \ -c prog \ -x PMREORDER_MARKER_NAME=ReorderPartial \ -p checker_binary checker_parameter ``` Checker binary will be used to run consistency checks on "store_log.log", output of pmemcheck tool. Any inconsistent stores found during **pmreorder** analysis will be logged to `pmreorder_out.log`. # SEE ALSO # **** pmdk-1.8/doc/Makefile0000664000000000000000000004122113615011243013216 0ustar rootroot# # Copyright 2014-2020, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # doc/Makefile -- Makefile for PMDK man pages # include ../src/common.inc MANPAGES_7_MD = libpmem/libpmem.7.md libpmemblk/libpmemblk.7.md libpmemlog/libpmemlog.7.md \ libpmemobj/libpmemobj.7.md libpmempool/libpmempool.7.md MANPAGES_5_MD = poolset/poolset.5.md pmem_ctl/pmem_ctl.5.md MANPAGES_3_MD = libpmem/pmem_flush.3.md libpmem/pmem_is_pmem.3.md libpmem/pmem_memmove_persist.3.md \ libpmemblk/pmemblk_bsize.3.md libpmemblk/pmemblk_create.3.md libpmemblk/pmemblk_ctl_get.3.md libpmemblk/pmemblk_read.3.md libpmemblk/pmemblk_set_zero.3.md \ libpmemlog/pmemlog_append.3.md libpmemlog/pmemlog_create.3.md libpmemlog/pmemlog_ctl_get.3.md libpmemlog/pmemlog_nbyte.3.md libpmemlog/pmemlog_tell.3.md \ libpmemobj/oid_is_null.3.md libpmemobj/pmemobj_action.3.md libpmemobj/pmemobj_alloc.3.md libpmemobj/pmemobj_ctl_get.3.md libpmemobj/pmemobj_first.3.md \ libpmemobj/pmemobj_list_insert.3.md libpmemobj/pmemobj_memcpy_persist.3.md libpmemobj/pmemobj_mutex_zero.3.md \ libpmemobj/pmemobj_open.3.md libpmemobj/pmemobj_root.3.md libpmemobj/pmemobj_tx_begin.3.md libpmemobj/pmemobj_tx_add_range.3.md \ libpmemobj/pmemobj_tx_alloc.3.md libpmemobj/pobj_layout_begin.3.md libpmemobj/pobj_list_head.3.md libpmemobj/toid_declare.3.md \ libpmempool/pmempool_check_init.3.md libpmempool/pmempool_feature_query.3.md libpmempool/pmempool_rm.3.md libpmempool/pmempool_sync.3.md MANPAGES_1_MD = pmempool/pmempool.1.md pmempool/pmempool-info.1.md pmempool/pmempool-create.1.md \ pmempool/pmempool-check.1.md pmempool/pmempool-dump.1.md pmempool/pmempool-rm.1.md \ pmempool/pmempool-convert.1.md pmempool/pmempool-sync.1.md pmempool/pmempool-transform.1.md \ pmempool/pmempool-feature.1.md pmreorder/pmreorder.1.md MANPAGES_3_DUMMY = libpmem/pmem_drain.3 libpmem/pmem_has_hw_drain.3 libpmem/pmem_has_auto_flush.3 \ libpmem/pmem_persist.3 libpmem/pmem_msync.3 libpmem/pmem_map_file.3 libpmem/pmem_deep_persist.3 libpmem/pmem_deep_flush.3 libpmem/pmem_deep_drain.3 libpmem/pmem_unmap.3 \ libpmem/pmem_memcpy_persist.3 libpmem/pmem_memset_persist.3 libpmem/pmem_memmove_nodrain.3 libpmem/pmem_memcpy_nodrain.3 libpmem/pmem_memset_nodrain.3 \ libpmem/pmem_memcpy.3 libpmem/pmem_memset.3 libpmem/pmem_memmove.3 \ libpmem/pmem_check_version.3 libpmem/pmem_errormsg.3 \ libpmemblk/pmemblk_nblock.3 \ libpmemblk/pmemblk_open.3 libpmemblk/pmemblk_close.3 \ libpmemblk/pmemblk_write.3 \ libpmemblk/pmemblk_set_error.3 \ libpmemblk/pmemblk_check_version.3 libpmemblk/pmemblk_check.3 libpmemblk/pmemblk_errormsg.3 libpmemblk/pmemblk_set_funcs.3 \ libpmemblk/pmemblk_ctl_set.3 libpmemblk/pmemblk_ctl_exec.3\ libpmemlog/pmemlog_rewind.3 libpmemlog/pmemlog_walk.3 \ libpmemlog/pmemlog_open.3 libpmemlog/pmemlog_close.3 \ libpmemlog/pmemlog_appendv.3 \ libpmemlog/pmemlog_check_version.3 libpmemlog/pmemlog_check.3 libpmemlog/pmemlog_errormsg.3 libpmemlog/pmemlog_set_funcs.3 \ libpmemlog/pmemlog_ctl_set.3 libpmemlog/pmemlog_ctl_exec.3\ libpmempool/pmempool_check.3 libpmempool/pmempool_check_end.3 \ libpmempool/pmempool_feature_enable.3 libpmempool/pmempool_feature_disable.3 \ libpmempool/pmempool_transform.3 \ libpmempool/pmempool_check_version.3 libpmempool/pmempool_errormsg.3 \ libpmemobj/oid_equals.3 libpmemobj/pmemobj_direct.3 libpmemobj/pmemobj_oid.3 libpmemobj/pmemobj_type_num.3 libpmemobj/pmemobj_pool_by_oid.3 libpmemobj/pmemobj_pool_by_ptr.3 libpmemobj/pmemobj_volatile.3\ libpmemobj/pmemobj_zalloc.3 libpmemobj/pmemobj_xalloc.3 libpmemobj/pmemobj_free.3 libpmemobj/pmemobj_realloc.3 libpmemobj/pmemobj_zrealloc.3 libpmemobj/pmemobj_strdup.3 libpmemobj/pmemobj_wcsdup.3 libpmemobj/pmemobj_alloc_usable_size.3 \ libpmemobj/pobj_new.3 libpmemobj/pobj_alloc.3 libpmemobj/pobj_znew.3 libpmemobj/pobj_zalloc.3 libpmemobj/pobj_realloc.3 libpmemobj/pobj_zrealloc.3 libpmemobj/pobj_free.3 \ libpmemobj/pobj_layout_toid.3 libpmemobj/pobj_layout_root.3 libpmemobj/pobj_layout_name.3 libpmemobj/pobj_layout_end.3 libpmemobj/pobj_layout_types_num.3 \ libpmemobj/pmemobj_ctl_set.3 libpmemobj/pmemobj_ctl_exec.3\ libpmemobj/pmemobj_create.3 libpmemobj/pmemobj_close.3 \ libpmemobj/pmemobj_list_insert_new.3 libpmemobj/pmemobj_list_remove.3 libpmemobj/pmemobj_list_move.3 \ libpmemobj/toid_declare_root.3 libpmemobj/toid.3 libpmemobj/toid_type_num.3 libpmemobj/toid_type_num_of.3 libpmemobj/toid_valid.3 libpmemobj/oid_instanceof.3 libpmemobj/toid_assign.3 libpmemobj/toid_is_null.3 libpmemobj/toid_equals.3 libpmemobj/toid_typeof.3 libpmemobj/toid_offsetof.3 libpmemobj/direct_rw.3 libpmemobj/d_rw.3 libpmemobj/direct_ro.3 libpmemobj/d_ro.3 \ libpmemobj/pmemobj_memcpy.3 libpmemobj/pmemobj_memmove.3 libpmemobj/pmemobj_memset.3 \ libpmemobj/pmemobj_memset_persist.3 libpmemobj/pmemobj_persist.3 libpmemobj/pmemobj_xpersist.3 libpmemobj/pmemobj_flush.3 libpmemobj/pmemobj_xflush.3 libpmemobj/pmemobj_drain.3 \ libpmemobj/pmemobj_tx_stage.3 libpmemobj/pmemobj_tx_lock.3 libpmemobj/pmemobj_tx_xlock.3 libpmemobj/pmemobj_tx_abort.3 libpmemobj/pmemobj_tx_commit.3 libpmemobj/pmemobj_tx_end.3 libpmemobj/pmemobj_tx_errno.3 \ libpmemobj/pmemobj_tx_process.3 libpmemobj/pmemobj_tx_add_range_direct.3 libpmemobj/pmemobj_tx_xadd_range.3 libpmemobj/pmemobj_tx_xadd_range_direct.3 \ libpmemobj/pmemobj_tx_zalloc.3 libpmemobj/pmemobj_tx_xalloc.3 libpmemobj/pmemobj_tx_realloc.3 libpmemobj/pmemobj_tx_zrealloc.3 libpmemobj/pmemobj_tx_strdup.3 libpmemobj/pmemobj_tx_xstrdup.3 libpmemobj/pmemobj_tx_wcsdup.3 libpmemobj/pmemobj_tx_xwcsdup.3 libpmemobj/pmemobj_tx_free.3 libpmemobj/pmemobj_tx_xfree.3\ libpmemobj/pmemobj_tx_log_append_buffer.3 libpmemobj/pmemobj_tx_xlog_append_buffer.3 libpmemobj/pmemobj_tx_log_auto_alloc.3 libpmemobj/pmemobj_tx_log_snapshots_max_size.3 libpmemobj/pmemobj_tx_log_intents_max_size.3 \ libpmemobj/tx_begin_param.3 libpmemobj/tx_begin_cb.3 libpmemobj/tx_begin.3 libpmemobj/tx_onabort.3 libpmemobj/tx_oncommit.3 libpmemobj/tx_finally.3 libpmemobj/tx_end.3 \ libpmemobj/tx_add.3 libpmemobj/tx_add_field.3 libpmemobj/tx_add_direct.3 libpmemobj/tx_add_field_direct.3 libpmemobj/tx_xadd.3 libpmemobj/tx_xadd_field.3 libpmemobj/tx_xadd_direct.3 libpmemobj/tx_xadd_field_direct.3 \ libpmemobj/tx_new.3 libpmemobj/tx_alloc.3 libpmemobj/tx_znew.3 libpmemobj/tx_zalloc.3 libpmemobj/tx_xalloc.3 libpmemobj/tx_realloc.3 libpmemobj/tx_zrealloc.3 libpmemobj/tx_strdup.3 libpmemobj/tx_wcsdup.3 libpmemobj/tx_free.3 libpmemobj/tx_set.3 libpmemobj/tx_set_direct.3 libpmemobj/tx_memcpy.3 libpmemobj/tx_memset.3 \ libpmemobj/pmemobj_f_mem_nodrain.3 libpmemobj/pmemobj_f_mem_nontemporal.3 libpmemobj/pmemobj_f_mem_temporal.3 libpmemobj/pmemobj_f_mem_wc.3 libpmemobj/pmemobj_f_mem_wb.3 libpmemobj/pmemobj_f_mem_noflush.3 libpmemobj/pmemobj_f_relaxed.3 \ libpmemobj/pmemobj_mutex_lock.3 libpmemobj/pmemobj_mutex_timedlock.3 libpmemobj/pmemobj_mutex_trylock.3 libpmemobj/pmemobj_mutex_unlock.3 \ libpmemobj/pmemobj_rwlock_zero.3 libpmemobj/pmemobj_rwlock_rdlock.3 libpmemobj/pmemobj_rwlock_wrlock.3 libpmemobj/pmemobj_rwlock_timedrdlock.3 libpmemobj/pmemobj_rwlock_timedwrlock.3 libpmemobj/pmemobj_rwlock_tryrdlock.3 libpmemobj/pmemobj_rwlock_trywrlock.3 libpmemobj/pmemobj_rwlock_unlock.3 \ libpmemobj/pmemobj_cond_zero.3 libpmemobj/pmemobj_cond_broadcast.3 libpmemobj/pmemobj_cond_signal.3 libpmemobj/pmemobj_cond_timedwait.3 libpmemobj/pmemobj_cond_wait.3 \ libpmemobj/pobj_list_entry.3 libpmemobj/pobj_list_first.3 libpmemobj/pobj_list_last.3 libpmemobj/pobj_list_empty.3 libpmemobj/pobj_list_next.3 libpmemobj/pobj_list_prev.3 libpmemobj/pobj_list_foreach.3 libpmemobj/pobj_list_foreach_reverse.3 \ libpmemobj/pobj_list_insert_head.3 libpmemobj/pobj_list_insert_tail.3 libpmemobj/pobj_list_insert_after.3 libpmemobj/pobj_list_insert_before.3 libpmemobj/pobj_list_insert_new_head.3 libpmemobj/pobj_list_insert_new_tail.3 \ libpmemobj/pobj_list_insert_new_after.3 libpmemobj/pobj_list_insert_new_before.3 libpmemobj/pobj_list_remove.3 libpmemobj/pobj_list_remove_free.3 \ libpmemobj/pobj_list_move_element_head.3 libpmemobj/pobj_list_move_element_tail.3 libpmemobj/pobj_list_move_element_after.3 libpmemobj/pobj_list_move_element_before.3 \ libpmemobj/pmemobj_next.3 libpmemobj/pobj_first_type_num.3 libpmemobj/pobj_first.3 libpmemobj/pobj_next_type_num.3 libpmemobj/pobj_next.3 libpmemobj/pobj_foreach.3 libpmemobj/pobj_foreach_safe.3 libpmemobj/pobj_foreach_type.3 libpmemobj/pobj_foreach_safe_type.3 \ libpmemobj/pmemobj_root_construct.3 libpmemobj/pobj_root.3 libpmemobj/pmemobj_root_size.3 \ libpmemobj/pmemobj_check_version.3 libpmemobj/pmemobj_check.3 libpmemobj/pmemobj_errormsg.3 libpmemobj/pmemobj_set_funcs.3 \ libpmemobj/pmemobj_reserve.3 libpmemobj/pmemobj_xreserve.3 libpmemobj/pmemobj_defer_free.3 libpmemobj/pmemobj_set_value.3 libpmemobj/pmemobj_publish.3 libpmemobj/pmemobj_tx_publish.3 libpmemobj/pmemobj_tx_xpublish.3 libpmemobj/pmemobj_cancel.3 libpmemobj/pobj_reserve_new.3 libpmemobj/pobj_reserve_alloc.3 libpmemobj/pobj_xreserve_new.3 libpmemobj/pobj_xreserve_alloc.3 \ libpmemobj/tx_xstrdup.3 libpmemobj/tx_xwcsdup.3 libpmemobj/tx_xfree.3 \ libpmemobj/pmemobj_defrag.3 libpmemobj/pmemobj_get_user_data.3 libpmemobj/pmemobj_set_user_data.3 libpmemobj/pmemobj_tx_get_user_data.3 libpmemobj/pmemobj_tx_set_user_data.3 MANPAGES_WEBDIR_LINUX = web_linux MANPAGES_WEBDIR_WINDOWS = web_windows # experimental MANPAGES_7_MD_EXP = MANPAGES_5_MD_EXP = MANPAGES_3_MD_EXP = MANPAGES_1_MD_EXP = MANPAGES_3_DUMMY_EXP = MANPAGES_1_DUMMY_EXP = # libpmem2 MANPAGES_7_MD_PMEM2 = libpmem2/libpmem2.7.md MANPAGES_5_MD_PMEM2 = MANPAGES_3_MD_PMEM2 = libpmem2/pmem2_errormsg.3.md libpmem2/pmem2_config.3.md libpmem2/pmem2_mapping.3.md MANPAGES_1_MD_PMEM2 = ifeq ($(PMEM2_INSTALL),y) MANPAGES_3_DUMMY += libpmem2/pmem2_config_new.3 libpmem2/pmem2_config_delete.3 libpmem2/pmem2_config_set_fd.3 libpmem2/pmem2_config_set_handle.3 \ libpmem2/pmem2_map.3 endif ifeq ($(BUILD_RPMEM),y) MANPAGES_7_MD += librpmem/librpmem.7.md MANPAGES_3_MD += librpmem/rpmem_create.3.md MANPAGES_3_MD += librpmem/rpmem_persist.3.md MANPAGES_1_MD += rpmemd/rpmemd.1.md MANPAGES_3_DUMMY += librpmem/rpmem_open.3 librpmem/rpmem_set_attr.3 librpmem/rpmem_close.3 \ librpmem/rpmem_read.3 librpmem/rpmem_remove.3 librpmem/rpmem_check_version.3 \ librpmem/rpmem_errormsg.3 librpmem/rpmem_deep_persist.3 librpmem/rpmem_flush.3 \ librpmem/rpmem_drain.3 endif ifeq ($(NDCTL_ENABLE),y) MANPAGES_1_MD += daxio/daxio.1.md endif MANPAGES_7_GROFF = $(MANPAGES_7_MD:.7.md=.7) MANPAGES_5_GROFF = $(MANPAGES_5_MD:.5.md=.5) MANPAGES_3_GROFF = $(MANPAGES_3_MD:.3.md=.3) MANPAGES_1_GROFF = $(MANPAGES_1_MD:.1.md=.1) MANPAGES_7_GROFF_EXP = $(MANPAGES_7_MD_EXP:.7.md=.7) MANPAGES_5_GROFF_EXP = $(MANPAGES_5_MD_EXP:.5.md=.5) MANPAGES_3_GROFF_EXP = $(MANPAGES_3_MD_EXP:.3.md=.3) MANPAGES_1_GROFF_EXP = $(MANPAGES_1_MD_EXP:.1.md=.1) MANPAGES_7_GROFF_PMEM2 = $(MANPAGES_7_MD_PMEM2:.7.md=.7) MANPAGES_5_GROFF_PMEM2 = $(MANPAGES_5_MD_PMEM2:.5.md=.5) MANPAGES_3_GROFF_PMEM2 = $(MANPAGES_3_MD_PMEM2:.3.md=.3) MANPAGES_1_GROFF_PMEM2 = $(MANPAGES_1_MD_PMEM2:.1.md=.1) ifeq ($(EXPERIMENTAL),y) MANPAGES_7_GROFF += $(MANPAGES_7_GROFF_EXP) MANPAGES_5_GROFF += $(MANPAGES_5_GROFF_EXP) MANPAGES_3_GROFF += $(MANPAGES_3_GROFF_EXP) MANPAGES_1_GROFF += $(MANPAGES_1_GROFF_EXP) else MANPAGES_7_NOINSTALL += $(MANPAGES_7_GROFF_EXP) MANPAGES_5_NOINSTALL += $(MANPAGES_5_GROFF_EXP) MANPAGES_3_NOINSTALL += $(MANPAGES_3_GROFF_EXP) MANPAGES_1_NOINSTALL += $(MANPAGES_1_GROFF_EXP) endif ifeq ($(PMEM2_INSTALL),y) MANPAGES_7_GROFF += $(MANPAGES_7_GROFF_PMEM2) MANPAGES_5_GROFF += $(MANPAGES_5_GROFF_PMEM2) MANPAGES_3_GROFF += $(MANPAGES_3_GROFF_PMEM2) MANPAGES_1_GROFF += $(MANPAGES_1_GROFF_PMEM2) else MANPAGES_7_NOINSTALL += $(MANPAGES_7_GROFF_PMEM2) MANPAGES_5_NOINSTALL += $(MANPAGES_5_GROFF_PMEM2) MANPAGES_3_NOINSTALL += $(MANPAGES_3_GROFF_PMEM2) MANPAGES_1_NOINSTALL += $(MANPAGES_1_GROFF_PMEM2) endif MANPAGES = $(MANPAGES_7_GROFF) $(MANPAGES_5_GROFF) $(MANPAGES_3_GROFF) $(MANPAGES_1_GROFF) \ $(MANPAGES_7_NOINSTALL) $(MANPAGES_5_NOINSTALL) $(MANPAGES_3_NOINSTALL) $(MANPAGES_1_NOINSTALL) HTMLFILES = $(MANPAGES:=.html) TXTFILES = $(MANPAGES:=.txt) GZFILES_7 = $(MANPAGES_7_GROFF:=.gz) GZFILES_5 = $(MANPAGES_5_GROFF:=.gz) GZFILES_3 = $(MANPAGES_3_GROFF:=.gz) GZFILES_1 = $(MANPAGES_1_GROFF:=.gz) GZFILES_7_NOINSTALL = $(MANPAGES_7_NOINSTALL:=.gz) GZFILES_5_NOINSTALL = $(MANPAGES_5_NOINSTALL:=.gz) GZFILES_3_NOINSTALL = $(MANPAGES_3_NOINSTALL:=.gz) GZFILES_1_NOINSTALL = $(MANPAGES_1_NOINSTALL:=.gz) GZFILES_3_DUMMY = $(MANPAGES_3_DUMMY:=.gz) GZFILES = $(GZFILES_7) $(GZFILES_5) $(GZFILES_3) $(GZFILES_1) \ $(GZFILES_7_NOINSTALL) $(GZFILES_5_NOINSTALL) $(GZFILES_3_NOINSTALL) $(GZFILES_1_NOINSTALL) \ $(GZFILES_3_DUMMY) MANPAGES_DESTDIR_7 = $(DESTDIR)$(man7dir) MANPAGES_DESTDIR_5 = $(DESTDIR)$(man5dir) MANPAGES_DESTDIR_3 = $(DESTDIR)$(man3dir) MANPAGES_DESTDIR_1 = $(DESTDIR)$(man1dir) DOCS_DESTDIR = $(DESTDIR)$(docdir) txt: $(TXTFILES) all: | $(MANPAGES) $(MANPAGES_WEBDIR_LINUX) $(MANPAGES_WEBDIR_WINDOWS): $(MKDIR) -p $@ %.1: %.1.md ../utils/md2man.sh $< default.man $@ %.3: %.3.md ../utils/md2man.sh $< default.man $@ %.5: %.5.md ../utils/md2man.sh $< default.man $@ %.7: %.7.md ../utils/md2man.sh $< default.man $@ %.txt: % man ./$< > $@ %.gz: % gzip -nc ./$* > $@ %.html: % groff -mandoc -Thtml ./$< > $@ html: $(HTMLFILES) web: $(MANPAGES) | $(MANPAGES_WEBDIR_LINUX) $(MANPAGES_WEBDIR_WINDOWS) $(MAKE) -C generated all $(foreach f, $(MANPAGES_7_MD), WEB=1 WIN32="" ../utils/md2man.sh $(f) default.man $(MANPAGES_WEBDIR_LINUX)/$(f);) $(foreach f, $(MANPAGES_5_MD), WEB=1 WIN32="" ../utils/md2man.sh $(f) default.man $(MANPAGES_WEBDIR_LINUX)/$(f);) $(foreach f, $(MANPAGES_3_MD), WEB=1 WIN32="" ../utils/md2man.sh $(f) default.man $(MANPAGES_WEBDIR_LINUX)/$(f);) $(foreach f, $(MANPAGES_1_MD), WEB=1 WIN32="" ../utils/md2man.sh $(f) default.man $(MANPAGES_WEBDIR_LINUX)/$(f);) $(foreach f, $(MANPAGES_7_MD), WEB=1 WIN32=1 ../utils/md2man.sh $(f) default.man $(MANPAGES_WEBDIR_WINDOWS)/$(f);) $(foreach f, $(MANPAGES_5_MD), WEB=1 WIN32=1 ../utils/md2man.sh $(f) default.man $(MANPAGES_WEBDIR_WINDOWS)/$(f);) $(foreach f, $(MANPAGES_3_MD), WEB=1 WIN32=1 ../utils/md2man.sh $(f) default.man $(MANPAGES_WEBDIR_WINDOWS)/$(f);) $(foreach f, $(MANPAGES_1_MD), WEB=1 WIN32=1 ../utils/md2man.sh $(f) default.man $(MANPAGES_WEBDIR_WINDOWS)/$(f);) compress: $(GZFILES) clean: clobber: clean $(RM) $(TXTFILES) $(HTMLFILES) $(GZFILES) $(MANPAGES) $(RM) -r $(MANPAGES_WEBDIR_LINUX) \ $(MANPAGES_WEBDIR_WINDOWS) install: compress install -d -v $(MANPAGES_DESTDIR_7) install -p -m 0644 $(GZFILES_7) $(MANPAGES_DESTDIR_7) install -d -v $(MANPAGES_DESTDIR_5) install -p -m 0644 $(GZFILES_5) $(MANPAGES_DESTDIR_5) install -d -v $(MANPAGES_DESTDIR_3) install -p -m 0644 $(GZFILES_3) $(GZFILES_3_DUMMY) $(MANPAGES_DESTDIR_3) install -d -v $(MANPAGES_DESTDIR_1) install -p -m 0644 $(GZFILES_1) $(MANPAGES_DESTDIR_1) uninstall: $(foreach f, $(notdir $(GZFILES_7)), $(RM) $(MANPAGES_DESTDIR_7)/$(f)) $(foreach f, $(notdir $(GZFILES_5)), $(RM) $(MANPAGES_DESTDIR_5)/$(f)) $(foreach f, $(notdir $(GZFILES_3) $(GZFILES_3_DUMMY)), $(RM) $(MANPAGES_DESTDIR_3)/$(f)) $(foreach f, $(notdir $(GZFILES_1)), $(RM) $(MANPAGES_DESTDIR_1)/$(f)) FORCE: .PHONY: all html clean compress clobber cstyle install uninstall pmdk-1.8/doc/poolset/0000775000000000000000000000000013615011416013245 5ustar rootrootpmdk-1.8/doc/poolset/.gitignore0000664000000000000000000000001213615011243015224 0ustar rootrootpoolset.5 pmdk-1.8/doc/poolset/poolset.50000644000000000000000000003027513615011416015025 0ustar rootroot.\" Automatically generated by Pandoc 2.2.1 .\" .TH "POOLSET" "5" "2020-01-31" "PMDK - poolset API version 1.0" "PMDK Programmer's Manual" .hy .\" Copyright 2017-2018, Intel Corporation .\" .\" Redistribution and use in source and binary forms, with or without .\" modification, are permitted provided that the following conditions .\" are met: .\" .\" * Redistributions of source code must retain the above copyright .\" notice, this list of conditions and the following disclaimer. .\" .\" * Redistributions in binary form must reproduce the above copyright .\" notice, this list of conditions and the following disclaimer in .\" the documentation and/or other materials provided with the .\" distribution. .\" .\" * Neither the name of the copyright holder nor the names of its .\" contributors may be used to endorse or promote products derived .\" from this software without specific prior written permission. .\" .\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS .\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT .\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR .\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT .\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, .\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT .\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, .\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY .\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT .\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE .\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. .SH NAME .PP poolset \- persistent memory pool configuration file format .SH SYNOPSIS .IP .nf \f[C] mypool.set \f[] .fi .SH DESCRIPTION .PP Depending on the configuration of the system, the available non\-volatile memory space may be divided into multiple memory devices. In such case, the maximum size of the transactional object store could be limited by the capacity of a single memory device. Therefore, \f[B]libpmemobj\f[](7), \f[B]libpmemblk\f[](7) and \f[B]libpmemlog\f[](7) allow building object stores spanning multiple memory devices by creation of persistent memory pools consisting of multiple files, where each part of such a \f[I]pool set\f[] may be stored on a different pmem\-aware filesystem. .PP To improve reliability and eliminate single point of failure, \f[B]libpmemobj\f[](7) also allows all the data written to a persistent memory pool to be copied to local or remote pool \f[I]replicas\f[], thereby providing backup for the persistent memory pool by producing a \f[I]mirrored pool set\f[]. In practice, the pool replicas may be considered as binary copies of the \[lq]master\[rq] pool set. Data replication is not supported in \f[B]libpmemblk\f[](7) and \f[B]libpmemlog\f[](7). .PP The \f[I]set\f[] file for each type of pool is a plain text file. Lines in the file are formatted as follows: .IP \[bu] 2 The first line of the file must be the literal string \[lq]PMEMPOOLSET\[rq] .IP \[bu] 2 The pool parts are specified, one per line, in the format: .RS 2 .PP \f[I]size\f[] \f[I]pathname\f[] .RE .IP \[bu] 2 \f[I]Replica\f[] sections, if any, start with the literal string \[lq]REPLICA\[rq]. See \f[B]REPLICAS\f[], below, for further details. .IP \[bu] 2 Pool set options, if any, start with literal string \f[I]OPTION\f[]. See \f[B]POOL SET OPTIONS\f[] below for details. .IP \[bu] 2 Lines starting with \[lq]#\[rq] are considered comments and are ignored. .PP The \f[I]size\f[] must be compliant with the format specified in IEC 80000\-13, IEEE 1541 or the Metric Interchange Format. These standards accept SI units with obligatory B \- kB, MB, GB, \&... (multiplier by 1000) suffixes, and IEC units with optional \[lq]iB\[rq] \- KiB, MiB, GiB, \&..., K, M, G, \&... \- (multiplier by 1024) suffixes. .PP \f[I]pathname\f[] must be an absolute pathname. .PP The \f[I]pathname\f[] of a part can point to a Device DAX. Device DAX is the device\-centric analogue of Filesystem DAX. It allows memory ranges to be allocated and mapped without need of an intervening file system. .PP Pools created on Device DAX have additional options and restrictions: .IP \[bu] 2 The \f[I]size\f[] may be set to \[lq]AUTO\[rq], in which case the size of the device will be automatically resolved at pool creation time. .IP \[bu] 2 To concatenate more than one Device DAX device into a single pool set, the configured internal alignment of the devices must be 4KiB, unless the \f[I]SINGLEHDR\f[] or \f[I]NOHDRS\f[] option is used in the pool set file. See \f[B]POOL SET OPTIONS\f[] below for details. .PP Please see \f[B]ndctl\-create\-namespace\f[](1) for more information on Device DAX, including how to configure desired alignment. .PP The minimum file size of each part of the pool set is defined as follows: .IP \[bu] 2 For block pools, as \f[B]PMEMBLK_MIN_PART\f[] in \f[B]\f[] .IP \[bu] 2 For object pools, as \f[B]PMEMOBJ_MIN_PART\f[] in \f[B]\f[] .IP \[bu] 2 For log pools, as \f[B]PMEMLOG_MIN_PART\f[] in \f[B]\f[] .PP The net pool size of the pool set is equal to: .IP .nf \f[C] net_pool_size\ =\ sum_over_all_parts(page_aligned_part_size\ \-\ 4KiB)\ +\ 4KiB \f[] .fi .PP where .IP .nf \f[C] page_aligned_part_size\ =\ part_size\ &\ ~(page_size\ \-\ 1) \f[] .fi .PP Note that page size is OS specific. For more information please see \f[B]sysconf\f[](3). .PP The minimum net pool size of a pool set is defined as follows: .IP \[bu] 2 For block pools, as \f[B]PMEMBLK_MIN_POOL\f[] in \f[B]\f[] .IP \[bu] 2 For object pools, as \f[B]PMEMOBJ_MIN_POOL\f[] in \f[B]\f[] .IP \[bu] 2 For log pools, as \f[B]PMEMLOG_MIN_POOL\f[] in \f[B]\f[] .PP Here is an example \[lq]mypool.set\[rq] file: .IP .nf \f[C] PMEMPOOLSET OPTION\ NOHDRS 100G\ /mountpoint0/myfile.part0 200G\ /mountpoint1/myfile.part1 400G\ /mountpoint2/myfile.part2 \f[] .fi .PP The files in the set may be created by running one of the following commands. To create a block pool: .IP .nf \f[C] $\ pmempool\ create\ blk\ \ mypool.set \f[] .fi .PP To create a log pool: .IP .nf \f[C] $\ pmempool\ create\ log\ mypool.set \f[] .fi .SH REPLICAS .PP Sections defining replica sets are optional. There may be multiple replica sections. .PP Local replica sections begin with a line containing only the literal string \[lq]REPLICA\[rq], followed by one or more pool part lines as described above. .PP Remote replica sections consist of the \f[I]REPLICA\f[] keyword, followed on the same line by the address of a remote host and a relative path to a remote pool set file: .IP .nf \f[C] REPLICA\ [\@]\ [/] \f[] .fi .IP \[bu] 2 \f[I]hostname\f[] must be in the format recognized by the \f[B]ssh\f[](1) remote login client .IP \[bu] 2 \f[I]pathname\f[] is relative to the root config directory on the target node \- see \f[B]librpmem\f[](7) .PP There are no other lines in the remote replica section \- the REPLICA line defines a remote replica entirely. .PP Here is an example \[lq]myobjpool.set\[rq] file with replicas: .IP .nf \f[C] PMEMPOOLSET 100G\ /mountpoint0/myfile.part0 200G\ /mountpoint1/myfile.part1 400G\ /mountpoint2/myfile.part2 #\ local\ replica REPLICA 500G\ /mountpoint3/mymirror.part0 200G\ /mountpoint4/mymirror.part1\ #\ remote\ replica REPLICA\ user\@example.com\ remote\-objpool.set \f[] .fi .PP The files in the object pool set may be created by running the following command: .IP .nf \f[C] $\ pmempool\ create\ \-\-layout="mylayout"\ obj\ myobjpool.set \f[] .fi .PP Remote replica cannot have replicas, i.e.\ a remote pool set file cannot define any replicas. .SH POOL SET OPTIONS .PP Pool set options can appear anywhere after the line with \f[I]PMEMPOOLSET\f[] string. Pool set file can contain several pool set options. The following options are supported: .IP \[bu] 2 \f[I]SINGLEHDR\f[] .IP \[bu] 2 \f[I]NOHDRS\f[] .PP If the \f[I]SINGLEHDR\f[] option is used, only the first part in each replica contains the pool part internal metadata. In that case the effective size of a replica is the sum of sizes of all its part files decreased once by 4096 bytes. .PP The \f[I]NOHDRS\f[] option can appear only in the remote pool set file, when \f[B]librpmem\f[] does not serve as a means of replication for \f[B]libpmemobj\f[] pool. In that case none of the pool parts contains internal metadata. The effective size of such a replica is the sum of sizes of all its part files. .PP Options \f[I]SINGLEHDR\f[] and \f[I]NOHDRS\f[] are mutually exclusive. If both are specified in a pool set file, creating or opening the pool will fail with an error. .PP When using the \f[I]SINGLEHDR\f[] or \f[I]NOHDRS\f[] option, one can concatenate more than one Device DAX devices with any internal alignments in one replica. .PP The \f[I]SINGLEHDR\f[] option concerns only replicas that are local to the pool set file. That is if one wants to create a pool set with the \f[I]SINGLEHDR\f[] option and with remote replicas, one has to add this option to the local pool set file as well as to every single remote pool set file. .PP Using the \f[I]SINGLEHDR\f[] and \f[I]NOHDRS\f[] options has important implications for data integrity checking and recoverability in case of a pool set damage. See \f[B]pmempool_sync\f[]() API for more information about pool set recovery. .SH DIRECTORIES .PP Providing a directory as a part's \f[I]pathname\f[] allows the pool to dynamically create files and consequently removes the user\-imposed limit on the size of the pool. .PP The \f[I]size\f[] argument of a part in a directory poolset becomes the size of the address space reservation required for the pool. In other words, the size argument is the maximum theoretical size of the mapping. This value can be freely increased between instances of the application, but decreasing it below the real required space will result in an error when attempting to open the pool. .PP The directory must NOT contain user created files with extension \f[I].pmem\f[], otherwise the behavior is undefined. If a file created by the library within the directory is in any way altered (resized, renamed) the behavior is undefined. .PP A directory poolset must exclusively use directories to specify paths \- combining files and directories will result in an error. A single replica can consist of one or more directories. If there are multiple directories, the address space reservation is equal to the sum of the sizes. .PP The order in which the files are created is unspecified, but the library will try to maintain equal usage of the directories. .PP By default pools grow in 128 megabyte increments. .PP Only poolsets with the \f[I]SINGLEHDR\f[] option can safely use directories. .SH NOTES .PP Creation of all the parts of the pool set and the associated replica sets can be done with the \f[B]pmemobj_create\f[](3), \f[B]pmemblk_create\f[](3) or \f[B]pmemlog_create\f[](3) function, or by using the \f[B]pmempool\f[](1) utility. .PP Restoring data from a local or remote replica can be done by using the \f[B]pmempool\-sync\f[](1) command or the \f[B]pmempool_sync\f[]() API from the \f[B]libpmempool\f[](7) library. .PP Modifications of a pool set file configuration can be done by using the \f[B]pmempool\-transform\f[](1) command or the \f[B]pmempool_transform\f[]() API from the \f[B]libpmempool\f[](7) library. .PP When creating a pool set consisting of multiple files, or when creating a replicated pool set, the \f[I]path\f[] argument passed to \f[B]pmemobj_create\f[](3), \f[B]pmemblk_create\f[](3) or \f[B]pmemlog_create\f[](3) must point to the special \f[I]set\f[] file that defines the pool layout and the location of all the parts of the pool set. .PP When opening a pool set consisting of multiple files, or when opening a replicated pool set, the \f[I]path\f[] argument passed to \f[B]pmemobj_open\f[](3), \f[B]pmemblk_open\f[](3) or \f[B]pmemlog_open\f[](3) must point to the same \f[I]set\f[] file that was used for pool set creation. .SH SEE ALSO .PP \f[B]ndctl\-create\-namespace\f[](1), \f[B]pmemblk_create\f[](3), \f[B]pmemlog_create\f[](3), \f[B]pmemobj_create\f[](3), \f[B]sysconf\f[](3), \f[B]libpmemblk\f[](7), \f[B]libpmemlog\f[](7), \f[B]libpmemobj\f[](7) and \f[B]\f[] pmdk-1.8/doc/poolset/poolset.5.md0000664000000000000000000002733713615011243015431 0ustar rootroot--- layout: manual Content-Style: 'text/css' title: _MP(POOLSET, 5) collection: poolset header: PMDK date: poolset API version 1.0 ... [comment]: <> (Copyright 2017-2018, Intel Corporation) [comment]: <> (Redistribution and use in source and binary forms, with or without) [comment]: <> (modification, are permitted provided that the following conditions) [comment]: <> (are met:) [comment]: <> ( * Redistributions of source code must retain the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer.) [comment]: <> ( * Redistributions in binary form must reproduce the above copyright) [comment]: <> ( notice, this list of conditions and the following disclaimer in) [comment]: <> ( the documentation and/or other materials provided with the) [comment]: <> ( distribution.) [comment]: <> ( * Neither the name of the copyright holder nor the names of its) [comment]: <> ( contributors may be used to endorse or promote products derived) [comment]: <> ( from this software without specific prior written permission.) [comment]: <> (THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS) [comment]: <> ("AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR) [comment]: <> (A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT) [comment]: <> (OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,) [comment]: <> (SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT) [comment]: <> (LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,) [comment]: <> (DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY) [comment]: <> (THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT) [comment]: <> ((INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE) [comment]: <> (OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.) [comment]: <> (poolset.5 -- man page that describes format of pool set file) [NAME](#name)
[SYNOPSIS](#synopsis)
[DESCRIPTION](#description)
[REPLICAS](#replicas)
[POOL SET OPTIONS](#pool-set-options)
[NOTES](#notes)
[SEE ALSO](#see-also)
# NAME # poolset - persistent memory pool configuration file format # SYNOPSIS # ```c mypool.set ``` # DESCRIPTION # Depending on the configuration of the system, the available non-volatile memory space may be divided into multiple memory devices. In such case, the maximum size of the transactional object store could be limited by the capacity of a single memory device. Therefore, **libpmemobj**(7), **libpmemblk**(7) and **libpmemlog**(7) allow building object stores spanning multiple memory devices by creation of persistent memory pools consisting of multiple files, where each part of such a *pool set* may be stored on a different pmem-aware filesystem. To improve reliability and eliminate single point of failure, **libpmemobj**(7) also allows all the data written to a persistent memory pool to be copied to local _WINUX(,or remote) pool *replicas*, thereby providing backup for the persistent memory pool by producing a *mirrored pool set*. In practice, the pool replicas may be considered as binary copies of the "master" pool set. Data replication is not supported in **libpmemblk**(7) and **libpmemlog**(7). The *set* file for each type of pool is a plain text file. Lines in the file are formatted as follows: + The first line of the file must be the literal string "PMEMPOOLSET" + The pool parts are specified, one per line, in the format: *size* *pathname* + *Replica* sections, if any, start with the literal string "REPLICA". See **REPLICAS**, below, for further details. + Pool set options, if any, start with literal string *OPTION*. See **POOL SET OPTIONS** below for details. + Lines starting with "#" are considered comments and are ignored. The *size* must be compliant with the format specified in IEC 80000-13, IEEE 1541 or the Metric Interchange Format. These standards accept SI units with obligatory B - kB, MB, GB, ... (multiplier by 1000) suffixes, and IEC units with optional "iB" - KiB, MiB, GiB, ..., K, M, G, ... - (multiplier by 1024) suffixes. *pathname* must be an absolute pathname. The *pathname* of a part can point to a Device DAX. Device DAX is the device-centric analogue of Filesystem DAX. It allows memory ranges to be allocated and mapped without need of an intervening file system. Pools created on Device DAX have additional options and restrictions: + The *size* may be set to "AUTO", in which case the size of the device will be automatically resolved at pool creation time. + To concatenate more than one Device DAX device into a single pool set, the configured internal alignment of the devices must be 4KiB, unless the *SINGLEHDR* or *NOHDRS* option is used in the pool set file. See **POOL SET OPTIONS** below for details. Please see **ndctl-create-namespace**(1) for more information on Device DAX, including how to configure desired alignment. The minimum file size of each part of the pool set is defined as follows: + For block pools, as **PMEMBLK_MIN_PART** in **\** + For object pools, as **PMEMOBJ_MIN_PART** in **\** + For log pools, as **PMEMLOG_MIN_PART** in **\** The net pool size of the pool set is equal to: ``` net_pool_size = sum_over_all_parts(page_aligned_part_size - 4KiB) + 4KiB ``` where ``` page_aligned_part_size = part_size & ~(page_size - 1) ``` Note that page size is OS specific. For more information please see **sysconf**(3). The minimum net pool size of a pool set is defined as follows: + For block pools, as **PMEMBLK_MIN_POOL** in **\** + For object pools, as **PMEMOBJ_MIN_POOL** in **\** + For log pools, as **PMEMLOG_MIN_POOL** in **\** Here is an example "mypool.set" file: ``` PMEMPOOLSET OPTION NOHDRS 100G /mountpoint0/myfile.part0 200G /mountpoint1/myfile.part1 400G /mountpoint2/myfile.part2 ``` The files in the set may be created by running one of the following commands. To create a block pool: ``` $ pmempool create blk mypool.set ``` To create a log pool: ``` $ pmempool create log mypool.set ``` # REPLICAS # Sections defining replica sets are optional. There may be multiple replica sections. Local replica sections begin with a line containing only the literal string "REPLICA", followed by one or more pool part lines as described above. _WINUX(, =q=Remote replica sections consist of the *REPLICA* keyword, followed on the same line by the address of a remote host and a relative path to a remote pool set file: ``` REPLICA [@] [/] ``` + *hostname* must be in the format recognized by the **ssh**(1) remote login client + *pathname* is relative to the root config directory on the target node - see **librpmem**(7) There are no other lines in the remote replica section - the REPLICA line defines a remote replica entirely. =e=) Here is an example "myobjpool.set" file with replicas: ``` PMEMPOOLSET 100G /mountpoint0/myfile.part0 200G /mountpoint1/myfile.part1 400G /mountpoint2/myfile.part2 # local replica REPLICA 500G /mountpoint3/mymirror.part0 200G /mountpoint4/mymirror.part1 _WINUX(,=q= # remote replica REPLICA user@example.com remote-objpool.set=e=) ``` The files in the object pool set may be created by running the following command: ``` $ pmempool create --layout="mylayout" obj myobjpool.set ``` _WINUX(, =q=Remote replica cannot have replicas, i.e. a remote pool set file cannot define any replicas.=e=) # POOL SET OPTIONS # Pool set options can appear anywhere after the line with *PMEMPOOLSET* string. Pool set file can contain several pool set options. The following options are supported: + *SINGLEHDR* + *NOHDRS* If the *SINGLEHDR* option is used, only the first part in each replica contains the pool part internal metadata. In that case the effective size of a replica is the sum of sizes of all its part files decreased once by 4096 bytes. The *NOHDRS* option can appear only in the remote pool set file, when **librpmem** does not serve as a means of replication for **libpmemobj** pool. In that case none of the pool parts contains internal metadata. The effective size of such a replica is the sum of sizes of all its part files. Options *SINGLEHDR* and *NOHDRS* are mutually exclusive. If both are specified in a pool set file, creating or opening the pool will fail with an error. When using the *SINGLEHDR* or *NOHDRS* option, one can concatenate more than one Device DAX devices with any internal alignments in one replica. The *SINGLEHDR* option concerns only replicas that are local to the pool set file. That is if one wants to create a pool set with the *SINGLEHDR* option and with remote replicas, one has to add this option to the local pool set file as well as to every single remote pool set file. Using the *SINGLEHDR* and *NOHDRS* options has important implications for data integrity checking and recoverability in case of a pool set damage. See _UW(pmempool_sync) API for more information about pool set recovery. # DIRECTORIES # Providing a directory as a part's *pathname* allows the pool to dynamically create files and consequently removes the user-imposed limit on the size of the pool. The *size* argument of a part in a directory poolset becomes the size of the address space reservation required for the pool. In other words, the size argument is the maximum theoretical size of the mapping. This value can be freely increased between instances of the application, but decreasing it below the real required space will result in an error when attempting to open the pool. The directory must NOT contain user created files with extension *.pmem*, otherwise the behavior is undefined. If a file created by the library within the directory is in any way altered (resized, renamed) the behavior is undefined. A directory poolset must exclusively use directories to specify paths - combining files and directories will result in an error. A single replica can consist of one or more directories. If there are multiple directories, the address space reservation is equal to the sum of the sizes. The order in which the files are created is unspecified, but the library will try to maintain equal usage of the directories. By default pools grow in 128 megabyte increments. Only poolsets with the *SINGLEHDR* option can safely use directories. # NOTES # Creation of all the parts of the pool set and the associated replica sets can be done with the **pmemobj_create**(3), **pmemblk_create**(3) or **pmemlog_create**(3) function, or by using the **pmempool**(1) utility. Restoring data from a local _WINUX(,or remote) replica can be done by using the **pmempool-sync**(1) command or the _UW(pmempool_sync) API from the **libpmempool**(7) library. Modifications of a pool set file configuration can be done by using the **pmempool-transform**(1) command or the _UW(pmempool_transform) API from the **libpmempool**(7) library. When creating a pool set consisting of multiple files, or when creating a replicated pool set, the *path* argument passed to **pmemobj_create**(3), **pmemblk_create**(3) or **pmemlog_create**(3) must point to the special *set* file that defines the pool layout and the location of all the parts of the pool set. When opening a pool set consisting of multiple files, or when opening a replicated pool set, the *path* argument passed to **pmemobj_open**(3), **pmemblk_open**(3) or **pmemlog_open**(3) must point to the same *set* file that was used for pool set creation. # SEE ALSO # **ndctl-create-namespace**(1), **pmemblk_create**(3), **pmemlog_create**(3), **pmemobj_create**(3), **sysconf**(3), **libpmemblk**(7), **libpmemlog**(7), **libpmemobj**(7) and **** pmdk-1.8/LICENSE0000664000000000000000000000353513615011243012024 0ustar rootrootCopyright 2014-2019, Intel Corporation Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. Everything in this source tree is covered by the previous license with the following exceptions: * src/common/valgrind/valgrind.h, src/common/valgrind/memcheck.h, src/common/valgrind/helgrind.h, src/common/valgrind/drd.h are covered by another similar BSD license variant, contained in those files. * utils/cstyle (used only during development) licensed under CDDL. pmdk-1.8/.skip-doc0000644000000000000000000000000013615011435012512 0ustar rootrootpmdk-1.8/CONTRIBUTING.md0000664000000000000000000001452513615011243013251 0ustar rootroot# Contributing to the Persistent Memory Development Kit Down below you'll find instructions on how to contribute to the Persistent Memory Development Kit. Your contributions are most welcome! You'll find it is best to begin with a conversation about your changes, rather than just writing a bunch of code and contributing it out of the blue. There are several good ways to suggest new features, offer to add a feature, or just begin a dialog about the Persistent Memory Development Kit: * Open an issue in our [GitHub Issues Database](https://github.com/pmem/issues/issues) * Suggest a feature, ask a question, start a discussion, etc. in our [pmem Google group](https://groups.google.com/group/pmem) * Chat with members of the PMDK team real-time on the **#pmem** IRC channel on [OFTC](https://www.oftc.net) **NOTE: If you do decide to implement code changes and contribute them, please make sure you agree your contribution can be made available under the [BSD-style License used for the Persistent Memory Development Kit](https://github.com/pmem/pmdk/blob/master/LICENSE).** **NOTE: Submitting your changes also means that you certify the following:** ``` Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: (a) The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or (b) The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or (c) The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it. (d) I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved. ``` In case of any doubt, the gatekeeper may ask you to certify the above in writing, i.e. via email or by including a `Signed-off-by:` line at the bottom of your commit comments. To improve tracking of who is the author of the contribution, we kindly ask you to use your real name (not an alias) when committing your changes to the Persistent Memory Development Kit: ``` Author: Random J Developer ``` ### Code Contributions Please feel free to use the forums mentioned above to ask for comments & questions on your code before submitting a pull request. The Persistent Memory Development Kit project uses the common *fork and merge* workflow used by most GitHub-hosted projects. The [Git Workflow blog article](https://pmem.io/2014/09/09/git-workflow.html) describes our workflow in more detail. #### Linux/FreeBSD Before contributing please remember to run: ``` $ make cstyle ``` This will check all C/C++ files in the tree for style issues. To check C++ files you have to have clang-format version 6.0, otherwise they will be skipped. If you want to run this target automatically at build time, you can pass CSTYLEON=1 to make. If you want cstyle to be run, but not fail the build, pass CSTYLEON=2 to make. There is also a target for automatic C++ code formatting, to do this run: ``` $ make format ``` There are cases, when you might have several clang-format-X.Y binaries and either no clang-format or it pointing to an older version. In such case run: ``` $ make CLANG_FORMAT=/path/to/clang-format cstyle|format ``` #### Windows On Windows to check the code for style issues, please run: ``` $ pmdk\utils\CSTYLE.ps1 ``` To check or format C++ files, you may use a standalone Visual Studio plugin for clang-format. The plugin installer can be downloaded from [LLVM Builds](https://llvm.org/builds) page. If you are actively working on an PMDK feature, please let other developers know by [creating an issue](https://github.com/pmem/issues/issues). Use the template `Feature` and assign it to yourself (due to the way GitHub permissions work, you may have to ask a team member to assign it to you). ### Bug Reports Bugs for the PMDK project are tracked in our [GitHub Issues Database](https://github.com/pmem/issues/issues). When reporting a new bug, please use `New issue` button, pick proper template and fill in all fields. Provide as much information as possible, including the product version: #### PMDK version Put the release name of the version of PMDK running when the bug was discovered in a bug comment. If you saw this bug in multiple PMDK versions, please put at least the most recent version and list the others if necessary. - Stable release names are in the form `#.#` (where `#` represents an integer); for example `0.3`. - Release names from working versions look like `#.#+b#` (adding a build #) or `#.#-rc#` (adding a release candidate number) If PMDK was built from source, the version number can be retrieved from git using this command: `git describe` For binary PMDK releases, use the entire package name. For RPMs, use `rpm -q pmdk` to display the name. For Deb packages, run `dpkg-query -W pmdk` and use the second (version) string. #### Priority Requested priority describes the urgency to resolve a defect and establishes the time frame for providing a verified resolution. Priorities are defined as: * **P1**: Showstopper bug, requiring a resolution before the next release of the library. * **P2**: High-priority bug, requiring a resolution although it may be decided that the bug does not prevent the next release of the library. * **P3**: Medium-priority bug. The expectation is that the bug will be evaluated and a plan will be made for when the bug will be resolved. * **P4**: Low-priority bug, the least urgent. Fixed when the resources are available. ### Other issues On our issues page we also gather feature requests and questions. Templates to use are `Feature` and `Question`, respectively. They should help deliver a meaningful description of a feature or ask a question to us (remember though we have different means of communication, as described at the top of the page). pmdk-1.8/src/0000775000000000000000000000000013615011243011600 5ustar rootrootpmdk-1.8/src/PMDK.sln0000664000000000000000000045027213615011243013063 0ustar rootrootMicrosoft Visual Studio Solution File, Format Version 12.00 # Visual Studio 15 VisualStudioVersion = 15.0.26730.15 MinimumVisualStudioVersion = 10.0.40219.1 Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmemlog_minimal", "examples\libpmemobj\pmemlog\obj_pmemlog_minimal.vcxproj", "{0056B0B6-CB3E-4F0E-B6DC-48D59CB8E235}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ctl_arenas", "test\obj_ctl_arenas\obj_ctl_arenas.vcxproj", "{019F5586-5558-4C87-B319-85906D4AE407}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_movnt_align", "test\pmem_movnt_align\pmem_movnt_align.vcxproj", "{025E7D51-41F2-4CBA-956E-C37A4443DB1B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "full_copy", "examples\libpmem\full_copy.vcxproj", "{0287C3DC-AE03-4714-AAFF-C52F062ECA6F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "traces_custom_function", "test\traces_custom_function\traces_custom_function.vcxproj", "{02BC3B44-C7F1-4793-86C1-6F36CA8A7F53}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ctl_stats", "test\obj_ctl_stats\obj_ctl_stats.vcxproj", "{03228F84-4F41-4BCC-8C2D-F329DC87B289}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_memblock", "test\obj_memblock\obj_memblock.vcxproj", "{0388E945-A655-41A7-AF27-8981CEE0E49A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_direct_volatile", "test\obj_direct_volatile\obj_direct_volatile.vcxproj", "{03B54A12-7793-4827-B820-C07491F7F45E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_callbacks", "test\obj_tx_callbacks\obj_tx_callbacks.vcxproj", "{0529575C-F6E8-44FD-BB82-82A29948D0F2}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "out_err_mt", "test\out_err_mt\out_err_mt.vcxproj", "{063037B2-CA35-4520-811C-19D9C4ED891E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmemlog_macros", "examples\libpmemobj\pmemlog\obj_pmemlog_macros.vcxproj", "{06877FED-15BA-421F-85C9-1A964FB97446}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_mt", "test\obj_tx_mt\obj_tx_mt.vcxproj", "{0703E813-9CC8-4DEA-AA33-42B099CD172D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_heap_interrupt", "test\obj_heap_interrupt\obj_heap_interrupt.vcxproj", "{07A153D9-DF17-4DE8-A3C2-EBF171B961AE}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_parse_size", "test\util_parse_size\util_parse_size.vcxproj", "{08B62E36-63D2-4FF1-A605-4BBABAEE73FB}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "log_include", "test\log_include\log_include.vcxproj", "{0A049EAD-652F-4E20-8026-90FD99AEE77A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmemlog", "libpmemlog\libpmemlog.vcxproj", "{0B1818EB-BDC8-4865-964F-DB8BF05CFD86}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "reader", "examples\libpmemobj\string_store\reader.vcxproj", "{0BFD78AA-FD94-4DB1-8495-8F5CC06D8F03}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Examples", "Examples", "{0CC6D525-806E-433F-AB4A-6CFD546418B1}" ProjectSection(SolutionItems) = preProject examples\ex_common.h = examples\ex_common.h EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_critnib", "test\obj_critnib\obj_critnib.vcxproj", "{0CDCEB97-3270-4939-A290-EA2D3BE34B0C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_deep_persist", "test\pmem_deep_persist\pmem_deep_persist.vcxproj", "{0D4E38EF-A9D5-4797-8994-5DBB1125C9EA}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "btree", "examples\libpmemobj\btree.vcxproj", "{0FB8F0FD-276C-413B-97A8-67ABE0C9043B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_direct", "test\obj_direct\obj_direct.vcxproj", "{10469175-EEF7-44A0-9961-AC4E45EFD800}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pi", "examples\libpmemobj\pi.vcxproj", "{11D76FBC-DFAA-4B31-9DB0-206E171E3F94}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmemspoil", "test\tools\pmemspoil\pmemspoil.vcxproj", "{11E158AE-C85A-4A6E-B66A-ED2994709276}" ProjectSection(ProjectDependencies) = postProject {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_map_file", "test\pmem_map_file\pmem_map_file.vcxproj", "{12A1A3EF-202C-4DD0-9B5A-F5126CAB078F}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmem", "libpmem", "{1434B17C-6165-4D42-BEA1-5A7730D5A6BB}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_recreate", "test\obj_recreate\obj_recreate.vcxproj", "{1464398A-100F-4518-BDB9-939A6362B6CF}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "dllview", "test\tools\dllview\dllview.vcxproj", "{179BEB5A-2C90-44F5-A734-FA756A5E668C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "rbtree_map", "examples\libpmemobj\tree_map\rbtree_map.vcxproj", "{17A4B817-68B1-4719-A9EF-BD8FAB747DE6}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "pmempool", "pmempool", "{181A4234-282C-41F0-85C2-2B7697B3CB1A}" ProjectSection(SolutionItems) = preProject ..\doc\pmempool\pmempool-check.1.md = ..\doc\pmempool\pmempool-check.1.md ..\doc\pmempool\pmempool-convert.1.md = ..\doc\pmempool\pmempool-convert.1.md ..\doc\pmempool\pmempool-create.1.md = ..\doc\pmempool\pmempool-create.1.md ..\doc\pmempool\pmempool-dump.1.md = ..\doc\pmempool\pmempool-dump.1.md ..\doc\pmempool\pmempool-info.1.md = ..\doc\pmempool\pmempool-info.1.md ..\doc\pmempool\pmempool-rm.1.md = ..\doc\pmempool\pmempool-rm.1.md ..\doc\pmempool\pmempool-sync.1.md = ..\doc\pmempool\pmempool-sync.1.md ..\doc\pmempool\pmempool-transform.1.md = ..\doc\pmempool\pmempool-transform.1.md ..\doc\pmempool\pmempool.1.md = ..\doc\pmempool\pmempool.1.md EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_non_zero", "test\blk_non_zero\blk_non_zero.vcxproj", "{18E90E1A-F2E0-40DF-9900-A14E560C9EB4}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {F7C6C6B6-4142-4C82-8699-4A9D8183181B} = {F7C6C6B6-4142-4C82-8699-4A9D8183181B} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemlog", "libpmemlog", "{1A36B57B-2E88-4D81-89C0-F575C9895E36}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "cmpmap", "test\tools\cmpmap\cmpmap.vcxproj", "{1B871BA2-3F70-4BC9-9DF4-725EB07F6628}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ex_libpmemblk", "test\ex_libpmemblk\ex_libpmemblk.vcxproj", "{1B9B0D6D-E530-44A6-ADAE-09EA2BDC47DE}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmemobj", "libpmemobj\libpmemobj.vcxproj", "{1BAA1617-93AE-4196-8A1A-BD492FB18AEF}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "writer", "examples\libpmemobj\string_store_tx_type\writer.vcxproj", "{1EB3DE5B-6357-498D-8CAC-EEC0209EA454}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "win_lists", "test\win_lists\win_lists.vcxproj", "{1F2E1C51-2B14-4047-BE6D-52E00FC3C780}" ProjectSection(ProjectDependencies) = postProject {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_recovery", "test\obj_recovery\obj_recovery.vcxproj", "{2498FCDA-E2CC-43EF-9A35-8CD63F253171}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bttdevice", "test\bttdevice\bttdevice.vcxproj", "{25758581-DD46-4AE4-99D9-11E736F72AD1}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_transform", "test\pmempool_transform\pmempool_transform.vcxproj", "{26166DF1-3C94-44AF-9075-BA31DCD2F6BB}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_test_win", "test\libpmempool_api_win\libpmempool_test_win.vcxproj", "{27FA11C6-431D-41D1-A417-FAB7C4F93DCA}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_toid", "test\obj_toid\obj_toid.vcxproj", "{296F3C5D-3951-423E-8E2F-FD4A37958C72}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_include", "test\blk_include\blk_include.vcxproj", "{29D9376B-DC36-4940-83F1-A7CBE38A2103}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_dump", "test\pmempool_dump\pmempool_dump.vcxproj", "{2A1D6AF2-7336-4966-A4B3-0BE9A24BAE00}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "out_err_mt_win", "test\out_err_mt_win\out_err_mt_win.vcxproj", "{2B1A5104-A324-4D02-B5C7-D021FB8F880C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_fragmentation2", "test\obj_fragmentation2\obj_fragmentation2.vcxproj", "{2B2DE575-1422-4FBF-97BE-35AEDA0AB465}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_include", "test\pmem_include\pmem_include.vcxproj", "{2B7772E6-9DAA-4F38-B0BC-7B2399366325}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "poolset", "poolset", "{2C24CC4F-B340-467D-908F-1BF2C69BC79F}" ProjectSection(SolutionItems) = preProject ..\doc\poolset\poolset.5.md = ..\doc\poolset\poolset.5.md EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "lists", "examples\libpmemobj\lists.vcxproj", "{2CD7408E-2F60-43C3-ACEB-C7D58CDD8462}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_locks", "test\obj_locks\obj_locks.vcxproj", "{2DE6B085-3C19-49B1-894A-AD9376000E09}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_action", "test\obj_action\obj_action.vcxproj", "{2ED26FDA-3C4E-4514-B387-5E77C302FF71}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmemdetect", "test\tools\pmemdetect\pmemdetect.vcxproj", "{2EFFC590-BF5E-46A2-AF04-E67E1D571D2E}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmempool", "libpmempool", "{2F543422-4B8A-4898-BE6B-590F52B4E9D1}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bttcreate", "test\tools\bttcreate\bttcreate.vcxproj", "{3142CB13-CADA-48D3-9A25-E6ACB243760A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_map_file_trunc", "test\pmem_map_file_trunc\pmem_map_file_trunc.vcxproj", "{34DB4951-DA08-45F1-938D-B08E5FF5AB46}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "list_map", "examples\libpmemobj\list_map\list_map.vcxproj", "{3799BA67-3C4F-4AE0-85DC-5BAAEA01A180}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "traces_pmem", "test\traces_pmem\traces_pmem.vcxproj", "{3B23831B-E5DE-4A62-9D0B-27D0D9F293F4}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "log_pool", "test\log_pool\log_pool.vcxproj", "{3CF270CD-0F56-48E3-AD84-82F369C568BF}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {0B1818EB-BDC8-4865-964F-DB8BF05CFD86} = {0B1818EB-BDC8-4865-964F-DB8BF05CFD86} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "sparsefile", "test\tools\sparsefile\sparsefile.vcxproj", "{3EC30D6A-BDA4-4971-879A-8814204EAE31}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pool", "test\obj_pool\obj_pool.vcxproj", "{3ECCB0F1-3ADF-486A-91C5-79DF0FC22F78}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "rtree_map", "examples\libpmemobj\tree_map\rtree_map.vcxproj", "{3ED56E55-84A6-422C-A8D4-A8439FB8F245}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_alloc", "test\obj_alloc\obj_alloc.vcxproj", "{42B97D47-F800-4100-BFA2-B3AC357E8B6B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_info", "test\pmempool_info\pmempool_info.vcxproj", "{42CCEF95-5ADD-460C-967E-DD5B2C744943}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "getopt", "test\getopt\getopt.vcxproj", "{433F7840-C597-4950-84C9-E4FF7DF6A298}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "sys", "sys", "{45027FC5-4A32-47BD-AC5B-66CC7616B1D2}" ProjectSection(SolutionItems) = preProject windows\include\sys\file.h = windows\include\sys\file.h windows\include\sys\mman.h = windows\include\sys\mman.h windows\include\sys\mount.h = windows\include\sys\mount.h windows\include\sys\param.h = windows\include\sys\param.h windows\include\sys\resource.h = windows\include\sys\resource.h windows\include\sys\statvfs.h = windows\include\sys\statvfs.h windows\include\sys\uio.h = windows\include\sys\uio.h windows\include\sys\wait.h = windows\include\sys\wait.h EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ctl_cow", "test\ctl_cow\ctl_cow.vcxproj", "{46B82069-10BE-432A-8D93-F4D995148555}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pool_lookup", "test\obj_pool_lookup\obj_pool_lookup.vcxproj", "{4850F425-9128-4E91-973C-5AE7BD97395B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmemcommon", "common\libpmemcommon.vcxproj", "{492BAA3D-0D5D-478E-9765-500463AE69AA}" ProjectSection(ProjectDependencies) = postProject {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libmap", "examples\libpmemobj\map\libmap.vcxproj", "{49A7CC5A-D5E7-4A07-917F-C6918B982BE8}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "util", "util", "{4C291EEB-3874-4724-9CC2-1335D13FF0EE}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_alloc", "test\obj_tx_alloc\obj_tx_alloc.vcxproj", "{4C429783-0B01-449F-A36F-C2019233890B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmemalloc", "test\tools\pmemalloc\pmemalloc.vcxproj", "{4C6E7F0A-7E6A-4713-B1D2-B7B4ADC992AF}" ProjectSection(ProjectDependencies) = postProject {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_include", "test\libpmempool_include\libpmempool_include.vcxproj", "{4E334022-7A71-4197-9E15-878F7EFC877E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "log_walker", "test\log_walker\log_walker.vcxproj", "{4FB4FF90-4E92-4CFB-A01F-C73D6861CA03}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_poolset_parse", "test\util_poolset_parse\util_poolset_parse.vcxproj", "{50FD1E47-2131-48D2-9435-5CB28DF6B15A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "asset_checkout", "examples\libpmemblk\assetdb\asset_checkout.vcxproj", "{513C4CFA-BD5B-4470-BA93-F6D43778A754}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "arch_flags", "test\arch_flags\arch_flags.vcxproj", "{53115A01-460C-4339-A2C8-AE1323A6E7EA}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mmap", "test\mmap\mmap.vcxproj", "{5580D11C-FDA6-4CF2-A0E8-1C2D3FBC11F1}" ProjectSection(ProjectDependencies) = postProject {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "asset_checkin", "examples\libpmemblk\assetdb\asset_checkin.vcxproj", "{581B3A58-F3F0-4765-91E5-D0C82816A528}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_basic_integration", "test\obj_basic_integration\obj_basic_integration.vcxproj", "{58386481-30B7-40FC-96AF-0723A4A7B228}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "pmempool", "pmempool", "{59AB6976-D16B-48D0-8D16-94360D3FE51D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "reader", "examples\libpmemobj\string_store_tx\reader.vcxproj", "{59D7A9CD-9912-40E4-96E1-8A873F777F62}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_memmove", "test\pmem_memmove\pmem_memmove.vcxproj", "{5A391A14-8E29-4788-93FC-EDADED31D32F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_map_file_win", "test\pmem_map_file_win\pmem_map_file_win.vcxproj", "{5AD07646-5E16-4CEF-B80A-BE5EE4D54FEF}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "data_store", "examples\libpmemobj\map\data_store.vcxproj", "{5B2B9C0D-1B6D-4357-8307-6DE1EE0A41A3}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_many_size_allocs", "test\obj_many_size_allocs\obj_many_size_allocs.vcxproj", "{5D362DB7-D2BD-4907-AAD8-4B8627E72282}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmemlog_simple", "examples\libpmemobj\pmemlog\obj_pmemlog_simple.vcxproj", "{5DB2E259-0D19-4A89-B8EC-B2912F39924D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_user_data", "test\obj_tx_user_data\obj_tx_user_data.vcxproj", "{5E7305DB-93E6-448B-AE44-90EAF916A776}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_sds", "test\util_sds\util_sds.vcxproj", "{5EC35099-9777-45E8-9520-EB2EE75BDF88}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_rm", "test\libpmempool_rm\libpmempool_rm.vcxproj", "{5F2B687A-1B42-439C-AEEC-135DD22FB851}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_is_pmem_windows", "test\pmem_is_pmem_windows\pmem_is_pmem_windows.vcxproj", "{5F8A56F8-2C5B-48B6-9654-DD642D3E5F5C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmemlog", "examples\libpmemobj\pmemlog\obj_pmemlog.vcxproj", "{60206D22-E132-4695-8486-10BECA32C5CC}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_backup", "test\libpmempool_backup\libpmempool_backup.vcxproj", "{60B463D4-8CD5-4BF6-A25B-01BE13B87590}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_fragmentation", "test\obj_fragmentation\obj_fragmentation.vcxproj", "{60EF55C7-8399-4543-B5B2-3AE2C532C67E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_rw_mt", "test\blk_rw_mt\blk_rw_mt.vcxproj", "{628FADA9-7047-4DD9-BD17-9FE4B5A1ADB0}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pool_lock", "test\obj_pool_lock\obj_pool_lock.vcxproj", "{63B8184D-85E0-4E6A-9729-558C567D1D1D}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemobj", "libpmemobj", "{63C9B3F8-437D-4AD9-B32D-D04AE38C35B6}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_strdup", "test\obj_tx_strdup\obj_tx_strdup.vcxproj", "{643B82A1-D009-46A9-92A0-2883399B05C2}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_sync", "test\obj_sync\obj_sync.vcxproj", "{6516D6CF-8000-4341-9487-312BC83EE370}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmalloc_basic", "test\obj_pmalloc_basic\obj_pmalloc_basic.vcxproj", "{65D92D98-97E1-48F7-AEF6-75221CF48EA4}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_memcpy", "test\pmem_memcpy\pmem_memcpy.vcxproj", "{673277EC-D26B-414D-92E3-84EE873316A8}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_rm_win", "test\libpmempool_rm_win\libpmempool_rm_win.vcxproj", "{67AC1343-98FD-4143-92C0-559C55F749F5}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_rw", "test\blk_rw\blk_rw.vcxproj", "{6851356E-A5D9-46A6-8262-A7E208729F18}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {F7C6C6B6-4142-4C82-8699-4A9D8183181B} = {F7C6C6B6-4142-4C82-8699-4A9D8183181B} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "win_common", "test\win_common\win_common.vcxproj", "{6AE1B8BE-D46A-4E99-87A2-F160FB950DCA}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_poolset_size", "test\util_poolset_size\util_poolset_size.vcxproj", "{6B492754-9F80-44B3-A2A7-1D98AF06F3B2}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_list_macro", "test\obj_list_macro\obj_list_macro.vcxproj", "{6BCEF2A5-0CEC-4CC6-9CB0-D3FBF871A408}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "string_store_tx", "string_store_tx", "{6D63CDF1-F62C-4614-AD8A-95B0A63AA070}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "set_funcs", "test\set_funcs\set_funcs.vcxproj", "{6D7C1169-3246-465F-B630-ECFEF4F3179A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ctl_alignment", "test\obj_ctl_alignment\obj_ctl_alignment.vcxproj", "{6DBD8C02-0C75-4DB0-BFDA-CD053B1B2D89}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_unmap", "test\pmem_unmap\pmem_unmap.vcxproj", "{6EC93484-AAF3-487E-84E4-5ABFBA0AFC53}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_poolset", "test\util_poolset\util_poolset.vcxproj", "{6F06A19B-0921-4B71-A3A5-B350B5FFEADB}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_locks_abort", "test\obj_tx_locks_abort\obj_tx_locks_abort.vcxproj", "{6F4953DA-FDC3-46CF-BF24-3752CCF2E1CB}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_feature", "test\libpmempool_feature\libpmempool_feature.vcxproj", "{6F776280-B383-4DCE-8F42-9670164D038D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_out_of_memory", "test\obj_out_of_memory\obj_out_of_memory.vcxproj", "{70EE1D40-0C65-4985-8EFC-BD40EE3A89B2}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_file_open", "test\util_file_open\util_file_open.vcxproj", "{715EADD7-0FFE-4F1F-94E7-49302968DF79}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_check", "test\obj_check\obj_check.vcxproj", "{71D182E0-345A-4375-B0FA-3536821B0EE3}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "array", "examples\libpmemobj\array\array.vcxproj", "{7264C8F6-73FB-4830-9306-1558D3EAC71B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_list", "test\obj_list\obj_list.vcxproj", "{729E3905-FF7D-49C5-9871-6D35D839183E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ravl", "test\obj_ravl\obj_ravl.vcxproj", "{72C9DB46-C665-48AD-B805-BA885B40CA3E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "writer", "examples\libpmemobj\string_store_tx\writer.vcxproj", "{7337E34A-97B0-44FC-988B-7E6AE7E0FBBF}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_memops", "test\obj_memops\obj_memops.vcxproj", "{740ED97D-005F-4F58-98B2-4EF5EF5776E8}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Tests", "Tests", "{746BA101-5C93-42A5-AC7A-64DCEB186572}" ProjectSection(SolutionItems) = preProject test\match = test\match test\RUNTESTLIB.PS1 = test\RUNTESTLIB.PS1 test\RUNTESTS.ps1 = test\RUNTESTS.ps1 test\unittest\unittest.ps1 = test\unittest\unittest.ps1 EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "reader", "examples\libpmemobj\string_store_tx_type\reader.vcxproj", "{74D655D5-F661-4887-A1EB-5A6222AF5FCA}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_critnib_mt", "test\obj_critnib_mt\obj_critnib_mt.vcxproj", "{7701627C-CFD9-48F6-942E-EAACC8D057FA}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "linux", "linux", "{774627B7-6532-4464-AEE4-02F72CA44F95}" ProjectSection(SolutionItems) = preProject windows\include\linux\limits.h = windows\include\linux\limits.h EndProjectSection EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemblk", "libpmemblk", "{7783BC49-A25B-468B-A6F8-AB6B39A91C65}" ProjectSection(SolutionItems) = preProject ..\doc\libpmemblk\libpmemblk.7.md = ..\doc\libpmemblk\libpmemblk.7.md ..\doc\libpmemblk\pmemblk_bsize.3.md = ..\doc\libpmemblk\pmemblk_bsize.3.md ..\doc\libpmemblk\pmemblk_create.3.md = ..\doc\libpmemblk\pmemblk_create.3.md ..\doc\libpmemblk\pmemblk_read.3.md = ..\doc\libpmemblk\pmemblk_read.3.md ..\doc\libpmemblk\pmemblk_set_zero.3.md = ..\doc\libpmemblk\pmemblk_set_zero.3.md EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_pool_lock", "test\blk_pool_lock\blk_pool_lock.vcxproj", "{779425B1-2211-499B-A7CC-4F9EC6CB0D25}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {F7C6C6B6-4142-4C82-8699-4A9D8183181B} = {F7C6C6B6-4142-4C82-8699-4A9D8183181B} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "btree_map", "examples\libpmemobj\tree_map\btree_map.vcxproj", "{79D37FFE-FF76-44B3-BB27-3DCAEFF2EBE9}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_extend", "test\obj_extend\obj_extend.vcxproj", "{7ABF755C-821B-49CD-8EDE-83C16594FF7F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool", "tools\pmempool\pmempool.vcxproj", "{7DC3B3DD-73ED-4602-9AF3-8D7053620DEA}" ProjectSection(ProjectDependencies) = postProject {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ex_libpmem", "test\ex_libpmem\ex_libpmem.vcxproj", "{7DFEB4A5-8B04-4302-9D09-8144918FCF81}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_free", "test\obj_tx_free\obj_tx_free.vcxproj", "{7F51CD29-3BCD-4DD8-B327-F384B5A616D1}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "asset_list", "examples\libpmemblk\assetdb\asset_list.vcxproj", "{8008010F-8718-4C5F-86B2-195AEBF73422}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "manpage", "examples\libpmemblk\manpage.vcxproj", "{8010BBB0-C71B-4EFF-95EB-65C01E5EC197}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_pool_win", "test\blk_pool_win\blk_pool_win.vcxproj", "{80AF1B7D-B8CE-4AF0-AE3B-1DABED1B57E7}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "win_poolset_unmap", "test\win_poolset_unmap\win_poolset_unmap.vcxproj", "{810DB909-6581-42D8-9616-906888F12149}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{853D45D8-980C-4991-B62A-DAC6FD245402}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_heap", "test\obj_heap\obj_heap.vcxproj", "{85D4076B-896B-4EBB-8F3A-8B44C24CD452}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_debug", "test\obj_debug\obj_debug.vcxproj", "{85DBDA9B-AEF6-43E7-B8B5-05FF2BEC61A3}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_heap_state", "test\obj_heap_state\obj_heap_state.vcxproj", "{86EE22CC-6D3C-4F81-ADC8-394946F0DA81}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Tools", "Tools", "{877E7D1D-8150-4FE5-A139-B6FBCEAEC393}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemobj", "libpmemobj", "{87A32959-E477-4CD5-8A1C-C85646D806B2}" ProjectSection(SolutionItems) = preProject ..\doc\libpmemobj\libpmemobj.7.md = ..\doc\libpmemobj\libpmemobj.7.md ..\doc\libpmemobj\oid_is_null.3.md = ..\doc\libpmemobj\oid_is_null.3.md ..\doc\libpmemobj\pmemobj_action.3.md = ..\doc\libpmemobj\pmemobj_action.3.md ..\doc\libpmemobj\pmemobj_alloc.3.md = ..\doc\libpmemobj\pmemobj_alloc.3.md ..\doc\libpmemobj\pmemobj_ctl_get.3.md = ..\doc\libpmemobj\pmemobj_ctl_get.3.md ..\doc\libpmemobj\pmemobj_first.3.md = ..\doc\libpmemobj\pmemobj_first.3.md ..\doc\libpmemobj\pmemobj_list_insert.3.md = ..\doc\libpmemobj\pmemobj_list_insert.3.md ..\doc\libpmemobj\pmemobj_memcpy_persist.3.md = ..\doc\libpmemobj\pmemobj_memcpy_persist.3.md ..\doc\libpmemobj\pmemobj_mutex_zero.3.md = ..\doc\libpmemobj\pmemobj_mutex_zero.3.md ..\doc\libpmemobj\pmemobj_open.3.md = ..\doc\libpmemobj\pmemobj_open.3.md ..\doc\libpmemobj\pmemobj_root.3.md = ..\doc\libpmemobj\pmemobj_root.3.md ..\doc\libpmemobj\pmemobj_tx_add_range.3.md = ..\doc\libpmemobj\pmemobj_tx_add_range.3.md ..\doc\libpmemobj\pmemobj_tx_alloc.3.md = ..\doc\libpmemobj\pmemobj_tx_alloc.3.md ..\doc\libpmemobj\pmemobj_tx_begin.3.md = ..\doc\libpmemobj\pmemobj_tx_begin.3.md ..\doc\libpmemobj\pobj_layout_begin.3.md = ..\doc\libpmemobj\pobj_layout_begin.3.md ..\doc\libpmemobj\pobj_list_head.3.md = ..\doc\libpmemobj\pobj_list_head.3.md ..\doc\libpmemobj\toid_declare.3.md = ..\doc\libpmemobj\toid_declare.3.md EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmalloc_oom_mt", "test\obj_pmalloc_oom_mt\obj_pmalloc_oom_mt.vcxproj", "{88D239E4-EB7D-4E0A-BE3A-AD78B9F408FC}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_add_range_direct", "test\obj_tx_add_range_direct\obj_tx_add_range_direct.vcxproj", "{89F947CA-DDEF-4131-8AFB-584ABA4A1302}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "out_err", "test\out_err\out_err.vcxproj", "{8A0FA780-068A-4534-AA2F-4FF4CF977AF2}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_bucket", "test\obj_bucket\obj_bucket.vcxproj", "{8A4872D7-A234-4B9B-8215-82C6BB15F3A2}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmemblk", "examples\libpmemobj\pmemblk\obj_pmemblk.vcxproj", "{8C42CA7C-1543-4F1B-A55F-28CD419C7D35}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_oid_thread", "test\obj_oid_thread\obj_oid_thread.vcxproj", "{8C6D73E0-0A6F-4487-A040-0EC78D7D6D9A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_persist_count", "test\obj_persist_count\obj_persist_count.vcxproj", "{8D75FA1A-EC74-4F88-8AC1-CE3F98E4D828}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_flow", "test\obj_tx_flow\obj_tx_flow.vcxproj", "{8E374371-30E1-4623-8755-2A2F3742170B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "srcversion", "windows\srcversion\srcversion.vcxproj", "{901F04DB-E1A5-4A41-8B81-9D31C19ACD59}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "getopt", "windows\getopt\getopt.vcxproj", "{9186EAC4-2F34-4F17-B940-6585D7869BCD}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemlog", "libpmemlog", "{91C30620-70CA-46C7-AC71-71F3C602690E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_realloc", "test\obj_realloc\obj_realloc.vcxproj", "{91E19AEB-7B75-43E0-B8B4-D2BB60D839EA}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_create", "test\pmempool_create\pmempool_create.vcxproj", "{92388A20-50FC-45F8-89E3-71F1618EFABB}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "compat_incompat_features", "test\compat_incompat_features\compat_incompat_features.vcxproj", "{924B2937-0B53-4DC6-B7E1-5F3102728F89}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_pool", "test\blk_pool\blk_pool.vcxproj", "{95B683BD-B9DC-400F-9BC0-8F1505F08BF5}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {F7C6C6B6-4142-4C82-8699-4A9D8183181B} = {F7C6C6B6-4142-4C82-8699-4A9D8183181B} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Windows", "Windows", "{95FAF291-03D1-42FC-9C10-424D551D475D}" ProjectSection(SolutionItems) = preProject common\common.rc = common\common.rc EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_movnt", "test\pmem_movnt\pmem_movnt.vcxproj", "{96D00A19-5CEF-4CC5-BDE8-E33C68BCE90F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_cpuid", "test\util_cpuid\util_cpuid.vcxproj", "{98ACBE5D-1A92-46F9-AA81-533412172952}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_rm", "test\pmempool_rm\pmempool_rm.vcxproj", "{99F7F00F-1DE5-45EA-992B-64BA282FAC76}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_uuid_generate", "test\util_uuid_generate\util_uuid_generate.vcxproj", "{9A4078F8-B8E4-4EC6-A6FF-4F29DAD9CE48}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "include", "include", "{9A8482A7-BF0C-423D-8266-189456ED41F6}" ProjectSection(SolutionItems) = preProject windows\include\dirent.h = windows\include\dirent.h windows\include\endian.h = windows\include\endian.h windows\include\err.h = windows\include\err.h windows\include\features.h = windows\include\features.h windows\include\libgen.h = windows\include\libgen.h windows\include\platform.h = windows\include\platform.h include\pmemcompat.h = include\pmemcompat.h windows\include\sched.h = windows\include\sched.h windows\include\srcversion.h = windows\include\srcversion.h windows\include\strings.h = windows\include\strings.h windows\include\unistd.h = windows\include\unistd.h windows\include\win_mmap.h = windows\include\win_mmap.h EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_realloc", "test\obj_tx_realloc\obj_tx_realloc.vcxproj", "{9AE2DAF9-10C4-4EC3-AE52-AD5EE9C77C55}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Benchmarks", "Benchmarks", "{9C37B8CC-F810-4787-924D-65BC227091A3}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_invalid", "test\obj_tx_invalid\obj_tx_invalid.vcxproj", "{9D9E33EB-4C24-4646-A3FB-35DA17247917}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmem", "libpmem\libpmem.vcxproj", "{9E9E3D25-2139-4A5D-9200-18148DDEAD45}" ProjectSection(ProjectDependencies) = postProject {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "manpage", "examples\libpmemlog\manpage.vcxproj", "{9FF51F3E-AF36-4F45-A797-C5F03A090298}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmalloc_mt", "test\obj_pmalloc_mt\obj_pmalloc_mt.vcxproj", "{9FF62356-30B4-42A1-8DC7-45262A18DD44}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmem2", "libpmem2", "{A14A4556-9092-430D-B9CA-B2B1223D56CB}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ddmap", "test\tools\ddmap\ddmap.vcxproj", "{A216BF23-FC5C-4426-BF20-8568A2AA5FA0}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_test", "test\libpmempool_api\libpmempool_test.vcxproj", "{A2A0FAEA-2B7C-4FC3-B904-1DB4DEACF88D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_nblock", "test\blk_nblock\blk_nblock.vcxproj", "{A38EFCDB-53D6-4474-97F3-0DDC6CE70D76}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {F7C6C6B6-4142-4C82-8699-4A9D8183181B} = {F7C6C6B6-4142-4C82-8699-4A9D8183181B} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_check_version", "test\libpmempool_check_version\libpmempool_check_version.vcxproj", "{A39D1640-8DBA-450D-9103-2533C248991A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "out_err_win", "test\out_err_win\out_err_win.vcxproj", "{A57D9365-172E-4782-ADC6-82A594E30943}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ctl_alloc_class_config", "test\obj_ctl_alloc_class_config\obj_ctl_alloc_class_config.vcxproj", "{A79E3093-B157-4B09-BABD-29266EA16407}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "addlog", "examples\libpmemlog\logfile\addlog.vcxproj", "{A7CA7975-CEDB-48E6-9AEB-1209DCBD07F2}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_include", "test\obj_include\obj_include.vcxproj", "{AB15A115-E429-4123-BEBF-206FBA4CF615}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_sync", "test\libpmempool_sync\libpmempool_sync.vcxproj", "{AE1C32FB-9B52-4760-ABFC-0D2FA2C7A6C8}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_defrag_advanced", "test\obj_defrag_advanced\obj_defrag_advanced.vcxproj", "{AE952763-5C84-43FC-B344-CACC950F056C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "signal_handle", "test\signal_handle\signal_handle.vcxproj", "{AE9E908D-BAEC-491F-9914-436B3CE35E94}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ctl_config", "test\obj_ctl_config\obj_ctl_config.vcxproj", "{AEAA72CD-E060-417C-9CA1-49B4738384E0}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_feature", "test\pmempool_feature\pmempool_feature.vcxproj", "{AF038868-2432-4159-A62F-941F11D12C5D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "checksum", "test\checksum\checksum.vcxproj", "{AF0B7480-EBE3-486B-B0C8-134910BC9324}" ProjectSection(ProjectDependencies) = postProject {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_transform_win", "test\libpmempool_transform_win\libpmempool_transform_win.vcxproj", "{B30C6212-A160-405A-8FE7-340E721738A2}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmemwrite", "test\tools\pmemwrite\pmemwrite.vcxproj", "{B35BFA09-DE68-483B-AB61-8790E8F060A8}" ProjectSection(ProjectDependencies) = postProject {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_memset", "test\pmem_memset\pmem_memset.vcxproj", "{B36F115C-8139-4C35-A3E7-E6BF9F3DA793}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ctl_heap_size", "test\obj_ctl_heap_size\obj_ctl_heap_size.vcxproj", "{B379539C-E130-460D-AE82-4EBDD1A97845}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_mem", "test\obj_mem\obj_mem.vcxproj", "{B3AF8A19-5802-4A34-9157-27BBE4E53C0A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ex_linkedlist", "test\ex_linkedlist\ex_linkedlist.vcxproj", "{B440BB05-37A8-42EA-98D3-D83EB113E497}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem2_include", "test\pmem2_include\pmem2_include.vcxproj", "{B6C0521B-EECA-47EF-BFA8-147F9C3F6DFF}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_sync_win", "test\libpmempool_sync_win\libpmempool_sync_win.vcxproj", "{B6DA6617-D98F-4A4D-A7C4-A317212924BF}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "tools_pmemspoil", "test\pmemspoil\pmemspoil.vcxproj", "{B6F4B85D-FE55-4A1B-AE97-D4A9ECFE195F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pool_win", "test\obj_pool_win\obj_pool_win.vcxproj", "{B775480C-5B32-4F64-B026-47367280EC56}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "windows", "windows", "{B870D8A6-12CD-4DD0-B843-833695C2310A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_list_recovery", "test\obj_list_recovery\obj_list_recovery.vcxproj", "{B887EA26-846C-4D6A-B0E4-432487506BC7}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemlog", "libpmemlog", "{B8A4320D-E9A3-4F89-A8AA-B16D746C158A}" ProjectSection(SolutionItems) = preProject ..\doc\libpmemlog\libpmemlog.7.md = ..\doc\libpmemlog\libpmemlog.7.md ..\doc\libpmemlog\pmemlog_append.3.md = ..\doc\libpmemlog\pmemlog_append.3.md ..\doc\libpmemlog\pmemlog_create.3.md = ..\doc\libpmemlog\pmemlog_create.3.md ..\doc\libpmemlog\pmemlog_nbyte.3.md = ..\doc\libpmemlog\pmemlog_nbyte.3.md ..\doc\libpmemlog\pmemlog_tell.3.md = ..\doc\libpmemlog\pmemlog_tell.3.md EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_first_next", "test\obj_first_next\obj_first_next.vcxproj", "{BABC6427-E533-4DCF-91E3-B5B2ED253F46}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_list_move", "test\obj_list_move\obj_list_move.vcxproj", "{BAE107BA-7618-4972-8188-2D3CDAAE0453}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_layout", "test\obj_layout\obj_layout.vcxproj", "{BB1120CF-B721-4EF9-8735-58F76AE51D2F}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mapcli", "examples\libpmemobj\map\mapcli.vcxproj", "{BB248BAC-6E1B-433C-A254-75140A273AB5}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "map", "map", "{BD6CC700-B36B-435B-BAF9-FC5AFCD766C9}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ctree_map", "examples\libpmemobj\tree_map\ctree_map.vcxproj", "{BE18F227-A9F0-4B38-B689-4E2F9F09CA5F}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemblk", "libpmemblk", "{BFBAB433-860E-4A28-96E3-A4B7AFE3B297}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "string_store", "string_store", "{BFEDF709-A700-4769-9056-ACA934D828A8}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "scope", "test\scope\scope.vcxproj", "{C0E811E0-8942-4CFD-A817-74D99E9E6577}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_list_insert", "test\obj_list_insert\obj_list_insert.vcxproj", "{C2C36D03-26EE-4BD8-8FFC-86CFE16C1218}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_strdup", "test\obj_strdup\obj_strdup.vcxproj", "{C2F94489-A483-4C44-B8A7-11A75F6AEC66}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ulog_size", "test\obj_ulog_size\obj_ulog_size.vcxproj", "{C35052AF-2383-4F9C-B18B-55A01829F2BF}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "printlog", "examples\libpmemlog\logfile\printlog.vcxproj", "{C3CEE34C-29E0-4A22-B258-3FBAF662AA19}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_sync", "test\pmempool_sync\pmempool_sync.vcxproj", "{C5E8B8DB-2507-4904-847F-A52196B075F0}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "log_pool_win", "test\log_pool_win\log_pool_win.vcxproj", "{C71DAF3E-9361-4723-93E2-C475D1D0C0D0}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemblk", "libpmemblk", "{C721EFBD-45DC-479E-9B99-E62FCC1FC6E5}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "asset_load", "examples\libpmemblk\assetdb\asset_load.vcxproj", "{C7E42AE1-052F-4024-B8BA-DE5DCE6BBEEC}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_is_absolute", "test\util_is_absolute\util_is_absolute.vcxproj", "{C973CD39-D63B-4F5C-BE1D-DED17388B5A4}" ProjectSection(ProjectDependencies) = postProject {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "traces", "test\traces\traces.vcxproj", "{CA4BBB24-D33E-42E2-A495-F10D80DE8C1D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmembench", "benchmarks\pmembench.vcxproj", "{CB906E89-1313-4929-AFF7-86FBF1CC301F}" ProjectSection(ProjectDependencies) = postProject {99F7F00F-1DE5-45EA-992B-64BA282FAC76} = {99F7F00F-1DE5-45EA-992B-64BA282FAC76} {67AC1343-98FD-4143-92C0-559C55F749F5} = {67AC1343-98FD-4143-92C0-559C55F749F5} {5F2B687A-1B42-439C-AEEC-135DD22FB851} = {5F2B687A-1B42-439C-AEEC-135DD22FB851} {CF9A0883-6334-44C7-AC29-349468C78E27} = {CF9A0883-6334-44C7-AC29-349468C78E27} {7DC3B3DD-73ED-4602-9AF3-8D7053620DEA} = {7DC3B3DD-73ED-4602-9AF3-8D7053620DEA} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_lane", "test\obj_lane\obj_lane.vcxproj", "{CCA9B681-D10B-45E4-98CC-531503D2EDE8}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_check", "test\pmempool_check\pmempool_check.vcxproj", "{CDD9DFC6-5C3D-42F7-B822-FE29A1C21752}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libut", "test\unittest\libut.vcxproj", "{CE3F2DFB-8470-4802-AD37-21CAF6CB2681}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool", "libpmempool\libpmempool.vcxproj", "{CF9A0883-6334-44C7-AC29-349468C78E27}" ProjectSection(ProjectDependencies) = postProject {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_zones", "test\obj_zones\obj_zones.vcxproj", "{CF9F4CEA-EC66-4E78-A086-107EB29E0637}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "simple_copy", "examples\libpmem\simple_copy.vcxproj", "{D062166F-0EC7-4C13-A772-0C7157EEFE41}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_pmalloc_rand_mt", "test\obj_pmalloc_rand_mt\obj_pmalloc_rand_mt.vcxproj", "{D140560D-FDEC-4D3D-8F58-BF5FD5E4DAA1}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "tools_pmemobjcli", "test\pmemobjcli\pmemobjcli.vcxproj", "{D28F5FF6-8401-4E0D-94F9-3A1FD7ED64E3}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmemobjcli", "test\tools\pmemobjcli\pmemobjcli.vcxproj", "{D2C30C7E-A7D3-487A-956E-418CECAFFE8E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "fifo", "examples\libpmemobj\linkedlist\fifo.vcxproj", "{D3A99F36-4B72-4766-ABCD-CCEDC26DD139}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmempool_help", "test\pmempool_help\pmempool_help.vcxproj", "{D4035736-1AD6-4100-9FA9-A8A0C1DAE0C7}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_file_create", "test\util_file_create\util_file_create.vcxproj", "{D829DB63-E046-474D-8EA3-43A6659294D8}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ex_libpmemlog", "test\ex_libpmemlog\ex_libpmemlog.vcxproj", "{D8317F1D-7A70-4A39-977A-EAB05A04A87B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_lock", "test\obj_tx_lock\obj_tx_lock.vcxproj", "{D88187D2-1977-4C5F-B0CD-83C69BD6C1BC}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "hashmap_tx", "examples\libpmemobj\hashmap\hashmap_tx.vcxproj", "{D93A2683-6D99-4F18-B378-91195D23E007}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem2_map", "test\pmem2_map\pmem2_map.vcxproj", "{D9A70E35-0C85-4A09-ACA8-B15B21B66F50}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "blk_recovery", "test\blk_recovery\blk_recovery.vcxproj", "{DB68AB21-510B-4BA1-9E6F-E5731D8647BC}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem2_config", "test\pmem2_config\pmem2_config.vcxproj", "{DE068BE1-A8E9-48A2-B216-92A7CE5EA4CE}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_has_auto_flush_win", "test\pmem_has_auto_flush_win\pmem_has_auto_flush_win.vcxproj", "{DEA3CD0A-8781-4ABE-9A7D-00B91132FED0}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_ctl_alloc_class", "test\obj_ctl_alloc_class\obj_ctl_alloc_class.vcxproj", "{E07C9A5F-B2E4-44FB-AA87-FBC885AC955D}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "examples", "examples", "{E23BB160-006E-44F2-8FB4-3A2240BBC20C}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "string_store_tx_type", "string_store_tx_type", "{E3229AF7-1FA2-4632-BB0B-B74F709F1A33}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "pmem_is_pmem", "test\pmem_is_pmem\pmem_is_pmem.vcxproj", "{E4E2EC33-7902-45D0-9C3C-ADBAFA46874A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_poolset_foreach", "test\util_poolset_foreach\util_poolset_foreach.vcxproj", "{E648732D-78FA-427A-928C-9A59222D37B7}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "log_pool_lock", "test\log_pool_lock\log_pool_lock.vcxproj", "{E68DEB59-C709-4945-AF80-EEBCADDED944}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_constructor", "test\obj_constructor\obj_constructor.vcxproj", "{E7691F81-86EF-467D-82E1-F5B9416386F9}" ProjectSection(ProjectDependencies) = postProject {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_ctl", "test\util_ctl\util_ctl.vcxproj", "{E796AA20-D664-4D05-ABD9-C93A4FBE3E5C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_bttdev", "test\libpmempool_bttdev\libpmempool_bttdev.vcxproj", "{E85E017F-04C0-4716-BF21-949C82C68912}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "log_recovery", "test\log_recovery\log_recovery.vcxproj", "{E901B756-EA72-4B8D-967F-85F109D0D1DE}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_locks", "test\obj_tx_locks\obj_tx_locks.vcxproj", "{E9E079D6-25BF-46E3-8075-7D733303DD59}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_map_flog", "test\libpmempool_map_flog\libpmempool_map_flog.vcxproj", "{ED2A831F-4AAF-4CF7-A953-3C45B0EC1BE6}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "manpage", "examples\libpmemobj\manpage.vcxproj", "{EDA88BAB-9FA7-4A2D-8974-EFCFA24B3FEB}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_sds", "test\obj_sds\obj_sds.vcxproj", "{EDD5FA29-69AF-445F-842A-132E65D3C92B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "win_mmap_dtor", "test\win_mmap_dtor\win_mmap_dtor.vcxproj", "{F03DABEE-A03E-4437-BFD3-D012836F2D94}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Tools", "Tools", "{F09A0864-9221-47AD-872F-D4538104D747}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_is_poolset", "test\util_is_poolset\util_is_poolset.vcxproj", "{F0B613C4-1D9A-4259-BD0E-C1B9FF2AA3A0}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "win_signal", "test\win_signal\win_signal.vcxproj", "{F13108C4-4C86-4D56-A317-A4E5892A8AF7}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Documentation", "Documentation", "{F18C84B3-7898-4324-9D75-99A6048F442D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_tx_add_range", "test\obj_tx_add_range\obj_tx_add_range.vcxproj", "{F3E5650D-834E-45E6-90C7-3FC2AA954929}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmemobj", "libpmemobj", "{F42C09CD-ABA5-4DA9-8383-5EA40FA4D763}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmem2", "libpmem2\libpmem2.vcxproj", "{F596C36C-5C96-4F08-B420-8908AF500954}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "writer", "examples\libpmemobj\string_store\writer.vcxproj", "{F5D850C9-D353-4B84-99BC-E336C231018C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "hashmap_rp", "examples\libpmemobj\hashmap\hashmap_rp.vcxproj", "{F5E2F6C4-19BA-497A-B754-232E4666E647}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "hashmap_atomic", "examples\libpmemobj\hashmap\hashmap_atomic.vcxproj", "{F5E2F6C4-19BA-497A-B754-232E469BE647}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "ex_libpmemobj", "test\ex_libpmemobj\ex_libpmemobj.vcxproj", "{F63FB47F-1DCE-48E5-9CBD-F3E0A354472B}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmemblk", "libpmemblk\libpmemblk.vcxproj", "{F7C6C6B6-4142-4C82-8699-4A9D8183181B}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {492BAA3D-0D5D-478E-9765-500463AE69AA} = {492BAA3D-0D5D-478E-9765-500463AE69AA} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} EndProjectSection EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "libpmem", "libpmem", "{F8373EDD-1B9E-462D-BF23-55638E23E98B}" EndProject Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Utils", "Utils", "{F8CCA5AE-2D75-4C79-BEAB-2588CD5956C8}" ProjectSection(SolutionItems) = preProject ..\appveyor.yml = ..\appveyor.yml ..\utils\CHECK_WHITESPACE.PS1 = ..\utils\CHECK_WHITESPACE.PS1 ..\utils\CREATE-ZIP.PS1 = ..\utils\CREATE-ZIP.PS1 ..\utils\cstyle = ..\utils\cstyle ..\utils\CSTYLE.ps1 = ..\utils\CSTYLE.ps1 EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libpmempool_transform", "test\libpmempool_transform\libpmempool_transform.vcxproj", "{FB2D2B18-E616-4639-8593-0E1AF2DA01A8}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "log_basic", "test\log_basic\log_basic.vcxproj", "{FBB77433-639E-42DC-9355-EA94CAE294D2}" ProjectSection(ProjectDependencies) = postProject {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {9E9E3D25-2139-4A5D-9200-18148DDEAD45} {0B1818EB-BDC8-4865-964F-DB8BF05CFD86} = {0B1818EB-BDC8-4865-964F-DB8BF05CFD86} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} EndProjectSection EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_root", "test\obj_root\obj_root.vcxproj", "{FC2248F5-3E9E-495B-9767-87F59614047C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "manpage", "examples\libpmem\manpage.vcxproj", "{FCD0587A-4504-4F5E-8E9C-468CC03D250A}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_vec", "test\util_vec\util_vec.vcxproj", "{FD726AA3-D4FA-4597-B435-08CC7752888C}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_is_zeroed", "test\util_is_zeroed\util_is_zeroed.vcxproj", "{FD726AA3-D4FA-4597-B435-08CC7752888D}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "util_vecq", "test\util_vecq\util_vecq.vcxproj", "{FD726AA3-D4FA-4597-B435-08CC7752888E}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "mmap_fixed", "test\mmap_fixed\mmap_fixed.vcxproj", "{FEA09B48-34C2-4963-8A5A-F97BDA136D72}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_list_remove", "test\obj_list_remove\obj_list_remove.vcxproj", "{FF6E5B0C-DC00-4C93-B9C2-63D1E858BA79}" EndProject Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "obj_defrag", "test\obj_defrag\obj_defrag.vcxproj", "{FF6E5B0C-DC00-4C93-B9C2-63D1E858BA80}" EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|x64 = Debug|x64 Release|x64 = Release|x64 EndGlobalSection GlobalSection(ProjectConfigurationPlatforms) = postSolution {0056B0B6-CB3E-4F0E-B6DC-48D59CB8E235}.Debug|x64.ActiveCfg = Debug|x64 {0056B0B6-CB3E-4F0E-B6DC-48D59CB8E235}.Debug|x64.Build.0 = Debug|x64 {0056B0B6-CB3E-4F0E-B6DC-48D59CB8E235}.Release|x64.ActiveCfg = Release|x64 {0056B0B6-CB3E-4F0E-B6DC-48D59CB8E235}.Release|x64.Build.0 = Release|x64 {019F5586-5558-4C87-B319-85906D4AE407}.Debug|x64.ActiveCfg = Debug|x64 {019F5586-5558-4C87-B319-85906D4AE407}.Debug|x64.Build.0 = Debug|x64 {019F5586-5558-4C87-B319-85906D4AE407}.Release|x64.ActiveCfg = Release|x64 {019F5586-5558-4C87-B319-85906D4AE407}.Release|x64.Build.0 = Release|x64 {025E7D51-41F2-4CBA-956E-C37A4443DB1B}.Debug|x64.ActiveCfg = Debug|x64 {025E7D51-41F2-4CBA-956E-C37A4443DB1B}.Debug|x64.Build.0 = Debug|x64 {025E7D51-41F2-4CBA-956E-C37A4443DB1B}.Release|x64.ActiveCfg = Release|x64 {025E7D51-41F2-4CBA-956E-C37A4443DB1B}.Release|x64.Build.0 = Release|x64 {0287C3DC-AE03-4714-AAFF-C52F062ECA6F}.Debug|x64.ActiveCfg = Debug|x64 {0287C3DC-AE03-4714-AAFF-C52F062ECA6F}.Debug|x64.Build.0 = Debug|x64 {0287C3DC-AE03-4714-AAFF-C52F062ECA6F}.Release|x64.ActiveCfg = Release|x64 {0287C3DC-AE03-4714-AAFF-C52F062ECA6F}.Release|x64.Build.0 = Release|x64 {02BC3B44-C7F1-4793-86C1-6F36CA8A7F53}.Debug|x64.ActiveCfg = Debug|x64 {02BC3B44-C7F1-4793-86C1-6F36CA8A7F53}.Debug|x64.Build.0 = Debug|x64 {02BC3B44-C7F1-4793-86C1-6F36CA8A7F53}.Release|x64.ActiveCfg = Release|x64 {03228F84-4F41-4BCC-8C2D-F329DC87B289}.Debug|x64.ActiveCfg = Debug|x64 {03228F84-4F41-4BCC-8C2D-F329DC87B289}.Debug|x64.Build.0 = Debug|x64 {03228F84-4F41-4BCC-8C2D-F329DC87B289}.Release|x64.ActiveCfg = Release|x64 {03228F84-4F41-4BCC-8C2D-F329DC87B289}.Release|x64.Build.0 = Release|x64 {0388E945-A655-41A7-AF27-8981CEE0E49A}.Debug|x64.ActiveCfg = Debug|x64 {0388E945-A655-41A7-AF27-8981CEE0E49A}.Debug|x64.Build.0 = Debug|x64 {0388E945-A655-41A7-AF27-8981CEE0E49A}.Release|x64.ActiveCfg = Release|x64 {0388E945-A655-41A7-AF27-8981CEE0E49A}.Release|x64.Build.0 = Release|x64 {03B54A12-7793-4827-B820-C07491F7F45E}.Debug|x64.ActiveCfg = Debug|x64 {03B54A12-7793-4827-B820-C07491F7F45E}.Debug|x64.Build.0 = Debug|x64 {03B54A12-7793-4827-B820-C07491F7F45E}.Release|x64.ActiveCfg = Release|x64 {03B54A12-7793-4827-B820-C07491F7F45E}.Release|x64.Build.0 = Release|x64 {0529575C-F6E8-44FD-BB82-82A29948D0F2}.Debug|x64.ActiveCfg = Debug|x64 {0529575C-F6E8-44FD-BB82-82A29948D0F2}.Debug|x64.Build.0 = Debug|x64 {0529575C-F6E8-44FD-BB82-82A29948D0F2}.Release|x64.ActiveCfg = Release|x64 {0529575C-F6E8-44FD-BB82-82A29948D0F2}.Release|x64.Build.0 = Release|x64 {063037B2-CA35-4520-811C-19D9C4ED891E}.Debug|x64.ActiveCfg = Debug|x64 {063037B2-CA35-4520-811C-19D9C4ED891E}.Debug|x64.Build.0 = Debug|x64 {063037B2-CA35-4520-811C-19D9C4ED891E}.Release|x64.ActiveCfg = Release|x64 {063037B2-CA35-4520-811C-19D9C4ED891E}.Release|x64.Build.0 = Release|x64 {06877FED-15BA-421F-85C9-1A964FB97446}.Debug|x64.ActiveCfg = Debug|x64 {06877FED-15BA-421F-85C9-1A964FB97446}.Debug|x64.Build.0 = Debug|x64 {06877FED-15BA-421F-85C9-1A964FB97446}.Release|x64.ActiveCfg = Release|x64 {06877FED-15BA-421F-85C9-1A964FB97446}.Release|x64.Build.0 = Release|x64 {0703E813-9CC8-4DEA-AA33-42B099CD172D}.Debug|x64.ActiveCfg = Debug|x64 {0703E813-9CC8-4DEA-AA33-42B099CD172D}.Debug|x64.Build.0 = Debug|x64 {0703E813-9CC8-4DEA-AA33-42B099CD172D}.Release|x64.ActiveCfg = Release|x64 {0703E813-9CC8-4DEA-AA33-42B099CD172D}.Release|x64.Build.0 = Release|x64 {07A153D9-DF17-4DE8-A3C2-EBF171B961AE}.Debug|x64.ActiveCfg = Debug|x64 {07A153D9-DF17-4DE8-A3C2-EBF171B961AE}.Debug|x64.Build.0 = Debug|x64 {07A153D9-DF17-4DE8-A3C2-EBF171B961AE}.Release|x64.ActiveCfg = Release|x64 {07A153D9-DF17-4DE8-A3C2-EBF171B961AE}.Release|x64.Build.0 = Release|x64 {08B62E36-63D2-4FF1-A605-4BBABAEE73FB}.Debug|x64.ActiveCfg = Debug|x64 {08B62E36-63D2-4FF1-A605-4BBABAEE73FB}.Debug|x64.Build.0 = Debug|x64 {08B62E36-63D2-4FF1-A605-4BBABAEE73FB}.Release|x64.ActiveCfg = Release|x64 {08B62E36-63D2-4FF1-A605-4BBABAEE73FB}.Release|x64.Build.0 = Release|x64 {0A049EAD-652F-4E20-8026-90FD99AEE77A}.Debug|x64.ActiveCfg = Debug|x64 {0A049EAD-652F-4E20-8026-90FD99AEE77A}.Debug|x64.Build.0 = Debug|x64 {0A049EAD-652F-4E20-8026-90FD99AEE77A}.Release|x64.ActiveCfg = Release|x64 {0A049EAD-652F-4E20-8026-90FD99AEE77A}.Release|x64.Build.0 = Release|x64 {0B1818EB-BDC8-4865-964F-DB8BF05CFD86}.Debug|x64.ActiveCfg = Debug|x64 {0B1818EB-BDC8-4865-964F-DB8BF05CFD86}.Debug|x64.Build.0 = Debug|x64 {0B1818EB-BDC8-4865-964F-DB8BF05CFD86}.Release|x64.ActiveCfg = Release|x64 {0B1818EB-BDC8-4865-964F-DB8BF05CFD86}.Release|x64.Build.0 = Release|x64 {0BFD78AA-FD94-4DB1-8495-8F5CC06D8F03}.Debug|x64.ActiveCfg = Debug|x64 {0BFD78AA-FD94-4DB1-8495-8F5CC06D8F03}.Debug|x64.Build.0 = Debug|x64 {0BFD78AA-FD94-4DB1-8495-8F5CC06D8F03}.Release|x64.ActiveCfg = Release|x64 {0BFD78AA-FD94-4DB1-8495-8F5CC06D8F03}.Release|x64.Build.0 = Release|x64 {0CDCEB97-3270-4939-A290-EA2D3BE34B0C}.Debug|x64.ActiveCfg = Debug|x64 {0CDCEB97-3270-4939-A290-EA2D3BE34B0C}.Debug|x64.Build.0 = Debug|x64 {0CDCEB97-3270-4939-A290-EA2D3BE34B0C}.Release|x64.ActiveCfg = Release|x64 {0CDCEB97-3270-4939-A290-EA2D3BE34B0C}.Release|x64.Build.0 = Release|x64 {0D4E38EF-A9D5-4797-8994-5DBB1125C9EA}.Debug|x64.ActiveCfg = Debug|x64 {0D4E38EF-A9D5-4797-8994-5DBB1125C9EA}.Debug|x64.Build.0 = Debug|x64 {0D4E38EF-A9D5-4797-8994-5DBB1125C9EA}.Release|x64.ActiveCfg = Release|x64 {0D4E38EF-A9D5-4797-8994-5DBB1125C9EA}.Release|x64.Build.0 = Release|x64 {0FB8F0FD-276C-413B-97A8-67ABE0C9043B}.Debug|x64.ActiveCfg = Debug|x64 {0FB8F0FD-276C-413B-97A8-67ABE0C9043B}.Debug|x64.Build.0 = Debug|x64 {0FB8F0FD-276C-413B-97A8-67ABE0C9043B}.Release|x64.ActiveCfg = Release|x64 {0FB8F0FD-276C-413B-97A8-67ABE0C9043B}.Release|x64.Build.0 = Release|x64 {10469175-EEF7-44A0-9961-AC4E45EFD800}.Debug|x64.ActiveCfg = Debug|x64 {10469175-EEF7-44A0-9961-AC4E45EFD800}.Debug|x64.Build.0 = Debug|x64 {10469175-EEF7-44A0-9961-AC4E45EFD800}.Release|x64.ActiveCfg = Release|x64 {10469175-EEF7-44A0-9961-AC4E45EFD800}.Release|x64.Build.0 = Release|x64 {11D76FBC-DFAA-4B31-9DB0-206E171E3F94}.Debug|x64.ActiveCfg = Debug|x64 {11D76FBC-DFAA-4B31-9DB0-206E171E3F94}.Debug|x64.Build.0 = Debug|x64 {11D76FBC-DFAA-4B31-9DB0-206E171E3F94}.Release|x64.ActiveCfg = Release|x64 {11D76FBC-DFAA-4B31-9DB0-206E171E3F94}.Release|x64.Build.0 = Release|x64 {11E158AE-C85A-4A6E-B66A-ED2994709276}.Debug|x64.ActiveCfg = Debug|x64 {11E158AE-C85A-4A6E-B66A-ED2994709276}.Debug|x64.Build.0 = Debug|x64 {11E158AE-C85A-4A6E-B66A-ED2994709276}.Release|x64.ActiveCfg = Release|x64 {11E158AE-C85A-4A6E-B66A-ED2994709276}.Release|x64.Build.0 = Release|x64 {12A1A3EF-202C-4DD0-9B5A-F5126CAB078F}.Debug|x64.ActiveCfg = Debug|x64 {12A1A3EF-202C-4DD0-9B5A-F5126CAB078F}.Debug|x64.Build.0 = Debug|x64 {12A1A3EF-202C-4DD0-9B5A-F5126CAB078F}.Release|x64.ActiveCfg = Release|x64 {12A1A3EF-202C-4DD0-9B5A-F5126CAB078F}.Release|x64.Build.0 = Release|x64 {1464398A-100F-4518-BDB9-939A6362B6CF}.Debug|x64.ActiveCfg = Debug|x64 {1464398A-100F-4518-BDB9-939A6362B6CF}.Debug|x64.Build.0 = Debug|x64 {1464398A-100F-4518-BDB9-939A6362B6CF}.Release|x64.ActiveCfg = Release|x64 {1464398A-100F-4518-BDB9-939A6362B6CF}.Release|x64.Build.0 = Release|x64 {179BEB5A-2C90-44F5-A734-FA756A5E668C}.Debug|x64.ActiveCfg = Debug|x64 {179BEB5A-2C90-44F5-A734-FA756A5E668C}.Debug|x64.Build.0 = Debug|x64 {179BEB5A-2C90-44F5-A734-FA756A5E668C}.Release|x64.ActiveCfg = Release|x64 {179BEB5A-2C90-44F5-A734-FA756A5E668C}.Release|x64.Build.0 = Release|x64 {17A4B817-68B1-4719-A9EF-BD8FAB747DE6}.Debug|x64.ActiveCfg = Debug|x64 {17A4B817-68B1-4719-A9EF-BD8FAB747DE6}.Debug|x64.Build.0 = Debug|x64 {17A4B817-68B1-4719-A9EF-BD8FAB747DE6}.Release|x64.ActiveCfg = Release|x64 {17A4B817-68B1-4719-A9EF-BD8FAB747DE6}.Release|x64.Build.0 = Release|x64 {18E90E1A-F2E0-40DF-9900-A14E560C9EB4}.Debug|x64.ActiveCfg = Debug|x64 {18E90E1A-F2E0-40DF-9900-A14E560C9EB4}.Debug|x64.Build.0 = Debug|x64 {18E90E1A-F2E0-40DF-9900-A14E560C9EB4}.Release|x64.ActiveCfg = Release|x64 {18E90E1A-F2E0-40DF-9900-A14E560C9EB4}.Release|x64.Build.0 = Release|x64 {1B871BA2-3F70-4BC9-9DF4-725EB07F6628}.Debug|x64.ActiveCfg = Debug|x64 {1B871BA2-3F70-4BC9-9DF4-725EB07F6628}.Debug|x64.Build.0 = Debug|x64 {1B871BA2-3F70-4BC9-9DF4-725EB07F6628}.Release|x64.ActiveCfg = Release|x64 {1B871BA2-3F70-4BC9-9DF4-725EB07F6628}.Release|x64.Build.0 = Release|x64 {1B9B0D6D-E530-44A6-ADAE-09EA2BDC47DE}.Debug|x64.ActiveCfg = Debug|x64 {1B9B0D6D-E530-44A6-ADAE-09EA2BDC47DE}.Debug|x64.Build.0 = Debug|x64 {1B9B0D6D-E530-44A6-ADAE-09EA2BDC47DE}.Release|x64.ActiveCfg = Release|x64 {1B9B0D6D-E530-44A6-ADAE-09EA2BDC47DE}.Release|x64.Build.0 = Release|x64 {1BAA1617-93AE-4196-8A1A-BD492FB18AEF}.Debug|x64.ActiveCfg = Debug|x64 {1BAA1617-93AE-4196-8A1A-BD492FB18AEF}.Debug|x64.Build.0 = Debug|x64 {1BAA1617-93AE-4196-8A1A-BD492FB18AEF}.Release|x64.ActiveCfg = Release|x64 {1BAA1617-93AE-4196-8A1A-BD492FB18AEF}.Release|x64.Build.0 = Release|x64 {1EB3DE5B-6357-498D-8CAC-EEC0209EA454}.Debug|x64.ActiveCfg = Debug|x64 {1EB3DE5B-6357-498D-8CAC-EEC0209EA454}.Debug|x64.Build.0 = Debug|x64 {1EB3DE5B-6357-498D-8CAC-EEC0209EA454}.Release|x64.ActiveCfg = Release|x64 {1EB3DE5B-6357-498D-8CAC-EEC0209EA454}.Release|x64.Build.0 = Release|x64 {1F2E1C51-2B14-4047-BE6D-52E00FC3C780}.Debug|x64.ActiveCfg = Debug|x64 {1F2E1C51-2B14-4047-BE6D-52E00FC3C780}.Debug|x64.Build.0 = Debug|x64 {1F2E1C51-2B14-4047-BE6D-52E00FC3C780}.Release|x64.ActiveCfg = Release|x64 {1F2E1C51-2B14-4047-BE6D-52E00FC3C780}.Release|x64.Build.0 = Release|x64 {2498FCDA-E2CC-43EF-9A35-8CD63F253171}.Debug|x64.ActiveCfg = Debug|x64 {2498FCDA-E2CC-43EF-9A35-8CD63F253171}.Debug|x64.Build.0 = Debug|x64 {2498FCDA-E2CC-43EF-9A35-8CD63F253171}.Release|x64.ActiveCfg = Release|x64 {2498FCDA-E2CC-43EF-9A35-8CD63F253171}.Release|x64.Build.0 = Release|x64 {25758581-DD46-4AE4-99D9-11E736F72AD1}.Debug|x64.ActiveCfg = Debug|x64 {25758581-DD46-4AE4-99D9-11E736F72AD1}.Debug|x64.Build.0 = Debug|x64 {25758581-DD46-4AE4-99D9-11E736F72AD1}.Release|x64.ActiveCfg = Release|x64 {25758581-DD46-4AE4-99D9-11E736F72AD1}.Release|x64.Build.0 = Release|x64 {26166DF1-3C94-44AF-9075-BA31DCD2F6BB}.Debug|x64.ActiveCfg = Debug|x64 {26166DF1-3C94-44AF-9075-BA31DCD2F6BB}.Debug|x64.Build.0 = Debug|x64 {26166DF1-3C94-44AF-9075-BA31DCD2F6BB}.Release|x64.ActiveCfg = Release|x64 {26166DF1-3C94-44AF-9075-BA31DCD2F6BB}.Release|x64.Build.0 = Release|x64 {27FA11C6-431D-41D1-A417-FAB7C4F93DCA}.Debug|x64.ActiveCfg = Debug|x64 {27FA11C6-431D-41D1-A417-FAB7C4F93DCA}.Debug|x64.Build.0 = Debug|x64 {27FA11C6-431D-41D1-A417-FAB7C4F93DCA}.Release|x64.ActiveCfg = Release|x64 {27FA11C6-431D-41D1-A417-FAB7C4F93DCA}.Release|x64.Build.0 = Release|x64 {296F3C5D-3951-423E-8E2F-FD4A37958C72}.Debug|x64.ActiveCfg = Debug|x64 {296F3C5D-3951-423E-8E2F-FD4A37958C72}.Debug|x64.Build.0 = Debug|x64 {296F3C5D-3951-423E-8E2F-FD4A37958C72}.Release|x64.ActiveCfg = Release|x64 {296F3C5D-3951-423E-8E2F-FD4A37958C72}.Release|x64.Build.0 = Release|x64 {29D9376B-DC36-4940-83F1-A7CBE38A2103}.Debug|x64.ActiveCfg = Debug|x64 {29D9376B-DC36-4940-83F1-A7CBE38A2103}.Debug|x64.Build.0 = Debug|x64 {29D9376B-DC36-4940-83F1-A7CBE38A2103}.Release|x64.ActiveCfg = Release|x64 {29D9376B-DC36-4940-83F1-A7CBE38A2103}.Release|x64.Build.0 = Release|x64 {2A1D6AF2-7336-4966-A4B3-0BE9A24BAE00}.Debug|x64.ActiveCfg = Debug|x64 {2A1D6AF2-7336-4966-A4B3-0BE9A24BAE00}.Debug|x64.Build.0 = Debug|x64 {2A1D6AF2-7336-4966-A4B3-0BE9A24BAE00}.Release|x64.ActiveCfg = Release|x64 {2A1D6AF2-7336-4966-A4B3-0BE9A24BAE00}.Release|x64.Build.0 = Release|x64 {2B1A5104-A324-4D02-B5C7-D021FB8F880C}.Debug|x64.ActiveCfg = Debug|x64 {2B1A5104-A324-4D02-B5C7-D021FB8F880C}.Debug|x64.Build.0 = Debug|x64 {2B1A5104-A324-4D02-B5C7-D021FB8F880C}.Release|x64.ActiveCfg = Release|x64 {2B1A5104-A324-4D02-B5C7-D021FB8F880C}.Release|x64.Build.0 = Release|x64 {2B2DE575-1422-4FBF-97BE-35AEDA0AB465}.Debug|x64.ActiveCfg = Debug|x64 {2B2DE575-1422-4FBF-97BE-35AEDA0AB465}.Debug|x64.Build.0 = Debug|x64 {2B2DE575-1422-4FBF-97BE-35AEDA0AB465}.Release|x64.ActiveCfg = Release|x64 {2B2DE575-1422-4FBF-97BE-35AEDA0AB465}.Release|x64.Build.0 = Release|x64 {2B7772E6-9DAA-4F38-B0BC-7B2399366325}.Debug|x64.ActiveCfg = Debug|x64 {2B7772E6-9DAA-4F38-B0BC-7B2399366325}.Debug|x64.Build.0 = Debug|x64 {2B7772E6-9DAA-4F38-B0BC-7B2399366325}.Release|x64.ActiveCfg = Release|x64 {2B7772E6-9DAA-4F38-B0BC-7B2399366325}.Release|x64.Build.0 = Release|x64 {2CD7408E-2F60-43C3-ACEB-C7D58CDD8462}.Debug|x64.ActiveCfg = Debug|x64 {2CD7408E-2F60-43C3-ACEB-C7D58CDD8462}.Debug|x64.Build.0 = Debug|x64 {2CD7408E-2F60-43C3-ACEB-C7D58CDD8462}.Release|x64.ActiveCfg = Release|x64 {2CD7408E-2F60-43C3-ACEB-C7D58CDD8462}.Release|x64.Build.0 = Release|x64 {2DE6B085-3C19-49B1-894A-AD9376000E09}.Debug|x64.ActiveCfg = Debug|x64 {2DE6B085-3C19-49B1-894A-AD9376000E09}.Debug|x64.Build.0 = Debug|x64 {2DE6B085-3C19-49B1-894A-AD9376000E09}.Release|x64.ActiveCfg = Release|x64 {2DE6B085-3C19-49B1-894A-AD9376000E09}.Release|x64.Build.0 = Release|x64 {2ED26FDA-3C4E-4514-B387-5E77C302FF71}.Debug|x64.ActiveCfg = Debug|x64 {2ED26FDA-3C4E-4514-B387-5E77C302FF71}.Debug|x64.Build.0 = Debug|x64 {2ED26FDA-3C4E-4514-B387-5E77C302FF71}.Release|x64.ActiveCfg = Release|x64 {2ED26FDA-3C4E-4514-B387-5E77C302FF71}.Release|x64.Build.0 = Release|x64 {2EFFC590-BF5E-46A2-AF04-E67E1D571D2E}.Debug|x64.ActiveCfg = Debug|x64 {2EFFC590-BF5E-46A2-AF04-E67E1D571D2E}.Debug|x64.Build.0 = Debug|x64 {2EFFC590-BF5E-46A2-AF04-E67E1D571D2E}.Release|x64.ActiveCfg = Release|x64 {2EFFC590-BF5E-46A2-AF04-E67E1D571D2E}.Release|x64.Build.0 = Release|x64 {3142CB13-CADA-48D3-9A25-E6ACB243760A}.Debug|x64.ActiveCfg = Debug|x64 {3142CB13-CADA-48D3-9A25-E6ACB243760A}.Debug|x64.Build.0 = Debug|x64 {3142CB13-CADA-48D3-9A25-E6ACB243760A}.Release|x64.ActiveCfg = Release|x64 {3142CB13-CADA-48D3-9A25-E6ACB243760A}.Release|x64.Build.0 = Release|x64 {34DB4951-DA08-45F1-938D-B08E5FF5AB46}.Debug|x64.ActiveCfg = Debug|x64 {34DB4951-DA08-45F1-938D-B08E5FF5AB46}.Debug|x64.Build.0 = Debug|x64 {34DB4951-DA08-45F1-938D-B08E5FF5AB46}.Release|x64.ActiveCfg = Release|x64 {34DB4951-DA08-45F1-938D-B08E5FF5AB46}.Release|x64.Build.0 = Release|x64 {3799BA67-3C4F-4AE0-85DC-5BAAEA01A180}.Debug|x64.ActiveCfg = Debug|x64 {3799BA67-3C4F-4AE0-85DC-5BAAEA01A180}.Debug|x64.Build.0 = Debug|x64 {3799BA67-3C4F-4AE0-85DC-5BAAEA01A180}.Release|x64.ActiveCfg = Release|x64 {3799BA67-3C4F-4AE0-85DC-5BAAEA01A180}.Release|x64.Build.0 = Release|x64 {3B23831B-E5DE-4A62-9D0B-27D0D9F293F4}.Debug|x64.ActiveCfg = Debug|x64 {3B23831B-E5DE-4A62-9D0B-27D0D9F293F4}.Debug|x64.Build.0 = Debug|x64 {3B23831B-E5DE-4A62-9D0B-27D0D9F293F4}.Release|x64.ActiveCfg = Release|x64 {3B23831B-E5DE-4A62-9D0B-27D0D9F293F4}.Release|x64.Build.0 = Release|x64 {3CF270CD-0F56-48E3-AD84-82F369C568BF}.Debug|x64.ActiveCfg = Debug|x64 {3CF270CD-0F56-48E3-AD84-82F369C568BF}.Debug|x64.Build.0 = Debug|x64 {3CF270CD-0F56-48E3-AD84-82F369C568BF}.Release|x64.ActiveCfg = Release|x64 {3CF270CD-0F56-48E3-AD84-82F369C568BF}.Release|x64.Build.0 = Release|x64 {3EC30D6A-BDA4-4971-879A-8814204EAE31}.Debug|x64.ActiveCfg = Debug|x64 {3EC30D6A-BDA4-4971-879A-8814204EAE31}.Debug|x64.Build.0 = Debug|x64 {3EC30D6A-BDA4-4971-879A-8814204EAE31}.Release|x64.ActiveCfg = Release|x64 {3EC30D6A-BDA4-4971-879A-8814204EAE31}.Release|x64.Build.0 = Release|x64 {3ECCB0F1-3ADF-486A-91C5-79DF0FC22F78}.Debug|x64.ActiveCfg = Debug|x64 {3ECCB0F1-3ADF-486A-91C5-79DF0FC22F78}.Debug|x64.Build.0 = Debug|x64 {3ECCB0F1-3ADF-486A-91C5-79DF0FC22F78}.Release|x64.ActiveCfg = Release|x64 {3ECCB0F1-3ADF-486A-91C5-79DF0FC22F78}.Release|x64.Build.0 = Release|x64 {3ED56E55-84A6-422C-A8D4-A8439FB8F245}.Debug|x64.ActiveCfg = Debug|x64 {3ED56E55-84A6-422C-A8D4-A8439FB8F245}.Debug|x64.Build.0 = Debug|x64 {3ED56E55-84A6-422C-A8D4-A8439FB8F245}.Release|x64.ActiveCfg = Release|x64 {3ED56E55-84A6-422C-A8D4-A8439FB8F245}.Release|x64.Build.0 = Release|x64 {42B97D47-F800-4100-BFA2-B3AC357E8B6B}.Debug|x64.ActiveCfg = Debug|x64 {42B97D47-F800-4100-BFA2-B3AC357E8B6B}.Debug|x64.Build.0 = Debug|x64 {42B97D47-F800-4100-BFA2-B3AC357E8B6B}.Release|x64.ActiveCfg = Release|x64 {42B97D47-F800-4100-BFA2-B3AC357E8B6B}.Release|x64.Build.0 = Release|x64 {42CCEF95-5ADD-460C-967E-DD5B2C744943}.Debug|x64.ActiveCfg = Debug|x64 {42CCEF95-5ADD-460C-967E-DD5B2C744943}.Debug|x64.Build.0 = Debug|x64 {42CCEF95-5ADD-460C-967E-DD5B2C744943}.Release|x64.ActiveCfg = Release|x64 {42CCEF95-5ADD-460C-967E-DD5B2C744943}.Release|x64.Build.0 = Release|x64 {433F7840-C597-4950-84C9-E4FF7DF6A298}.Debug|x64.ActiveCfg = Debug|x64 {433F7840-C597-4950-84C9-E4FF7DF6A298}.Debug|x64.Build.0 = Debug|x64 {433F7840-C597-4950-84C9-E4FF7DF6A298}.Release|x64.ActiveCfg = Release|x64 {433F7840-C597-4950-84C9-E4FF7DF6A298}.Release|x64.Build.0 = Release|x64 {46B82069-10BE-432A-8D93-F4D995148555}.Debug|x64.ActiveCfg = Debug|x64 {46B82069-10BE-432A-8D93-F4D995148555}.Debug|x64.Build.0 = Debug|x64 {46B82069-10BE-432A-8D93-F4D995148555}.Release|x64.ActiveCfg = Release|x64 {46B82069-10BE-432A-8D93-F4D995148555}.Release|x64.Build.0 = Release|x64 {4850F425-9128-4E91-973C-5AE7BD97395B}.Debug|x64.ActiveCfg = Debug|x64 {4850F425-9128-4E91-973C-5AE7BD97395B}.Debug|x64.Build.0 = Debug|x64 {4850F425-9128-4E91-973C-5AE7BD97395B}.Release|x64.ActiveCfg = Release|x64 {4850F425-9128-4E91-973C-5AE7BD97395B}.Release|x64.Build.0 = Release|x64 {492BAA3D-0D5D-478E-9765-500463AE69AA}.Debug|x64.ActiveCfg = Debug|x64 {492BAA3D-0D5D-478E-9765-500463AE69AA}.Debug|x64.Build.0 = Debug|x64 {492BAA3D-0D5D-478E-9765-500463AE69AA}.Release|x64.ActiveCfg = Release|x64 {492BAA3D-0D5D-478E-9765-500463AE69AA}.Release|x64.Build.0 = Release|x64 {49A7CC5A-D5E7-4A07-917F-C6918B982BE8}.Debug|x64.ActiveCfg = Debug|x64 {49A7CC5A-D5E7-4A07-917F-C6918B982BE8}.Debug|x64.Build.0 = Debug|x64 {49A7CC5A-D5E7-4A07-917F-C6918B982BE8}.Release|x64.ActiveCfg = Release|x64 {49A7CC5A-D5E7-4A07-917F-C6918B982BE8}.Release|x64.Build.0 = Release|x64 {4C429783-0B01-449F-A36F-C2019233890B}.Debug|x64.ActiveCfg = Debug|x64 {4C429783-0B01-449F-A36F-C2019233890B}.Debug|x64.Build.0 = Debug|x64 {4C429783-0B01-449F-A36F-C2019233890B}.Release|x64.ActiveCfg = Release|x64 {4C429783-0B01-449F-A36F-C2019233890B}.Release|x64.Build.0 = Release|x64 {4C6E7F0A-7E6A-4713-B1D2-B7B4ADC992AF}.Debug|x64.ActiveCfg = Debug|x64 {4C6E7F0A-7E6A-4713-B1D2-B7B4ADC992AF}.Debug|x64.Build.0 = Debug|x64 {4C6E7F0A-7E6A-4713-B1D2-B7B4ADC992AF}.Release|x64.ActiveCfg = Release|x64 {4C6E7F0A-7E6A-4713-B1D2-B7B4ADC992AF}.Release|x64.Build.0 = Release|x64 {4E334022-7A71-4197-9E15-878F7EFC877E}.Debug|x64.ActiveCfg = Debug|x64 {4E334022-7A71-4197-9E15-878F7EFC877E}.Debug|x64.Build.0 = Debug|x64 {4E334022-7A71-4197-9E15-878F7EFC877E}.Release|x64.ActiveCfg = Release|x64 {4E334022-7A71-4197-9E15-878F7EFC877E}.Release|x64.Build.0 = Release|x64 {4FB4FF90-4E92-4CFB-A01F-C73D6861CA03}.Debug|x64.ActiveCfg = Debug|x64 {4FB4FF90-4E92-4CFB-A01F-C73D6861CA03}.Debug|x64.Build.0 = Debug|x64 {4FB4FF90-4E92-4CFB-A01F-C73D6861CA03}.Release|x64.ActiveCfg = Release|x64 {4FB4FF90-4E92-4CFB-A01F-C73D6861CA03}.Release|x64.Build.0 = Release|x64 {50FD1E47-2131-48D2-9435-5CB28DF6B15A}.Debug|x64.ActiveCfg = Debug|x64 {50FD1E47-2131-48D2-9435-5CB28DF6B15A}.Debug|x64.Build.0 = Debug|x64 {50FD1E47-2131-48D2-9435-5CB28DF6B15A}.Release|x64.ActiveCfg = Release|x64 {50FD1E47-2131-48D2-9435-5CB28DF6B15A}.Release|x64.Build.0 = Release|x64 {513C4CFA-BD5B-4470-BA93-F6D43778A754}.Debug|x64.ActiveCfg = Debug|x64 {513C4CFA-BD5B-4470-BA93-F6D43778A754}.Debug|x64.Build.0 = Debug|x64 {513C4CFA-BD5B-4470-BA93-F6D43778A754}.Release|x64.ActiveCfg = Release|x64 {513C4CFA-BD5B-4470-BA93-F6D43778A754}.Release|x64.Build.0 = Release|x64 {53115A01-460C-4339-A2C8-AE1323A6E7EA}.Debug|x64.ActiveCfg = Debug|x64 {53115A01-460C-4339-A2C8-AE1323A6E7EA}.Debug|x64.Build.0 = Debug|x64 {53115A01-460C-4339-A2C8-AE1323A6E7EA}.Release|x64.ActiveCfg = Release|x64 {53115A01-460C-4339-A2C8-AE1323A6E7EA}.Release|x64.Build.0 = Release|x64 {5580D11C-FDA6-4CF2-A0E8-1C2D3FBC11F1}.Debug|x64.ActiveCfg = Debug|x64 {5580D11C-FDA6-4CF2-A0E8-1C2D3FBC11F1}.Debug|x64.Build.0 = Debug|x64 {5580D11C-FDA6-4CF2-A0E8-1C2D3FBC11F1}.Release|x64.ActiveCfg = Release|x64 {5580D11C-FDA6-4CF2-A0E8-1C2D3FBC11F1}.Release|x64.Build.0 = Release|x64 {581B3A58-F3F0-4765-91E5-D0C82816A528}.Debug|x64.ActiveCfg = Debug|x64 {581B3A58-F3F0-4765-91E5-D0C82816A528}.Debug|x64.Build.0 = Debug|x64 {581B3A58-F3F0-4765-91E5-D0C82816A528}.Release|x64.ActiveCfg = Release|x64 {581B3A58-F3F0-4765-91E5-D0C82816A528}.Release|x64.Build.0 = Release|x64 {58386481-30B7-40FC-96AF-0723A4A7B228}.Debug|x64.ActiveCfg = Debug|x64 {58386481-30B7-40FC-96AF-0723A4A7B228}.Debug|x64.Build.0 = Debug|x64 {58386481-30B7-40FC-96AF-0723A4A7B228}.Release|x64.ActiveCfg = Release|x64 {58386481-30B7-40FC-96AF-0723A4A7B228}.Release|x64.Build.0 = Release|x64 {59D7A9CD-9912-40E4-96E1-8A873F777F62}.Debug|x64.ActiveCfg = Debug|x64 {59D7A9CD-9912-40E4-96E1-8A873F777F62}.Debug|x64.Build.0 = Debug|x64 {59D7A9CD-9912-40E4-96E1-8A873F777F62}.Release|x64.ActiveCfg = Release|x64 {59D7A9CD-9912-40E4-96E1-8A873F777F62}.Release|x64.Build.0 = Release|x64 {5A391A14-8E29-4788-93FC-EDADED31D32F}.Debug|x64.ActiveCfg = Debug|x64 {5A391A14-8E29-4788-93FC-EDADED31D32F}.Debug|x64.Build.0 = Debug|x64 {5A391A14-8E29-4788-93FC-EDADED31D32F}.Release|x64.ActiveCfg = Release|x64 {5A391A14-8E29-4788-93FC-EDADED31D32F}.Release|x64.Build.0 = Release|x64 {5AD07646-5E16-4CEF-B80A-BE5EE4D54FEF}.Debug|x64.ActiveCfg = Debug|x64 {5AD07646-5E16-4CEF-B80A-BE5EE4D54FEF}.Debug|x64.Build.0 = Debug|x64 {5AD07646-5E16-4CEF-B80A-BE5EE4D54FEF}.Release|x64.ActiveCfg = Release|x64 {5AD07646-5E16-4CEF-B80A-BE5EE4D54FEF}.Release|x64.Build.0 = Release|x64 {5B2B9C0D-1B6D-4357-8307-6DE1EE0A41A3}.Debug|x64.ActiveCfg = Debug|x64 {5B2B9C0D-1B6D-4357-8307-6DE1EE0A41A3}.Debug|x64.Build.0 = Debug|x64 {5B2B9C0D-1B6D-4357-8307-6DE1EE0A41A3}.Release|x64.ActiveCfg = Release|x64 {5B2B9C0D-1B6D-4357-8307-6DE1EE0A41A3}.Release|x64.Build.0 = Release|x64 {5D362DB7-D2BD-4907-AAD8-4B8627E72282}.Debug|x64.ActiveCfg = Debug|x64 {5D362DB7-D2BD-4907-AAD8-4B8627E72282}.Debug|x64.Build.0 = Debug|x64 {5D362DB7-D2BD-4907-AAD8-4B8627E72282}.Release|x64.ActiveCfg = Release|x64 {5D362DB7-D2BD-4907-AAD8-4B8627E72282}.Release|x64.Build.0 = Release|x64 {5DB2E259-0D19-4A89-B8EC-B2912F39924D}.Debug|x64.ActiveCfg = Debug|x64 {5DB2E259-0D19-4A89-B8EC-B2912F39924D}.Debug|x64.Build.0 = Debug|x64 {5DB2E259-0D19-4A89-B8EC-B2912F39924D}.Release|x64.ActiveCfg = Release|x64 {5DB2E259-0D19-4A89-B8EC-B2912F39924D}.Release|x64.Build.0 = Release|x64 {5E7305DB-93E6-448B-AE44-90EAF916A776}.Debug|x64.ActiveCfg = Debug|x64 {5E7305DB-93E6-448B-AE44-90EAF916A776}.Debug|x64.Build.0 = Debug|x64 {5E7305DB-93E6-448B-AE44-90EAF916A776}.Release|x64.ActiveCfg = Release|x64 {5E7305DB-93E6-448B-AE44-90EAF916A776}.Release|x64.Build.0 = Release|x64 {5EC35099-9777-45E8-9520-EB2EE75BDF88}.Debug|x64.ActiveCfg = Debug|x64 {5EC35099-9777-45E8-9520-EB2EE75BDF88}.Debug|x64.Build.0 = Debug|x64 {5EC35099-9777-45E8-9520-EB2EE75BDF88}.Release|x64.ActiveCfg = Release|x64 {5EC35099-9777-45E8-9520-EB2EE75BDF88}.Release|x64.Build.0 = Release|x64 {5F2B687A-1B42-439C-AEEC-135DD22FB851}.Debug|x64.ActiveCfg = Debug|x64 {5F2B687A-1B42-439C-AEEC-135DD22FB851}.Debug|x64.Build.0 = Debug|x64 {5F2B687A-1B42-439C-AEEC-135DD22FB851}.Release|x64.ActiveCfg = Release|x64 {5F2B687A-1B42-439C-AEEC-135DD22FB851}.Release|x64.Build.0 = Release|x64 {5F8A56F8-2C5B-48B6-9654-DD642D3E5F5C}.Debug|x64.ActiveCfg = Debug|x64 {5F8A56F8-2C5B-48B6-9654-DD642D3E5F5C}.Debug|x64.Build.0 = Debug|x64 {5F8A56F8-2C5B-48B6-9654-DD642D3E5F5C}.Release|x64.ActiveCfg = Release|x64 {5F8A56F8-2C5B-48B6-9654-DD642D3E5F5C}.Release|x64.Build.0 = Release|x64 {60206D22-E132-4695-8486-10BECA32C5CC}.Debug|x64.ActiveCfg = Debug|x64 {60206D22-E132-4695-8486-10BECA32C5CC}.Debug|x64.Build.0 = Debug|x64 {60206D22-E132-4695-8486-10BECA32C5CC}.Release|x64.ActiveCfg = Release|x64 {60206D22-E132-4695-8486-10BECA32C5CC}.Release|x64.Build.0 = Release|x64 {60B463D4-8CD5-4BF6-A25B-01BE13B87590}.Debug|x64.ActiveCfg = Debug|x64 {60B463D4-8CD5-4BF6-A25B-01BE13B87590}.Debug|x64.Build.0 = Debug|x64 {60B463D4-8CD5-4BF6-A25B-01BE13B87590}.Release|x64.ActiveCfg = Release|x64 {60B463D4-8CD5-4BF6-A25B-01BE13B87590}.Release|x64.Build.0 = Release|x64 {60EF55C7-8399-4543-B5B2-3AE2C532C67E}.Debug|x64.ActiveCfg = Debug|x64 {60EF55C7-8399-4543-B5B2-3AE2C532C67E}.Debug|x64.Build.0 = Debug|x64 {60EF55C7-8399-4543-B5B2-3AE2C532C67E}.Release|x64.ActiveCfg = Release|x64 {60EF55C7-8399-4543-B5B2-3AE2C532C67E}.Release|x64.Build.0 = Release|x64 {628FADA9-7047-4DD9-BD17-9FE4B5A1ADB0}.Debug|x64.ActiveCfg = Debug|x64 {628FADA9-7047-4DD9-BD17-9FE4B5A1ADB0}.Debug|x64.Build.0 = Debug|x64 {628FADA9-7047-4DD9-BD17-9FE4B5A1ADB0}.Release|x64.ActiveCfg = Release|x64 {628FADA9-7047-4DD9-BD17-9FE4B5A1ADB0}.Release|x64.Build.0 = Release|x64 {63B8184D-85E0-4E6A-9729-558C567D1D1D}.Debug|x64.ActiveCfg = Debug|x64 {63B8184D-85E0-4E6A-9729-558C567D1D1D}.Debug|x64.Build.0 = Debug|x64 {63B8184D-85E0-4E6A-9729-558C567D1D1D}.Release|x64.ActiveCfg = Release|x64 {63B8184D-85E0-4E6A-9729-558C567D1D1D}.Release|x64.Build.0 = Release|x64 {643B82A1-D009-46A9-92A0-2883399B05C2}.Debug|x64.ActiveCfg = Debug|x64 {643B82A1-D009-46A9-92A0-2883399B05C2}.Debug|x64.Build.0 = Debug|x64 {643B82A1-D009-46A9-92A0-2883399B05C2}.Release|x64.ActiveCfg = Release|x64 {643B82A1-D009-46A9-92A0-2883399B05C2}.Release|x64.Build.0 = Release|x64 {6516D6CF-8000-4341-9487-312BC83EE370}.Debug|x64.ActiveCfg = Debug|x64 {6516D6CF-8000-4341-9487-312BC83EE370}.Debug|x64.Build.0 = Debug|x64 {6516D6CF-8000-4341-9487-312BC83EE370}.Release|x64.ActiveCfg = Release|x64 {6516D6CF-8000-4341-9487-312BC83EE370}.Release|x64.Build.0 = Release|x64 {65D92D98-97E1-48F7-AEF6-75221CF48EA4}.Debug|x64.ActiveCfg = Debug|x64 {65D92D98-97E1-48F7-AEF6-75221CF48EA4}.Debug|x64.Build.0 = Debug|x64 {65D92D98-97E1-48F7-AEF6-75221CF48EA4}.Release|x64.ActiveCfg = Release|x64 {65D92D98-97E1-48F7-AEF6-75221CF48EA4}.Release|x64.Build.0 = Release|x64 {673277EC-D26B-414D-92E3-84EE873316A8}.Debug|x64.ActiveCfg = Debug|x64 {673277EC-D26B-414D-92E3-84EE873316A8}.Debug|x64.Build.0 = Debug|x64 {673277EC-D26B-414D-92E3-84EE873316A8}.Release|x64.ActiveCfg = Release|x64 {673277EC-D26B-414D-92E3-84EE873316A8}.Release|x64.Build.0 = Release|x64 {67AC1343-98FD-4143-92C0-559C55F749F5}.Debug|x64.ActiveCfg = Debug|x64 {67AC1343-98FD-4143-92C0-559C55F749F5}.Debug|x64.Build.0 = Debug|x64 {67AC1343-98FD-4143-92C0-559C55F749F5}.Release|x64.ActiveCfg = Release|x64 {67AC1343-98FD-4143-92C0-559C55F749F5}.Release|x64.Build.0 = Release|x64 {6851356E-A5D9-46A6-8262-A7E208729F18}.Debug|x64.ActiveCfg = Debug|x64 {6851356E-A5D9-46A6-8262-A7E208729F18}.Debug|x64.Build.0 = Debug|x64 {6851356E-A5D9-46A6-8262-A7E208729F18}.Release|x64.ActiveCfg = Release|x64 {6851356E-A5D9-46A6-8262-A7E208729F18}.Release|x64.Build.0 = Release|x64 {6AE1B8BE-D46A-4E99-87A2-F160FB950DCA}.Debug|x64.ActiveCfg = Debug|x64 {6AE1B8BE-D46A-4E99-87A2-F160FB950DCA}.Debug|x64.Build.0 = Debug|x64 {6AE1B8BE-D46A-4E99-87A2-F160FB950DCA}.Release|x64.ActiveCfg = Release|x64 {6AE1B8BE-D46A-4E99-87A2-F160FB950DCA}.Release|x64.Build.0 = Release|x64 {6B492754-9F80-44B3-A2A7-1D98AF06F3B2}.Debug|x64.ActiveCfg = Debug|x64 {6B492754-9F80-44B3-A2A7-1D98AF06F3B2}.Debug|x64.Build.0 = Debug|x64 {6B492754-9F80-44B3-A2A7-1D98AF06F3B2}.Release|x64.ActiveCfg = Release|x64 {6B492754-9F80-44B3-A2A7-1D98AF06F3B2}.Release|x64.Build.0 = Release|x64 {6BCEF2A5-0CEC-4CC6-9CB0-D3FBF871A408}.Debug|x64.ActiveCfg = Debug|x64 {6BCEF2A5-0CEC-4CC6-9CB0-D3FBF871A408}.Debug|x64.Build.0 = Debug|x64 {6BCEF2A5-0CEC-4CC6-9CB0-D3FBF871A408}.Release|x64.ActiveCfg = Release|x64 {6BCEF2A5-0CEC-4CC6-9CB0-D3FBF871A408}.Release|x64.Build.0 = Release|x64 {6D7C1169-3246-465F-B630-ECFEF4F3179A}.Debug|x64.ActiveCfg = Debug|x64 {6D7C1169-3246-465F-B630-ECFEF4F3179A}.Debug|x64.Build.0 = Debug|x64 {6D7C1169-3246-465F-B630-ECFEF4F3179A}.Release|x64.ActiveCfg = Release|x64 {6D7C1169-3246-465F-B630-ECFEF4F3179A}.Release|x64.Build.0 = Release|x64 {6DBD8C02-0C75-4DB0-BFDA-CD053B1B2D89}.Debug|x64.ActiveCfg = Debug|x64 {6DBD8C02-0C75-4DB0-BFDA-CD053B1B2D89}.Debug|x64.Build.0 = Debug|x64 {6DBD8C02-0C75-4DB0-BFDA-CD053B1B2D89}.Release|x64.ActiveCfg = Release|x64 {6DBD8C02-0C75-4DB0-BFDA-CD053B1B2D89}.Release|x64.Build.0 = Release|x64 {6EC93484-AAF3-487E-84E4-5ABFBA0AFC53}.Debug|x64.ActiveCfg = Debug|x64 {6EC93484-AAF3-487E-84E4-5ABFBA0AFC53}.Debug|x64.Build.0 = Debug|x64 {6EC93484-AAF3-487E-84E4-5ABFBA0AFC53}.Release|x64.ActiveCfg = Release|x64 {6EC93484-AAF3-487E-84E4-5ABFBA0AFC53}.Release|x64.Build.0 = Release|x64 {6F06A19B-0921-4B71-A3A5-B350B5FFEADB}.Debug|x64.ActiveCfg = Debug|x64 {6F06A19B-0921-4B71-A3A5-B350B5FFEADB}.Debug|x64.Build.0 = Debug|x64 {6F06A19B-0921-4B71-A3A5-B350B5FFEADB}.Release|x64.ActiveCfg = Release|x64 {6F06A19B-0921-4B71-A3A5-B350B5FFEADB}.Release|x64.Build.0 = Release|x64 {6F4953DA-FDC3-46CF-BF24-3752CCF2E1CB}.Debug|x64.ActiveCfg = Debug|x64 {6F4953DA-FDC3-46CF-BF24-3752CCF2E1CB}.Debug|x64.Build.0 = Debug|x64 {6F4953DA-FDC3-46CF-BF24-3752CCF2E1CB}.Release|x64.ActiveCfg = Release|x64 {6F4953DA-FDC3-46CF-BF24-3752CCF2E1CB}.Release|x64.Build.0 = Release|x64 {6F776280-B383-4DCE-8F42-9670164D038D}.Debug|x64.ActiveCfg = Debug|x64 {6F776280-B383-4DCE-8F42-9670164D038D}.Debug|x64.Build.0 = Debug|x64 {6F776280-B383-4DCE-8F42-9670164D038D}.Release|x64.ActiveCfg = Release|x64 {6F776280-B383-4DCE-8F42-9670164D038D}.Release|x64.Build.0 = Release|x64 {70EE1D40-0C65-4985-8EFC-BD40EE3A89B2}.Debug|x64.ActiveCfg = Debug|x64 {70EE1D40-0C65-4985-8EFC-BD40EE3A89B2}.Debug|x64.Build.0 = Debug|x64 {70EE1D40-0C65-4985-8EFC-BD40EE3A89B2}.Release|x64.ActiveCfg = Release|x64 {70EE1D40-0C65-4985-8EFC-BD40EE3A89B2}.Release|x64.Build.0 = Release|x64 {715EADD7-0FFE-4F1F-94E7-49302968DF79}.Debug|x64.ActiveCfg = Debug|x64 {715EADD7-0FFE-4F1F-94E7-49302968DF79}.Debug|x64.Build.0 = Debug|x64 {715EADD7-0FFE-4F1F-94E7-49302968DF79}.Release|x64.ActiveCfg = Release|x64 {715EADD7-0FFE-4F1F-94E7-49302968DF79}.Release|x64.Build.0 = Release|x64 {71D182E0-345A-4375-B0FA-3536821B0EE3}.Debug|x64.ActiveCfg = Debug|x64 {71D182E0-345A-4375-B0FA-3536821B0EE3}.Debug|x64.Build.0 = Debug|x64 {71D182E0-345A-4375-B0FA-3536821B0EE3}.Release|x64.ActiveCfg = Release|x64 {71D182E0-345A-4375-B0FA-3536821B0EE3}.Release|x64.Build.0 = Release|x64 {7264C8F6-73FB-4830-9306-1558D3EAC71B}.Debug|x64.ActiveCfg = Debug|x64 {7264C8F6-73FB-4830-9306-1558D3EAC71B}.Debug|x64.Build.0 = Debug|x64 {7264C8F6-73FB-4830-9306-1558D3EAC71B}.Release|x64.ActiveCfg = Release|x64 {7264C8F6-73FB-4830-9306-1558D3EAC71B}.Release|x64.Build.0 = Release|x64 {729E3905-FF7D-49C5-9871-6D35D839183E}.Debug|x64.ActiveCfg = Debug|x64 {729E3905-FF7D-49C5-9871-6D35D839183E}.Debug|x64.Build.0 = Debug|x64 {729E3905-FF7D-49C5-9871-6D35D839183E}.Release|x64.ActiveCfg = Release|x64 {729E3905-FF7D-49C5-9871-6D35D839183E}.Release|x64.Build.0 = Release|x64 {72C9DB46-C665-48AD-B805-BA885B40CA3E}.Debug|x64.ActiveCfg = Debug|x64 {72C9DB46-C665-48AD-B805-BA885B40CA3E}.Debug|x64.Build.0 = Debug|x64 {72C9DB46-C665-48AD-B805-BA885B40CA3E}.Release|x64.ActiveCfg = Release|x64 {72C9DB46-C665-48AD-B805-BA885B40CA3E}.Release|x64.Build.0 = Release|x64 {7337E34A-97B0-44FC-988B-7E6AE7E0FBBF}.Debug|x64.ActiveCfg = Debug|x64 {7337E34A-97B0-44FC-988B-7E6AE7E0FBBF}.Debug|x64.Build.0 = Debug|x64 {7337E34A-97B0-44FC-988B-7E6AE7E0FBBF}.Release|x64.ActiveCfg = Release|x64 {7337E34A-97B0-44FC-988B-7E6AE7E0FBBF}.Release|x64.Build.0 = Release|x64 {740ED97D-005F-4F58-98B2-4EF5EF5776E8}.Debug|x64.ActiveCfg = Debug|x64 {740ED97D-005F-4F58-98B2-4EF5EF5776E8}.Debug|x64.Build.0 = Debug|x64 {740ED97D-005F-4F58-98B2-4EF5EF5776E8}.Release|x64.ActiveCfg = Release|x64 {740ED97D-005F-4F58-98B2-4EF5EF5776E8}.Release|x64.Build.0 = Release|x64 {74D655D5-F661-4887-A1EB-5A6222AF5FCA}.Debug|x64.ActiveCfg = Debug|x64 {74D655D5-F661-4887-A1EB-5A6222AF5FCA}.Debug|x64.Build.0 = Debug|x64 {74D655D5-F661-4887-A1EB-5A6222AF5FCA}.Release|x64.ActiveCfg = Release|x64 {74D655D5-F661-4887-A1EB-5A6222AF5FCA}.Release|x64.Build.0 = Release|x64 {7701627C-CFD9-48F6-942E-EAACC8D057FA}.Debug|x64.ActiveCfg = Debug|x64 {7701627C-CFD9-48F6-942E-EAACC8D057FA}.Debug|x64.Build.0 = Debug|x64 {7701627C-CFD9-48F6-942E-EAACC8D057FA}.Release|x64.ActiveCfg = Release|x64 {7701627C-CFD9-48F6-942E-EAACC8D057FA}.Release|x64.Build.0 = Release|x64 {779425B1-2211-499B-A7CC-4F9EC6CB0D25}.Debug|x64.ActiveCfg = Debug|x64 {779425B1-2211-499B-A7CC-4F9EC6CB0D25}.Debug|x64.Build.0 = Debug|x64 {779425B1-2211-499B-A7CC-4F9EC6CB0D25}.Release|x64.ActiveCfg = Release|x64 {779425B1-2211-499B-A7CC-4F9EC6CB0D25}.Release|x64.Build.0 = Release|x64 {79D37FFE-FF76-44B3-BB27-3DCAEFF2EBE9}.Debug|x64.ActiveCfg = Debug|x64 {79D37FFE-FF76-44B3-BB27-3DCAEFF2EBE9}.Debug|x64.Build.0 = Debug|x64 {79D37FFE-FF76-44B3-BB27-3DCAEFF2EBE9}.Release|x64.ActiveCfg = Release|x64 {79D37FFE-FF76-44B3-BB27-3DCAEFF2EBE9}.Release|x64.Build.0 = Release|x64 {7ABF755C-821B-49CD-8EDE-83C16594FF7F}.Debug|x64.ActiveCfg = Debug|x64 {7ABF755C-821B-49CD-8EDE-83C16594FF7F}.Debug|x64.Build.0 = Debug|x64 {7ABF755C-821B-49CD-8EDE-83C16594FF7F}.Release|x64.ActiveCfg = Release|x64 {7ABF755C-821B-49CD-8EDE-83C16594FF7F}.Release|x64.Build.0 = Release|x64 {7DC3B3DD-73ED-4602-9AF3-8D7053620DEA}.Debug|x64.ActiveCfg = Debug|x64 {7DC3B3DD-73ED-4602-9AF3-8D7053620DEA}.Debug|x64.Build.0 = Debug|x64 {7DC3B3DD-73ED-4602-9AF3-8D7053620DEA}.Release|x64.ActiveCfg = Release|x64 {7DC3B3DD-73ED-4602-9AF3-8D7053620DEA}.Release|x64.Build.0 = Release|x64 {7DFEB4A5-8B04-4302-9D09-8144918FCF81}.Debug|x64.ActiveCfg = Debug|x64 {7DFEB4A5-8B04-4302-9D09-8144918FCF81}.Debug|x64.Build.0 = Debug|x64 {7DFEB4A5-8B04-4302-9D09-8144918FCF81}.Release|x64.ActiveCfg = Release|x64 {7DFEB4A5-8B04-4302-9D09-8144918FCF81}.Release|x64.Build.0 = Release|x64 {7F51CD29-3BCD-4DD8-B327-F384B5A616D1}.Debug|x64.ActiveCfg = Debug|x64 {7F51CD29-3BCD-4DD8-B327-F384B5A616D1}.Debug|x64.Build.0 = Debug|x64 {7F51CD29-3BCD-4DD8-B327-F384B5A616D1}.Release|x64.ActiveCfg = Release|x64 {7F51CD29-3BCD-4DD8-B327-F384B5A616D1}.Release|x64.Build.0 = Release|x64 {8008010F-8718-4C5F-86B2-195AEBF73422}.Debug|x64.ActiveCfg = Debug|x64 {8008010F-8718-4C5F-86B2-195AEBF73422}.Debug|x64.Build.0 = Debug|x64 {8008010F-8718-4C5F-86B2-195AEBF73422}.Release|x64.ActiveCfg = Release|x64 {8008010F-8718-4C5F-86B2-195AEBF73422}.Release|x64.Build.0 = Release|x64 {8010BBB0-C71B-4EFF-95EB-65C01E5EC197}.Debug|x64.ActiveCfg = Debug|x64 {8010BBB0-C71B-4EFF-95EB-65C01E5EC197}.Debug|x64.Build.0 = Debug|x64 {8010BBB0-C71B-4EFF-95EB-65C01E5EC197}.Release|x64.ActiveCfg = Release|x64 {8010BBB0-C71B-4EFF-95EB-65C01E5EC197}.Release|x64.Build.0 = Release|x64 {80AF1B7D-B8CE-4AF0-AE3B-1DABED1B57E7}.Debug|x64.ActiveCfg = Debug|x64 {80AF1B7D-B8CE-4AF0-AE3B-1DABED1B57E7}.Debug|x64.Build.0 = Debug|x64 {80AF1B7D-B8CE-4AF0-AE3B-1DABED1B57E7}.Release|x64.ActiveCfg = Release|x64 {80AF1B7D-B8CE-4AF0-AE3B-1DABED1B57E7}.Release|x64.Build.0 = Release|x64 {810DB909-6581-42D8-9616-906888F12149}.Debug|x64.ActiveCfg = Debug|x64 {810DB909-6581-42D8-9616-906888F12149}.Debug|x64.Build.0 = Debug|x64 {810DB909-6581-42D8-9616-906888F12149}.Release|x64.ActiveCfg = Release|x64 {810DB909-6581-42D8-9616-906888F12149}.Release|x64.Build.0 = Release|x64 {85D4076B-896B-4EBB-8F3A-8B44C24CD452}.Debug|x64.ActiveCfg = Debug|x64 {85D4076B-896B-4EBB-8F3A-8B44C24CD452}.Debug|x64.Build.0 = Debug|x64 {85D4076B-896B-4EBB-8F3A-8B44C24CD452}.Release|x64.ActiveCfg = Release|x64 {85D4076B-896B-4EBB-8F3A-8B44C24CD452}.Release|x64.Build.0 = Release|x64 {85DBDA9B-AEF6-43E7-B8B5-05FF2BEC61A3}.Debug|x64.ActiveCfg = Debug|x64 {85DBDA9B-AEF6-43E7-B8B5-05FF2BEC61A3}.Debug|x64.Build.0 = Debug|x64 {85DBDA9B-AEF6-43E7-B8B5-05FF2BEC61A3}.Release|x64.ActiveCfg = Release|x64 {85DBDA9B-AEF6-43E7-B8B5-05FF2BEC61A3}.Release|x64.Build.0 = Release|x64 {86EE22CC-6D3C-4F81-ADC8-394946F0DA81}.Debug|x64.ActiveCfg = Debug|x64 {86EE22CC-6D3C-4F81-ADC8-394946F0DA81}.Debug|x64.Build.0 = Debug|x64 {86EE22CC-6D3C-4F81-ADC8-394946F0DA81}.Release|x64.ActiveCfg = Release|x64 {86EE22CC-6D3C-4F81-ADC8-394946F0DA81}.Release|x64.Build.0 = Release|x64 {88D239E4-EB7D-4E0A-BE3A-AD78B9F408FC}.Debug|x64.ActiveCfg = Debug|x64 {88D239E4-EB7D-4E0A-BE3A-AD78B9F408FC}.Debug|x64.Build.0 = Debug|x64 {88D239E4-EB7D-4E0A-BE3A-AD78B9F408FC}.Release|x64.ActiveCfg = Release|x64 {88D239E4-EB7D-4E0A-BE3A-AD78B9F408FC}.Release|x64.Build.0 = Release|x64 {89F947CA-DDEF-4131-8AFB-584ABA4A1302}.Debug|x64.ActiveCfg = Debug|x64 {89F947CA-DDEF-4131-8AFB-584ABA4A1302}.Debug|x64.Build.0 = Debug|x64 {89F947CA-DDEF-4131-8AFB-584ABA4A1302}.Release|x64.ActiveCfg = Release|x64 {89F947CA-DDEF-4131-8AFB-584ABA4A1302}.Release|x64.Build.0 = Release|x64 {8A0FA780-068A-4534-AA2F-4FF4CF977AF2}.Debug|x64.ActiveCfg = Debug|x64 {8A0FA780-068A-4534-AA2F-4FF4CF977AF2}.Debug|x64.Build.0 = Debug|x64 {8A0FA780-068A-4534-AA2F-4FF4CF977AF2}.Release|x64.ActiveCfg = Release|x64 {8A4872D7-A234-4B9B-8215-82C6BB15F3A2}.Debug|x64.ActiveCfg = Debug|x64 {8A4872D7-A234-4B9B-8215-82C6BB15F3A2}.Debug|x64.Build.0 = Debug|x64 {8A4872D7-A234-4B9B-8215-82C6BB15F3A2}.Release|x64.ActiveCfg = Release|x64 {8A4872D7-A234-4B9B-8215-82C6BB15F3A2}.Release|x64.Build.0 = Release|x64 {8C42CA7C-1543-4F1B-A55F-28CD419C7D35}.Debug|x64.ActiveCfg = Debug|x64 {8C42CA7C-1543-4F1B-A55F-28CD419C7D35}.Debug|x64.Build.0 = Debug|x64 {8C42CA7C-1543-4F1B-A55F-28CD419C7D35}.Release|x64.ActiveCfg = Release|x64 {8C42CA7C-1543-4F1B-A55F-28CD419C7D35}.Release|x64.Build.0 = Release|x64 {8C6D73E0-0A6F-4487-A040-0EC78D7D6D9A}.Debug|x64.ActiveCfg = Debug|x64 {8C6D73E0-0A6F-4487-A040-0EC78D7D6D9A}.Debug|x64.Build.0 = Debug|x64 {8C6D73E0-0A6F-4487-A040-0EC78D7D6D9A}.Release|x64.ActiveCfg = Release|x64 {8C6D73E0-0A6F-4487-A040-0EC78D7D6D9A}.Release|x64.Build.0 = Release|x64 {8D75FA1A-EC74-4F88-8AC1-CE3F98E4D828}.Debug|x64.ActiveCfg = Debug|x64 {8D75FA1A-EC74-4F88-8AC1-CE3F98E4D828}.Debug|x64.Build.0 = Debug|x64 {8D75FA1A-EC74-4F88-8AC1-CE3F98E4D828}.Release|x64.ActiveCfg = Release|x64 {8D75FA1A-EC74-4F88-8AC1-CE3F98E4D828}.Release|x64.Build.0 = Release|x64 {8E374371-30E1-4623-8755-2A2F3742170B}.Debug|x64.ActiveCfg = Debug|x64 {8E374371-30E1-4623-8755-2A2F3742170B}.Debug|x64.Build.0 = Debug|x64 {8E374371-30E1-4623-8755-2A2F3742170B}.Release|x64.ActiveCfg = Release|x64 {8E374371-30E1-4623-8755-2A2F3742170B}.Release|x64.Build.0 = Release|x64 {901F04DB-E1A5-4A41-8B81-9D31C19ACD59}.Debug|x64.ActiveCfg = Debug|x64 {901F04DB-E1A5-4A41-8B81-9D31C19ACD59}.Debug|x64.Build.0 = Debug|x64 {901F04DB-E1A5-4A41-8B81-9D31C19ACD59}.Release|x64.ActiveCfg = Release|x64 {901F04DB-E1A5-4A41-8B81-9D31C19ACD59}.Release|x64.Build.0 = Release|x64 {9186EAC4-2F34-4F17-B940-6585D7869BCD}.Debug|x64.ActiveCfg = Debug|x64 {9186EAC4-2F34-4F17-B940-6585D7869BCD}.Debug|x64.Build.0 = Debug|x64 {9186EAC4-2F34-4F17-B940-6585D7869BCD}.Release|x64.ActiveCfg = Release|x64 {9186EAC4-2F34-4F17-B940-6585D7869BCD}.Release|x64.Build.0 = Release|x64 {91E19AEB-7B75-43E0-B8B4-D2BB60D839EA}.Debug|x64.ActiveCfg = Debug|x64 {91E19AEB-7B75-43E0-B8B4-D2BB60D839EA}.Debug|x64.Build.0 = Debug|x64 {91E19AEB-7B75-43E0-B8B4-D2BB60D839EA}.Release|x64.ActiveCfg = Release|x64 {91E19AEB-7B75-43E0-B8B4-D2BB60D839EA}.Release|x64.Build.0 = Release|x64 {92388A20-50FC-45F8-89E3-71F1618EFABB}.Debug|x64.ActiveCfg = Debug|x64 {92388A20-50FC-45F8-89E3-71F1618EFABB}.Debug|x64.Build.0 = Debug|x64 {92388A20-50FC-45F8-89E3-71F1618EFABB}.Release|x64.ActiveCfg = Release|x64 {92388A20-50FC-45F8-89E3-71F1618EFABB}.Release|x64.Build.0 = Release|x64 {924B2937-0B53-4DC6-B7E1-5F3102728F89}.Debug|x64.ActiveCfg = Debug|x64 {924B2937-0B53-4DC6-B7E1-5F3102728F89}.Debug|x64.Build.0 = Debug|x64 {924B2937-0B53-4DC6-B7E1-5F3102728F89}.Release|x64.ActiveCfg = Release|x64 {924B2937-0B53-4DC6-B7E1-5F3102728F89}.Release|x64.Build.0 = Release|x64 {95B683BD-B9DC-400F-9BC0-8F1505F08BF5}.Debug|x64.ActiveCfg = Debug|x64 {95B683BD-B9DC-400F-9BC0-8F1505F08BF5}.Debug|x64.Build.0 = Debug|x64 {95B683BD-B9DC-400F-9BC0-8F1505F08BF5}.Release|x64.ActiveCfg = Release|x64 {95B683BD-B9DC-400F-9BC0-8F1505F08BF5}.Release|x64.Build.0 = Release|x64 {96D00A19-5CEF-4CC5-BDE8-E33C68BCE90F}.Debug|x64.ActiveCfg = Debug|x64 {96D00A19-5CEF-4CC5-BDE8-E33C68BCE90F}.Debug|x64.Build.0 = Debug|x64 {96D00A19-5CEF-4CC5-BDE8-E33C68BCE90F}.Release|x64.ActiveCfg = Release|x64 {96D00A19-5CEF-4CC5-BDE8-E33C68BCE90F}.Release|x64.Build.0 = Release|x64 {98ACBE5D-1A92-46F9-AA81-533412172952}.Debug|x64.ActiveCfg = Debug|x64 {98ACBE5D-1A92-46F9-AA81-533412172952}.Debug|x64.Build.0 = Debug|x64 {98ACBE5D-1A92-46F9-AA81-533412172952}.Release|x64.ActiveCfg = Release|x64 {98ACBE5D-1A92-46F9-AA81-533412172952}.Release|x64.Build.0 = Release|x64 {99F7F00F-1DE5-45EA-992B-64BA282FAC76}.Debug|x64.ActiveCfg = Debug|x64 {99F7F00F-1DE5-45EA-992B-64BA282FAC76}.Debug|x64.Build.0 = Debug|x64 {99F7F00F-1DE5-45EA-992B-64BA282FAC76}.Release|x64.ActiveCfg = Release|x64 {99F7F00F-1DE5-45EA-992B-64BA282FAC76}.Release|x64.Build.0 = Release|x64 {9A4078F8-B8E4-4EC6-A6FF-4F29DAD9CE48}.Debug|x64.ActiveCfg = Debug|x64 {9A4078F8-B8E4-4EC6-A6FF-4F29DAD9CE48}.Debug|x64.Build.0 = Debug|x64 {9A4078F8-B8E4-4EC6-A6FF-4F29DAD9CE48}.Release|x64.ActiveCfg = Release|x64 {9A4078F8-B8E4-4EC6-A6FF-4F29DAD9CE48}.Release|x64.Build.0 = Release|x64 {9AE2DAF9-10C4-4EC3-AE52-AD5EE9C77C55}.Debug|x64.ActiveCfg = Debug|x64 {9AE2DAF9-10C4-4EC3-AE52-AD5EE9C77C55}.Debug|x64.Build.0 = Debug|x64 {9AE2DAF9-10C4-4EC3-AE52-AD5EE9C77C55}.Release|x64.ActiveCfg = Release|x64 {9AE2DAF9-10C4-4EC3-AE52-AD5EE9C77C55}.Release|x64.Build.0 = Release|x64 {9D9E33EB-4C24-4646-A3FB-35DA17247917}.Debug|x64.ActiveCfg = Debug|x64 {9D9E33EB-4C24-4646-A3FB-35DA17247917}.Debug|x64.Build.0 = Debug|x64 {9D9E33EB-4C24-4646-A3FB-35DA17247917}.Release|x64.ActiveCfg = Release|x64 {9D9E33EB-4C24-4646-A3FB-35DA17247917}.Release|x64.Build.0 = Release|x64 {9E9E3D25-2139-4A5D-9200-18148DDEAD45}.Debug|x64.ActiveCfg = Debug|x64 {9E9E3D25-2139-4A5D-9200-18148DDEAD45}.Debug|x64.Build.0 = Debug|x64 {9E9E3D25-2139-4A5D-9200-18148DDEAD45}.Release|x64.ActiveCfg = Release|x64 {9E9E3D25-2139-4A5D-9200-18148DDEAD45}.Release|x64.Build.0 = Release|x64 {9FF51F3E-AF36-4F45-A797-C5F03A090298}.Debug|x64.ActiveCfg = Debug|x64 {9FF51F3E-AF36-4F45-A797-C5F03A090298}.Debug|x64.Build.0 = Debug|x64 {9FF51F3E-AF36-4F45-A797-C5F03A090298}.Release|x64.ActiveCfg = Release|x64 {9FF51F3E-AF36-4F45-A797-C5F03A090298}.Release|x64.Build.0 = Release|x64 {9FF62356-30B4-42A1-8DC7-45262A18DD44}.Debug|x64.ActiveCfg = Debug|x64 {9FF62356-30B4-42A1-8DC7-45262A18DD44}.Debug|x64.Build.0 = Debug|x64 {9FF62356-30B4-42A1-8DC7-45262A18DD44}.Release|x64.ActiveCfg = Release|x64 {9FF62356-30B4-42A1-8DC7-45262A18DD44}.Release|x64.Build.0 = Release|x64 {A216BF23-FC5C-4426-BF20-8568A2AA5FA0}.Debug|x64.ActiveCfg = Debug|x64 {A216BF23-FC5C-4426-BF20-8568A2AA5FA0}.Debug|x64.Build.0 = Debug|x64 {A216BF23-FC5C-4426-BF20-8568A2AA5FA0}.Release|x64.ActiveCfg = Release|x64 {A216BF23-FC5C-4426-BF20-8568A2AA5FA0}.Release|x64.Build.0 = Release|x64 {A2A0FAEA-2B7C-4FC3-B904-1DB4DEACF88D}.Debug|x64.ActiveCfg = Debug|x64 {A2A0FAEA-2B7C-4FC3-B904-1DB4DEACF88D}.Debug|x64.Build.0 = Debug|x64 {A2A0FAEA-2B7C-4FC3-B904-1DB4DEACF88D}.Release|x64.ActiveCfg = Release|x64 {A2A0FAEA-2B7C-4FC3-B904-1DB4DEACF88D}.Release|x64.Build.0 = Release|x64 {A38EFCDB-53D6-4474-97F3-0DDC6CE70D76}.Debug|x64.ActiveCfg = Debug|x64 {A38EFCDB-53D6-4474-97F3-0DDC6CE70D76}.Debug|x64.Build.0 = Debug|x64 {A38EFCDB-53D6-4474-97F3-0DDC6CE70D76}.Release|x64.ActiveCfg = Release|x64 {A38EFCDB-53D6-4474-97F3-0DDC6CE70D76}.Release|x64.Build.0 = Release|x64 {A39D1640-8DBA-450D-9103-2533C248991A}.Debug|x64.ActiveCfg = Debug|x64 {A39D1640-8DBA-450D-9103-2533C248991A}.Debug|x64.Build.0 = Debug|x64 {A39D1640-8DBA-450D-9103-2533C248991A}.Release|x64.ActiveCfg = Release|x64 {A39D1640-8DBA-450D-9103-2533C248991A}.Release|x64.Build.0 = Release|x64 {A57D9365-172E-4782-ADC6-82A594E30943}.Debug|x64.ActiveCfg = Debug|x64 {A57D9365-172E-4782-ADC6-82A594E30943}.Debug|x64.Build.0 = Debug|x64 {A57D9365-172E-4782-ADC6-82A594E30943}.Release|x64.ActiveCfg = Release|x64 {A57D9365-172E-4782-ADC6-82A594E30943}.Release|x64.Build.0 = Release|x64 {A79E3093-B157-4B09-BABD-29266EA16407}.Debug|x64.ActiveCfg = Debug|x64 {A79E3093-B157-4B09-BABD-29266EA16407}.Debug|x64.Build.0 = Debug|x64 {A79E3093-B157-4B09-BABD-29266EA16407}.Release|x64.ActiveCfg = Release|x64 {A79E3093-B157-4B09-BABD-29266EA16407}.Release|x64.Build.0 = Release|x64 {A7CA7975-CEDB-48E6-9AEB-1209DCBD07F2}.Debug|x64.ActiveCfg = Debug|x64 {A7CA7975-CEDB-48E6-9AEB-1209DCBD07F2}.Debug|x64.Build.0 = Debug|x64 {A7CA7975-CEDB-48E6-9AEB-1209DCBD07F2}.Release|x64.ActiveCfg = Release|x64 {A7CA7975-CEDB-48E6-9AEB-1209DCBD07F2}.Release|x64.Build.0 = Release|x64 {AB15A115-E429-4123-BEBF-206FBA4CF615}.Debug|x64.ActiveCfg = Debug|x64 {AB15A115-E429-4123-BEBF-206FBA4CF615}.Debug|x64.Build.0 = Debug|x64 {AB15A115-E429-4123-BEBF-206FBA4CF615}.Release|x64.ActiveCfg = Release|x64 {AB15A115-E429-4123-BEBF-206FBA4CF615}.Release|x64.Build.0 = Release|x64 {AE1C32FB-9B52-4760-ABFC-0D2FA2C7A6C8}.Debug|x64.ActiveCfg = Debug|x64 {AE1C32FB-9B52-4760-ABFC-0D2FA2C7A6C8}.Debug|x64.Build.0 = Debug|x64 {AE1C32FB-9B52-4760-ABFC-0D2FA2C7A6C8}.Release|x64.ActiveCfg = Release|x64 {AE1C32FB-9B52-4760-ABFC-0D2FA2C7A6C8}.Release|x64.Build.0 = Release|x64 {AE952763-5C84-43FC-B344-CACC950F056C}.Debug|x64.ActiveCfg = Debug|x64 {AE952763-5C84-43FC-B344-CACC950F056C}.Debug|x64.Build.0 = Debug|x64 {AE952763-5C84-43FC-B344-CACC950F056C}.Release|x64.ActiveCfg = Release|x64 {AE952763-5C84-43FC-B344-CACC950F056C}.Release|x64.Build.0 = Release|x64 {AE9E908D-BAEC-491F-9914-436B3CE35E94}.Debug|x64.ActiveCfg = Debug|x64 {AE9E908D-BAEC-491F-9914-436B3CE35E94}.Debug|x64.Build.0 = Debug|x64 {AE9E908D-BAEC-491F-9914-436B3CE35E94}.Release|x64.ActiveCfg = Release|x64 {AE9E908D-BAEC-491F-9914-436B3CE35E94}.Release|x64.Build.0 = Release|x64 {AEAA72CD-E060-417C-9CA1-49B4738384E0}.Debug|x64.ActiveCfg = Debug|x64 {AEAA72CD-E060-417C-9CA1-49B4738384E0}.Debug|x64.Build.0 = Debug|x64 {AEAA72CD-E060-417C-9CA1-49B4738384E0}.Release|x64.ActiveCfg = Release|x64 {AEAA72CD-E060-417C-9CA1-49B4738384E0}.Release|x64.Build.0 = Release|x64 {AF038868-2432-4159-A62F-941F11D12C5D}.Debug|x64.ActiveCfg = Debug|x64 {AF038868-2432-4159-A62F-941F11D12C5D}.Debug|x64.Build.0 = Debug|x64 {AF038868-2432-4159-A62F-941F11D12C5D}.Release|x64.ActiveCfg = Release|x64 {AF038868-2432-4159-A62F-941F11D12C5D}.Release|x64.Build.0 = Release|x64 {AF0B7480-EBE3-486B-B0C8-134910BC9324}.Debug|x64.ActiveCfg = Debug|x64 {AF0B7480-EBE3-486B-B0C8-134910BC9324}.Debug|x64.Build.0 = Debug|x64 {AF0B7480-EBE3-486B-B0C8-134910BC9324}.Release|x64.ActiveCfg = Release|x64 {AF0B7480-EBE3-486B-B0C8-134910BC9324}.Release|x64.Build.0 = Release|x64 {B30C6212-A160-405A-8FE7-340E721738A2}.Debug|x64.ActiveCfg = Debug|x64 {B30C6212-A160-405A-8FE7-340E721738A2}.Debug|x64.Build.0 = Debug|x64 {B30C6212-A160-405A-8FE7-340E721738A2}.Release|x64.ActiveCfg = Release|x64 {B30C6212-A160-405A-8FE7-340E721738A2}.Release|x64.Build.0 = Release|x64 {B35BFA09-DE68-483B-AB61-8790E8F060A8}.Debug|x64.ActiveCfg = Debug|x64 {B35BFA09-DE68-483B-AB61-8790E8F060A8}.Debug|x64.Build.0 = Debug|x64 {B35BFA09-DE68-483B-AB61-8790E8F060A8}.Release|x64.ActiveCfg = Release|x64 {B35BFA09-DE68-483B-AB61-8790E8F060A8}.Release|x64.Build.0 = Release|x64 {B36F115C-8139-4C35-A3E7-E6BF9F3DA793}.Debug|x64.ActiveCfg = Debug|x64 {B36F115C-8139-4C35-A3E7-E6BF9F3DA793}.Debug|x64.Build.0 = Debug|x64 {B36F115C-8139-4C35-A3E7-E6BF9F3DA793}.Release|x64.ActiveCfg = Release|x64 {B36F115C-8139-4C35-A3E7-E6BF9F3DA793}.Release|x64.Build.0 = Release|x64 {B379539C-E130-460D-AE82-4EBDD1A97845}.Debug|x64.ActiveCfg = Debug|x64 {B379539C-E130-460D-AE82-4EBDD1A97845}.Debug|x64.Build.0 = Debug|x64 {B379539C-E130-460D-AE82-4EBDD1A97845}.Release|x64.ActiveCfg = Release|x64 {B379539C-E130-460D-AE82-4EBDD1A97845}.Release|x64.Build.0 = Release|x64 {B3AF8A19-5802-4A34-9157-27BBE4E53C0A}.Debug|x64.ActiveCfg = Debug|x64 {B3AF8A19-5802-4A34-9157-27BBE4E53C0A}.Debug|x64.Build.0 = Debug|x64 {B3AF8A19-5802-4A34-9157-27BBE4E53C0A}.Release|x64.ActiveCfg = Release|x64 {B3AF8A19-5802-4A34-9157-27BBE4E53C0A}.Release|x64.Build.0 = Release|x64 {B440BB05-37A8-42EA-98D3-D83EB113E497}.Debug|x64.ActiveCfg = Debug|x64 {B440BB05-37A8-42EA-98D3-D83EB113E497}.Debug|x64.Build.0 = Debug|x64 {B440BB05-37A8-42EA-98D3-D83EB113E497}.Release|x64.ActiveCfg = Release|x64 {B440BB05-37A8-42EA-98D3-D83EB113E497}.Release|x64.Build.0 = Release|x64 {B6C0521B-EECA-47EF-BFA8-147F9C3F6DFF}.Debug|x64.ActiveCfg = Debug|x64 {B6C0521B-EECA-47EF-BFA8-147F9C3F6DFF}.Debug|x64.Build.0 = Debug|x64 {B6C0521B-EECA-47EF-BFA8-147F9C3F6DFF}.Release|x64.ActiveCfg = Release|x64 {B6C0521B-EECA-47EF-BFA8-147F9C3F6DFF}.Release|x64.Build.0 = Release|x64 {B6DA6617-D98F-4A4D-A7C4-A317212924BF}.Debug|x64.ActiveCfg = Debug|x64 {B6DA6617-D98F-4A4D-A7C4-A317212924BF}.Debug|x64.Build.0 = Debug|x64 {B6DA6617-D98F-4A4D-A7C4-A317212924BF}.Release|x64.ActiveCfg = Release|x64 {B6DA6617-D98F-4A4D-A7C4-A317212924BF}.Release|x64.Build.0 = Release|x64 {B6F4B85D-FE55-4A1B-AE97-D4A9ECFE195F}.Debug|x64.ActiveCfg = Debug|x64 {B6F4B85D-FE55-4A1B-AE97-D4A9ECFE195F}.Debug|x64.Build.0 = Debug|x64 {B6F4B85D-FE55-4A1B-AE97-D4A9ECFE195F}.Release|x64.ActiveCfg = Release|x64 {B6F4B85D-FE55-4A1B-AE97-D4A9ECFE195F}.Release|x64.Build.0 = Release|x64 {B775480C-5B32-4F64-B026-47367280EC56}.Debug|x64.ActiveCfg = Debug|x64 {B775480C-5B32-4F64-B026-47367280EC56}.Debug|x64.Build.0 = Debug|x64 {B775480C-5B32-4F64-B026-47367280EC56}.Release|x64.ActiveCfg = Release|x64 {B775480C-5B32-4F64-B026-47367280EC56}.Release|x64.Build.0 = Release|x64 {B887EA26-846C-4D6A-B0E4-432487506BC7}.Debug|x64.ActiveCfg = Debug|x64 {B887EA26-846C-4D6A-B0E4-432487506BC7}.Debug|x64.Build.0 = Debug|x64 {B887EA26-846C-4D6A-B0E4-432487506BC7}.Release|x64.ActiveCfg = Release|x64 {B887EA26-846C-4D6A-B0E4-432487506BC7}.Release|x64.Build.0 = Release|x64 {BABC6427-E533-4DCF-91E3-B5B2ED253F46}.Debug|x64.ActiveCfg = Debug|x64 {BABC6427-E533-4DCF-91E3-B5B2ED253F46}.Debug|x64.Build.0 = Debug|x64 {BABC6427-E533-4DCF-91E3-B5B2ED253F46}.Release|x64.ActiveCfg = Release|x64 {BABC6427-E533-4DCF-91E3-B5B2ED253F46}.Release|x64.Build.0 = Release|x64 {BAE107BA-7618-4972-8188-2D3CDAAE0453}.Debug|x64.ActiveCfg = Debug|x64 {BAE107BA-7618-4972-8188-2D3CDAAE0453}.Debug|x64.Build.0 = Debug|x64 {BAE107BA-7618-4972-8188-2D3CDAAE0453}.Release|x64.ActiveCfg = Release|x64 {BAE107BA-7618-4972-8188-2D3CDAAE0453}.Release|x64.Build.0 = Release|x64 {BB1120CF-B721-4EF9-8735-58F76AE51D2F}.Debug|x64.ActiveCfg = Debug|x64 {BB1120CF-B721-4EF9-8735-58F76AE51D2F}.Debug|x64.Build.0 = Debug|x64 {BB1120CF-B721-4EF9-8735-58F76AE51D2F}.Release|x64.ActiveCfg = Release|x64 {BB1120CF-B721-4EF9-8735-58F76AE51D2F}.Release|x64.Build.0 = Release|x64 {BB248BAC-6E1B-433C-A254-75140A273AB5}.Debug|x64.ActiveCfg = Debug|x64 {BB248BAC-6E1B-433C-A254-75140A273AB5}.Debug|x64.Build.0 = Debug|x64 {BB248BAC-6E1B-433C-A254-75140A273AB5}.Release|x64.ActiveCfg = Release|x64 {BB248BAC-6E1B-433C-A254-75140A273AB5}.Release|x64.Build.0 = Release|x64 {BE18F227-A9F0-4B38-B689-4E2F9F09CA5F}.Debug|x64.ActiveCfg = Debug|x64 {BE18F227-A9F0-4B38-B689-4E2F9F09CA5F}.Debug|x64.Build.0 = Debug|x64 {BE18F227-A9F0-4B38-B689-4E2F9F09CA5F}.Release|x64.ActiveCfg = Release|x64 {BE18F227-A9F0-4B38-B689-4E2F9F09CA5F}.Release|x64.Build.0 = Release|x64 {C0E811E0-8942-4CFD-A817-74D99E9E6577}.Debug|x64.ActiveCfg = Debug|x64 {C0E811E0-8942-4CFD-A817-74D99E9E6577}.Debug|x64.Build.0 = Debug|x64 {C0E811E0-8942-4CFD-A817-74D99E9E6577}.Release|x64.ActiveCfg = Release|x64 {C0E811E0-8942-4CFD-A817-74D99E9E6577}.Release|x64.Build.0 = Release|x64 {C2C36D03-26EE-4BD8-8FFC-86CFE16C1218}.Debug|x64.ActiveCfg = Debug|x64 {C2C36D03-26EE-4BD8-8FFC-86CFE16C1218}.Debug|x64.Build.0 = Debug|x64 {C2C36D03-26EE-4BD8-8FFC-86CFE16C1218}.Release|x64.ActiveCfg = Release|x64 {C2C36D03-26EE-4BD8-8FFC-86CFE16C1218}.Release|x64.Build.0 = Release|x64 {C2F94489-A483-4C44-B8A7-11A75F6AEC66}.Debug|x64.ActiveCfg = Debug|x64 {C2F94489-A483-4C44-B8A7-11A75F6AEC66}.Debug|x64.Build.0 = Debug|x64 {C2F94489-A483-4C44-B8A7-11A75F6AEC66}.Release|x64.ActiveCfg = Release|x64 {C2F94489-A483-4C44-B8A7-11A75F6AEC66}.Release|x64.Build.0 = Release|x64 {C35052AF-2383-4F9C-B18B-55A01829F2BF}.Debug|x64.ActiveCfg = Debug|x64 {C35052AF-2383-4F9C-B18B-55A01829F2BF}.Debug|x64.Build.0 = Debug|x64 {C35052AF-2383-4F9C-B18B-55A01829F2BF}.Release|x64.ActiveCfg = Release|x64 {C35052AF-2383-4F9C-B18B-55A01829F2BF}.Release|x64.Build.0 = Release|x64 {C3CEE34C-29E0-4A22-B258-3FBAF662AA19}.Debug|x64.ActiveCfg = Debug|x64 {C3CEE34C-29E0-4A22-B258-3FBAF662AA19}.Debug|x64.Build.0 = Debug|x64 {C3CEE34C-29E0-4A22-B258-3FBAF662AA19}.Release|x64.ActiveCfg = Release|x64 {C3CEE34C-29E0-4A22-B258-3FBAF662AA19}.Release|x64.Build.0 = Release|x64 {C5E8B8DB-2507-4904-847F-A52196B075F0}.Debug|x64.ActiveCfg = Debug|x64 {C5E8B8DB-2507-4904-847F-A52196B075F0}.Debug|x64.Build.0 = Debug|x64 {C5E8B8DB-2507-4904-847F-A52196B075F0}.Release|x64.ActiveCfg = Release|x64 {C5E8B8DB-2507-4904-847F-A52196B075F0}.Release|x64.Build.0 = Release|x64 {C71DAF3E-9361-4723-93E2-C475D1D0C0D0}.Debug|x64.ActiveCfg = Debug|x64 {C71DAF3E-9361-4723-93E2-C475D1D0C0D0}.Debug|x64.Build.0 = Debug|x64 {C71DAF3E-9361-4723-93E2-C475D1D0C0D0}.Release|x64.ActiveCfg = Release|x64 {C71DAF3E-9361-4723-93E2-C475D1D0C0D0}.Release|x64.Build.0 = Release|x64 {C7E42AE1-052F-4024-B8BA-DE5DCE6BBEEC}.Debug|x64.ActiveCfg = Debug|x64 {C7E42AE1-052F-4024-B8BA-DE5DCE6BBEEC}.Debug|x64.Build.0 = Debug|x64 {C7E42AE1-052F-4024-B8BA-DE5DCE6BBEEC}.Release|x64.ActiveCfg = Release|x64 {C7E42AE1-052F-4024-B8BA-DE5DCE6BBEEC}.Release|x64.Build.0 = Release|x64 {C973CD39-D63B-4F5C-BE1D-DED17388B5A4}.Debug|x64.ActiveCfg = Debug|x64 {C973CD39-D63B-4F5C-BE1D-DED17388B5A4}.Debug|x64.Build.0 = Debug|x64 {C973CD39-D63B-4F5C-BE1D-DED17388B5A4}.Release|x64.ActiveCfg = Release|x64 {C973CD39-D63B-4F5C-BE1D-DED17388B5A4}.Release|x64.Build.0 = Release|x64 {CA4BBB24-D33E-42E2-A495-F10D80DE8C1D}.Debug|x64.ActiveCfg = Debug|x64 {CA4BBB24-D33E-42E2-A495-F10D80DE8C1D}.Debug|x64.Build.0 = Debug|x64 {CA4BBB24-D33E-42E2-A495-F10D80DE8C1D}.Release|x64.ActiveCfg = Release|x64 {CB906E89-1313-4929-AFF7-86FBF1CC301F}.Debug|x64.ActiveCfg = Debug|x64 {CB906E89-1313-4929-AFF7-86FBF1CC301F}.Debug|x64.Build.0 = Debug|x64 {CB906E89-1313-4929-AFF7-86FBF1CC301F}.Release|x64.ActiveCfg = Release|x64 {CB906E89-1313-4929-AFF7-86FBF1CC301F}.Release|x64.Build.0 = Release|x64 {CCA9B681-D10B-45E4-98CC-531503D2EDE8}.Debug|x64.ActiveCfg = Debug|x64 {CCA9B681-D10B-45E4-98CC-531503D2EDE8}.Debug|x64.Build.0 = Debug|x64 {CCA9B681-D10B-45E4-98CC-531503D2EDE8}.Release|x64.ActiveCfg = Release|x64 {CCA9B681-D10B-45E4-98CC-531503D2EDE8}.Release|x64.Build.0 = Release|x64 {CDD9DFC6-5C3D-42F7-B822-FE29A1C21752}.Debug|x64.ActiveCfg = Debug|x64 {CDD9DFC6-5C3D-42F7-B822-FE29A1C21752}.Debug|x64.Build.0 = Debug|x64 {CDD9DFC6-5C3D-42F7-B822-FE29A1C21752}.Release|x64.ActiveCfg = Release|x64 {CDD9DFC6-5C3D-42F7-B822-FE29A1C21752}.Release|x64.Build.0 = Release|x64 {CE3F2DFB-8470-4802-AD37-21CAF6CB2681}.Debug|x64.ActiveCfg = Debug|x64 {CE3F2DFB-8470-4802-AD37-21CAF6CB2681}.Debug|x64.Build.0 = Debug|x64 {CE3F2DFB-8470-4802-AD37-21CAF6CB2681}.Release|x64.ActiveCfg = Release|x64 {CE3F2DFB-8470-4802-AD37-21CAF6CB2681}.Release|x64.Build.0 = Release|x64 {CF9A0883-6334-44C7-AC29-349468C78E27}.Debug|x64.ActiveCfg = Debug|x64 {CF9A0883-6334-44C7-AC29-349468C78E27}.Debug|x64.Build.0 = Debug|x64 {CF9A0883-6334-44C7-AC29-349468C78E27}.Release|x64.ActiveCfg = Release|x64 {CF9A0883-6334-44C7-AC29-349468C78E27}.Release|x64.Build.0 = Release|x64 {CF9F4CEA-EC66-4E78-A086-107EB29E0637}.Debug|x64.ActiveCfg = Debug|x64 {CF9F4CEA-EC66-4E78-A086-107EB29E0637}.Debug|x64.Build.0 = Debug|x64 {CF9F4CEA-EC66-4E78-A086-107EB29E0637}.Release|x64.ActiveCfg = Release|x64 {CF9F4CEA-EC66-4E78-A086-107EB29E0637}.Release|x64.Build.0 = Release|x64 {D062166F-0EC7-4C13-A772-0C7157EEFE41}.Debug|x64.ActiveCfg = Debug|x64 {D062166F-0EC7-4C13-A772-0C7157EEFE41}.Debug|x64.Build.0 = Debug|x64 {D062166F-0EC7-4C13-A772-0C7157EEFE41}.Release|x64.ActiveCfg = Release|x64 {D062166F-0EC7-4C13-A772-0C7157EEFE41}.Release|x64.Build.0 = Release|x64 {D140560D-FDEC-4D3D-8F58-BF5FD5E4DAA1}.Debug|x64.ActiveCfg = Debug|x64 {D140560D-FDEC-4D3D-8F58-BF5FD5E4DAA1}.Debug|x64.Build.0 = Debug|x64 {D140560D-FDEC-4D3D-8F58-BF5FD5E4DAA1}.Release|x64.ActiveCfg = Release|x64 {D140560D-FDEC-4D3D-8F58-BF5FD5E4DAA1}.Release|x64.Build.0 = Release|x64 {D28F5FF6-8401-4E0D-94F9-3A1FD7ED64E3}.Debug|x64.ActiveCfg = Debug|x64 {D28F5FF6-8401-4E0D-94F9-3A1FD7ED64E3}.Debug|x64.Build.0 = Debug|x64 {D28F5FF6-8401-4E0D-94F9-3A1FD7ED64E3}.Release|x64.ActiveCfg = Release|x64 {D28F5FF6-8401-4E0D-94F9-3A1FD7ED64E3}.Release|x64.Build.0 = Release|x64 {D2C30C7E-A7D3-487A-956E-418CECAFFE8E}.Debug|x64.ActiveCfg = Debug|x64 {D2C30C7E-A7D3-487A-956E-418CECAFFE8E}.Debug|x64.Build.0 = Debug|x64 {D2C30C7E-A7D3-487A-956E-418CECAFFE8E}.Release|x64.ActiveCfg = Release|x64 {D2C30C7E-A7D3-487A-956E-418CECAFFE8E}.Release|x64.Build.0 = Release|x64 {D3A99F36-4B72-4766-ABCD-CCEDC26DD139}.Debug|x64.ActiveCfg = Debug|x64 {D3A99F36-4B72-4766-ABCD-CCEDC26DD139}.Debug|x64.Build.0 = Debug|x64 {D3A99F36-4B72-4766-ABCD-CCEDC26DD139}.Release|x64.ActiveCfg = Release|x64 {D3A99F36-4B72-4766-ABCD-CCEDC26DD139}.Release|x64.Build.0 = Release|x64 {D4035736-1AD6-4100-9FA9-A8A0C1DAE0C7}.Debug|x64.ActiveCfg = Debug|x64 {D4035736-1AD6-4100-9FA9-A8A0C1DAE0C7}.Debug|x64.Build.0 = Debug|x64 {D4035736-1AD6-4100-9FA9-A8A0C1DAE0C7}.Release|x64.ActiveCfg = Release|x64 {D4035736-1AD6-4100-9FA9-A8A0C1DAE0C7}.Release|x64.Build.0 = Release|x64 {D829DB63-E046-474D-8EA3-43A6659294D8}.Debug|x64.ActiveCfg = Debug|x64 {D829DB63-E046-474D-8EA3-43A6659294D8}.Debug|x64.Build.0 = Debug|x64 {D829DB63-E046-474D-8EA3-43A6659294D8}.Release|x64.ActiveCfg = Release|x64 {D829DB63-E046-474D-8EA3-43A6659294D8}.Release|x64.Build.0 = Release|x64 {D8317F1D-7A70-4A39-977A-EAB05A04A87B}.Debug|x64.ActiveCfg = Debug|x64 {D8317F1D-7A70-4A39-977A-EAB05A04A87B}.Debug|x64.Build.0 = Debug|x64 {D8317F1D-7A70-4A39-977A-EAB05A04A87B}.Release|x64.ActiveCfg = Release|x64 {D8317F1D-7A70-4A39-977A-EAB05A04A87B}.Release|x64.Build.0 = Release|x64 {D88187D2-1977-4C5F-B0CD-83C69BD6C1BC}.Debug|x64.ActiveCfg = Debug|x64 {D88187D2-1977-4C5F-B0CD-83C69BD6C1BC}.Debug|x64.Build.0 = Debug|x64 {D88187D2-1977-4C5F-B0CD-83C69BD6C1BC}.Release|x64.ActiveCfg = Release|x64 {D88187D2-1977-4C5F-B0CD-83C69BD6C1BC}.Release|x64.Build.0 = Release|x64 {D93A2683-6D99-4F18-B378-91195D23E007}.Debug|x64.ActiveCfg = Debug|x64 {D93A2683-6D99-4F18-B378-91195D23E007}.Debug|x64.Build.0 = Debug|x64 {D93A2683-6D99-4F18-B378-91195D23E007}.Release|x64.ActiveCfg = Release|x64 {D93A2683-6D99-4F18-B378-91195D23E007}.Release|x64.Build.0 = Release|x64 {D9A70E35-0C85-4A09-ACA8-B15B21B66F50}.Debug|x64.ActiveCfg = Debug|x64 {D9A70E35-0C85-4A09-ACA8-B15B21B66F50}.Debug|x64.Build.0 = Debug|x64 {D9A70E35-0C85-4A09-ACA8-B15B21B66F50}.Release|x64.ActiveCfg = Release|x64 {D9A70E35-0C85-4A09-ACA8-B15B21B66F50}.Release|x64.Build.0 = Release|x64 {DB68AB21-510B-4BA1-9E6F-E5731D8647BC}.Debug|x64.ActiveCfg = Debug|x64 {DB68AB21-510B-4BA1-9E6F-E5731D8647BC}.Debug|x64.Build.0 = Debug|x64 {DB68AB21-510B-4BA1-9E6F-E5731D8647BC}.Release|x64.ActiveCfg = Release|x64 {DB68AB21-510B-4BA1-9E6F-E5731D8647BC}.Release|x64.Build.0 = Release|x64 {DE068BE1-A8E9-48A2-B216-92A7CE5EA4CE}.Debug|x64.ActiveCfg = Debug|x64 {DE068BE1-A8E9-48A2-B216-92A7CE5EA4CE}.Debug|x64.Build.0 = Debug|x64 {DE068BE1-A8E9-48A2-B216-92A7CE5EA4CE}.Release|x64.ActiveCfg = Release|x64 {DE068BE1-A8E9-48A2-B216-92A7CE5EA4CE}.Release|x64.Build.0 = Release|x64 {DEA3CD0A-8781-4ABE-9A7D-00B91132FED0}.Debug|x64.ActiveCfg = Debug|x64 {DEA3CD0A-8781-4ABE-9A7D-00B91132FED0}.Debug|x64.Build.0 = Debug|x64 {DEA3CD0A-8781-4ABE-9A7D-00B91132FED0}.Release|x64.ActiveCfg = Release|x64 {DEA3CD0A-8781-4ABE-9A7D-00B91132FED0}.Release|x64.Build.0 = Release|x64 {E07C9A5F-B2E4-44FB-AA87-FBC885AC955D}.Debug|x64.ActiveCfg = Debug|x64 {E07C9A5F-B2E4-44FB-AA87-FBC885AC955D}.Debug|x64.Build.0 = Debug|x64 {E07C9A5F-B2E4-44FB-AA87-FBC885AC955D}.Release|x64.ActiveCfg = Release|x64 {E07C9A5F-B2E4-44FB-AA87-FBC885AC955D}.Release|x64.Build.0 = Release|x64 {E4E2EC33-7902-45D0-9C3C-ADBAFA46874A}.Debug|x64.ActiveCfg = Debug|x64 {E4E2EC33-7902-45D0-9C3C-ADBAFA46874A}.Debug|x64.Build.0 = Debug|x64 {E4E2EC33-7902-45D0-9C3C-ADBAFA46874A}.Release|x64.ActiveCfg = Release|x64 {E4E2EC33-7902-45D0-9C3C-ADBAFA46874A}.Release|x64.Build.0 = Release|x64 {E648732D-78FA-427A-928C-9A59222D37B7}.Debug|x64.ActiveCfg = Debug|x64 {E648732D-78FA-427A-928C-9A59222D37B7}.Debug|x64.Build.0 = Debug|x64 {E648732D-78FA-427A-928C-9A59222D37B7}.Release|x64.ActiveCfg = Release|x64 {E648732D-78FA-427A-928C-9A59222D37B7}.Release|x64.Build.0 = Release|x64 {E68DEB59-C709-4945-AF80-EEBCADDED944}.Debug|x64.ActiveCfg = Debug|x64 {E68DEB59-C709-4945-AF80-EEBCADDED944}.Debug|x64.Build.0 = Debug|x64 {E68DEB59-C709-4945-AF80-EEBCADDED944}.Release|x64.ActiveCfg = Release|x64 {E68DEB59-C709-4945-AF80-EEBCADDED944}.Release|x64.Build.0 = Release|x64 {E7691F81-86EF-467D-82E1-F5B9416386F9}.Debug|x64.ActiveCfg = Debug|x64 {E7691F81-86EF-467D-82E1-F5B9416386F9}.Debug|x64.Build.0 = Debug|x64 {E7691F81-86EF-467D-82E1-F5B9416386F9}.Release|x64.ActiveCfg = Release|x64 {E7691F81-86EF-467D-82E1-F5B9416386F9}.Release|x64.Build.0 = Release|x64 {E796AA20-D664-4D05-ABD9-C93A4FBE3E5C}.Debug|x64.ActiveCfg = Debug|x64 {E796AA20-D664-4D05-ABD9-C93A4FBE3E5C}.Debug|x64.Build.0 = Debug|x64 {E796AA20-D664-4D05-ABD9-C93A4FBE3E5C}.Release|x64.ActiveCfg = Release|x64 {E796AA20-D664-4D05-ABD9-C93A4FBE3E5C}.Release|x64.Build.0 = Release|x64 {E85E017F-04C0-4716-BF21-949C82C68912}.Debug|x64.ActiveCfg = Debug|x64 {E85E017F-04C0-4716-BF21-949C82C68912}.Debug|x64.Build.0 = Debug|x64 {E85E017F-04C0-4716-BF21-949C82C68912}.Release|x64.ActiveCfg = Release|x64 {E85E017F-04C0-4716-BF21-949C82C68912}.Release|x64.Build.0 = Release|x64 {E901B756-EA72-4B8D-967F-85F109D0D1DE}.Debug|x64.ActiveCfg = Debug|x64 {E901B756-EA72-4B8D-967F-85F109D0D1DE}.Debug|x64.Build.0 = Debug|x64 {E901B756-EA72-4B8D-967F-85F109D0D1DE}.Release|x64.ActiveCfg = Release|x64 {E901B756-EA72-4B8D-967F-85F109D0D1DE}.Release|x64.Build.0 = Release|x64 {E9E079D6-25BF-46E3-8075-7D733303DD59}.Debug|x64.ActiveCfg = Debug|x64 {E9E079D6-25BF-46E3-8075-7D733303DD59}.Debug|x64.Build.0 = Debug|x64 {E9E079D6-25BF-46E3-8075-7D733303DD59}.Release|x64.ActiveCfg = Release|x64 {E9E079D6-25BF-46E3-8075-7D733303DD59}.Release|x64.Build.0 = Release|x64 {ED2A831F-4AAF-4CF7-A953-3C45B0EC1BE6}.Debug|x64.ActiveCfg = Debug|x64 {ED2A831F-4AAF-4CF7-A953-3C45B0EC1BE6}.Debug|x64.Build.0 = Debug|x64 {ED2A831F-4AAF-4CF7-A953-3C45B0EC1BE6}.Release|x64.ActiveCfg = Release|x64 {ED2A831F-4AAF-4CF7-A953-3C45B0EC1BE6}.Release|x64.Build.0 = Release|x64 {EDA88BAB-9FA7-4A2D-8974-EFCFA24B3FEB}.Debug|x64.ActiveCfg = Debug|x64 {EDA88BAB-9FA7-4A2D-8974-EFCFA24B3FEB}.Debug|x64.Build.0 = Debug|x64 {EDA88BAB-9FA7-4A2D-8974-EFCFA24B3FEB}.Release|x64.ActiveCfg = Release|x64 {EDA88BAB-9FA7-4A2D-8974-EFCFA24B3FEB}.Release|x64.Build.0 = Release|x64 {EDD5FA29-69AF-445F-842A-132E65D3C92B}.Debug|x64.ActiveCfg = Debug|x64 {EDD5FA29-69AF-445F-842A-132E65D3C92B}.Debug|x64.Build.0 = Debug|x64 {EDD5FA29-69AF-445F-842A-132E65D3C92B}.Release|x64.ActiveCfg = Release|x64 {EDD5FA29-69AF-445F-842A-132E65D3C92B}.Release|x64.Build.0 = Release|x64 {F03DABEE-A03E-4437-BFD3-D012836F2D94}.Debug|x64.ActiveCfg = Debug|x64 {F03DABEE-A03E-4437-BFD3-D012836F2D94}.Debug|x64.Build.0 = Debug|x64 {F03DABEE-A03E-4437-BFD3-D012836F2D94}.Release|x64.ActiveCfg = Release|x64 {F03DABEE-A03E-4437-BFD3-D012836F2D94}.Release|x64.Build.0 = Release|x64 {F0B613C4-1D9A-4259-BD0E-C1B9FF2AA3A0}.Debug|x64.ActiveCfg = Debug|x64 {F0B613C4-1D9A-4259-BD0E-C1B9FF2AA3A0}.Debug|x64.Build.0 = Debug|x64 {F0B613C4-1D9A-4259-BD0E-C1B9FF2AA3A0}.Release|x64.ActiveCfg = Release|x64 {F0B613C4-1D9A-4259-BD0E-C1B9FF2AA3A0}.Release|x64.Build.0 = Release|x64 {F13108C4-4C86-4D56-A317-A4E5892A8AF7}.Debug|x64.ActiveCfg = Debug|x64 {F13108C4-4C86-4D56-A317-A4E5892A8AF7}.Debug|x64.Build.0 = Debug|x64 {F13108C4-4C86-4D56-A317-A4E5892A8AF7}.Release|x64.ActiveCfg = Release|x64 {F13108C4-4C86-4D56-A317-A4E5892A8AF7}.Release|x64.Build.0 = Release|x64 {F3E5650D-834E-45E6-90C7-3FC2AA954929}.Debug|x64.ActiveCfg = Debug|x64 {F3E5650D-834E-45E6-90C7-3FC2AA954929}.Debug|x64.Build.0 = Debug|x64 {F3E5650D-834E-45E6-90C7-3FC2AA954929}.Release|x64.ActiveCfg = Release|x64 {F3E5650D-834E-45E6-90C7-3FC2AA954929}.Release|x64.Build.0 = Release|x64 {F596C36C-5C96-4F08-B420-8908AF500954}.Debug|x64.ActiveCfg = Debug|x64 {F596C36C-5C96-4F08-B420-8908AF500954}.Debug|x64.Build.0 = Debug|x64 {F596C36C-5C96-4F08-B420-8908AF500954}.Release|x64.ActiveCfg = Release|x64 {F596C36C-5C96-4F08-B420-8908AF500954}.Release|x64.Build.0 = Release|x64 {F5D850C9-D353-4B84-99BC-E336C231018C}.Debug|x64.ActiveCfg = Debug|x64 {F5D850C9-D353-4B84-99BC-E336C231018C}.Debug|x64.Build.0 = Debug|x64 {F5D850C9-D353-4B84-99BC-E336C231018C}.Release|x64.ActiveCfg = Release|x64 {F5D850C9-D353-4B84-99BC-E336C231018C}.Release|x64.Build.0 = Release|x64 {F5E2F6C4-19BA-497A-B754-232E4666E647}.Debug|x64.ActiveCfg = Debug|x64 {F5E2F6C4-19BA-497A-B754-232E4666E647}.Debug|x64.Build.0 = Debug|x64 {F5E2F6C4-19BA-497A-B754-232E4666E647}.Release|x64.ActiveCfg = Release|x64 {F5E2F6C4-19BA-497A-B754-232E4666E647}.Release|x64.Build.0 = Release|x64 {F5E2F6C4-19BA-497A-B754-232E469BE647}.Debug|x64.ActiveCfg = Debug|x64 {F5E2F6C4-19BA-497A-B754-232E469BE647}.Debug|x64.Build.0 = Debug|x64 {F5E2F6C4-19BA-497A-B754-232E469BE647}.Release|x64.ActiveCfg = Release|x64 {F5E2F6C4-19BA-497A-B754-232E469BE647}.Release|x64.Build.0 = Release|x64 {F63FB47F-1DCE-48E5-9CBD-F3E0A354472B}.Debug|x64.ActiveCfg = Debug|x64 {F63FB47F-1DCE-48E5-9CBD-F3E0A354472B}.Debug|x64.Build.0 = Debug|x64 {F63FB47F-1DCE-48E5-9CBD-F3E0A354472B}.Release|x64.ActiveCfg = Release|x64 {F63FB47F-1DCE-48E5-9CBD-F3E0A354472B}.Release|x64.Build.0 = Release|x64 {F7C6C6B6-4142-4C82-8699-4A9D8183181B}.Debug|x64.ActiveCfg = Debug|x64 {F7C6C6B6-4142-4C82-8699-4A9D8183181B}.Debug|x64.Build.0 = Debug|x64 {F7C6C6B6-4142-4C82-8699-4A9D8183181B}.Release|x64.ActiveCfg = Release|x64 {F7C6C6B6-4142-4C82-8699-4A9D8183181B}.Release|x64.Build.0 = Release|x64 {FB2D2B18-E616-4639-8593-0E1AF2DA01A8}.Debug|x64.ActiveCfg = Debug|x64 {FB2D2B18-E616-4639-8593-0E1AF2DA01A8}.Debug|x64.Build.0 = Debug|x64 {FB2D2B18-E616-4639-8593-0E1AF2DA01A8}.Release|x64.ActiveCfg = Release|x64 {FB2D2B18-E616-4639-8593-0E1AF2DA01A8}.Release|x64.Build.0 = Release|x64 {FBB77433-639E-42DC-9355-EA94CAE294D2}.Debug|x64.ActiveCfg = Debug|x64 {FBB77433-639E-42DC-9355-EA94CAE294D2}.Debug|x64.Build.0 = Debug|x64 {FBB77433-639E-42DC-9355-EA94CAE294D2}.Release|x64.ActiveCfg = Release|x64 {FBB77433-639E-42DC-9355-EA94CAE294D2}.Release|x64.Build.0 = Release|x64 {FC2248F5-3E9E-495B-9767-87F59614047C}.Debug|x64.ActiveCfg = Debug|x64 {FC2248F5-3E9E-495B-9767-87F59614047C}.Debug|x64.Build.0 = Debug|x64 {FC2248F5-3E9E-495B-9767-87F59614047C}.Release|x64.ActiveCfg = Release|x64 {FC2248F5-3E9E-495B-9767-87F59614047C}.Release|x64.Build.0 = Release|x64 {FCD0587A-4504-4F5E-8E9C-468CC03D250A}.Debug|x64.ActiveCfg = Debug|x64 {FCD0587A-4504-4F5E-8E9C-468CC03D250A}.Debug|x64.Build.0 = Debug|x64 {FCD0587A-4504-4F5E-8E9C-468CC03D250A}.Release|x64.ActiveCfg = Release|x64 {FCD0587A-4504-4F5E-8E9C-468CC03D250A}.Release|x64.Build.0 = Release|x64 {FD726AA3-D4FA-4597-B435-08CC7752888C}.Debug|x64.ActiveCfg = Debug|x64 {FD726AA3-D4FA-4597-B435-08CC7752888C}.Debug|x64.Build.0 = Debug|x64 {FD726AA3-D4FA-4597-B435-08CC7752888C}.Release|x64.ActiveCfg = Release|x64 {FD726AA3-D4FA-4597-B435-08CC7752888C}.Release|x64.Build.0 = Release|x64 {FD726AA3-D4FA-4597-B435-08CC7752888D}.Debug|x64.ActiveCfg = Debug|x64 {FD726AA3-D4FA-4597-B435-08CC7752888D}.Debug|x64.Build.0 = Debug|x64 {FD726AA3-D4FA-4597-B435-08CC7752888D}.Release|x64.ActiveCfg = Release|x64 {FD726AA3-D4FA-4597-B435-08CC7752888D}.Release|x64.Build.0 = Release|x64 {FD726AA3-D4FA-4597-B435-08CC7752888E}.Debug|x64.ActiveCfg = Debug|x64 {FD726AA3-D4FA-4597-B435-08CC7752888E}.Debug|x64.Build.0 = Debug|x64 {FD726AA3-D4FA-4597-B435-08CC7752888E}.Release|x64.ActiveCfg = Release|x64 {FD726AA3-D4FA-4597-B435-08CC7752888E}.Release|x64.Build.0 = Release|x64 {FEA09B48-34C2-4963-8A5A-F97BDA136D72}.Debug|x64.ActiveCfg = Debug|x64 {FEA09B48-34C2-4963-8A5A-F97BDA136D72}.Debug|x64.Build.0 = Debug|x64 {FEA09B48-34C2-4963-8A5A-F97BDA136D72}.Release|x64.ActiveCfg = Release|x64 {FEA09B48-34C2-4963-8A5A-F97BDA136D72}.Release|x64.Build.0 = Release|x64 {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA79}.Debug|x64.ActiveCfg = Debug|x64 {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA79}.Debug|x64.Build.0 = Debug|x64 {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA79}.Release|x64.ActiveCfg = Release|x64 {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA79}.Release|x64.Build.0 = Release|x64 {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA80}.Debug|x64.ActiveCfg = Debug|x64 {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA80}.Debug|x64.Build.0 = Debug|x64 {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA80}.Release|x64.ActiveCfg = Release|x64 {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA80}.Release|x64.Build.0 = Release|x64 EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE EndGlobalSection GlobalSection(NestedProjects) = preSolution {0056B0B6-CB3E-4F0E-B6DC-48D59CB8E235} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {019F5586-5558-4C87-B319-85906D4AE407} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {025E7D51-41F2-4CBA-956E-C37A4443DB1B} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {0287C3DC-AE03-4714-AAFF-C52F062ECA6F} = {1434B17C-6165-4D42-BEA1-5A7730D5A6BB} {02BC3B44-C7F1-4793-86C1-6F36CA8A7F53} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {03228F84-4F41-4BCC-8C2D-F329DC87B289} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {0388E945-A655-41A7-AF27-8981CEE0E49A} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {03B54A12-7793-4827-B820-C07491F7F45E} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {0529575C-F6E8-44FD-BB82-82A29948D0F2} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {063037B2-CA35-4520-811C-19D9C4ED891E} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {06877FED-15BA-421F-85C9-1A964FB97446} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {0703E813-9CC8-4DEA-AA33-42B099CD172D} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {07A153D9-DF17-4DE8-A3C2-EBF171B961AE} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {08B62E36-63D2-4FF1-A605-4BBABAEE73FB} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {0A049EAD-652F-4E20-8026-90FD99AEE77A} = {1A36B57B-2E88-4D81-89C0-F575C9895E36} {0B1818EB-BDC8-4865-964F-DB8BF05CFD86} = {853D45D8-980C-4991-B62A-DAC6FD245402} {0BFD78AA-FD94-4DB1-8495-8F5CC06D8F03} = {BFEDF709-A700-4769-9056-ACA934D828A8} {0CC6D525-806E-433F-AB4A-6CFD546418B1} = {853D45D8-980C-4991-B62A-DAC6FD245402} {0CDCEB97-3270-4939-A290-EA2D3BE34B0C} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {0D4E38EF-A9D5-4797-8994-5DBB1125C9EA} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {0FB8F0FD-276C-413B-97A8-67ABE0C9043B} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {10469175-EEF7-44A0-9961-AC4E45EFD800} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {11D76FBC-DFAA-4B31-9DB0-206E171E3F94} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {11E158AE-C85A-4A6E-B66A-ED2994709276} = {F09A0864-9221-47AD-872F-D4538104D747} {12A1A3EF-202C-4DD0-9B5A-F5126CAB078F} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {1434B17C-6165-4D42-BEA1-5A7730D5A6BB} = {0CC6D525-806E-433F-AB4A-6CFD546418B1} {1464398A-100F-4518-BDB9-939A6362B6CF} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {179BEB5A-2C90-44F5-A734-FA756A5E668C} = {F09A0864-9221-47AD-872F-D4538104D747} {17A4B817-68B1-4719-A9EF-BD8FAB747DE6} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {181A4234-282C-41F0-85C2-2B7697B3CB1A} = {F18C84B3-7898-4324-9D75-99A6048F442D} {18E90E1A-F2E0-40DF-9900-A14E560C9EB4} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {1A36B57B-2E88-4D81-89C0-F575C9895E36} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {1B871BA2-3F70-4BC9-9DF4-725EB07F6628} = {F09A0864-9221-47AD-872F-D4538104D747} {1B9B0D6D-E530-44A6-ADAE-09EA2BDC47DE} = {E23BB160-006E-44F2-8FB4-3A2240BBC20C} {1BAA1617-93AE-4196-8A1A-BD492FB18AEF} = {853D45D8-980C-4991-B62A-DAC6FD245402} {1EB3DE5B-6357-498D-8CAC-EEC0209EA454} = {E3229AF7-1FA2-4632-BB0B-B74F709F1A33} {1F2E1C51-2B14-4047-BE6D-52E00FC3C780} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {2498FCDA-E2CC-43EF-9A35-8CD63F253171} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {25758581-DD46-4AE4-99D9-11E736F72AD1} = {F09A0864-9221-47AD-872F-D4538104D747} {26166DF1-3C94-44AF-9075-BA31DCD2F6BB} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {27FA11C6-431D-41D1-A417-FAB7C4F93DCA} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {296F3C5D-3951-423E-8E2F-FD4A37958C72} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {29D9376B-DC36-4940-83F1-A7CBE38A2103} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {2A1D6AF2-7336-4966-A4B3-0BE9A24BAE00} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {2B1A5104-A324-4D02-B5C7-D021FB8F880C} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {2B2DE575-1422-4FBF-97BE-35AEDA0AB465} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {2B7772E6-9DAA-4F38-B0BC-7B2399366325} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {2C24CC4F-B340-467D-908F-1BF2C69BC79F} = {F18C84B3-7898-4324-9D75-99A6048F442D} {2CD7408E-2F60-43C3-ACEB-C7D58CDD8462} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {2DE6B085-3C19-49B1-894A-AD9376000E09} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {2ED26FDA-3C4E-4514-B387-5E77C302FF71} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {2EFFC590-BF5E-46A2-AF04-E67E1D571D2E} = {F09A0864-9221-47AD-872F-D4538104D747} {2F543422-4B8A-4898-BE6B-590F52B4E9D1} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {3142CB13-CADA-48D3-9A25-E6ACB243760A} = {F09A0864-9221-47AD-872F-D4538104D747} {34DB4951-DA08-45F1-938D-B08E5FF5AB46} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {3799BA67-3C4F-4AE0-85DC-5BAAEA01A180} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {3B23831B-E5DE-4A62-9D0B-27D0D9F293F4} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {3CF270CD-0F56-48E3-AD84-82F369C568BF} = {1A36B57B-2E88-4D81-89C0-F575C9895E36} {3EC30D6A-BDA4-4971-879A-8814204EAE31} = {F09A0864-9221-47AD-872F-D4538104D747} {3ECCB0F1-3ADF-486A-91C5-79DF0FC22F78} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {3ED56E55-84A6-422C-A8D4-A8439FB8F245} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {42B97D47-F800-4100-BFA2-B3AC357E8B6B} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {42CCEF95-5ADD-460C-967E-DD5B2C744943} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {433F7840-C597-4950-84C9-E4FF7DF6A298} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {45027FC5-4A32-47BD-AC5B-66CC7616B1D2} = {9A8482A7-BF0C-423D-8266-189456ED41F6} {46B82069-10BE-432A-8D93-F4D995148555} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {4850F425-9128-4E91-973C-5AE7BD97395B} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {492BAA3D-0D5D-478E-9765-500463AE69AA} = {853D45D8-980C-4991-B62A-DAC6FD245402} {49A7CC5A-D5E7-4A07-917F-C6918B982BE8} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {4C291EEB-3874-4724-9CC2-1335D13FF0EE} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {4C429783-0B01-449F-A36F-C2019233890B} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {4C6E7F0A-7E6A-4713-B1D2-B7B4ADC992AF} = {F09A0864-9221-47AD-872F-D4538104D747} {4E334022-7A71-4197-9E15-878F7EFC877E} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {4FB4FF90-4E92-4CFB-A01F-C73D6861CA03} = {1A36B57B-2E88-4D81-89C0-F575C9895E36} {50FD1E47-2131-48D2-9435-5CB28DF6B15A} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {513C4CFA-BD5B-4470-BA93-F6D43778A754} = {C721EFBD-45DC-479E-9B99-E62FCC1FC6E5} {53115A01-460C-4339-A2C8-AE1323A6E7EA} = {F09A0864-9221-47AD-872F-D4538104D747} {5580D11C-FDA6-4CF2-A0E8-1C2D3FBC11F1} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {581B3A58-F3F0-4765-91E5-D0C82816A528} = {C721EFBD-45DC-479E-9B99-E62FCC1FC6E5} {58386481-30B7-40FC-96AF-0723A4A7B228} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {59AB6976-D16B-48D0-8D16-94360D3FE51D} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {59D7A9CD-9912-40E4-96E1-8A873F777F62} = {6D63CDF1-F62C-4614-AD8A-95B0A63AA070} {5A391A14-8E29-4788-93FC-EDADED31D32F} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {5AD07646-5E16-4CEF-B80A-BE5EE4D54FEF} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {5B2B9C0D-1B6D-4357-8307-6DE1EE0A41A3} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {5D362DB7-D2BD-4907-AAD8-4B8627E72282} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {5DB2E259-0D19-4A89-B8EC-B2912F39924D} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {5E7305DB-93E6-448B-AE44-90EAF916A776} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {5EC35099-9777-45E8-9520-EB2EE75BDF88} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {5F2B687A-1B42-439C-AEEC-135DD22FB851} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {5F8A56F8-2C5B-48B6-9654-DD642D3E5F5C} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {60206D22-E132-4695-8486-10BECA32C5CC} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {60B463D4-8CD5-4BF6-A25B-01BE13B87590} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {60EF55C7-8399-4543-B5B2-3AE2C532C67E} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {628FADA9-7047-4DD9-BD17-9FE4B5A1ADB0} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {63B8184D-85E0-4E6A-9729-558C567D1D1D} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {643B82A1-D009-46A9-92A0-2883399B05C2} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {6516D6CF-8000-4341-9487-312BC83EE370} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {65D92D98-97E1-48F7-AEF6-75221CF48EA4} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {673277EC-D26B-414D-92E3-84EE873316A8} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {67AC1343-98FD-4143-92C0-559C55F749F5} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {6851356E-A5D9-46A6-8262-A7E208729F18} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {6AE1B8BE-D46A-4E99-87A2-F160FB950DCA} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {6B492754-9F80-44B3-A2A7-1D98AF06F3B2} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {6BCEF2A5-0CEC-4CC6-9CB0-D3FBF871A408} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {6D63CDF1-F62C-4614-AD8A-95B0A63AA070} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {6D7C1169-3246-465F-B630-ECFEF4F3179A} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {6DBD8C02-0C75-4DB0-BFDA-CD053B1B2D89} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {6EC93484-AAF3-487E-84E4-5ABFBA0AFC53} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {6F06A19B-0921-4B71-A3A5-B350B5FFEADB} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {6F4953DA-FDC3-46CF-BF24-3752CCF2E1CB} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {6F776280-B383-4DCE-8F42-9670164D038D} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {70EE1D40-0C65-4985-8EFC-BD40EE3A89B2} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {715EADD7-0FFE-4F1F-94E7-49302968DF79} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {71D182E0-345A-4375-B0FA-3536821B0EE3} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {7264C8F6-73FB-4830-9306-1558D3EAC71B} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {729E3905-FF7D-49C5-9871-6D35D839183E} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {72C9DB46-C665-48AD-B805-BA885B40CA3E} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {7337E34A-97B0-44FC-988B-7E6AE7E0FBBF} = {6D63CDF1-F62C-4614-AD8A-95B0A63AA070} {740ED97D-005F-4F58-98B2-4EF5EF5776E8} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {746BA101-5C93-42A5-AC7A-64DCEB186572} = {853D45D8-980C-4991-B62A-DAC6FD245402} {74D655D5-F661-4887-A1EB-5A6222AF5FCA} = {E3229AF7-1FA2-4632-BB0B-B74F709F1A33} {7701627C-CFD9-48F6-942E-EAACC8D057FA} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {774627B7-6532-4464-AEE4-02F72CA44F95} = {9A8482A7-BF0C-423D-8266-189456ED41F6} {7783BC49-A25B-468B-A6F8-AB6B39A91C65} = {F18C84B3-7898-4324-9D75-99A6048F442D} {779425B1-2211-499B-A7CC-4F9EC6CB0D25} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {79D37FFE-FF76-44B3-BB27-3DCAEFF2EBE9} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {7ABF755C-821B-49CD-8EDE-83C16594FF7F} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {7DC3B3DD-73ED-4602-9AF3-8D7053620DEA} = {877E7D1D-8150-4FE5-A139-B6FBCEAEC393} {7DFEB4A5-8B04-4302-9D09-8144918FCF81} = {E23BB160-006E-44F2-8FB4-3A2240BBC20C} {7F51CD29-3BCD-4DD8-B327-F384B5A616D1} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {8008010F-8718-4C5F-86B2-195AEBF73422} = {C721EFBD-45DC-479E-9B99-E62FCC1FC6E5} {8010BBB0-C71B-4EFF-95EB-65C01E5EC197} = {C721EFBD-45DC-479E-9B99-E62FCC1FC6E5} {80AF1B7D-B8CE-4AF0-AE3B-1DABED1B57E7} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {810DB909-6581-42D8-9616-906888F12149} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {85D4076B-896B-4EBB-8F3A-8B44C24CD452} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {85DBDA9B-AEF6-43E7-B8B5-05FF2BEC61A3} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {86EE22CC-6D3C-4F81-ADC8-394946F0DA81} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {877E7D1D-8150-4FE5-A139-B6FBCEAEC393} = {853D45D8-980C-4991-B62A-DAC6FD245402} {87A32959-E477-4CD5-8A1C-C85646D806B2} = {F18C84B3-7898-4324-9D75-99A6048F442D} {88D239E4-EB7D-4E0A-BE3A-AD78B9F408FC} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {89F947CA-DDEF-4131-8AFB-584ABA4A1302} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {8A0FA780-068A-4534-AA2F-4FF4CF977AF2} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {8A4872D7-A234-4B9B-8215-82C6BB15F3A2} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {8C42CA7C-1543-4F1B-A55F-28CD419C7D35} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {8C6D73E0-0A6F-4487-A040-0EC78D7D6D9A} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {8D75FA1A-EC74-4F88-8AC1-CE3F98E4D828} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {8E374371-30E1-4623-8755-2A2F3742170B} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} = {95FAF291-03D1-42FC-9C10-424D551D475D} {9186EAC4-2F34-4F17-B940-6585D7869BCD} = {95FAF291-03D1-42FC-9C10-424D551D475D} {91C30620-70CA-46C7-AC71-71F3C602690E} = {0CC6D525-806E-433F-AB4A-6CFD546418B1} {91E19AEB-7B75-43E0-B8B4-D2BB60D839EA} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {92388A20-50FC-45F8-89E3-71F1618EFABB} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {924B2937-0B53-4DC6-B7E1-5F3102728F89} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {95B683BD-B9DC-400F-9BC0-8F1505F08BF5} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {95FAF291-03D1-42FC-9C10-424D551D475D} = {853D45D8-980C-4991-B62A-DAC6FD245402} {96D00A19-5CEF-4CC5-BDE8-E33C68BCE90F} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {98ACBE5D-1A92-46F9-AA81-533412172952} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {99F7F00F-1DE5-45EA-992B-64BA282FAC76} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {9A4078F8-B8E4-4EC6-A6FF-4F29DAD9CE48} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {9A8482A7-BF0C-423D-8266-189456ED41F6} = {95FAF291-03D1-42FC-9C10-424D551D475D} {9AE2DAF9-10C4-4EC3-AE52-AD5EE9C77C55} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {9C37B8CC-F810-4787-924D-65BC227091A3} = {853D45D8-980C-4991-B62A-DAC6FD245402} {9D9E33EB-4C24-4646-A3FB-35DA17247917} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {9E9E3D25-2139-4A5D-9200-18148DDEAD45} = {853D45D8-980C-4991-B62A-DAC6FD245402} {9FF51F3E-AF36-4F45-A797-C5F03A090298} = {91C30620-70CA-46C7-AC71-71F3C602690E} {9FF62356-30B4-42A1-8DC7-45262A18DD44} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {A14A4556-9092-430D-B9CA-B2B1223D56CB} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {A216BF23-FC5C-4426-BF20-8568A2AA5FA0} = {F09A0864-9221-47AD-872F-D4538104D747} {A2A0FAEA-2B7C-4FC3-B904-1DB4DEACF88D} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {A38EFCDB-53D6-4474-97F3-0DDC6CE70D76} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {A39D1640-8DBA-450D-9103-2533C248991A} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {A57D9365-172E-4782-ADC6-82A594E30943} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {A79E3093-B157-4B09-BABD-29266EA16407} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {A7CA7975-CEDB-48E6-9AEB-1209DCBD07F2} = {91C30620-70CA-46C7-AC71-71F3C602690E} {AB15A115-E429-4123-BEBF-206FBA4CF615} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {AE1C32FB-9B52-4760-ABFC-0D2FA2C7A6C8} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {AE952763-5C84-43FC-B344-CACC950F056C} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {AE9E908D-BAEC-491F-9914-436B3CE35E94} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {AEAA72CD-E060-417C-9CA1-49B4738384E0} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {AF038868-2432-4159-A62F-941F11D12C5D} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {AF0B7480-EBE3-486B-B0C8-134910BC9324} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {B30C6212-A160-405A-8FE7-340E721738A2} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {B35BFA09-DE68-483B-AB61-8790E8F060A8} = {F09A0864-9221-47AD-872F-D4538104D747} {B36F115C-8139-4C35-A3E7-E6BF9F3DA793} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {B379539C-E130-460D-AE82-4EBDD1A97845} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {B3AF8A19-5802-4A34-9157-27BBE4E53C0A} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {B440BB05-37A8-42EA-98D3-D83EB113E497} = {E23BB160-006E-44F2-8FB4-3A2240BBC20C} {B6C0521B-EECA-47EF-BFA8-147F9C3F6DFF} = {A14A4556-9092-430D-B9CA-B2B1223D56CB} {B6DA6617-D98F-4A4D-A7C4-A317212924BF} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {B6F4B85D-FE55-4A1B-AE97-D4A9ECFE195F} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {B775480C-5B32-4F64-B026-47367280EC56} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {B870D8A6-12CD-4DD0-B843-833695C2310A} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {B887EA26-846C-4D6A-B0E4-432487506BC7} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {B8A4320D-E9A3-4F89-A8AA-B16D746C158A} = {F18C84B3-7898-4324-9D75-99A6048F442D} {BABC6427-E533-4DCF-91E3-B5B2ED253F46} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {BAE107BA-7618-4972-8188-2D3CDAAE0453} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {BB1120CF-B721-4EF9-8735-58F76AE51D2F} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {BB248BAC-6E1B-433C-A254-75140A273AB5} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {BE18F227-A9F0-4B38-B689-4E2F9F09CA5F} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {BFEDF709-A700-4769-9056-ACA934D828A8} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {C0E811E0-8942-4CFD-A817-74D99E9E6577} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {C2C36D03-26EE-4BD8-8FFC-86CFE16C1218} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {C2F94489-A483-4C44-B8A7-11A75F6AEC66} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {C35052AF-2383-4F9C-B18B-55A01829F2BF} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {C3CEE34C-29E0-4A22-B258-3FBAF662AA19} = {91C30620-70CA-46C7-AC71-71F3C602690E} {C5E8B8DB-2507-4904-847F-A52196B075F0} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {C71DAF3E-9361-4723-93E2-C475D1D0C0D0} = {1A36B57B-2E88-4D81-89C0-F575C9895E36} {C721EFBD-45DC-479E-9B99-E62FCC1FC6E5} = {0CC6D525-806E-433F-AB4A-6CFD546418B1} {C7E42AE1-052F-4024-B8BA-DE5DCE6BBEEC} = {C721EFBD-45DC-479E-9B99-E62FCC1FC6E5} {C973CD39-D63B-4F5C-BE1D-DED17388B5A4} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {CA4BBB24-D33E-42E2-A495-F10D80DE8C1D} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {CB906E89-1313-4929-AFF7-86FBF1CC301F} = {9C37B8CC-F810-4787-924D-65BC227091A3} {CCA9B681-D10B-45E4-98CC-531503D2EDE8} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {CDD9DFC6-5C3D-42F7-B822-FE29A1C21752} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {CE3F2DFB-8470-4802-AD37-21CAF6CB2681} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {CF9A0883-6334-44C7-AC29-349468C78E27} = {853D45D8-980C-4991-B62A-DAC6FD245402} {CF9F4CEA-EC66-4E78-A086-107EB29E0637} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {D062166F-0EC7-4C13-A772-0C7157EEFE41} = {1434B17C-6165-4D42-BEA1-5A7730D5A6BB} {D140560D-FDEC-4D3D-8F58-BF5FD5E4DAA1} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {D28F5FF6-8401-4E0D-94F9-3A1FD7ED64E3} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {D2C30C7E-A7D3-487A-956E-418CECAFFE8E} = {F09A0864-9221-47AD-872F-D4538104D747} {D3A99F36-4B72-4766-ABCD-CCEDC26DD139} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {D4035736-1AD6-4100-9FA9-A8A0C1DAE0C7} = {59AB6976-D16B-48D0-8D16-94360D3FE51D} {D829DB63-E046-474D-8EA3-43A6659294D8} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {D8317F1D-7A70-4A39-977A-EAB05A04A87B} = {E23BB160-006E-44F2-8FB4-3A2240BBC20C} {D88187D2-1977-4C5F-B0CD-83C69BD6C1BC} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {D93A2683-6D99-4F18-B378-91195D23E007} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {D9A70E35-0C85-4A09-ACA8-B15B21B66F50} = {A14A4556-9092-430D-B9CA-B2B1223D56CB} {DB68AB21-510B-4BA1-9E6F-E5731D8647BC} = {BFBAB433-860E-4A28-96E3-A4B7AFE3B297} {DE068BE1-A8E9-48A2-B216-92A7CE5EA4CE} = {A14A4556-9092-430D-B9CA-B2B1223D56CB} {DEA3CD0A-8781-4ABE-9A7D-00B91132FED0} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {E07C9A5F-B2E4-44FB-AA87-FBC885AC955D} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {E23BB160-006E-44F2-8FB4-3A2240BBC20C} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {E3229AF7-1FA2-4632-BB0B-B74F709F1A33} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {E4E2EC33-7902-45D0-9C3C-ADBAFA46874A} = {F8373EDD-1B9E-462D-BF23-55638E23E98B} {E648732D-78FA-427A-928C-9A59222D37B7} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {E68DEB59-C709-4945-AF80-EEBCADDED944} = {1A36B57B-2E88-4D81-89C0-F575C9895E36} {E7691F81-86EF-467D-82E1-F5B9416386F9} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {E796AA20-D664-4D05-ABD9-C93A4FBE3E5C} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {E85E017F-04C0-4716-BF21-949C82C68912} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {E901B756-EA72-4B8D-967F-85F109D0D1DE} = {1A36B57B-2E88-4D81-89C0-F575C9895E36} {E9E079D6-25BF-46E3-8075-7D733303DD59} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {ED2A831F-4AAF-4CF7-A953-3C45B0EC1BE6} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {EDA88BAB-9FA7-4A2D-8974-EFCFA24B3FEB} = {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} {EDD5FA29-69AF-445F-842A-132E65D3C92B} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {F03DABEE-A03E-4437-BFD3-D012836F2D94} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {F09A0864-9221-47AD-872F-D4538104D747} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {F0B613C4-1D9A-4259-BD0E-C1B9FF2AA3A0} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {F13108C4-4C86-4D56-A317-A4E5892A8AF7} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {F3E5650D-834E-45E6-90C7-3FC2AA954929} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {F42C09CD-ABA5-4DA9-8383-5EA40FA4D763} = {0CC6D525-806E-433F-AB4A-6CFD546418B1} {F596C36C-5C96-4F08-B420-8908AF500954} = {853D45D8-980C-4991-B62A-DAC6FD245402} {F5D850C9-D353-4B84-99BC-E336C231018C} = {BFEDF709-A700-4769-9056-ACA934D828A8} {F5E2F6C4-19BA-497A-B754-232E4666E647} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {F5E2F6C4-19BA-497A-B754-232E469BE647} = {BD6CC700-B36B-435B-BAF9-FC5AFCD766C9} {F63FB47F-1DCE-48E5-9CBD-F3E0A354472B} = {E23BB160-006E-44F2-8FB4-3A2240BBC20C} {F7C6C6B6-4142-4C82-8699-4A9D8183181B} = {853D45D8-980C-4991-B62A-DAC6FD245402} {F8373EDD-1B9E-462D-BF23-55638E23E98B} = {746BA101-5C93-42A5-AC7A-64DCEB186572} {F8CCA5AE-2D75-4C79-BEAB-2588CD5956C8} = {853D45D8-980C-4991-B62A-DAC6FD245402} {FB2D2B18-E616-4639-8593-0E1AF2DA01A8} = {2F543422-4B8A-4898-BE6B-590F52B4E9D1} {FBB77433-639E-42DC-9355-EA94CAE294D2} = {1A36B57B-2E88-4D81-89C0-F575C9895E36} {FC2248F5-3E9E-495B-9767-87F59614047C} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {FCD0587A-4504-4F5E-8E9C-468CC03D250A} = {1434B17C-6165-4D42-BEA1-5A7730D5A6BB} {FD726AA3-D4FA-4597-B435-08CC7752888C} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {FD726AA3-D4FA-4597-B435-08CC7752888D} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {FD726AA3-D4FA-4597-B435-08CC7752888E} = {4C291EEB-3874-4724-9CC2-1335D13FF0EE} {FEA09B48-34C2-4963-8A5A-F97BDA136D72} = {B870D8A6-12CD-4DD0-B843-833695C2310A} {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA79} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} {FF6E5B0C-DC00-4C93-B9C2-63D1E858BA80} = {63C9B3F8-437D-4AD9-B32D-D04AE38C35B6} EndGlobalSection GlobalSection(ExtensibilityGlobals) = postSolution SolutionGuid = {5E690324-2D48-486A-8D3C-DCB520D3F693} EndGlobalSection EndGlobal pmdk-1.8/src/libvmmalloc/0000775000000000000000000000000013615011243014101 5ustar rootrootpmdk-1.8/src/libvmmalloc/README.md0000664000000000000000000000012613615011243015357 0ustar rootrootThis library has been moved to a [separate repository](https://github.com/pmem/vmem). pmdk-1.8/src/freebsd/0000775000000000000000000000000013615011243013212 5ustar rootrootpmdk-1.8/src/freebsd/README0000664000000000000000000000115713615011243014076 0ustar rootrootPersistent Memory Development Kit This is src/freebsd/README. This directory contains FreeBSD-specific files for the Persistent Memory Development Kit. The subdirectory "include" contains header files that have no equivalents on FreeBSD. Most of these files are empty, which is a cheap trick to avoid preprocessor errors when including non-existing files. Others are redirects for files that are in different locations on FreeBSD. This way we don't need a lot of preprocessor conditionals in all the source code files, although it does require conditionals in the Makefiles (which could be addressed by using autoconf). pmdk-1.8/src/freebsd/include/0000775000000000000000000000000013615011243014635 5ustar rootrootpmdk-1.8/src/freebsd/include/features.h0000664000000000000000000000315113615011243016624 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * features.h -- Empty file redirect */ pmdk-1.8/src/freebsd/include/sys/0000775000000000000000000000000013615011243015453 5ustar rootrootpmdk-1.8/src/freebsd/include/sys/sysmacros.h0000664000000000000000000000315613615011243017654 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * sys/sysmacros.h -- Empty file redirect */ pmdk-1.8/src/freebsd/include/endian.h0000664000000000000000000000322013615011243016241 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * endian.h -- redirect for FreeBSD */ #include pmdk-1.8/src/freebsd/include/linux/0000775000000000000000000000000013615011243015774 5ustar rootrootpmdk-1.8/src/freebsd/include/linux/limits.h0000664000000000000000000000315513615011243017452 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * linux/limits.h -- Empty file redirect */ pmdk-1.8/src/freebsd/include/linux/kdev_t.h0000664000000000000000000000315513615011243017425 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * linux/kdev_t.h -- Empty file redirect */ pmdk-1.8/src/LongPath.manifest0000664000000000000000000000052313615011243015044 0ustar rootroot true pmdk-1.8/src/libpmemblk/0000775000000000000000000000000013615011243013716 5ustar rootrootpmdk-1.8/src/libpmemblk/btt.h0000664000000000000000000000653713615011243014673 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * btt.h -- btt module definitions */ #ifndef BTT_H #define BTT_H 1 #ifdef __cplusplus extern "C" { #endif /* callback functions passed to btt_init() */ struct ns_callback { int (*nsread)(void *ns, unsigned lane, void *buf, size_t count, uint64_t off); int (*nswrite)(void *ns, unsigned lane, const void *buf, size_t count, uint64_t off); int (*nszero)(void *ns, unsigned lane, size_t count, uint64_t off); ssize_t (*nsmap)(void *ns, unsigned lane, void **addrp, size_t len, uint64_t off); void (*nssync)(void *ns, unsigned lane, void *addr, size_t len); int ns_is_zeroed; }; struct btt_info; struct btt *btt_init(uint64_t rawsize, uint32_t lbasize, uint8_t parent_uuid[], unsigned maxlane, void *ns, const struct ns_callback *ns_cbp); unsigned btt_nlane(struct btt *bttp); size_t btt_nlba(struct btt *bttp); int btt_read(struct btt *bttp, unsigned lane, uint64_t lba, void *buf); int btt_write(struct btt *bttp, unsigned lane, uint64_t lba, const void *buf); int btt_set_zero(struct btt *bttp, unsigned lane, uint64_t lba); int btt_set_error(struct btt *bttp, unsigned lane, uint64_t lba); int btt_check(struct btt *bttp); void btt_fini(struct btt *bttp); uint64_t btt_flog_size(uint32_t nfree); uint64_t btt_map_size(uint32_t external_nlba); uint64_t btt_arena_datasize(uint64_t arena_size, uint32_t nfree); int btt_info_set(struct btt_info *info, uint32_t external_lbasize, uint32_t nfree, uint64_t arena_size, uint64_t space_left); struct btt_flog *btt_flog_get_valid(struct btt_flog *flog_pair, int *next); int map_entry_is_initial(uint32_t map_entry); void btt_info_convert2h(struct btt_info *infop); void btt_info_convert2le(struct btt_info *infop); void btt_flog_convert2h(struct btt_flog *flogp); void btt_flog_convert2le(struct btt_flog *flogp); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmemblk/libpmemblk_main.c0000664000000000000000000000421013615011243017201 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmemblk_main.c -- entry point for libpmemblk.dll * * XXX - This is a placeholder. All the library initialization/cleanup * that is done in library ctors/dtors, as well as TLS initialization * should be moved here. */ void libpmemblk_init(void); void libpmemblk_fini(void); int APIENTRY DllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID lpReserved) { switch (dwReason) { case DLL_PROCESS_ATTACH: libpmemblk_init(); break; case DLL_THREAD_ATTACH: case DLL_THREAD_DETACH: break; case DLL_PROCESS_DETACH: libpmemblk_fini(); break; } return TRUE; } pmdk-1.8/src/libpmemblk/blk.h0000664000000000000000000000763613615011243014653 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * blk.h -- internal definitions for libpmem blk module */ #ifndef BLK_H #define BLK_H 1 #include #include "ctl.h" #include "os_thread.h" #include "pool_hdr.h" #include "page_size.h" #ifdef __cplusplus extern "C" { #endif #include "alloc.h" #include "fault_injection.h" #define PMEMBLK_LOG_PREFIX "libpmemblk" #define PMEMBLK_LOG_LEVEL_VAR "PMEMBLK_LOG_LEVEL" #define PMEMBLK_LOG_FILE_VAR "PMEMBLK_LOG_FILE" /* attributes of the blk memory pool format for the pool header */ #define BLK_HDR_SIG "PMEMBLK" /* must be 8 bytes including '\0' */ #define BLK_FORMAT_MAJOR 1 #define BLK_FORMAT_FEAT_DEFAULT \ {POOL_FEAT_COMPAT_DEFAULT, POOL_FEAT_INCOMPAT_DEFAULT, 0x0000} #define BLK_FORMAT_FEAT_CHECK \ {POOL_FEAT_COMPAT_VALID, POOL_FEAT_INCOMPAT_VALID, 0x0000} static const features_t blk_format_feat_default = BLK_FORMAT_FEAT_DEFAULT; struct pmemblk { struct pool_hdr hdr; /* memory pool header */ /* root info for on-media format... */ uint32_t bsize; /* block size */ /* flag indicating if the pool was zero-initialized */ int is_zeroed; /* some run-time state, allocated out of memory pool... */ void *addr; /* mapped region */ size_t size; /* size of mapped region */ int is_pmem; /* true if pool is PMEM */ int rdonly; /* true if pool is opened read-only */ void *data; /* post-header data area */ size_t datasize; /* size of data area */ size_t nlba; /* number of LBAs in pool */ struct btt *bttp; /* btt handle */ unsigned nlane; /* number of lanes */ unsigned next_lane; /* used to rotate through lanes */ os_mutex_t *locks; /* one per lane */ int is_dev_dax; /* true if mapped on device dax */ struct ctl *ctl; /* top level node of the ctl tree structure */ struct pool_set *set; /* pool set info */ #ifdef DEBUG /* held during read/write mprotected sections */ os_mutex_t write_lock; #endif }; /* data area starts at this alignment after the struct pmemblk above */ #define BLK_FORMAT_DATA_ALIGN ((uintptr_t)PMEM_PAGESIZE) #if FAULT_INJECTION void pmemblk_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at); int pmemblk_fault_injection_enabled(void); #else static inline void pmemblk_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { abort(); } static inline int pmemblk_fault_injection_enabled(void) { return 0; } #endif #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmemblk/btt.c0000664000000000000000000016263513615011243014670 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * btt.c -- block translation table providing atomic block updates * * This is a user-space implementation of the BTT mechanism providing * single block powerfail write atomicity, as described by: * The NVDIMM Namespace Specification * * To use this module, the caller must provide five routines for * accessing the namespace containing the data (in this context, * "namespace" refers to the storage containing the BTT layout, such * as a file). All namespace I/O is done by these callbacks: * * nsread Read count bytes from namespace at offset off * nswrite Write count bytes to namespace at offset off * nszero Zero count bytes in namespace at offset off * nsmap Return direct access to a range of a namespace * nssync Flush changes made to an nsmap'd range * * Data written by the nswrite callback is flushed out to the media * (made durable) when the call returns. Data written directly via * the nsmap callback must be flushed explicitly using nssync. * * The caller passes these callbacks, along with information such as * namespace size and UUID to btt_init() and gets back an opaque handle * which is then used with the rest of the entry points. * * Here is a brief list of the entry points to this module: * * btt_nlane Returns number of concurrent threads allowed * * btt_nlba Returns the usable size, as a count of LBAs * * btt_read Reads a single block at a given LBA * * btt_write Writes a single block (atomically) at a given LBA * * btt_set_zero Sets a block to read back as zeros * * btt_set_error Sets a block to return error on read * * btt_check Checks the BTT metadata for consistency * * btt_fini Frees run-time state, done using namespace * * If the caller is multi-threaded, it must only allow btt_nlane() threads * to enter this module at a time, each assigned a unique "lane" number * between 0 and btt_nlane() - 1. * * There are a number of static routines defined in this module. Here's * a brief overview of the most important routines: * * read_layout Checks for valid BTT layout and builds run-time state. * A number of helper functions are used by read_layout * to handle various parts of the metadata: * read_info * read_arenas * read_arena * read_flogs * read_flog_pair * * write_layout Generates a new BTT layout when one doesn't exist. * Once a new layout is written, write_layout uses * the same helper functions above to construct the * run-time state. * * invalid_lba Range check done by each entry point that takes * an LBA. * * lba_to_arena_lba * Find the arena and LBA in that arena for a given * external LBA. This is the heart of the arena * range matching logic. * * flog_update Update the BTT free list/log combined data structure * (known as the "flog"). This is the heart of the * logic that makes writes powerfail atomic. * * map_lock These routines provide atomic access to the BTT map * map_unlock data structure in an area. * map_abort * * map_entry_setf Common code for btt_set_zero() and btt_set_error(). * * zero_block Generate a block of all zeros (instead of actually * doing a read), when the metadata indicates the * block should read as zeros. * * build_rtt These routines construct the run-time tracking * build_map_locks data structures used during I/O. */ #include #include #include #include #include #include #include #include #include "out.h" #include "uuid.h" #include "btt.h" #include "btt_layout.h" #include "sys_util.h" #include "util.h" #include "alloc.h" /* * The opaque btt handle containing state tracked by this module * for the btt namespace. This is created by btt_init(), handed to * all the other btt_* entry points, and deleted by btt_fini(). */ struct btt { unsigned nlane; /* number of concurrent threads allowed per btt */ /* * The laidout flag indicates whether the namespace contains valid BTT * metadata. It is initialized by read_layout() and if no valid layout * is found, all reads return zeros and the first write will write the * BTT layout. The layout_write_mutex protects the laidout flag so * only one write threads ends up writing the initial metadata by * calling write_layout(). */ os_mutex_t layout_write_mutex; int laidout; /* * UUID of the BTT */ uint8_t uuid[BTTINFO_UUID_LEN]; /* * UUID of the containing namespace, used to validate BTT metadata. */ uint8_t parent_uuid[BTTINFO_UUID_LEN]; /* * Parameters controlling/describing the BTT layout. */ uint64_t rawsize; /* size of containing namespace */ uint32_t lbasize; /* external LBA size */ uint32_t nfree; /* available flog entries */ uint64_t nlba; /* total number of external LBAs */ unsigned narena; /* number of arenas */ /* run-time state kept for each arena */ struct arena { uint32_t flags; /* arena flags (btt_info) */ uint32_t external_nlba; /* LBAs that live in this arena */ uint32_t internal_lbasize; uint32_t internal_nlba; /* * The following offsets are relative to the beginning of * the encapsulating namespace. This is different from * how these offsets are stored on-media, where they are * relative to the start of the arena. The offset are * converted by read_layout() to make them more convenient * for run-time use. */ uint64_t startoff; /* offset to start of arena */ uint64_t dataoff; /* offset to arena data area */ uint64_t mapoff; /* offset to area map */ uint64_t flogoff; /* offset to area flog */ uint64_t nextoff; /* offset to next arena */ /* * Run-time flog state. Indexed by lane. * * The write path uses the flog to find the free block * it writes to before atomically making it the new * active block for an external LBA. * * The read path doesn't use the flog at all. */ struct flog_runtime { struct btt_flog flog; /* current info */ uint64_t entries[2]; /* offsets for flog pair */ int next; /* next write (0 or 1) */ } *flogs; /* * Read tracking table. Indexed by lane. * * Before using a free block found in the flog, the write path * scans the rtt to see if there are any outstanding reads on * that block (reads that started before the block was freed by * a concurrent write). Unused slots in the rtt are indicated * by setting the error bit, BTT_MAP_ENTRY_ERROR, so that the * entry won't match any post-map LBA when checked. */ uint32_t volatile *rtt; /* * Map locking. Indexed by pre-map LBA modulo nlane. */ os_mutex_t *map_locks; /* * Arena info block locking. */ os_mutex_t info_lock; } *arenas; /* * Callbacks for doing I/O to namespace. These are provided by * the code calling the BTT module, which passes them in to * btt_init(). All namespace I/O is done using these. * * The opaque namespace handle "ns" was provided by the code calling * the BTT module and is passed to each callback to identify the * namespace being accessed. */ void *ns; const struct ns_callback *ns_cbp; }; /* * Signature for arena info blocks. Total size is 16 bytes, including * the '\0' added to the string by the declaration (the last two bytes * of the string are '\0'). */ static const char Sig[] = BTTINFO_SIG; /* * Zeroed out flog entry, used when initializing the flog. */ static const struct btt_flog Zflog; /* * Lookup table and macro for looking up sequence numbers. These are * the 2-bit numbers that cycle between 01, 10, and 11. * * To advance a sequence number to the next number, use something like: * seq = NSEQ(seq); */ static const unsigned Nseq[] = { 0, 2, 3, 1 }; #define NSEQ(seq) (Nseq[(seq) & 3]) /* * get_map_lock_num -- (internal) Calculate offset into map_locks[] * * map_locks[] contains nfree locks which are used to protect the map * from concurrent access to the same cache line. The index into * map_locks[] is calculated by looking at the byte offset into the map * (premap_lba * BTT_MAP_ENTRY_SIZE), figuring out how many cache lines * that is into the map that is (dividing by BTT_MAP_LOCK_ALIGN), and * then selecting one of nfree locks (the modulo at the end). * * The extra cast is to keep gcc from generating a false positive * 64-32 bit conversion error when -fsanitize is set. */ static inline uint32_t get_map_lock_num(uint32_t premap_lba, uint32_t nfree) { return (uint32_t)(premap_lba * BTT_MAP_ENTRY_SIZE / BTT_MAP_LOCK_ALIGN) % nfree; } /* * invalid_lba -- (internal) set errno and return true if lba is invalid * * This function is used at the top of the entry points where an external * LBA is provided, like this: * * if (invalid_lba(bttp, lba)) * return -1; */ static int invalid_lba(struct btt *bttp, uint64_t lba) { LOG(3, "bttp %p lba %" PRIu64, bttp, lba); if (lba >= bttp->nlba) { ERR("lba out of range (nlba %" PRIu64 ")", bttp->nlba); errno = EINVAL; return 1; } return 0; } /* * read_info -- (internal) convert btt_info to host byte order & validate * * Returns true if info block is valid, and all the integer fields are * converted to host byte order. If the info block is not valid, this * routine returns false and the info block passed in is left in an * unknown state. */ static int read_info(struct btt *bttp, struct btt_info *infop) { LOG(3, "infop %p", infop); if (memcmp(infop->sig, Sig, BTTINFO_SIG_LEN)) { LOG(3, "signature invalid"); return 0; } if (memcmp(infop->parent_uuid, bttp->parent_uuid, BTTINFO_UUID_LEN)) { LOG(3, "parent UUID mismatch"); return 0; } /* to be valid, the fields must checksum correctly */ if (!util_checksum(infop, sizeof(*infop), &infop->checksum, 0, 0)) { LOG(3, "invalid checksum"); return 0; } /* to be valid, info block must have a major version of at least 1 */ if ((infop->major = le16toh(infop->major)) == 0) { LOG(3, "invalid major version (0)"); return 0; } infop->flags = le32toh(infop->flags); infop->minor = le16toh(infop->minor); infop->external_lbasize = le32toh(infop->external_lbasize); infop->external_nlba = le32toh(infop->external_nlba); infop->internal_lbasize = le32toh(infop->internal_lbasize); infop->internal_nlba = le32toh(infop->internal_nlba); infop->nfree = le32toh(infop->nfree); infop->infosize = le32toh(infop->infosize); infop->nextoff = le64toh(infop->nextoff); infop->dataoff = le64toh(infop->dataoff); infop->mapoff = le64toh(infop->mapoff); infop->flogoff = le64toh(infop->flogoff); infop->infooff = le64toh(infop->infooff); return 1; } /* * map_entry_is_zero -- (internal) checks if map_entry is in zero state */ static inline int map_entry_is_zero(uint32_t map_entry) { return (map_entry & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_ZERO; } /* * map_entry_is_error -- (internal) checks if map_entry is in error state */ static inline int map_entry_is_error(uint32_t map_entry) { return (map_entry & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_ERROR; } /* * map_entry_is_initial -- checks if map_entry is in initial state */ int map_entry_is_initial(uint32_t map_entry) { return (map_entry & ~BTT_MAP_ENTRY_LBA_MASK) == 0; } /* * map_entry_is_zero_or_initial -- (internal) checks if map_entry is in initial * or zero state */ static inline int map_entry_is_zero_or_initial(uint32_t map_entry) { uint32_t entry_flags = map_entry & ~BTT_MAP_ENTRY_LBA_MASK; return entry_flags == 0 || entry_flags == BTT_MAP_ENTRY_ZERO; } /* * btt_flog_get_valid -- return valid and current flog entry */ struct btt_flog * btt_flog_get_valid(struct btt_flog *flog_pair, int *next) { /* * Interesting cases: * - no valid seq numbers: layout consistency error * - one valid seq number: that's the current entry * - two valid seq numbers: higher number is current entry * - identical seq numbers: layout consistency error */ if (flog_pair[0].seq == flog_pair[1].seq) { return NULL; } else if (flog_pair[0].seq == 0) { /* singleton valid flog at flog_pair[1] */ *next = 0; return &flog_pair[1]; } else if (flog_pair[1].seq == 0) { /* singleton valid flog at flog_pair[0] */ *next = 1; return &flog_pair[0]; } else if (NSEQ(flog_pair[0].seq) == flog_pair[1].seq) { /* flog_pair[1] has the later sequence number */ *next = 0; return &flog_pair[1]; } else { /* flog_pair[0] has the later sequence number */ *next = 1; return &flog_pair[0]; } } /* * read_flog_pair -- (internal) load up a single flog pair * * Zero is returned on success, otherwise -1/errno. */ static int read_flog_pair(struct btt *bttp, unsigned lane, struct arena *arenap, uint64_t flog_off, struct flog_runtime *flog_runtimep, uint32_t flognum) { LOG(5, "bttp %p lane %u arenap %p flog_off %" PRIu64 " runtimep %p " "flognum %u", bttp, lane, arenap, flog_off, flog_runtimep, flognum); flog_runtimep->entries[0] = flog_off; flog_runtimep->entries[1] = flog_off + sizeof(struct btt_flog); if (lane >= bttp->nfree) { ERR("invalid lane %u among nfree %d", lane, bttp->nfree); errno = EINVAL; return -1; } if (flog_off == 0) { ERR("invalid flog offset %" PRIu64, flog_off); errno = EINVAL; return -1; } struct btt_flog flog_pair[2]; if ((*bttp->ns_cbp->nsread)(bttp->ns, lane, flog_pair, sizeof(flog_pair), flog_off) < 0) return -1; btt_flog_convert2h(&flog_pair[0]); if (invalid_lba(bttp, flog_pair[0].lba)) return -1; btt_flog_convert2h(&flog_pair[1]); if (invalid_lba(bttp, flog_pair[1].lba)) return -1; LOG(6, "flog_pair[0] flog_off %" PRIu64 " old_map %u new_map %u seq %u", flog_off, flog_pair[0].old_map, flog_pair[0].new_map, flog_pair[0].seq); LOG(6, "flog_pair[1] old_map %u new_map %u seq %u", flog_pair[1].old_map, flog_pair[1].new_map, flog_pair[1].seq); struct btt_flog *currentp = btt_flog_get_valid(flog_pair, &flog_runtimep->next); if (currentp == NULL) { ERR("flog layout error: bad seq numbers %d %d", flog_pair[0].seq, flog_pair[1].seq); arenap->flags |= BTTINFO_FLAG_ERROR; return 0; } LOG(6, "run-time flog next is %d", flog_runtimep->next); /* copy current flog into run-time flog state */ flog_runtimep->flog = *currentp; LOG(9, "read flog[%u]: lba %u old %u%s%s%s new %u%s%s%s", flognum, currentp->lba, currentp->old_map & BTT_MAP_ENTRY_LBA_MASK, (map_entry_is_error(currentp->old_map)) ? " ERROR" : "", (map_entry_is_zero(currentp->old_map)) ? " ZERO" : "", (map_entry_is_initial(currentp->old_map)) ? " INIT" : "", currentp->new_map & BTT_MAP_ENTRY_LBA_MASK, (map_entry_is_error(currentp->new_map)) ? " ERROR" : "", (map_entry_is_zero(currentp->new_map)) ? " ZERO" : "", (map_entry_is_initial(currentp->new_map)) ? " INIT" : ""); /* * Decide if the current flog info represents a completed * operation or an incomplete operation. If completed, the * old_map field will contain the free block to be used for * the next write. But if the operation didn't complete (indicated * by the map entry not being updated), then the operation is * completed now by updating the map entry. * * A special case, used by flog entries when first created, is * when old_map == new_map. This counts as a complete entry * and doesn't require reading the map to see if recovery is * required. */ if (currentp->old_map == currentp->new_map) { LOG(9, "flog[%u] entry complete (initial state)", flognum); return 0; } /* convert pre-map LBA into an offset into the map */ uint64_t map_entry_off = arenap->mapoff + BTT_MAP_ENTRY_SIZE * currentp->lba; /* read current map entry */ uint32_t entry; if ((*bttp->ns_cbp->nsread)(bttp->ns, lane, &entry, sizeof(entry), map_entry_off) < 0) return -1; entry = le32toh(entry); /* map entry in initial state */ if (map_entry_is_initial(entry)) entry = currentp->lba | BTT_MAP_ENTRY_NORMAL; if (currentp->new_map != entry && currentp->old_map == entry) { /* last update didn't complete */ LOG(9, "recover flog[%u]: map[%u]: %u", flognum, currentp->lba, currentp->new_map); /* * Recovery step is to complete the transaction by * updating the map entry. */ entry = htole32(currentp->new_map); if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &entry, sizeof(uint32_t), map_entry_off) < 0) return -1; } return 0; } /* * flog_update -- (internal) write out an updated flog entry * * The flog entries are not checksummed. Instead, increasing sequence * numbers are used to atomically switch the active flog entry between * the first and second struct btt_flog in each slot. In order for this * to work, the sequence number must be updated only after all the other * fields in the flog are updated. So the writes to the flog are broken * into two writes, one for the first three fields (lba, old_map, new_map) * and, only after those fields are known to be written durably, the * second write for the seq field is done. * * Returns 0 on success, otherwise -1/errno. */ static int flog_update(struct btt *bttp, unsigned lane, struct arena *arenap, uint32_t lba, uint32_t old_map, uint32_t new_map) { LOG(3, "bttp %p lane %u arenap %p lba %u old_map %u new_map %u", bttp, lane, arenap, lba, old_map, new_map); /* construct new flog entry in little-endian byte order */ struct btt_flog new_flog; new_flog.lba = lba; new_flog.old_map = old_map; new_flog.new_map = new_map; new_flog.seq = NSEQ(arenap->flogs[lane].flog.seq); btt_flog_convert2le(&new_flog); uint64_t new_flog_off = arenap->flogs[lane].entries[arenap->flogs[lane].next]; /* write out first two fields first */ if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &new_flog, sizeof(uint32_t) * 2, new_flog_off) < 0) return -1; new_flog_off += sizeof(uint32_t) * 2; /* write out new_map and seq field to make it active */ if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &new_flog.new_map, sizeof(uint32_t) * 2, new_flog_off) < 0) return -1; /* flog entry written successfully, update run-time state */ arenap->flogs[lane].next = 1 - arenap->flogs[lane].next; arenap->flogs[lane].flog.lba = lba; arenap->flogs[lane].flog.old_map = old_map; arenap->flogs[lane].flog.new_map = new_map; arenap->flogs[lane].flog.seq = NSEQ(arenap->flogs[lane].flog.seq); LOG(9, "update flog[%u]: lba %u old %u%s%s%s new %u%s%s%s", lane, lba, old_map & BTT_MAP_ENTRY_LBA_MASK, (map_entry_is_error(old_map)) ? " ERROR" : "", (map_entry_is_zero(old_map)) ? " ZERO" : "", (map_entry_is_initial(old_map)) ? " INIT" : "", new_map & BTT_MAP_ENTRY_LBA_MASK, (map_entry_is_error(new_map)) ? " ERROR" : "", (map_entry_is_zero(new_map)) ? " ZERO" : "", (map_entry_is_initial(new_map)) ? " INIT" : ""); return 0; } /* * arena_setf -- (internal) updates the given flag for the arena info block */ static int arena_setf(struct btt *bttp, struct arena *arenap, unsigned lane, uint32_t setf) { LOG(3, "bttp %p arenap %p lane %u setf 0x%x", bttp, arenap, lane, setf); /* update runtime state */ util_fetch_and_or32(&arenap->flags, setf); if (!bttp->laidout) { /* no layout yet to update */ return 0; } /* * Read, modify and write out the info block * at both the beginning and end of the arena. */ uint64_t arena_off = arenap->startoff; struct btt_info info; /* protect from simultaneous writes to the layout */ util_mutex_lock(&arenap->info_lock); if ((*bttp->ns_cbp->nsread)(bttp->ns, lane, &info, sizeof(info), arena_off) < 0) { goto err; } uint64_t infooff = le64toh(info.infooff); /* update flags */ info.flags |= htole32(setf); /* update checksum */ util_checksum(&info, sizeof(info), &info.checksum, 1, 0); if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &info, sizeof(info), arena_off) < 0) { goto err; } if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &info, sizeof(info), arena_off + infooff) < 0) { goto err; } util_mutex_unlock(&arenap->info_lock); return 0; err: util_mutex_unlock(&arenap->info_lock); return -1; } /* * set_arena_error -- (internal) set the error flag for the given arena */ static int set_arena_error(struct btt *bttp, struct arena *arenap, unsigned lane) { LOG(3, "bttp %p arena %p lane %u", bttp, arenap, lane); return arena_setf(bttp, arenap, lane, BTTINFO_FLAG_ERROR); } /* * read_flogs -- (internal) load up all the flog entries for an arena * * Zero is returned on success, otherwise -1/errno. */ static int read_flogs(struct btt *bttp, unsigned lane, struct arena *arenap) { if ((arenap->flogs = Zalloc(bttp->nfree * sizeof(struct flog_runtime))) == NULL) { ERR("!Malloc for %u flog entries", bttp->nfree); return -1; } /* * Load up the flog state. read_flog_pair() will determine if * any recovery steps are required take them on the in-memory * data structures it creates. Sets error flag when it * determines an invalid state. */ uint64_t flog_off = arenap->flogoff; struct flog_runtime *flog_runtimep = arenap->flogs; for (uint32_t i = 0; i < bttp->nfree; i++) { if (read_flog_pair(bttp, lane, arenap, flog_off, flog_runtimep, i) < 0) { set_arena_error(bttp, arenap, lane); return -1; } /* prepare for next time around the loop */ flog_off += roundup(2 * sizeof(struct btt_flog), BTT_FLOG_PAIR_ALIGN); flog_runtimep++; } return 0; } /* * build_rtt -- (internal) construct a read tracking table for an arena * * Zero is returned on success, otherwise -1/errno. * * The rtt is big enough to hold an entry for each free block (nfree) * since nlane can't be bigger than nfree. nlane may end up smaller, * in which case some of the high rtt entries will be unused. */ static int build_rtt(struct btt *bttp, struct arena *arenap) { if ((arenap->rtt = Malloc(bttp->nfree * sizeof(uint32_t))) == NULL) { ERR("!Malloc for %d rtt entries", bttp->nfree); return -1; } for (uint32_t lane = 0; lane < bttp->nfree; lane++) arenap->rtt[lane] = BTT_MAP_ENTRY_ERROR; util_synchronize(); return 0; } /* * build_map_locks -- (internal) construct map locks * * Zero is returned on success, otherwise -1/errno. */ static int build_map_locks(struct btt *bttp, struct arena *arenap) { if ((arenap->map_locks = Malloc(bttp->nfree * sizeof(*arenap->map_locks))) == NULL) { ERR("!Malloc for %d map_lock entries", bttp->nfree); return -1; } for (uint32_t lane = 0; lane < bttp->nfree; lane++) util_mutex_init(&arenap->map_locks[lane]); return 0; } /* * read_arena -- (internal) load up an arena and build run-time state * * Zero is returned on success, otherwise -1/errno. */ static int read_arena(struct btt *bttp, unsigned lane, uint64_t arena_off, struct arena *arenap) { LOG(3, "bttp %p lane %u arena_off %" PRIu64 " arenap %p", bttp, lane, arena_off, arenap); struct btt_info info; if ((*bttp->ns_cbp->nsread)(bttp->ns, lane, &info, sizeof(info), arena_off) < 0) return -1; arenap->flags = le32toh(info.flags); arenap->external_nlba = le32toh(info.external_nlba); arenap->internal_lbasize = le32toh(info.internal_lbasize); arenap->internal_nlba = le32toh(info.internal_nlba); arenap->startoff = arena_off; arenap->dataoff = arena_off + le64toh(info.dataoff); arenap->mapoff = arena_off + le64toh(info.mapoff); arenap->flogoff = arena_off + le64toh(info.flogoff); arenap->nextoff = arena_off + le64toh(info.nextoff); if (read_flogs(bttp, lane, arenap) < 0) return -1; if (build_rtt(bttp, arenap) < 0) return -1; if (build_map_locks(bttp, arenap) < 0) return -1; /* initialize the per arena info block lock */ util_mutex_init(&arenap->info_lock); return 0; } /* * util_convert2h_btt_info -- convert btt_info to host byte order */ void btt_info_convert2h(struct btt_info *infop) { infop->flags = le32toh(infop->flags); infop->major = le16toh(infop->major); infop->minor = le16toh(infop->minor); infop->external_lbasize = le32toh(infop->external_lbasize); infop->external_nlba = le32toh(infop->external_nlba); infop->internal_lbasize = le32toh(infop->internal_lbasize); infop->internal_nlba = le32toh(infop->internal_nlba); infop->nfree = le32toh(infop->nfree); infop->infosize = le32toh(infop->infosize); infop->nextoff = le64toh(infop->nextoff); infop->dataoff = le64toh(infop->dataoff); infop->mapoff = le64toh(infop->mapoff); infop->flogoff = le64toh(infop->flogoff); infop->infooff = le64toh(infop->infooff); } /* * btt_info_convert2le -- convert btt_info to little-endian byte order */ void btt_info_convert2le(struct btt_info *infop) { infop->flags = le32toh(infop->flags); infop->major = le16toh(infop->major); infop->minor = le16toh(infop->minor); infop->external_lbasize = le32toh(infop->external_lbasize); infop->external_nlba = le32toh(infop->external_nlba); infop->internal_lbasize = le32toh(infop->internal_lbasize); infop->internal_nlba = le32toh(infop->internal_nlba); infop->nfree = le32toh(infop->nfree); infop->infosize = le32toh(infop->infosize); infop->nextoff = le64toh(infop->nextoff); infop->dataoff = le64toh(infop->dataoff); infop->mapoff = le64toh(infop->mapoff); infop->flogoff = le64toh(infop->flogoff); infop->infooff = le64toh(infop->infooff); } /* * btt_flog_convert2h -- convert btt_flog to host byte order */ void btt_flog_convert2h(struct btt_flog *flogp) { flogp->lba = le32toh(flogp->lba); flogp->old_map = le32toh(flogp->old_map); flogp->new_map = le32toh(flogp->new_map); flogp->seq = le32toh(flogp->seq); } /* * btt_flog_convert2le -- convert btt_flog to LE byte order */ void btt_flog_convert2le(struct btt_flog *flogp) { flogp->lba = htole32(flogp->lba); flogp->old_map = htole32(flogp->old_map); flogp->new_map = htole32(flogp->new_map); flogp->seq = htole32(flogp->seq); } /* * read_arenas -- (internal) load up all arenas and build run-time state * * On entry, layout must be known to be valid, and the number of arenas * must be known. Zero is returned on success, otherwise -1/errno. */ static int read_arenas(struct btt *bttp, unsigned lane, unsigned narena) { LOG(3, "bttp %p lane %u narena %d", bttp, lane, narena); if ((bttp->arenas = Zalloc(narena * sizeof(*bttp->arenas))) == NULL) { ERR("!Malloc for %u arenas", narena); goto err; } uint64_t arena_off = 0; struct arena *arenap = bttp->arenas; for (unsigned i = 0; i < narena; i++) { if (read_arena(bttp, lane, arena_off, arenap) < 0) goto err; /* prepare for next time around the loop */ arena_off = arenap->nextoff; arenap++; } bttp->laidout = 1; return 0; err: LOG(4, "error clean up"); int oerrno = errno; if (bttp->arenas) { for (unsigned i = 0; i < bttp->narena; i++) { if (bttp->arenas[i].flogs) Free(bttp->arenas[i].flogs); if (bttp->arenas[i].rtt) Free((void *)bttp->arenas[i].rtt); if (bttp->arenas[i].map_locks) Free((void *)bttp->arenas[i].map_locks); } Free(bttp->arenas); bttp->arenas = NULL; } errno = oerrno; return -1; } /* * internal_lbasize -- (internal) calculate internal LBA size */ static inline uint32_t internal_lbasize(uint32_t external_lbasize) { uint32_t internal_lbasize = external_lbasize; if (internal_lbasize < BTT_MIN_LBA_SIZE) internal_lbasize = BTT_MIN_LBA_SIZE; internal_lbasize = roundup(internal_lbasize, BTT_INTERNAL_LBA_ALIGNMENT); /* check for overflow */ if (internal_lbasize < BTT_INTERNAL_LBA_ALIGNMENT) { errno = EINVAL; ERR("!Invalid lba size after alignment: %u ", internal_lbasize); return 0; } return internal_lbasize; } /* * btt_flog_size -- calculate flog data size */ uint64_t btt_flog_size(uint32_t nfree) { uint64_t flog_size = nfree * roundup(2 * sizeof(struct btt_flog), BTT_FLOG_PAIR_ALIGN); return roundup(flog_size, BTT_ALIGNMENT); } /* * btt_map_size -- calculate map data size */ uint64_t btt_map_size(uint32_t external_nlba) { return roundup(external_nlba * BTT_MAP_ENTRY_SIZE, BTT_ALIGNMENT); } /* * btt_arena_datasize -- whole arena size without BTT Info header, backup and * flog means size of blocks and map */ uint64_t btt_arena_datasize(uint64_t arena_size, uint32_t nfree) { return arena_size - 2 * sizeof(struct btt_info) - btt_flog_size(nfree); } /* * btt_info_set_params -- (internal) calculate and set BTT Info * external_lbasize, internal_lbasize, nfree, infosize, external_nlba and * internal_nlba */ static int btt_info_set_params(struct btt_info *info, uint32_t external_lbasize, uint32_t internal_lbasize, uint32_t nfree, uint64_t arena_size) { info->external_lbasize = external_lbasize; info->internal_lbasize = internal_lbasize; info->nfree = nfree; info->infosize = sizeof(*info); uint64_t arena_data_size = btt_arena_datasize(arena_size, nfree); /* allow for map alignment padding */ uint64_t internal_nlba = (arena_data_size - BTT_ALIGNMENT) / (info->internal_lbasize + BTT_MAP_ENTRY_SIZE); /* ensure the number of blocks is at least 2*nfree */ if (internal_nlba < 2 * nfree) { errno = EINVAL; ERR("!number of internal blocks: %" PRIu64 " expected at least %u", internal_nlba, 2 * nfree); return -1; } ASSERT(internal_nlba <= UINT32_MAX); uint32_t internal_nlba_u32 = (uint32_t)internal_nlba; info->internal_nlba = internal_nlba_u32; /* external LBA does not include free blocks */ info->external_nlba = internal_nlba_u32 - info->nfree; ASSERT((arena_data_size - btt_map_size(info->external_nlba)) / internal_lbasize >= internal_nlba); return 0; } /* * btt_info_set_offs -- (internal) calculate and set the BTT Info dataoff, * nextoff, infooff, flogoff and mapoff. These are all relative to the * beginning of the arena. */ static void btt_info_set_offs(struct btt_info *info, uint64_t arena_size, uint64_t space_left) { info->dataoff = info->infosize; /* set offset to next valid arena */ if (space_left >= BTT_MIN_SIZE) info->nextoff = arena_size; else info->nextoff = 0; info->infooff = arena_size - sizeof(struct btt_info); info->flogoff = info->infooff - btt_flog_size(info->nfree); info->mapoff = info->flogoff - btt_map_size(info->external_nlba); ASSERTeq(btt_arena_datasize(arena_size, info->nfree) - btt_map_size(info->external_nlba), info->mapoff - info->dataoff); } /* * btt_info_set -- set BTT Info params and offsets */ int btt_info_set(struct btt_info *info, uint32_t external_lbasize, uint32_t nfree, uint64_t arena_size, uint64_t space_left) { /* calculate internal LBA size */ uint32_t internal_lba_size = internal_lbasize(external_lbasize); if (internal_lba_size == 0) return -1; /* set params and offsets */ if (btt_info_set_params(info, external_lbasize, internal_lba_size, nfree, arena_size)) return -1; btt_info_set_offs(info, arena_size, space_left); return 0; } /* * write_layout -- (internal) write out the initial btt metadata layout * * Called with write == 1 only once in the life time of a btt namespace, when * the first write happens. The caller of this routine is responsible for * locking out multiple threads. This routine doesn't read anything -- by the * time it is called, it is known there's no layout in the namespace and a new * layout should be written. * * Calling with write == 0 tells this routine to do the calculations for * bttp->narena and bttp->nlba, but don't write out any metadata. * * If successful, sets bttp->layout to 1 and returns 0. Otherwise -1 * is returned and errno is set, and bttp->layout remains 0 so that * later attempts to write will try again to create the layout. */ static int write_layout(struct btt *bttp, unsigned lane, int write) { LOG(3, "bttp %p lane %u write %d", bttp, lane, write); ASSERT(bttp->rawsize >= BTT_MIN_SIZE); ASSERT(bttp->nfree); /* * If a new layout is being written, generate the BTT's UUID. */ if (write) { int ret = util_uuid_generate(bttp->uuid); if (ret < 0) { LOG(2, "util_uuid_generate failed"); return -1; } } /* * The number of arenas is the number of full arena of * size BTT_MAX_ARENA that fit into rawsize and then, if * the remainder is at least BTT_MIN_SIZE in size, then * that adds one more arena. */ bttp->narena = (unsigned)(bttp->rawsize / BTT_MAX_ARENA); if (bttp->rawsize % BTT_MAX_ARENA >= BTT_MIN_SIZE) bttp->narena++; LOG(4, "narena %u", bttp->narena); uint32_t internal_lba_size = internal_lbasize(bttp->lbasize); if (internal_lba_size == 0) return -1; LOG(4, "adjusted internal_lbasize %u", internal_lba_size); uint64_t total_nlba = 0; uint64_t rawsize = bttp->rawsize; unsigned arena_num = 0; uint64_t arena_off = 0; /* * for each arena... */ while (rawsize >= BTT_MIN_SIZE) { LOG(4, "layout arena %u", arena_num); uint64_t arena_rawsize = rawsize; if (arena_rawsize > BTT_MAX_ARENA) { arena_rawsize = BTT_MAX_ARENA; } rawsize -= arena_rawsize; arena_num++; struct btt_info info; memset(&info, '\0', sizeof(info)); if (btt_info_set_params(&info, bttp->lbasize, internal_lba_size, bttp->nfree, arena_rawsize)) return -1; LOG(4, "internal_nlba %u external_nlba %u", info.internal_nlba, info.external_nlba); total_nlba += info.external_nlba; /* * The rest of the loop body calculates metadata structures * and lays it out for this arena. So only continue if * the write flag is set. */ if (!write) continue; btt_info_set_offs(&info, arena_rawsize, rawsize); LOG(4, "nextoff 0x%016" PRIx64, info.nextoff); LOG(4, "dataoff 0x%016" PRIx64, info.dataoff); LOG(4, "mapoff 0x%016" PRIx64, info.mapoff); LOG(4, "flogoff 0x%016" PRIx64, info.flogoff); LOG(4, "infooff 0x%016" PRIx64, info.infooff); /* zero map if ns is not zero-initialized */ if (!bttp->ns_cbp->ns_is_zeroed) { uint64_t mapsize = btt_map_size(info.external_nlba); if ((*bttp->ns_cbp->nszero)(bttp->ns, lane, mapsize, info.mapoff) < 0) return -1; } /* write out the initial flog */ uint64_t flog_entry_off = arena_off + info.flogoff; uint32_t next_free_lba = info.external_nlba; for (uint32_t i = 0; i < bttp->nfree; i++) { struct btt_flog flog; flog.lba = htole32(i); flog.old_map = flog.new_map = htole32(next_free_lba | BTT_MAP_ENTRY_ZERO); flog.seq = htole32(1); /* * Write both btt_flog structs in the pair, writing * the second one as all zeros. */ LOG(6, "flog[%u] entry off %" PRIu64 " initial %u + zero = %u", i, flog_entry_off, next_free_lba, next_free_lba | BTT_MAP_ENTRY_ZERO); if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &flog, sizeof(flog), flog_entry_off) < 0) return -1; flog_entry_off += sizeof(flog); LOG(6, "flog[%u] entry off %" PRIu64 " zeros", i, flog_entry_off); if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &Zflog, sizeof(Zflog), flog_entry_off) < 0) return -1; flog_entry_off += sizeof(flog); flog_entry_off = roundup(flog_entry_off, BTT_FLOG_PAIR_ALIGN); next_free_lba++; } /* * Construct the BTT info block and write it out * at both the beginning and end of the arena. */ memcpy(info.sig, Sig, BTTINFO_SIG_LEN); memcpy(info.uuid, bttp->uuid, BTTINFO_UUID_LEN); memcpy(info.parent_uuid, bttp->parent_uuid, BTTINFO_UUID_LEN); info.major = BTTINFO_MAJOR_VERSION; info.minor = BTTINFO_MINOR_VERSION; btt_info_convert2le(&info); util_checksum(&info, sizeof(info), &info.checksum, 1, 0); if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &info, sizeof(info), arena_off) < 0) return -1; if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, &info, sizeof(info), arena_off + info.infooff) < 0) return -1; arena_off += info.nextoff; } ASSERTeq(bttp->narena, arena_num); bttp->nlba = total_nlba; if (write) { /* * The layout is written now, so load up the arenas. */ return read_arenas(bttp, lane, bttp->narena); } return 0; } /* * read_layout -- (internal) load up layout info from btt namespace * * Called once when the btt namespace is opened for use. * Sets bttp->layout to 0 if no valid layout is found, 1 otherwise. * * Any recovery actions required (as indicated by the flog state) are * performed by this routine. * * Any quick checks for layout consistency are performed by this routine * (quick enough to be done each time a BTT area is opened for use, not * like the slow consistency checks done by btt_check()). * * Returns 0 if no errors are encountered accessing the namespace (in this * context, detecting there's no layout is not an error if the nsread function * didn't have any problems doing the reads). Otherwise, -1 is returned * and errno is set. */ static int read_layout(struct btt *bttp, unsigned lane) { LOG(3, "bttp %p", bttp); ASSERT(bttp->rawsize >= BTT_MIN_SIZE); unsigned narena = 0; uint32_t smallest_nfree = UINT32_MAX; uint64_t rawsize = bttp->rawsize; uint64_t total_nlba = 0; uint64_t arena_off = 0; bttp->nfree = BTT_DEFAULT_NFREE; /* * For each arena, see if there's a valid info block */ while (rawsize >= BTT_MIN_SIZE) { narena++; struct btt_info info; if ((*bttp->ns_cbp->nsread)(bttp->ns, lane, &info, sizeof(info), arena_off) < 0) return -1; if (!read_info(bttp, &info)) { /* * Failed to find complete BTT metadata. Just * calculate the narena and nlba values that will * result when write_layout() gets called. This * allows checks against nlba to work correctly * even before the layout is written. */ return write_layout(bttp, lane, 0); } if (info.external_lbasize != bttp->lbasize) { /* can't read it assuming the wrong block size */ ERR("inconsistent lbasize"); errno = EINVAL; return -1; } if (info.nfree == 0) { ERR("invalid nfree"); errno = EINVAL; return -1; } if (info.external_nlba == 0) { ERR("invalid external_nlba"); errno = EINVAL; return -1; } if (info.nextoff && (info.nextoff != BTT_MAX_ARENA)) { ERR("invalid arena size"); errno = EINVAL; return -1; } if (info.nfree < smallest_nfree) smallest_nfree = info.nfree; total_nlba += info.external_nlba; arena_off += info.nextoff; if (info.nextoff == 0) break; if (info.nextoff > rawsize) { ERR("invalid next arena offset"); errno = EINVAL; return -1; } rawsize -= info.nextoff; } ASSERT(narena); bttp->narena = narena; bttp->nlba = total_nlba; /* * All arenas were valid. nfree should be the smallest value found * among different arenas. */ if (smallest_nfree < bttp->nfree) bttp->nfree = smallest_nfree; /* * Load up arenas. */ return read_arenas(bttp, lane, narena); } /* * zero_block -- (internal) satisfy a read with a block of zeros * * Returns 0 on success, otherwise -1/errno. */ static int zero_block(struct btt *bttp, void *buf) { LOG(3, "bttp %p", bttp); memset(buf, '\0', bttp->lbasize); return 0; } /* * lba_to_arena_lba -- (internal) calculate the arena & pre-map LBA * * This routine takes the external LBA and matches it to the * appropriate arena, adjusting the lba for use within that arena. * * If successful, zero is returned, *arenapp is a pointer to the appropriate * arena struct in the run-time state, and *premap_lbap is the LBA adjusted * to an arena-internal LBA (also known as the pre-map LBA). Otherwise * -1/errno. */ static int lba_to_arena_lba(struct btt *bttp, uint64_t lba, struct arena **arenapp, uint32_t *premap_lbap) { LOG(3, "bttp %p lba %" PRIu64, bttp, lba); ASSERT(bttp->laidout); unsigned arena; for (arena = 0; arena < bttp->narena; arena++) if (lba < bttp->arenas[arena].external_nlba) break; else lba -= bttp->arenas[arena].external_nlba; ASSERT(arena < bttp->narena); *arenapp = &bttp->arenas[arena]; ASSERT(lba <= UINT32_MAX); *premap_lbap = (uint32_t)lba; LOG(3, "arenap %p pre-map LBA %u", *arenapp, *premap_lbap); return 0; } /* * btt_init -- prepare a btt namespace for use, returning an opaque handle * * Returns handle on success, otherwise NULL/errno. * * When submitted a pristine namespace it will be formatted implicitly when * touched for the first time. * * If arenas have different nfree values, we will be using the lowest one * found as limiting to the overall "bandwidth". */ struct btt * btt_init(uint64_t rawsize, uint32_t lbasize, uint8_t parent_uuid[], unsigned maxlane, void *ns, const struct ns_callback *ns_cbp) { LOG(3, "rawsize %" PRIu64 " lbasize %u", rawsize, lbasize); if (rawsize < BTT_MIN_SIZE) { ERR("rawsize smaller than BTT_MIN_SIZE %u", BTT_MIN_SIZE); errno = EINVAL; return NULL; } struct btt *bttp = Zalloc(sizeof(*bttp)); if (bttp == NULL) { ERR("!Malloc %zu bytes", sizeof(*bttp)); return NULL; } util_mutex_init(&bttp->layout_write_mutex); memcpy(bttp->parent_uuid, parent_uuid, BTTINFO_UUID_LEN); bttp->rawsize = rawsize; bttp->lbasize = lbasize; bttp->ns = ns; bttp->ns_cbp = ns_cbp; /* * Load up layout, if it exists. * * Whether read_layout() finds a valid layout or not, it finishes * updating these layout-related fields: * bttp->nfree * bttp->nlba * bttp->narena * since these fields are used even before a valid layout it written. */ if (read_layout(bttp, 0) < 0) { btt_fini(bttp); /* free up any allocations */ return NULL; } bttp->nlane = bttp->nfree; /* maxlane, if provided, is an upper bound on nlane */ if (maxlane && bttp->nlane > maxlane) bttp->nlane = maxlane; LOG(3, "success, bttp %p nlane %u", bttp, bttp->nlane); return bttp; } /* * btt_nlane -- return the number of "lanes" for this btt namespace * * The number of lanes is the number of threads allowed in this module * concurrently for a given btt. Each thread executing this code must * have a unique "lane" number assigned to it between 0 and btt_nlane() - 1. */ unsigned btt_nlane(struct btt *bttp) { LOG(3, "bttp %p", bttp); return bttp->nlane; } /* * btt_nlba -- return the number of usable blocks in a btt namespace * * Valid LBAs to pass to btt_read() and btt_write() are 0 through * btt_nlba() - 1. */ size_t btt_nlba(struct btt *bttp) { LOG(3, "bttp %p", bttp); return bttp->nlba; } /* * btt_read -- read a block from a btt namespace * * Returns 0 on success, otherwise -1/errno. */ int btt_read(struct btt *bttp, unsigned lane, uint64_t lba, void *buf) { LOG(3, "bttp %p lane %u lba %" PRIu64, bttp, lane, lba); if (invalid_lba(bttp, lba)) return -1; /* if there's no layout written yet, all reads come back as zeros */ if (!bttp->laidout) return zero_block(bttp, buf); /* find which arena LBA lives in, and the offset to the map entry */ struct arena *arenap; uint32_t premap_lba; uint64_t map_entry_off; if (lba_to_arena_lba(bttp, lba, &arenap, &premap_lba) < 0) return -1; /* convert pre-map LBA into an offset into the map */ map_entry_off = arenap->mapoff + BTT_MAP_ENTRY_SIZE * premap_lba; /* * Read the current map entry to get the post-map LBA for the data * block read. */ uint32_t entry; if ((*bttp->ns_cbp->nsread)(bttp->ns, lane, &entry, sizeof(entry), map_entry_off) < 0) return -1; entry = le32toh(entry); /* * Retries come back to the top of this loop (for a rare case where * the map is changed by another thread doing writes to the same LBA). */ while (1) { if (map_entry_is_error(entry)) { ERR("EIO due to map entry error flag"); errno = EIO; return -1; } if (map_entry_is_zero_or_initial(entry)) return zero_block(bttp, buf); /* * Record the post-map LBA in the read tracking table during * the read. The write will check entries in the read tracking * table before allocating a block for a write, waiting for * outstanding reads on that block to complete. * * Since we already checked for error, zero, and initial * states above, the entry must have both error and zero * bits set at this point (BTT_MAP_ENTRY_NORMAL). We store * the entry that way, with those bits set, in the rtt and * btt_write() will check for it the same way, with the bits * both set. */ arenap->rtt[lane] = entry; util_synchronize(); /* * In case this thread was preempted between reading entry and * storing it in the rtt, check to see if the map changed. If * it changed, the block about to be read is at least free now * (in the flog, but that's okay since the data will still be * undisturbed) and potentially allocated and being used for * another write (data disturbed, so not okay to continue). */ uint32_t latest_entry; if ((*bttp->ns_cbp->nsread)(bttp->ns, lane, &latest_entry, sizeof(latest_entry), map_entry_off) < 0) { arenap->rtt[lane] = BTT_MAP_ENTRY_ERROR; return -1; } latest_entry = le32toh(latest_entry); if (entry == latest_entry) break; /* map stayed the same */ else entry = latest_entry; /* try again */ } /* * It is safe to read the block now, since the rtt protects the * block from getting re-allocated to something else by a write. */ uint64_t data_block_off = arenap->dataoff + (uint64_t)(entry & BTT_MAP_ENTRY_LBA_MASK) * arenap->internal_lbasize; int readret = (*bttp->ns_cbp->nsread)(bttp->ns, lane, buf, bttp->lbasize, data_block_off); /* done with read, so clear out rtt entry */ arenap->rtt[lane] = BTT_MAP_ENTRY_ERROR; return readret; } /* * map_lock -- (internal) grab the map_lock and read a map entry */ static int map_lock(struct btt *bttp, unsigned lane, struct arena *arenap, uint32_t *entryp, uint32_t premap_lba) { LOG(3, "bttp %p lane %u arenap %p premap_lba %u", bttp, lane, arenap, premap_lba); uint64_t map_entry_off = arenap->mapoff + BTT_MAP_ENTRY_SIZE * premap_lba; uint32_t map_lock_num = get_map_lock_num(premap_lba, bttp->nfree); util_mutex_lock(&arenap->map_locks[map_lock_num]); /* read the old map entry */ if ((*bttp->ns_cbp->nsread)(bttp->ns, lane, entryp, sizeof(uint32_t), map_entry_off) < 0) { util_mutex_unlock(&arenap->map_locks[map_lock_num]); return -1; } /* if map entry is in its initial state return premap_lba */ if (map_entry_is_initial(*entryp)) *entryp = htole32(premap_lba | BTT_MAP_ENTRY_NORMAL); LOG(9, "locked map[%d]: %u%s%s", premap_lba, *entryp & BTT_MAP_ENTRY_LBA_MASK, (map_entry_is_error(*entryp)) ? " ERROR" : "", (map_entry_is_zero(*entryp)) ? " ZERO" : ""); return 0; } /* * map_abort -- (internal) drop the map_lock without updating the entry */ static void map_abort(struct btt *bttp, unsigned lane, struct arena *arenap, uint32_t premap_lba) { LOG(3, "bttp %p lane %u arenap %p premap_lba %u", bttp, lane, arenap, premap_lba); util_mutex_unlock(&arenap->map_locks[get_map_lock_num(premap_lba, bttp->nfree)]); } /* * map_unlock -- (internal) update the map and drop the map_lock */ static int map_unlock(struct btt *bttp, unsigned lane, struct arena *arenap, uint32_t entry, uint32_t premap_lba) { LOG(3, "bttp %p lane %u arenap %p entry %u premap_lba %u", bttp, lane, arenap, entry, premap_lba); uint64_t map_entry_off = arenap->mapoff + BTT_MAP_ENTRY_SIZE * premap_lba; /* write the new map entry */ int err = (*bttp->ns_cbp->nswrite)(bttp->ns, lane, &entry, sizeof(uint32_t), map_entry_off); util_mutex_unlock(&arenap->map_locks[get_map_lock_num(premap_lba, bttp->nfree)]); LOG(9, "unlocked map[%d]: %u%s%s", premap_lba, entry & BTT_MAP_ENTRY_LBA_MASK, (map_entry_is_error(entry)) ? " ERROR" : "", (map_entry_is_zero(entry)) ? " ZERO" : ""); return err; } /* * btt_write -- write a block to a btt namespace * * Returns 0 on success, otherwise -1/errno. */ int btt_write(struct btt *bttp, unsigned lane, uint64_t lba, const void *buf) { LOG(3, "bttp %p lane %u lba %" PRIu64, bttp, lane, lba); if (invalid_lba(bttp, lba)) return -1; /* first write through here will initialize the metadata layout */ if (!bttp->laidout) { int err = 0; util_mutex_lock(&bttp->layout_write_mutex); if (!bttp->laidout) err = write_layout(bttp, lane, 1); util_mutex_unlock(&bttp->layout_write_mutex); if (err < 0) return err; } /* find which arena LBA lives in, and the offset to the map entry */ struct arena *arenap; uint32_t premap_lba; if (lba_to_arena_lba(bttp, lba, &arenap, &premap_lba) < 0) return -1; /* if the arena is in an error state, writing is not allowed */ if (arenap->flags & BTTINFO_FLAG_ERROR_MASK) { ERR("EIO due to btt_info error flags 0x%x", arenap->flags & BTTINFO_FLAG_ERROR_MASK); errno = EIO; return -1; } /* * This routine was passed a unique "lane" which is an index * into the flog. That means the free block held by flog[lane] * is assigned to this thread and to no other threads (no additional * locking required). So start by performing the write to the * free block. It is only safe to write to a free block if it * doesn't appear in the read tracking table, so scan that first * and if found, wait for the thread reading from it to finish. */ uint32_t free_entry = (arenap->flogs[lane].flog.old_map & BTT_MAP_ENTRY_LBA_MASK) | BTT_MAP_ENTRY_NORMAL; LOG(3, "free_entry %u (before mask %u)", free_entry, arenap->flogs[lane].flog.old_map); /* wait for other threads to finish any reads on free block */ for (unsigned i = 0; i < bttp->nlane; i++) while (arenap->rtt[i] == free_entry) ; /* it is now safe to perform write to the free block */ uint64_t data_block_off = arenap->dataoff + (uint64_t)(free_entry & BTT_MAP_ENTRY_LBA_MASK) * arenap->internal_lbasize; if ((*bttp->ns_cbp->nswrite)(bttp->ns, lane, buf, bttp->lbasize, data_block_off) < 0) return -1; /* * Make the new block active atomically by updating the on-media flog * and then updating the map. */ uint32_t old_entry; if (map_lock(bttp, lane, arenap, &old_entry, premap_lba) < 0) return -1; old_entry = le32toh(old_entry); /* update the flog */ if (flog_update(bttp, lane, arenap, premap_lba, old_entry, free_entry) < 0) { map_abort(bttp, lane, arenap, premap_lba); return -1; } if (map_unlock(bttp, lane, arenap, htole32(free_entry), premap_lba) < 0) { /* * A critical write error occurred, set the arena's * info block error bit. */ set_arena_error(bttp, arenap, lane); errno = EIO; return -1; } return 0; } /* * map_entry_setf -- (internal) set a given flag on a map entry * * Returns 0 on success, otherwise -1/errno. */ static int map_entry_setf(struct btt *bttp, unsigned lane, uint64_t lba, uint32_t setf) { LOG(3, "bttp %p lane %u lba %" PRIu64 " setf 0x%x", bttp, lane, lba, setf); if (invalid_lba(bttp, lba)) return -1; if (!bttp->laidout) { /* * No layout is written yet. If the flag being set * is the zero flag, it is superfluous since all blocks * read as zero at this point. */ if (setf == BTT_MAP_ENTRY_ZERO) return 0; /* * Treat this like the first write and write out * the metadata layout at this point. */ int err = 0; util_mutex_lock(&bttp->layout_write_mutex); if (!bttp->laidout) err = write_layout(bttp, lane, 1); util_mutex_unlock(&bttp->layout_write_mutex); if (err < 0) return err; } /* find which arena LBA lives in, and the offset to the map entry */ struct arena *arenap; uint32_t premap_lba; if (lba_to_arena_lba(bttp, lba, &arenap, &premap_lba) < 0) return -1; /* if the arena is in an error state, writing is not allowed */ if (arenap->flags & BTTINFO_FLAG_ERROR_MASK) { ERR("EIO due to btt_info error flags 0x%x", arenap->flags & BTTINFO_FLAG_ERROR_MASK); errno = EIO; return -1; } /* * Set the flags in the map entry. To do this, read the * current map entry, set the flags, and write out the update. */ uint32_t old_entry; uint32_t new_entry; if (map_lock(bttp, lane, arenap, &old_entry, premap_lba) < 0) return -1; old_entry = le32toh(old_entry); if (setf == BTT_MAP_ENTRY_ZERO && map_entry_is_zero_or_initial(old_entry)) { map_abort(bttp, lane, arenap, premap_lba); return 0; /* block already zero, nothing to do */ } /* create the new map entry */ new_entry = (old_entry & BTT_MAP_ENTRY_LBA_MASK) | setf; if (map_unlock(bttp, lane, arenap, htole32(new_entry), premap_lba) < 0) return -1; return 0; } /* * btt_set_zero -- mark a block as zeroed in a btt namespace * * Returns 0 on success, otherwise -1/errno. */ int btt_set_zero(struct btt *bttp, unsigned lane, uint64_t lba) { LOG(3, "bttp %p lane %u lba %" PRIu64, bttp, lane, lba); return map_entry_setf(bttp, lane, lba, BTT_MAP_ENTRY_ZERO); } /* * btt_set_error -- mark a block as in an error state in a btt namespace * * Returns 0 on success, otherwise -1/errno. */ int btt_set_error(struct btt *bttp, unsigned lane, uint64_t lba) { LOG(3, "bttp %p lane %u lba %" PRIu64, bttp, lane, lba); return map_entry_setf(bttp, lane, lba, BTT_MAP_ENTRY_ERROR); } /* * check_arena -- (internal) perform a consistency check on an arena */ static int check_arena(struct btt *bttp, struct arena *arenap) { LOG(3, "bttp %p arenap %p", bttp, arenap); int consistent = 1; uint64_t map_entry_off = arenap->mapoff; uint32_t bitmapsize = howmany(arenap->internal_nlba, 8); uint8_t *bitmap = Zalloc(bitmapsize); if (bitmap == NULL) { ERR("!Malloc for bitmap"); return -1; } /* * Go through every post-map LBA mentioned in the map and make sure * there are no duplicates. bitmap is used to track which LBAs have * been seen so far. */ uint32_t *mapp = NULL; ssize_t mlen; int next_index = 0; size_t remaining = 0; for (uint32_t i = 0; i < arenap->external_nlba; i++) { uint32_t entry; if (remaining == 0) { /* request a mapping of remaining map area */ size_t req_len = (arenap->external_nlba - i) * sizeof(uint32_t); mlen = (*bttp->ns_cbp->nsmap)(bttp->ns, 0, (void **)&mapp, req_len, map_entry_off); if (mlen < 0) return -1; remaining = (size_t)mlen; next_index = 0; } entry = le32toh(mapp[next_index]); /* for debug, dump non-zero map entries at log level 11 */ if (map_entry_is_zero_or_initial(entry) == 0) LOG(11, "map[%d]: %u%s", i, entry & BTT_MAP_ENTRY_LBA_MASK, (map_entry_is_error(entry)) ? " ERROR" : ""); /* this is an uninitialized map entry, set the default value */ if (map_entry_is_initial(entry)) entry = i; else entry &= BTT_MAP_ENTRY_LBA_MASK; /* check if entry is valid */ if (entry >= arenap->internal_nlba) { ERR("map[%d] entry out of bounds: %u", i, entry); errno = EINVAL; return -1; } if (util_isset(bitmap, entry)) { ERR("map[%d] duplicate entry: %u", i, entry); consistent = 0; } else util_setbit(bitmap, entry); map_entry_off += sizeof(uint32_t); next_index++; ASSERT(remaining >= sizeof(uint32_t)); remaining -= sizeof(uint32_t); } /* * Go through the free blocks in the flog, adding them to bitmap * and checking for duplications. It is sufficient to read the * run-time flog here, avoiding more calls to nsread. */ for (uint32_t i = 0; i < bttp->nfree; i++) { uint32_t entry = arenap->flogs[i].flog.old_map; entry &= BTT_MAP_ENTRY_LBA_MASK; if (util_isset(bitmap, entry)) { ERR("flog[%u] duplicate entry: %u", i, entry); consistent = 0; } else util_setbit(bitmap, entry); } /* * Make sure every possible post-map LBA was accounted for * in the two loops above. */ for (uint32_t i = 0; i < arenap->internal_nlba; i++) if (util_isclr(bitmap, i)) { ERR("unreferenced lba: %d", i); consistent = 0; } Free(bitmap); return consistent; } /* * btt_check -- perform a consistency check on a btt namespace * * This routine contains a fairly high-impact set of consistency checks. * It may use a good amount of dynamic memory and CPU time performing * the checks. Any lightweight, quick consistency checks are included * in read_layout() so they happen every time the BTT area is opened * for use. * * Returns true if consistent, zero if inconsistent, -1/error if checking * cannot happen due to other errors. * * No lane number required here because only one thread is allowed -- all * other threads must be locked out of all btt routines for this btt * namespace while this is running. */ int btt_check(struct btt *bttp) { LOG(3, "bttp %p", bttp); int consistent = 1; if (!bttp->laidout) { /* consistent by definition */ LOG(3, "no layout yet"); return consistent; } /* XXX report issues found during read_layout (from flags) */ /* for each arena... */ struct arena *arenap = bttp->arenas; for (unsigned i = 0; i < bttp->narena; i++, arenap++) { /* * Perform the consistency checks for the arena. */ int retval = check_arena(bttp, arenap); if (retval < 0) return retval; else if (retval == 0) consistent = 0; } /* XXX stub */ return consistent; } /* * btt_fini -- delete opaque btt info, done using btt namespace */ void btt_fini(struct btt *bttp) { LOG(3, "bttp %p", bttp); if (bttp->arenas) { for (unsigned i = 0; i < bttp->narena; i++) { if (bttp->arenas[i].flogs) Free(bttp->arenas[i].flogs); if (bttp->arenas[i].rtt) Free((void *)bttp->arenas[i].rtt); if (bttp->arenas[i].rtt) Free((void *)bttp->arenas[i].map_locks); } Free(bttp->arenas); } Free(bttp); } pmdk-1.8/src/libpmemblk/btt_layout.h0000664000000000000000000001115013615011243016253 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * btt_layout.h -- block translation table on-media layout definitions */ /* * Layout of BTT info block. All integers are stored little-endian. */ #ifndef BTT_LAYOUT_H #define BTT_LAYOUT_H 1 #ifdef __cplusplus extern "C" { #endif #define BTT_ALIGNMENT ((uintptr_t)4096) /* alignment of all BTT structures */ #define BTTINFO_SIG_LEN 16 #define BTTINFO_UUID_LEN 16 #define BTTINFO_UNUSED_LEN 3968 #define BTTINFO_SIG "BTT_ARENA_INFO\0" struct btt_info { char sig[BTTINFO_SIG_LEN]; /* must be "BTT_ARENA_INFO\0\0" */ uint8_t uuid[BTTINFO_UUID_LEN]; /* BTT UUID */ uint8_t parent_uuid[BTTINFO_UUID_LEN]; /* UUID of container */ uint32_t flags; /* see flag bits below */ uint16_t major; /* major version */ uint16_t minor; /* minor version */ uint32_t external_lbasize; /* advertised LBA size (bytes) */ uint32_t external_nlba; /* advertised LBAs in this arena */ uint32_t internal_lbasize; /* size of data area blocks (bytes) */ uint32_t internal_nlba; /* number of blocks in data area */ uint32_t nfree; /* number of free blocks */ uint32_t infosize; /* size of this info block */ /* * The following offsets are relative to the beginning of * the btt_info block. */ uint64_t nextoff; /* offset to next arena (or zero) */ uint64_t dataoff; /* offset to arena data area */ uint64_t mapoff; /* offset to area map */ uint64_t flogoff; /* offset to area flog */ uint64_t infooff; /* offset to backup info block */ char unused[BTTINFO_UNUSED_LEN]; /* must be zero */ uint64_t checksum; /* Fletcher64 of all fields */ }; /* * Definitions for flags mask for btt_info structure above. */ #define BTTINFO_FLAG_ERROR 0x00000001 /* error state (read-only) */ #define BTTINFO_FLAG_ERROR_MASK 0x00000001 /* all error bits */ /* * Current on-media format versions. */ #define BTTINFO_MAJOR_VERSION 1 #define BTTINFO_MINOR_VERSION 1 /* * Layout of a BTT "flog" entry. All integers are stored little-endian. * * The "nfree" field in the BTT info block determines how many of these * flog entries there are, and each entry consists of two of the following * structs (entry updates alternate between the two structs), padded up * to a cache line boundary to isolate adjacent updates. */ #define BTT_FLOG_PAIR_ALIGN ((uintptr_t)64) struct btt_flog { uint32_t lba; /* last pre-map LBA using this entry */ uint32_t old_map; /* old post-map LBA (the freed block) */ uint32_t new_map; /* new post-map LBA */ uint32_t seq; /* sequence number (01, 10, 11) */ }; /* * Layout of a BTT "map" entry. 4-byte internal LBA offset, little-endian. */ #define BTT_MAP_ENTRY_SIZE 4 #define BTT_MAP_ENTRY_ERROR 0x40000000U #define BTT_MAP_ENTRY_ZERO 0x80000000U #define BTT_MAP_ENTRY_NORMAL 0xC0000000U #define BTT_MAP_ENTRY_LBA_MASK 0x3fffffffU #define BTT_MAP_LOCK_ALIGN ((uintptr_t)64) /* * BTT layout properties... */ #define BTT_MIN_SIZE ((1u << 20) * 16) #define BTT_MAX_ARENA (1ull << 39) /* 512GB per arena */ #define BTT_MIN_LBA_SIZE (size_t)512 #define BTT_INTERNAL_LBA_ALIGNMENT 256U #define BTT_DEFAULT_NFREE 256 #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmemblk/libpmemblk.rc0000664000000000000000000000344113615011243016364 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmemblk.rc -- libpmemblk resource file */ #include #define FILE_NAME "libpmemblk.dll" #define DESCRIPTION "libpmemblk - persistent memory resident array of blocks" #define TYPE VFT_DLL #include pmdk-1.8/src/libpmemblk/blk.c0000664000000000000000000005235113615011243014640 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * blk.c -- block memory pool entry points for libpmem */ #include #include #include #include #include #include #include #include #include #include #include #include "libpmem.h" #include "libpmemblk.h" #include "mmap.h" #include "set.h" #include "out.h" #include "btt.h" #include "blk.h" #include "util.h" #include "sys_util.h" #include "util_pmem.h" #include "valgrind_internal.h" static const struct pool_attr Blk_create_attr = { BLK_HDR_SIG, BLK_FORMAT_MAJOR, BLK_FORMAT_FEAT_DEFAULT, {0}, {0}, {0}, {0}, {0} }; static const struct pool_attr Blk_open_attr = { BLK_HDR_SIG, BLK_FORMAT_MAJOR, BLK_FORMAT_FEAT_CHECK, {0}, {0}, {0}, {0}, {0} }; /* * lane_enter -- (internal) acquire a unique lane number */ static void lane_enter(PMEMblkpool *pbp, unsigned *lane) { unsigned mylane; mylane = util_fetch_and_add32(&pbp->next_lane, 1) % pbp->nlane; /* lane selected, grab the per-lane lock */ util_mutex_lock(&pbp->locks[mylane]); *lane = mylane; } /* * lane_exit -- (internal) drop lane lock */ static void lane_exit(PMEMblkpool *pbp, unsigned mylane) { util_mutex_unlock(&pbp->locks[mylane]); } /* * nsread -- (internal) read data from the namespace encapsulating the BTT * * This routine is provided to btt_init() to allow the btt module to * do I/O on the memory pool containing the BTT layout. */ static int nsread(void *ns, unsigned lane, void *buf, size_t count, uint64_t off) { struct pmemblk *pbp = (struct pmemblk *)ns; LOG(13, "pbp %p lane %u count %zu off %" PRIu64, pbp, lane, count, off); if (off + count > pbp->datasize) { ERR("offset + count (%zu) past end of data area (%zu)", (size_t)off + count, pbp->datasize); errno = EINVAL; return -1; } memcpy(buf, (char *)pbp->data + off, count); return 0; } /* * nswrite -- (internal) write data to the namespace encapsulating the BTT * * This routine is provided to btt_init() to allow the btt module to * do I/O on the memory pool containing the BTT layout. */ static int nswrite(void *ns, unsigned lane, const void *buf, size_t count, uint64_t off) { struct pmemblk *pbp = (struct pmemblk *)ns; LOG(13, "pbp %p lane %u count %zu off %" PRIu64, pbp, lane, count, off); if (off + count > pbp->datasize) { ERR("offset + count (%zu) past end of data area (%zu)", (size_t)off + count, pbp->datasize); errno = EINVAL; return -1; } void *dest = (char *)pbp->data + off; #ifdef DEBUG /* grab debug write lock */ util_mutex_lock(&pbp->write_lock); #endif /* unprotect the memory (debug version only) */ RANGE_RW(dest, count, pbp->is_dev_dax); if (pbp->is_pmem) pmem_memcpy_nodrain(dest, buf, count); else memcpy(dest, buf, count); /* protect the memory again (debug version only) */ RANGE_RO(dest, count, pbp->is_dev_dax); #ifdef DEBUG /* release debug write lock */ util_mutex_unlock(&pbp->write_lock); #endif if (pbp->is_pmem) pmem_drain(); else pmem_msync(dest, count); return 0; } /* * nsmap -- (internal) allow direct access to a range of a namespace * * The caller requests a range to be "mapped" but the return value * may indicate a smaller amount (in which case the caller is expected * to call back later for another mapping). * * This routine is provided to btt_init() to allow the btt module to * do I/O on the memory pool containing the BTT layout. */ static ssize_t nsmap(void *ns, unsigned lane, void **addrp, size_t len, uint64_t off) { struct pmemblk *pbp = (struct pmemblk *)ns; LOG(12, "pbp %p lane %u len %zu off %" PRIu64, pbp, lane, len, off); ASSERT(((ssize_t)len) >= 0); if (off + len >= pbp->datasize) { ERR("offset + len (%zu) past end of data area (%zu)", (size_t)off + len, pbp->datasize - 1); errno = EINVAL; return -1; } /* * Since the entire file is memory-mapped, this callback * can always provide the entire length requested. */ *addrp = (char *)pbp->data + off; LOG(12, "returning addr %p", *addrp); return (ssize_t)len; } /* * nssync -- (internal) flush changes made to a namespace range * * This is used in conjunction with the addresses handed out by * nsmap() above. There's no need to sync things written via * nswrite() since those changes are flushed each time nswrite() * is called. * * This routine is provided to btt_init() to allow the btt module to * do I/O on the memory pool containing the BTT layout. */ static void nssync(void *ns, unsigned lane, void *addr, size_t len) { struct pmemblk *pbp = (struct pmemblk *)ns; LOG(12, "pbp %p lane %u addr %p len %zu", pbp, lane, addr, len); if (pbp->is_pmem) pmem_persist(addr, len); else pmem_msync(addr, len); } /* * nszero -- (internal) zero data in the namespace encapsulating the BTT * * This routine is provided to btt_init() to allow the btt module to * zero the memory pool containing the BTT layout. */ static int nszero(void *ns, unsigned lane, size_t count, uint64_t off) { struct pmemblk *pbp = (struct pmemblk *)ns; LOG(13, "pbp %p lane %u count %zu off %" PRIu64, pbp, lane, count, off); if (off + count > pbp->datasize) { ERR("offset + count (%zu) past end of data area (%zu)", (size_t)off + count, pbp->datasize); errno = EINVAL; return -1; } void *dest = (char *)pbp->data + off; /* unprotect the memory (debug version only) */ RANGE_RW(dest, count, pbp->is_dev_dax); pmem_memset_persist(dest, 0, count); /* protect the memory again (debug version only) */ RANGE_RO(dest, count, pbp->is_dev_dax); return 0; } /* callbacks for btt_init() */ static struct ns_callback ns_cb = { .nsread = nsread, .nswrite = nswrite, .nszero = nszero, .nsmap = nsmap, .nssync = nssync, .ns_is_zeroed = 0 }; /* * blk_descr_create -- (internal) create block memory pool descriptor */ static void blk_descr_create(PMEMblkpool *pbp, uint32_t bsize, int zeroed) { LOG(3, "pbp %p bsize %u zeroed %d", pbp, bsize, zeroed); /* create the required metadata */ pbp->bsize = htole32(bsize); util_persist(pbp->is_pmem, &pbp->bsize, sizeof(bsize)); pbp->is_zeroed = zeroed; util_persist(pbp->is_pmem, &pbp->is_zeroed, sizeof(pbp->is_zeroed)); } /* * blk_descr_check -- (internal) validate block memory pool descriptor */ static int blk_descr_check(PMEMblkpool *pbp, size_t *bsize) { LOG(3, "pbp %p bsize %zu", pbp, *bsize); size_t hdr_bsize = le32toh(pbp->bsize); if (*bsize && *bsize != hdr_bsize) { ERR("wrong bsize (%zu), pool created with bsize %zu", *bsize, hdr_bsize); errno = EINVAL; return -1; } *bsize = hdr_bsize; LOG(3, "using block size from header: %zu", *bsize); return 0; } /* * blk_runtime_init -- (internal) initialize block memory pool runtime data */ static int blk_runtime_init(PMEMblkpool *pbp, size_t bsize, int rdonly) { LOG(3, "pbp %p bsize %zu rdonly %d", pbp, bsize, rdonly); /* remove volatile part of header */ VALGRIND_REMOVE_PMEM_MAPPING(&pbp->addr, sizeof(struct pmemblk) - sizeof(struct pool_hdr) - sizeof(pbp->bsize) - sizeof(pbp->is_zeroed)); /* * Use some of the memory pool area for run-time info. This * run-time state is never loaded from the file, it is always * created here, so no need to worry about byte-order. */ pbp->rdonly = rdonly; pbp->data = (char *)pbp->addr + roundup(sizeof(*pbp), BLK_FORMAT_DATA_ALIGN); ASSERT(((char *)pbp->addr + pbp->size) >= (char *)pbp->data); pbp->datasize = (size_t) (((char *)pbp->addr + pbp->size) - (char *)pbp->data); LOG(4, "data area %p data size %zu bsize %zu", pbp->data, pbp->datasize, bsize); long ncpus = sysconf(_SC_NPROCESSORS_ONLN); if (ncpus < 1) ncpus = 1; ns_cb.ns_is_zeroed = pbp->is_zeroed; /* things free by "goto err" if not NULL */ struct btt *bttp = NULL; os_mutex_t *locks = NULL; bttp = btt_init(pbp->datasize, (uint32_t)bsize, pbp->hdr.poolset_uuid, (unsigned)ncpus * 2, pbp, &ns_cb); if (bttp == NULL) goto err; /* btt_init set errno, called LOG */ pbp->bttp = bttp; pbp->nlane = btt_nlane(pbp->bttp); pbp->next_lane = 0; if ((locks = Malloc(pbp->nlane * sizeof(*locks))) == NULL) { ERR("!Malloc for lane locks"); goto err; } for (unsigned i = 0; i < pbp->nlane; i++) util_mutex_init(&locks[i]); pbp->locks = locks; #ifdef DEBUG /* initialize debug lock */ util_mutex_init(&pbp->write_lock); #endif /* * If possible, turn off all permissions on the pool header page. * * The prototype PMFS doesn't allow this when large pages are in * use. It is not considered an error if this fails. */ RANGE_NONE(pbp->addr, sizeof(struct pool_hdr), pbp->is_dev_dax); /* the data area should be kept read-only for debug version */ RANGE_RO(pbp->data, pbp->datasize, pbp->is_dev_dax); return 0; err: LOG(4, "error clean up"); int oerrno = errno; if (bttp) btt_fini(bttp); errno = oerrno; return -1; } /* * pmemblk_createU -- create a block memory pool */ #ifndef _WIN32 static inline #endif PMEMblkpool * pmemblk_createU(const char *path, size_t bsize, size_t poolsize, mode_t mode) { LOG(3, "path %s bsize %zu poolsize %zu mode %o", path, bsize, poolsize, mode); /* check if bsize is valid */ if (bsize == 0) { ERR("Invalid block size %zu", bsize); errno = EINVAL; return NULL; } if (bsize > UINT32_MAX) { ERR("Invalid block size %zu", bsize); errno = EINVAL; return NULL; } struct pool_set *set; struct pool_attr adj_pool_attr = Blk_create_attr; /* force set SDS feature */ if (SDS_at_create) adj_pool_attr.features.incompat |= POOL_FEAT_SDS; else adj_pool_attr.features.incompat &= ~POOL_FEAT_SDS; if (util_pool_create(&set, path, poolsize, PMEMBLK_MIN_POOL, PMEMBLK_MIN_PART, &adj_pool_attr, NULL, REPLICAS_DISABLED) != 0) { LOG(2, "cannot create pool or pool set"); return NULL; } ASSERT(set->nreplicas > 0); struct pool_replica *rep = set->replica[0]; PMEMblkpool *pbp = rep->part[0].addr; VALGRIND_REMOVE_PMEM_MAPPING(&pbp->addr, sizeof(struct pmemblk) - ((uintptr_t)&pbp->addr - (uintptr_t)&pbp->hdr)); pbp->addr = pbp; pbp->size = rep->repsize; pbp->set = set; pbp->is_pmem = rep->is_pmem; pbp->is_dev_dax = rep->part[0].is_dev_dax; /* is_dev_dax implies is_pmem */ ASSERT(!pbp->is_dev_dax || pbp->is_pmem); /* create pool descriptor */ blk_descr_create(pbp, (uint32_t)bsize, set->zeroed); /* initialize runtime parts */ if (blk_runtime_init(pbp, bsize, 0) != 0) { ERR("pool initialization failed"); goto err; } if (util_poolset_chmod(set, mode)) goto err; util_poolset_fdclose(set); LOG(3, "pbp %p", pbp); return pbp; err: LOG(4, "error clean up"); int oerrno = errno; util_poolset_close(set, DELETE_CREATED_PARTS); errno = oerrno; return NULL; } #ifndef _WIN32 /* * pmemblk_create -- create a block memory pool */ PMEMblkpool * pmemblk_create(const char *path, size_t bsize, size_t poolsize, mode_t mode) { return pmemblk_createU(path, bsize, poolsize, mode); } #else /* * pmemblk_createW -- create a block memory pool */ PMEMblkpool * pmemblk_createW(const wchar_t *path, size_t bsize, size_t poolsize, mode_t mode) { char *upath = util_toUTF8(path); if (upath == NULL) return NULL; PMEMblkpool *ret = pmemblk_createU(upath, bsize, poolsize, mode); util_free_UTF8(upath); return ret; } #endif /* * blk_open_common -- (internal) open a block memory pool * * This routine does all the work, but takes a cow flag so internal * calls can map a read-only pool if required. * * Passing in bsize == 0 means a valid pool header must exist (which * will supply the block size). */ static PMEMblkpool * blk_open_common(const char *path, size_t bsize, unsigned flags) { LOG(3, "path %s bsize %zu flags 0x%x", path, bsize, flags); struct pool_set *set; if (util_pool_open(&set, path, PMEMBLK_MIN_PART, &Blk_open_attr, NULL, NULL, flags) != 0) { LOG(2, "cannot open pool or pool set"); return NULL; } ASSERT(set->nreplicas > 0); struct pool_replica *rep = set->replica[0]; PMEMblkpool *pbp = rep->part[0].addr; VALGRIND_REMOVE_PMEM_MAPPING(&pbp->addr, sizeof(struct pmemblk) - ((uintptr_t)&pbp->addr - (uintptr_t)&pbp->hdr)); pbp->addr = pbp; pbp->size = rep->repsize; pbp->set = set; pbp->is_pmem = rep->is_pmem; pbp->is_dev_dax = rep->part[0].is_dev_dax; /* is_dev_dax implies is_pmem */ ASSERT(!pbp->is_dev_dax || pbp->is_pmem); if (set->nreplicas > 1) { errno = ENOTSUP; ERR("!replicas not supported"); goto err; } /* validate pool descriptor */ if (blk_descr_check(pbp, &bsize) != 0) { LOG(2, "descriptor check failed"); goto err; } /* initialize runtime parts */ if (blk_runtime_init(pbp, bsize, set->rdonly) != 0) { ERR("pool initialization failed"); goto err; } util_poolset_fdclose(set); LOG(3, "pbp %p", pbp); return pbp; err: LOG(4, "error clean up"); int oerrno = errno; util_poolset_close(set, DO_NOT_DELETE_PARTS); errno = oerrno; return NULL; } /* * pmemblk_openU -- open a block memory pool */ #ifndef _WIN32 static inline #endif PMEMblkpool * pmemblk_openU(const char *path, size_t bsize) { LOG(3, "path %s bsize %zu", path, bsize); return blk_open_common(path, bsize, COW_at_open ? POOL_OPEN_COW : 0); } #ifndef _WIN32 /* * pmemblk_open -- open a block memory pool */ PMEMblkpool * pmemblk_open(const char *path, size_t bsize) { return pmemblk_openU(path, bsize); } #else /* * pmemblk_openW -- open a block memory pool */ PMEMblkpool * pmemblk_openW(const wchar_t *path, size_t bsize) { char *upath = util_toUTF8(path); if (upath == NULL) return NULL; PMEMblkpool *ret = pmemblk_openU(upath, bsize); util_free_UTF8(upath); return ret; } #endif /* * pmemblk_close -- close a block memory pool */ void pmemblk_close(PMEMblkpool *pbp) { LOG(3, "pbp %p", pbp); btt_fini(pbp->bttp); if (pbp->locks) { for (unsigned i = 0; i < pbp->nlane; i++) util_mutex_destroy(&pbp->locks[i]); Free((void *)pbp->locks); } #ifdef DEBUG /* destroy debug lock */ util_mutex_destroy(&pbp->write_lock); #endif util_poolset_close(pbp->set, DO_NOT_DELETE_PARTS); } /* * pmemblk_bsize -- return size of block for specified pool */ size_t pmemblk_bsize(PMEMblkpool *pbp) { LOG(3, "pbp %p", pbp); return le32toh(pbp->bsize); } /* * pmemblk_nblock -- return number of usable blocks in a block memory pool */ size_t pmemblk_nblock(PMEMblkpool *pbp) { LOG(3, "pbp %p", pbp); return btt_nlba(pbp->bttp); } /* * pmemblk_read -- read a block in a block memory pool */ int pmemblk_read(PMEMblkpool *pbp, void *buf, long long blockno) { LOG(3, "pbp %p buf %p blockno %lld", pbp, buf, blockno); if (blockno < 0) { ERR("negative block number"); errno = EINVAL; return -1; } unsigned lane; lane_enter(pbp, &lane); int err = btt_read(pbp->bttp, lane, (uint64_t)blockno, buf); lane_exit(pbp, lane); return err; } /* * pmemblk_write -- write a block (atomically) in a block memory pool */ int pmemblk_write(PMEMblkpool *pbp, const void *buf, long long blockno) { LOG(3, "pbp %p buf %p blockno %lld", pbp, buf, blockno); if (pbp->rdonly) { ERR("EROFS (pool is read-only)"); errno = EROFS; return -1; } if (blockno < 0) { ERR("negative block number"); errno = EINVAL; return -1; } unsigned lane; lane_enter(pbp, &lane); int err = btt_write(pbp->bttp, lane, (uint64_t)blockno, buf); lane_exit(pbp, lane); return err; } /* * pmemblk_set_zero -- zero a block in a block memory pool */ int pmemblk_set_zero(PMEMblkpool *pbp, long long blockno) { LOG(3, "pbp %p blockno %lld", pbp, blockno); if (pbp->rdonly) { ERR("EROFS (pool is read-only)"); errno = EROFS; return -1; } if (blockno < 0) { ERR("negative block number"); errno = EINVAL; return -1; } unsigned lane; lane_enter(pbp, &lane); int err = btt_set_zero(pbp->bttp, lane, (uint64_t)blockno); lane_exit(pbp, lane); return err; } /* * pmemblk_set_error -- set the error state on a block in a block memory pool */ int pmemblk_set_error(PMEMblkpool *pbp, long long blockno) { LOG(3, "pbp %p blockno %lld", pbp, blockno); if (pbp->rdonly) { ERR("EROFS (pool is read-only)"); errno = EROFS; return -1; } if (blockno < 0) { ERR("negative block number"); errno = EINVAL; return -1; } unsigned lane; lane_enter(pbp, &lane); int err = btt_set_error(pbp->bttp, lane, (uint64_t)blockno); lane_exit(pbp, lane); return err; } /* * pmemblk_checkU -- block memory pool consistency check */ #ifndef _WIN32 static inline #endif int pmemblk_checkU(const char *path, size_t bsize) { LOG(3, "path \"%s\" bsize %zu", path, bsize); /* map the pool read-only */ PMEMblkpool *pbp = blk_open_common(path, bsize, POOL_OPEN_COW); if (pbp == NULL) return -1; /* errno set by blk_open_common() */ int retval = btt_check(pbp->bttp); int oerrno = errno; pmemblk_close(pbp); errno = oerrno; return retval; } #ifndef _WIN32 /* * pmemblk_check -- block memory pool consistency check */ int pmemblk_check(const char *path, size_t bsize) { return pmemblk_checkU(path, bsize); } #else /* * pmemblk_checkW -- block memory pool consistency check */ int pmemblk_checkW(const wchar_t *path, size_t bsize) { char *upath = util_toUTF8(path); if (upath == NULL) return -1; int ret = pmemblk_checkU(upath, bsize); util_free_UTF8(upath); return ret; } #endif /* * pmemblk_ctl_getU -- programmatically executes a read ctl query */ #ifndef _WIN32 static inline #endif int pmemblk_ctl_getU(PMEMblkpool *pbp, const char *name, void *arg) { LOG(3, "pbp %p name %s arg %p", pbp, name, arg); return ctl_query(pbp == NULL ? NULL : pbp->ctl, pbp, CTL_QUERY_PROGRAMMATIC, name, CTL_QUERY_READ, arg); } /* * pmemblk_ctl_setU -- programmatically executes a write ctl query */ #ifndef _WIN32 static inline #endif int pmemblk_ctl_setU(PMEMblkpool *pbp, const char *name, void *arg) { LOG(3, "pbp %p name %s arg %p", pbp, name, arg); return ctl_query(pbp == NULL ? NULL : pbp->ctl, pbp, CTL_QUERY_PROGRAMMATIC, name, CTL_QUERY_WRITE, arg); } /* * pmemblk_ctl_execU -- programmatically executes a runnable ctl query */ #ifndef _WIN32 static inline #endif int pmemblk_ctl_execU(PMEMblkpool *pbp, const char *name, void *arg) { LOG(3, "pbp %p name %s arg %p", pbp, name, arg); return ctl_query(pbp == NULL ? NULL : pbp->ctl, pbp, CTL_QUERY_PROGRAMMATIC, name, CTL_QUERY_RUNNABLE, arg); } #ifndef _WIN32 /* * pmemblk_ctl_get -- programmatically executes a read ctl query */ int pmemblk_ctl_get(PMEMblkpool *pbp, const char *name, void *arg) { return pmemblk_ctl_getU(pbp, name, arg); } /* * pmemblk_ctl_set -- programmatically executes a write ctl query */ int pmemblk_ctl_set(PMEMblkpool *pbp, const char *name, void *arg) { return pmemblk_ctl_setU(pbp, name, arg); } /* * pmemblk_ctl_exec -- programmatically executes a runnable ctl query */ int pmemblk_ctl_exec(PMEMblkpool *pbp, const char *name, void *arg) { return pmemblk_ctl_execU(pbp, name, arg); } #else /* * pmemblk_ctl_getW -- programmatically executes a read ctl query */ int pmemblk_ctl_getW(PMEMblkpool *pbp, const wchar_t *name, void *arg) { char *uname = util_toUTF8(name); if (uname == NULL) return -1; int ret = pmemblk_ctl_getU(pbp, uname, arg); util_free_UTF8(uname); return ret; } /* * pmemblk_ctl_setW -- programmatically executes a write ctl query */ int pmemblk_ctl_setW(PMEMblkpool *pbp, const wchar_t *name, void *arg) { char *uname = util_toUTF8(name); if (uname == NULL) return -1; int ret = pmemblk_ctl_setU(pbp, uname, arg); util_free_UTF8(uname); return ret; } /* * pmemblk_ctl_execW -- programmatically executes a runnable ctl query */ int pmemblk_ctl_execW(PMEMblkpool *pbp, const wchar_t *name, void *arg) { char *uname = util_toUTF8(name); if (uname == NULL) return -1; int ret = pmemblk_ctl_execU(pbp, uname, arg); util_free_UTF8(uname); return ret; } #endif #if FAULT_INJECTION void pmemblk_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { common_inject_fault_at(type, nth, at); } int pmemblk_fault_injection_enabled(void) { return common_fault_injection_enabled(); } #endif pmdk-1.8/src/libpmemblk/libpmemblk.vcxproj.filters0000664000000000000000000001413213615011243021121 0ustar rootroot {5f4b56cf-a674-4f35-abfa-d867d9d91f68} {dee0ff57-9af8-485a-888b-0087d6e11cf8} Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Source Files Source Files pmdk-1.8/src/libpmemblk/libpmemblk.link.in0000664000000000000000000000371013615011243017321 0ustar rootroot# # Copyright 2014-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # src/libpmemblk.link -- linker link file for libpmemblk # LIBPMEMBLK_1.0 { global: pmemblk_check_version; pmemblk_set_funcs; pmemblk_errormsg; pmemblk_create; pmemblk_open; pmemblk_close; pmemblk_check; pmemblk_ctl_exec; pmemblk_ctl_get; pmemblk_ctl_set; pmemblk_nblock; pmemblk_read; pmemblk_write; pmemblk_set_zero; pmemblk_set_error; pmemblk_bsize; fault_injection; local: *; }; pmdk-1.8/src/libpmemblk/libpmemblk.def0000664000000000000000000000407113615011243016516 0ustar rootroot;;;; Begin Copyright Notice ; ; Copyright 2015-2018, Intel Corporation ; ; Redistribution and use in source and binary forms, with or without ; modification, are permitted provided that the following conditions ; are met: ; ; * Redistributions of source code must retain the above copyright ; notice, this list of conditions and the following disclaimer. ; ; * Redistributions in binary form must reproduce the above copyright ; notice, this list of conditions and the following disclaimer in ; the documentation and/or other materials provided with the ; distribution. ; ; * Neither the name of the copyright holder nor the names of its ; contributors may be used to endorse or promote products derived ; from this software without specific prior written permission. ; ; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ; "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT ; LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ; A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT ; OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, ; SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT ; LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, ; DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY ; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT ; (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE ; OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ; ;;;; End Copyright Notice LIBRARY libpmemblk VERSION 1.0 EXPORTS pmemblk_check_versionU pmemblk_check_versionW pmemblk_set_funcs pmemblk_errormsgU pmemblk_errormsgW pmemblk_createU pmemblk_createW pmemblk_openU pmemblk_openW pmemblk_close pmemblk_checkU pmemblk_checkW pmemblk_ctl_execU; pmemblk_ctl_execW; pmemblk_ctl_getU; pmemblk_ctl_getW; pmemblk_ctl_setU; pmemblk_ctl_setW; pmemblk_bsize pmemblk_nblock pmemblk_read pmemblk_write pmemblk_set_zero pmemblk_set_error DllMain pmdk-1.8/src/libpmemblk/libpmemblk.vcxproj0000664000000000000000000001405313615011243017454 0ustar rootroot Debug x64 Release x64 {9e9e3d25-2139-4a5d-9200-18148ddead45} {901f04db-e1a5-4a41-8b81-9d31c19acd59} {f7c6c6b6-4142-4c82-8699-4a9d8183181b} DynamicLibrary libpmemblk libpmemblk en-US 14.0 10.0.16299.0 10.0.10240.0 DynamicLibrary true v140 DynamicLibrary false false v140 pmdk-1.8/src/libpmemblk/Makefile0000664000000000000000000000351013615011243015355 0ustar rootroot# Copyright 2014-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/libpmemblk/Makefile -- Makefile for libpmemblk # LIBRARY_NAME = pmemblk LIBRARY_SO_VERSION = 1 LIBRARY_VERSION = 0.0 include ../common/pmemcommon.inc SOURCE +=\ blk.c\ btt.c\ libpmemblk.c include ../Makefile.inc CFLAGS += $(LIBNDCTL_CFLAGS) LIBS += -pthread -lpmem $(LIBNDCTL_LIBS) pmdk-1.8/src/libpmemblk/libpmemblk.c0000664000000000000000000001331113615011243016177 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmemblk.c -- pmem entry points for libpmemblk */ #include #include #include "libpmemblk.h" #include "ctl_global.h" #include "pmemcommon.h" #include "blk.h" /* * The variable from which the config is directly loaded. The string * cannot contain any comments or extraneous white characters. */ #define BLK_CONFIG_ENV_VARIABLE "PMEMBLK_CONF" /* * The variable that points to a config file from which the config is loaded. */ #define BLK_CONFIG_FILE_ENV_VARIABLE "PMEMBLK_CONF_FILE" /* * blk_ctl_init_and_load -- (static) initializes CTL and loads configuration * from env variable and file */ static int blk_ctl_init_and_load(PMEMblkpool *pbp) { LOG(3, "pbp %p", pbp); if (pbp != NULL && (pbp->ctl = ctl_new()) == NULL) { LOG(2, "!ctl_new"); return -1; } char *env_config = os_getenv(BLK_CONFIG_ENV_VARIABLE); if (env_config != NULL) { if (ctl_load_config_from_string(pbp ? pbp->ctl : NULL, pbp, env_config) != 0) { LOG(2, "unable to parse config stored in %s " "environment variable", BLK_CONFIG_ENV_VARIABLE); goto err; } } char *env_config_file = os_getenv(BLK_CONFIG_FILE_ENV_VARIABLE); if (env_config_file != NULL && env_config_file[0] != '\0') { if (ctl_load_config_from_file(pbp ? pbp->ctl : NULL, pbp, env_config_file) != 0) { LOG(2, "unable to parse config stored in %s " "file (from %s environment variable)", env_config_file, BLK_CONFIG_FILE_ENV_VARIABLE); goto err; } } return 0; err: if (pbp) ctl_delete(pbp->ctl); return -1; } /* * libpmemblk_init -- (internal) load-time initialization for blk * * Called automatically by the run-time loader. */ ATTR_CONSTRUCTOR void libpmemblk_init(void) { ctl_global_register(); if (blk_ctl_init_and_load(NULL)) FATAL("error: %s", pmemblk_errormsg()); common_init(PMEMBLK_LOG_PREFIX, PMEMBLK_LOG_LEVEL_VAR, PMEMBLK_LOG_FILE_VAR, PMEMBLK_MAJOR_VERSION, PMEMBLK_MINOR_VERSION); LOG(3, NULL); } /* * libpmemblk_fini -- libpmemblk cleanup routine * * Called automatically when the process terminates. */ ATTR_DESTRUCTOR void libpmemblk_fini(void) { LOG(3, NULL); common_fini(); } /* * pmemblk_check_versionU -- see if lib meets application version requirements */ #ifndef _WIN32 static inline #endif const char * pmemblk_check_versionU(unsigned major_required, unsigned minor_required) { LOG(3, "major_required %u minor_required %u", major_required, minor_required); if (major_required != PMEMBLK_MAJOR_VERSION) { ERR("libpmemblk major version mismatch (need %u, found %u)", major_required, PMEMBLK_MAJOR_VERSION); return out_get_errormsg(); } if (minor_required > PMEMBLK_MINOR_VERSION) { ERR("libpmemblk minor version mismatch (need %u, found %u)", minor_required, PMEMBLK_MINOR_VERSION); return out_get_errormsg(); } return NULL; } #ifndef _WIN32 /* * pmemblk_check_version -- see if lib meets application version requirements */ const char * pmemblk_check_version(unsigned major_required, unsigned minor_required) { return pmemblk_check_versionU(major_required, minor_required); } #else /* * pmemblk_check_versionW -- see if lib meets application version requirements */ const wchar_t * pmemblk_check_versionW(unsigned major_required, unsigned minor_required) { if (pmemblk_check_versionU(major_required, minor_required) != NULL) return out_get_errormsgW(); else return NULL; } #endif /* * pmemblk_set_funcs -- allow overriding libpmemblk's call to malloc, etc. */ void pmemblk_set_funcs( void *(*malloc_func)(size_t size), void (*free_func)(void *ptr), void *(*realloc_func)(void *ptr, size_t size), char *(*strdup_func)(const char *s)) { LOG(3, NULL); util_set_alloc_funcs(malloc_func, free_func, realloc_func, strdup_func); } /* * pmemblk_errormsgU -- return last error message */ #ifndef _WIN32 static inline #endif const char * pmemblk_errormsgU(void) { return out_get_errormsg(); } #ifndef _WIN32 /* * pmemblk_errormsg -- return last error message */ const char * pmemblk_errormsg(void) { return pmemblk_errormsgU(); } #else /* * pmemblk_errormsgW -- return last error message as wchar_t */ const wchar_t * pmemblk_errormsgW(void) { return out_get_errormsgW(); } #endif pmdk-1.8/src/Makefile.inc0000664000000000000000000002144013615011243014011 0ustar rootroot# Copyright 2014-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/Makefile.inc -- common Makefile rules for PMDK # TOP := $(dir $(lastword $(MAKEFILE_LIST))).. include $(TOP)/src/common.inc INCLUDE = $(TOP)/src/include RPMEM_COMMON = $(TOP)/src/rpmem_common vpath %.c $(RPMEM_COMMON) COMMON = $(TOP)/src/common vpath %.c $(COMMON) INCS += -I../include -I../common/ $(OS_INCS) # default CFLAGS DEFAULT_CFLAGS += -std=gnu99 DEFAULT_CFLAGS += -Wall DEFAULT_CFLAGS += -Werror DEFAULT_CFLAGS += -Wmissing-prototypes DEFAULT_CFLAGS += -Wpointer-arith DEFAULT_CFLAGS += -Wsign-conversion DEFAULT_CFLAGS += -Wsign-compare ifeq ($(WCONVERSION_AVAILABLE), y) DEFAULT_CFLAGS += -Wconversion endif ifeq ($(IS_ICC), n) DEFAULT_CFLAGS += -Wunused-macros DEFAULT_CFLAGS += -Wmissing-field-initializers endif ifeq ($(WUNREACHABLE_CODE_RETURN_AVAILABLE), y) DEFAULT_CFLAGS += -Wunreachable-code-return endif ifeq ($(WMISSING_VARIABLE_DECLARATIONS_AVAILABLE), y) DEFAULT_CFLAGS += -Wmissing-variable-declarations endif ifeq ($(WFLOAT_EQUAL_AVAILABLE), y) DEFAULT_CFLAGS += -Wfloat-equal endif ifeq ($(WSWITCH_DEFAULT_AVAILABLE), y) DEFAULT_CFLAGS += -Wswitch-default endif ifeq ($(WCAST_FUNCTION_TYPE_AVAILABLE), y) DEFAULT_CFLAGS += -Wcast-function-type endif ifeq ($(WSTRINGOP_TRUNCATION_AVAILABLE), y) DEFAULT_CFLAGS += -DSTRINGOP_TRUNCATION_SUPPORTED endif ifeq ($(DEBUG),1) # Undefine _FORTIFY_SOURCE in case it's set in system-default or # user-defined CFLAGS as it conflicts with -O0. DEBUG_CFLAGS += -Wp,-U_FORTIFY_SOURCE DEBUG_CFLAGS += -O0 -ggdb -DDEBUG LIB_SUBDIR = /pmdk_debug OBJDIR = debug else DEFAULT_CFLAGS += -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 LIB_SUBDIR = OBJDIR = nondebug endif # use defaults, if system or user-defined CFLAGS are not specified CFLAGS ?= $(DEFAULT_CFLAGS) CFLAGS += -std=gnu99 CFLAGS += -fno-common CFLAGS += -pthread CFLAGS += -DSRCVERSION=\"$(SRCVERSION)\" ifeq ($(OS_DIMM),ndctl) CFLAGS += -DSDS_ENABLED CFLAGS += $(OS_DIMM_CFLAG) endif ifeq ($(COVERAGE),1) CFLAGS += $(GCOV_CFLAGS) LDFLAGS += $(GCOV_LDFLAGS) LIBS += $(GCOV_LIBS) endif ifeq ($(VALGRIND),0) CFLAGS += -DVALGRIND_ENABLED=0 CXXFLAGS += -DVALGRIND_ENABLED=0 endif ifeq ($(FAULT_INJECTION),1) CFLAGS += -DFAULT_INJECTION=1 CXXFLAGS += -DFAULT_INJECTION=1 endif ifneq ($(SANITIZE),) CFLAGS += -fsanitize=$(tsanitize) LDFLAGS += -fsanitize=$(tsanitize) endif CFLAGS += $(EXTRA_CFLAGS) ifeq ($(DEBUG),1) CFLAGS += $(EXTRA_CFLAGS_DEBUG) $(DEBUG_CFLAGS) else CFLAGS += $(EXTRA_CFLAGS_RELEASE) endif LDFLAGS += -Wl,-z,relro -Wl,--fatal-warnings -Wl,--warn-common $(EXTRA_LDFLAGS) ifneq ($(NORPATH),1) LDFLAGS += -Wl,-rpath=$(libdir)$(LIB_SUBDIR):$(LIBNDCTL_LD_LIBRARY_PATHS) endif ifeq ($(LIBRT_NEEDED), y) LIBS += -lrt endif define arch32_error_msg ################################################## ### 32-bit builds of PMDK are not supported! ### ### Please, use 64-bit platform/compiler. ### ################################################## endef TESTCMD := $(CC) $(CFLAGS) -dM -E -x c /dev/null -o /dev/null TESTBUILD := $(shell $(TESTCMD) && echo 1 || echo 0) ifneq ($(TESTBUILD), 1) $(error "$(TESTCMD)" failed) endif ifeq ($(filter $(ARCH), x86_64 aarch64 ppc64),) $(error unsupported architecture: $(ARCH)) endif LP64 := $(shell $(CC) $(CFLAGS) -dM -E -x c /dev/null | grep -Ec "__SIZEOF_LONG__.+8|__SIZEOF_POINTER__.+8" ) ifneq ($(LP64), 2) $(error $(arch32_error_msg)) endif LIBS_DESTDIR = $(DESTDIR)$(libdir)$(LIB_SUBDIR) DIRNAME = $(shell basename $(CURDIR)) ifeq ($(OBJDIR),$(abspath $(OBJDIR))) objdir = $(OBJDIR)/$(DIRNAME) else objdir = ../$(OBJDIR)/$(DIRNAME) endif LIB_OUTDIR ?= $(objdir)/.. ifneq ($(LIB_OUTDIR),) LDFLAGS += -L$(LIB_OUTDIR) endif ifneq ($(SOURCE),) _OBJS = $(SOURCE:.c=.o) _OBJS_COMMON = $(patsubst $(COMMON)/%, %, $(_OBJS)) _OBJS_RPMEM_COMMON = $(patsubst $(RPMEM_COMMON)/%, %, $(_OBJS_COMMON)) OBJS += $(addprefix $(objdir)/, $(_OBJS_RPMEM_COMMON)) endif ifneq ($(HEADERS),) ifneq ($(filter 1 2, $(CSTYLEON)),) TMP_HEADERS := $(addsuffix tmp, $(HEADERS)) TMP_HEADERS := $(addprefix $(objdir)/, $(TMP_HEADERS)) endif endif ifneq ($(LIBRARY_NAME),) LIB_NAME = lib$(LIBRARY_NAME) endif ifneq ($(LIBRARY_SO_VERSION),) LIB_LINK = $(LIB_NAME).link LIB_SONAME = $(LIB_NAME).so.$(LIBRARY_SO_VERSION) LIB_SO = $(LIB_OUTDIR)/$(LIB_NAME).so LIB_SO_SONAME = $(LIB_SO).$(LIBRARY_SO_VERSION) ifneq ($(LIBRARY_VERSION),) LIB_SO_REAL = $(LIB_SO_SONAME).$(LIBRARY_VERSION) else $(error LIBRARY_VERSION not set) endif TARGET_LIBS = $(LIB_SO_REAL) TARGET_LINKS = $(LIB_SO_SONAME) $(LIB_SO) endif ifneq ($(LIB_NAME),) LIB_AR = $(LIB_OUTDIR)/$(LIB_NAME).a LIB_AR_UNSCOPED = $(objdir)/$(LIB_NAME)_unscoped.o LIB_AR_ALL = $(objdir)/$(LIB_NAME)_all.o TARGET_LIBS += $(LIB_AR) endif ifneq ($(EXTRA_TARGETS),) EXTRA_TARGETS_CLEAN = $(EXTRA_TARGETS:=-clean) EXTRA_TARGETS_CLOBBER = $(EXTRA_TARGETS:=-clobber) endif PMEMLOG_PRIV_OBJ=$(LIB_OUTDIR)/libpmemlog/libpmemlog_unscoped.o PMEMBLK_PRIV_OBJ=$(LIB_OUTDIR)/libpmemblk/libpmemblk_unscoped.o ifneq ($(LIBPMEMLOG_PRIV_FUNCS),) OBJS += pmemlog_priv_funcs.o endif ifneq ($(LIBPMEMBLK_PRIV_FUNCS),) OBJS += pmemblk_priv_funcs.o endif MAKEFILE_DEPS=../Makefile.inc Makefile $(TOP)/src/common.inc all: $(objdir) $(LIB_OUTDIR) $(EXTRA_TARGETS) $(LIB_AR) $(LIB_SO_SONAME) $(LIB_SO_REAL) $(LIB_SO) $(TMP_HEADERS) $(objdir) $(LIB_OUTDIR): $(MKDIR) -p $@ $(LIB_SO_REAL): $(OBJS) $(EXTRA_OBJS) $(LIB_LINK) $(MAKEFILE_DEPS) $(CC) $(LDFLAGS) -shared -Wl,--version-script=$(LIB_LINK),-soname,$(LIB_SONAME) -o $@ $(OBJS) $(EXTRA_OBJS) $(LIBS) $(LIB_SO_SONAME): $(LIB_SO_REAL) $(MAKEFILE_DEPS) $(LN) -sf $(shell basename $<) $@ $(LIB_SO): $(LIB_SO_SONAME) $(MAKEFILE_DEPS) $(LN) -sf $(shell basename $<) $@ $(LIB_AR_UNSCOPED): $(OBJS) $(EXTRA_OBJS) $(MAKEFILE_DEPS) $(LD) -o $@ -r $(OBJS) $(EXTRA_OBJS) ifeq ($(LIB_LINK),) $(LIB_AR_ALL): $(LIB_AR_UNSCOPED) $(MAKEFILE_DEPS) $(OBJCOPY) $< $@ else $(LIB_AR_ALL): $(LIB_AR_UNSCOPED) $(LIB_LINK) $(MAKEFILE_DEPS) $(OBJCOPY) --localize-hidden `sed -n 's/^ *\([a-zA-Z0-9_]*\);$$/-G \1/p' $(LIB_LINK)` $< $@ endif $(LIB_AR): $(LIB_AR_ALL) $(MAKEFILE_DEPS) $(AR) rv $@ $(LIB_AR_ALL) $(PMEMBLK_PRIV_OBJ): $(MAKE) -C $(LIBSDIR) libpmemblk install: all ifneq ($(LIBRARY_NAME),) $(INSTALL) -d $(LIBS_DESTDIR) $(INSTALL) -p -m 0755 $(TARGET_LIBS) $(LIBS_DESTDIR) $(CP) -d $(TARGET_LINKS) $(LIBS_DESTDIR) endif uninstall: ifneq ($(LIBRARY_NAME),) $(foreach f, $(TARGET_LIBS), $(RM) $(LIBS_DESTDIR)/$(notdir $(f))) $(foreach f, $(TARGET_LINKS), $(RM) $(LIBS_DESTDIR)/$(notdir $(f))) endif clean: $(EXTRA_TARGETS_CLEAN) ifneq ($(LIBRARY_NAME),) $(RM) $(OBJS) $(TMP_HEADERS) $(RM) $(LIB_AR_ALL) $(LIB_AR_UNSCOPED) endif clobber: clean $(EXTRA_TARGETS_CLOBBER) ifneq ($(LIBRARY_NAME),) $(RM) $(LIB_AR) $(LIB_SO_SONAME) $(LIB_SO_REAL) $(LIB_SO) $(RM) -r $(objdir)/.deps $(RM) -f *.link endif $(eval $(cstyle-rule)) $(objdir)/%.o: %.c $(MAKEFILE_DEPS) $(call check-cstyle, $<) @mkdir -p $(objdir)/.deps $(CC) -MD -c -o $@ $(CFLAGS) $(INCS) -fPIC $(call coverage-path, $<) $(call check-os, $@, $<) $(create-deps) sparse: $(if $(SOURCE), $(sparse-c)) $(objdir)/%.htmp: %.h $(call check-cstyle, $<, $@) .PHONY: all clean clobber install uninstall cstyle -include $(objdir)/.deps/*.P %.link: %.link.in ifeq ($(FAULT_INJECTION),1) @sed 's/fault_injection;/$(LIBRARY_NAME)_inject_fault_at;\n\t\t$(LIBRARY_NAME)_fault_injection_enabled;/g' $< > $@_temp else @sed '/fault_injection;/d' $< > $@_temp endif @mv $@_temp $@ pmdk-1.8/src/windows/0000775000000000000000000000000013615011243013272 5ustar rootrootpmdk-1.8/src/windows/libs_debug.props0000664000000000000000000000353713615011243016466 0ustar rootroot $(SolutionDir)$(Platform)\$(Configuration)\libs\ $(FrameworkSDKdir)bin\$(TargetPlatformVersion)\$(Platform);$(ExecutablePath) $(SolutionDir)\include;$(SolutionDir)\windows\include;$(SolutionDir)\common;$(SolutionDir)\$(TargetName) PMDK_UTF8_API;SDS_ENABLED;NTDDI_VERSION=NTDDI_WIN10_RS1;_CRT_SECURE_NO_WARNINGS;_WINDLL;_DEBUG;%(PreprocessorDefinitions) CompileAsC true platform.h Level3 true true false true shlwapi.lib;ntdll.lib;%(AdditionalDependencies) $(TargetName).def true true false _DEBUG $(SolutionDir)\common;$(SolutionDir)\windows\include pmdk-1.8/src/windows/README0000664000000000000000000000143213615011243014152 0ustar rootrootPersistent Memory Development Kit This is src/windows/README. This directory contains the Windows-specific source for the PMDK. The subdirectory "include" contains header files that have no equivalents on Windows OS, when building PMDK using VC++ compiler. Some of those files are empty, which is a cheap trick to avoid preprocessor errors when including non-existing files. This way we don't need a lot of preprocessor conditionals in all the source code files. The "platform.h" file contains definitions of all the basic types and macros that are not available under VC++. When building PMDK with Visual Studio, "platform.h" file is included to each source file using "/FI" (forced include) option. The subdirectory "getopt" contains a windows implementation of getopt and getopt_long pmdk-1.8/src/windows/getopt/0000775000000000000000000000000013615011243014574 5ustar rootrootpmdk-1.8/src/windows/getopt/getopt.c0000664000000000000000000002321513615011243016245 0ustar rootroot/* * *Copyright (c) 2012, Kim Gräsman * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * Neither the name of Kim Gräsman nor the * names of contributors may be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL KIM GRÄSMAN BE LIABLE FOR ANY * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "getopt.h" #include #include #include char* optarg; int optopt; /* The variable optind [...] shall be initialized to 1 by the system. */ int optind = 1; int opterr; static char* optcursor = NULL; static char *first = NULL; /* rotates argv array */ static void rotate(char **argv, int argc) { if (argc <= 1) return; char *tmp = argv[0]; memmove(argv, argv + 1, (argc - 1) * sizeof(char *)); argv[argc - 1] = tmp; } /* Implemented based on [1] and [2] for optional arguments. optopt is handled FreeBSD-style, per [3]. Other GNU and FreeBSD extensions are purely accidental. [1] https://pubs.opengroup.org/onlinepubs/000095399/functions/getopt.html [2] https://www.kernel.org/doc/man-pages/online/pages/man3/getopt.3.html [3] https://www.freebsd.org/cgi/man.cgi?query=getopt&sektion=3&manpath=FreeBSD+9.0-RELEASE */ int getopt(int argc, char* const argv[], const char* optstring) { int optchar = -1; const char* optdecl = NULL; optarg = NULL; opterr = 0; optopt = 0; /* Unspecified, but we need it to avoid overrunning the argv bounds. */ if (optind >= argc) goto no_more_optchars; /* If, when getopt() is called argv[optind] is a null pointer, getopt() shall return -1 without changing optind. */ if (argv[optind] == NULL) goto no_more_optchars; /* If, when getopt() is called *argv[optind] is not the character '-', permute argv to move non options to the end */ if (*argv[optind] != '-') { if (argc - optind <= 1) goto no_more_optchars; if (!first) first = argv[optind]; do { rotate((char **)(argv + optind), argc - optind); } while (*argv[optind] != '-' && argv[optind] != first); if (argv[optind] == first) goto no_more_optchars; } /* If, when getopt() is called argv[optind] points to the string "-", getopt() shall return -1 without changing optind. */ if (strcmp(argv[optind], "-") == 0) goto no_more_optchars; /* If, when getopt() is called argv[optind] points to the string "--", getopt() shall return -1 after incrementing optind. */ if (strcmp(argv[optind], "--") == 0) { ++optind; if (first) { do { rotate((char **)(argv + optind), argc - optind); } while (argv[optind] != first); } goto no_more_optchars; } if (optcursor == NULL || *optcursor == '\0') optcursor = argv[optind] + 1; optchar = *optcursor; /* FreeBSD: The variable optopt saves the last known option character returned by getopt(). */ optopt = optchar; /* The getopt() function shall return the next option character (if one is found) from argv that matches a character in optstring, if there is one that matches. */ optdecl = strchr(optstring, optchar); if (optdecl) { /* [I]f a character is followed by a colon, the option takes an argument. */ if (optdecl[1] == ':') { optarg = ++optcursor; if (*optarg == '\0') { /* GNU extension: Two colons mean an option takes an optional arg; if there is text in the current argv-element (i.e., in the same word as the option name itself, for example, "-oarg"), then it is returned in optarg, otherwise optarg is set to zero. */ if (optdecl[2] != ':') { /* If the option was the last character in the string pointed to by an element of argv, then optarg shall contain the next element of argv, and optind shall be incremented by 2. If the resulting value of optind is greater than argc, this indicates a missing option-argument, and getopt() shall return an error indication. Otherwise, optarg shall point to the string following the option character in that element of argv, and optind shall be incremented by 1. */ if (++optind < argc) { optarg = argv[optind]; } else { /* If it detects a missing option-argument, it shall return the colon character ( ':' ) if the first character of optstring was a colon, or a question-mark character ( '?' ) otherwise. */ optarg = NULL; fprintf(stderr, "%s: option requires an argument -- '%c'\n", argv[0], optchar); optchar = (optstring[0] == ':') ? ':' : '?'; } } else { optarg = NULL; } } optcursor = NULL; } } else { fprintf(stderr,"%s: invalid option -- '%c'\n", argv[0], optchar); /* If getopt() encounters an option character that is not contained in optstring, it shall return the question-mark ( '?' ) character. */ optchar = '?'; } if (optcursor == NULL || *++optcursor == '\0') ++optind; return optchar; no_more_optchars: optcursor = NULL; first = NULL; return -1; } /* Implementation based on [1]. [1] https://www.kernel.org/doc/man-pages/online/pages/man3/getopt.3.html */ int getopt_long(int argc, char* const argv[], const char* optstring, const struct option* longopts, int* longindex) { const struct option* o = longopts; const struct option* match = NULL; int num_matches = 0; size_t argument_name_length = 0; const char* current_argument = NULL; int retval = -1; optarg = NULL; optopt = 0; if (optind >= argc) return -1; /* If, when getopt() is called argv[optind] is a null pointer, getopt_long() shall return -1 without changing optind. */ if (argv[optind] == NULL) goto no_more_optchars; /* If, when getopt_long() is called *argv[optind] is not the character '-', permute argv to move non options to the end */ if (*argv[optind] != '-') { if (argc - optind <= 1) goto no_more_optchars; if (!first) first = argv[optind]; do { rotate((char **)(argv + optind), argc - optind); } while (*argv[optind] != '-' && argv[optind] != first); if (argv[optind] == first) goto no_more_optchars; } if (strlen(argv[optind]) < 3 || strncmp(argv[optind], "--", 2) != 0) return getopt(argc, argv, optstring); /* It's an option; starts with -- and is longer than two chars. */ current_argument = argv[optind] + 2; argument_name_length = strcspn(current_argument, "="); for (; o->name; ++o) { if (strncmp(o->name, current_argument, argument_name_length) == 0) { match = o; ++num_matches; if (strlen(o->name) == argument_name_length) { /* found match is exactly the one which we are looking for */ num_matches = 1; break; } } } if (num_matches == 1) { /* If longindex is not NULL, it points to a variable which is set to the index of the long option relative to longopts. */ if (longindex) *longindex = (int)(match - longopts); /* If flag is NULL, then getopt_long() shall return val. Otherwise, getopt_long() returns 0, and flag shall point to a variable which shall be set to val if the option is found, but left unchanged if the option is not found. */ if (match->flag) *(match->flag) = match->val; retval = match->flag ? 0 : match->val; if (match->has_arg != no_argument) { optarg = strchr(argv[optind], '='); if (optarg != NULL) ++optarg; if (match->has_arg == required_argument) { /* Only scan the next argv for required arguments. Behavior is not specified, but has been observed with Ubuntu and Mac OSX. */ if (optarg == NULL && ++optind < argc) { optarg = argv[optind]; } if (optarg == NULL) retval = ':'; } } else if (strchr(argv[optind], '=')) { /* An argument was provided to a non-argument option. I haven't seen this specified explicitly, but both GNU and BSD-based implementations show this behavior. */ retval = '?'; } } else { /* Unknown option or ambiguous match. */ retval = '?'; if (num_matches == 0) { fprintf(stderr, "%s: unrecognized option -- '%s'\n", argv[0], argv[optind]); } else { fprintf(stderr, "%s: option '%s' is ambiguous\n", argv[0], argv[optind]); } } ++optind; return retval; no_more_optchars: first = NULL; return -1; } pmdk-1.8/src/windows/getopt/getopt.h0000664000000000000000000000413413615011243016251 0ustar rootroot/* * *Copyright (c) 2012, Kim Gräsman * All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are met: * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * * Neither the name of Kim Gräsman nor the * names of contributors may be used to endorse or promote products * derived from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL KIM GRÄSMAN BE LIABLE FOR ANY * DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF * THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef INCLUDED_GETOPT_PORT_H #define INCLUDED_GETOPT_PORT_H #if defined(__cplusplus) extern "C" { #endif #define no_argument 0 #define required_argument 1 #define optional_argument 2 extern char* optarg; extern int optind, opterr, optopt; struct option { const char* name; int has_arg; int* flag; int val; }; int getopt(int argc, char* const argv[], const char* optstring); int getopt_long(int argc, char* const argv[], const char* optstring, const struct option* longopts, int* longindex); #if defined(__cplusplus) } #endif #endif // INCLUDED_GETOPT_PORT_H pmdk-1.8/src/windows/getopt/README0000664000000000000000000000040213615011243015450 0ustar rootrootPersistent Memory Development Kit This is src/windows/getopt/README. This is directory contains windows getopt implementation downloaded from: https://github.com/kimgr/getopt_port with changes applied to compile it with "compile as c code(/TC)" option. pmdk-1.8/src/windows/getopt/getopt.vcxproj.filters0000664000000000000000000000144013615011243021161 0ustar rootroot {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {93995380-89BD-4b04-88EB-625FBE52EBFB} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files Header Files pmdk-1.8/src/windows/getopt/LICENSE.txt0000664000000000000000000000271313615011243016422 0ustar rootrootCopyright (c) 2012, Kim Gräsman All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of Kim Gräsman nor the names of contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL KIM GRÄSMAN BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. pmdk-1.8/src/windows/getopt/getopt.vcxproj0000664000000000000000000000762313615011243017523 0ustar rootroot Debug x64 Release x64 {9186EAC4-2F34-4F17-B940-6585D7869BCD} getopt 10.0.16299.0 StaticLibrary true v140 NotSet StaticLibrary false v140 NotSet Level3 Disabled CompileAsC true NTDDI_VERSION=NTDDI_WIN10_RS1;_DEBUG;_CRT_SECURE_NO_WARNINGS;_MBCS;%(PreprocessorDefinitions) 4819 true Level3 MaxSpeed true true CompileAsC true NTDDI_VERSION=NTDDI_WIN10_RS1;NDEBUG;_CRT_SECURE_NO_WARNINGS;_MBCS;%(PreprocessorDefinitions) true true true pmdk-1.8/src/windows/getopt/.cstyleignore0000664000000000000000000000002213615011243017276 0ustar rootrootgetopt.c getopt.h pmdk-1.8/src/windows/include/0000775000000000000000000000000013615011243014715 5ustar rootrootpmdk-1.8/src/windows/include/libgen.h0000664000000000000000000000312513615011243016327 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake libgen.h */ pmdk-1.8/src/windows/include/features.h0000664000000000000000000000312713615011243016707 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake features.h */ pmdk-1.8/src/windows/include/dirent.h0000664000000000000000000000313213615011243016352 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake dirent.h */ pmdk-1.8/src/windows/include/unistd.h0000664000000000000000000000757213615011243016407 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * unistd.h -- compatibility layer for POSIX operating system API */ #ifndef UNISTD_H #define UNISTD_H 1 #include #define _SC_PAGESIZE 0 #define _SC_NPROCESSORS_ONLN 1 #define R_OK 04 #define W_OK 02 #define X_OK 00 /* execute permission doesn't exist on Windows */ #define F_OK 00 /* * sysconf -- get configuration information at run time */ static __inline long sysconf(int p) { SYSTEM_INFO si; int ret = 0; switch (p) { case _SC_PAGESIZE: GetSystemInfo(&si); return si.dwPageSize; case _SC_NPROCESSORS_ONLN: for (int i = 0; i < GetActiveProcessorGroupCount(); i++) { ret += GetActiveProcessorCount(i); } return ret; default: return 0; } } #define getpid _getpid /* * pread -- read from a file descriptor at given offset */ static ssize_t pread(int fd, void *buf, size_t count, os_off_t offset) { __int64 position = _lseeki64(fd, 0, SEEK_CUR); _lseeki64(fd, offset, SEEK_SET); int ret = _read(fd, buf, (unsigned)count); _lseeki64(fd, position, SEEK_SET); return ret; } /* * pwrite -- write to a file descriptor at given offset */ static ssize_t pwrite(int fd, const void *buf, size_t count, os_off_t offset) { __int64 position = _lseeki64(fd, 0, SEEK_CUR); _lseeki64(fd, offset, SEEK_SET); int ret = _write(fd, buf, (unsigned)count); _lseeki64(fd, position, SEEK_SET); return ret; } #define S_ISBLK(x) 0 /* BLK devices not exist on Windows */ /* * basename -- parse pathname and return filename component */ static char * basename(char *path) { char fname[_MAX_FNAME]; char ext[_MAX_EXT]; _splitpath(path, NULL, NULL, fname, ext); sprintf(path, "%s%s", fname, ext); return path; } /* * dirname -- parse pathname and return directory component */ static char * dirname(char *path) { if (path == NULL) return "."; size_t len = strlen(path); if (len == 0) return "."; char *end = path + len; /* strip trailing forslashes and backslashes */ while ((--end) > path) { if (*end != '\\' && *end != '/') { *(end + 1) = '\0'; break; } } /* strip basename */ while ((--end) > path) { if (*end == '\\' || *end == '/') { *end = '\0'; break; } } if (end != path) { return path; /* handle edge cases */ } else if (*end == '\\' || *end == '/') { *(end + 1) = '\0'; } else { *end++ = '.'; *end = '\0'; } return path; } #endif /* UNISTD_H */ pmdk-1.8/src/windows/include/sched.h0000664000000000000000000000312413615011243016154 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake sched.h */ pmdk-1.8/src/windows/include/sys/0000775000000000000000000000000013615011243015533 5ustar rootrootpmdk-1.8/src/windows/include/sys/param.h0000664000000000000000000000411713615011243017007 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * sys/param.h -- a few useful macros */ #ifndef SYS_PARAM_H #define SYS_PARAM_H 1 #define roundup(x, y) ((((x) + ((y) - 1)) / (y)) * (y)) #define howmany(x, y) (((x) + ((y) - 1)) / (y)) #define BPB 8 /* bits per byte */ #define setbit(b, i) ((b)[(i) / BPB] |= 1 << ((i) % BPB)) #define isset(b, i) ((b)[(i) / BPB] & (1 << ((i) % BPB))) #define isclr(b, i) (((b)[(i) / BPB] & (1 << ((i) % BPB))) == 0) #define MIN(a, b) (((a) < (b)) ? (a) : (b)) #define MAX(a, b) (((a) > (b)) ? (a) : (b)) #endif /* SYS_PARAM_H */ pmdk-1.8/src/windows/include/sys/statvfs.h0000664000000000000000000000312613615011243017400 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake statvfs.h */ pmdk-1.8/src/windows/include/sys/resource.h0000664000000000000000000000313313615011243017533 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake sys/resource.h */ pmdk-1.8/src/windows/include/sys/uio.h0000664000000000000000000000352213615011243016502 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * sys/uio.h -- definition of iovec structure */ #ifndef SYS_UIO_H #define SYS_UIO_H 1 #include #ifdef __cplusplus extern "C" { #endif ssize_t writev(int fd, const struct iovec *iov, int iovcnt); #ifdef __cplusplus } #endif #endif /* SYS_UIO_H */ pmdk-1.8/src/windows/include/sys/file.h0000664000000000000000000000325213615011243016625 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * sys/file.h -- file locking */ pmdk-1.8/src/windows/include/sys/mount.h0000664000000000000000000000313513615011243017050 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake sys/mount.h */ pmdk-1.8/src/windows/include/sys/mman.h0000664000000000000000000000446513615011243016645 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * sys/mman.h -- memory-mapped files for Windows */ #ifndef SYS_MMAN_H #define SYS_MMAN_H 1 #ifdef __cplusplus extern "C" { #endif #define PROT_NONE 0x0 #define PROT_READ 0x1 #define PROT_WRITE 0x2 #define PROT_EXEC 0x4 #define MAP_SHARED 0x1 #define MAP_PRIVATE 0x2 #define MAP_FIXED 0x10 #define MAP_ANONYMOUS 0x20 #define MAP_ANON MAP_ANONYMOUS #define MAP_NORESERVE 0x04000 #define MS_ASYNC 1 #define MS_SYNC 4 #define MS_INVALIDATE 2 #define MAP_FAILED ((void *)(-1)) void *mmap(void *addr, size_t len, int prot, int flags, int fd, os_off_t offset); int munmap(void *addr, size_t len); int msync(void *addr, size_t len, int flags); int mprotect(void *addr, size_t len, int prot); #ifdef __cplusplus } #endif #endif /* SYS_MMAN_H */ pmdk-1.8/src/windows/include/sys/wait.h0000664000000000000000000000313413615011243016651 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake sys/wait.h */ pmdk-1.8/src/windows/include/win_mmap.h0000664000000000000000000000541313615011243016700 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * win_mmap.h -- (internal) tracks the regions mapped by mmap */ #ifndef WIN_MMAP_H #define WIN_MMAP_H 1 #include "queue.h" #define roundup(x, y) ((((x) + ((y) - 1)) / (y)) * (y)) #define rounddown(x, y) (((x) / (y)) * (y)) void win_mmap_init(void); void win_mmap_fini(void); /* allocation/mmap granularity */ extern unsigned long long Mmap_align; typedef enum FILE_MAPPING_TRACKER_FLAGS { FILE_MAPPING_TRACKER_FLAG_DIRECT_MAPPED = 0x0001, /* * This should hold the value of all flags ORed for debug purpose. */ FILE_MAPPING_TRACKER_FLAGS_MASK = FILE_MAPPING_TRACKER_FLAG_DIRECT_MAPPED } FILE_MAPPING_TRACKER_FLAGS; /* * this structure tracks the file mappings outstanding per file handle */ typedef struct FILE_MAPPING_TRACKER { PMDK_SORTEDQ_ENTRY(FILE_MAPPING_TRACKER) ListEntry; HANDLE FileHandle; HANDLE FileMappingHandle; void *BaseAddress; void *EndAddress; DWORD Access; os_off_t Offset; size_t FileLen; FILE_MAPPING_TRACKER_FLAGS Flags; } FILE_MAPPING_TRACKER, *PFILE_MAPPING_TRACKER; extern SRWLOCK FileMappingQLock; extern PMDK_SORTEDQ_HEAD(FMLHead, FILE_MAPPING_TRACKER) FileMappingQHead; #endif /* WIN_MMAP_H */ pmdk-1.8/src/windows/include/err.h0000664000000000000000000000421613615011243015661 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * err.h - error and warning messages */ #ifndef ERR_H #define ERR_H 1 #include #include #include /* * err - windows implementation of unix err function */ __declspec(noreturn) static void err(int eval, const char *fmt, ...) { va_list vl; va_start(vl, fmt); vfprintf(stderr, fmt, vl); va_end(vl); exit(eval); } /* * warn - windows implementation of unix warn function */ static void warn(const char *fmt, ...) { va_list vl; va_start(vl, fmt); fprintf(stderr, "Warning: "); vfprintf(stderr, fmt, vl); va_end(vl); } #endif /* ERR_H */ pmdk-1.8/src/windows/include/endian.h0000664000000000000000000000424313615011243016327 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * endian.h -- convert values between host and big-/little-endian byte order */ #ifndef ENDIAN_H #define ENDIAN_H 1 /* * XXX: On Windows we can assume little-endian architecture */ #include #define htole16(a) (a) #define htole32(a) (a) #define htole64(a) (a) #define le16toh(a) (a) #define le32toh(a) (a) #define le64toh(a) (a) #define htobe16(x) _byteswap_ushort(x) #define htobe32(x) _byteswap_ulong(x) #define htobe64(x) _byteswap_uint64(x) #define be16toh(x) _byteswap_ushort(x) #define be32toh(x) _byteswap_ulong(x) #define be64toh(x) _byteswap_uint64(x) #endif /* ENDIAN_H */ pmdk-1.8/src/windows/include/platform.h0000664000000000000000000001241313615011243016713 0ustar rootroot/* * Copyright 2015-2020, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * platform.h -- dirty hacks to compile Linux code on Windows using VC++ * * This is included to each source file using "/FI" (forced include) option. * * XXX - it is a subject for refactoring */ #ifndef PLATFORM_H #define PLATFORM_H 1 #pragma warning(disable : 4996) #pragma warning(disable : 4200) /* allow flexible array member */ #pragma warning(disable : 4819) /* non unicode characters */ #ifdef __cplusplus extern "C" { #endif /* Prevent PMDK compilation for 32-bit platforms */ #if defined(_WIN32) && !defined(_WIN64) #error "32-bit builds of PMDK are not supported!" #endif #define _CRT_RAND_S /* rand_s() */ #include #include #include #include #include #include #include #include #include #include #include /* use uuid_t definition from util.h */ #ifdef uuid_t #undef uuid_t #endif /* a few trivial substitutions */ #define PATH_MAX MAX_PATH #define __thread __declspec(thread) #define __func__ __FUNCTION__ #ifdef _DEBUG #define DEBUG #endif /* * The inline keyword is available only in VC++. * https://msdn.microsoft.com/en-us/library/bw1hbe6y.aspx */ #ifndef __cplusplus #define inline __inline #endif /* XXX - no equivalents in VC++ */ #define __attribute__(a) #define __builtin_constant_p(cnd) 0 /* * missing definitions */ /* errno.h */ #define ELIBACC 79 /* cannot access a needed shared library */ /* sys/stat.h */ #define S_IRUSR S_IREAD #define S_IWUSR S_IWRITE #define S_IRGRP S_IRUSR #define S_IWGRP S_IWUSR #define O_SYNC 0 typedef int mode_t; #define fchmod(fd, mode) 0 /* XXX - dummy */ #define setlinebuf(fp) setvbuf(fp, NULL, _IOLBF, BUFSIZ); /* unistd.h */ typedef long long os_off_t; typedef long long ssize_t; int setenv(const char *name, const char *value, int overwrite); int unsetenv(const char *name); /* fcntl.h */ int posix_fallocate(int fd, os_off_t offset, os_off_t len); /* string.h */ #define strtok_r strtok_s /* time.h */ #define CLOCK_MONOTONIC 1 #define CLOCK_REALTIME 2 int clock_gettime(int id, struct timespec *ts); /* signal.h */ typedef unsigned long long sigset_t; /* one bit for each signal */ C_ASSERT(NSIG <= sizeof(sigset_t) * 8); struct sigaction { void (*sa_handler) (int signum); /* void (*sa_sigaction)(int, siginfo_t *, void *); */ sigset_t sa_mask; int sa_flags; void (*sa_restorer) (void); }; __inline int sigemptyset(sigset_t *set) { *set = 0; return 0; } __inline int sigfillset(sigset_t *set) { *set = ~0; return 0; } __inline int sigaddset(sigset_t *set, int signum) { if (signum <= 0 || signum >= NSIG) { errno = EINVAL; return -1; } *set |= (1ULL << (signum - 1)); return 0; } __inline int sigdelset(sigset_t *set, int signum) { if (signum <= 0 || signum >= NSIG) { errno = EINVAL; return -1; } *set &= ~(1ULL << (signum - 1)); return 0; } __inline int sigismember(const sigset_t *set, int signum) { if (signum <= 0 || signum >= NSIG) { errno = EINVAL; return -1; } return ((*set & (1ULL << (signum - 1))) ? 1 : 0); } /* sched.h */ /* * sched_yield -- yield the processor */ __inline int sched_yield(void) { SwitchToThread(); return 0; /* always succeeds */ } /* * helper macros for library ctor/dtor function declarations */ #define MSVC_CONSTR(func) \ void func(void); \ __pragma(comment(linker, "/include:_" #func)) \ __pragma(section(".CRT$XCU", read)) \ __declspec(allocate(".CRT$XCU")) \ const void (WINAPI *_##func)(void) = (const void (WINAPI *)(void))func; #define MSVC_DESTR(func) \ void func(void); \ static void _##func##_reg(void) { atexit(func); }; \ MSVC_CONSTR(_##func##_reg) #ifdef __cplusplus } #endif #endif /* PLATFORM_H */ pmdk-1.8/src/windows/include/linux/0000775000000000000000000000000013615011243016054 5ustar rootrootpmdk-1.8/src/windows/include/linux/limits.h0000664000000000000000000000370213615011243017530 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * linux/limits.h -- fake header file */ /* * XXX - The only purpose of this empty file is to avoid preprocessor * errors when including a Linux-specific header file that has no equivalent * on Windows. With this cheap trick, we don't need a lot of preprocessor * conditionals in all the source code files. * * In the future, this will be addressed in some other way. */ pmdk-1.8/src/windows/include/strings.h0000664000000000000000000000313313615011243016557 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fake strings.h */ pmdk-1.8/src/windows/include/.cstyleignore0000664000000000000000000000001513615011243017421 0ustar rootrootsrcversion.h pmdk-1.8/src/windows/libs_release.props0000664000000000000000000000363513615011243017017 0ustar rootroot $(SolutionDir)$(Platform)\$(Configuration)\libs\ $(FrameworkSDKdir)bin\$(TargetPlatformVersion)\$(Platform);$(ExecutablePath) $(SolutionDir)\include;$(SolutionDir)\windows\include;$(SolutionDir)\common;$(SolutionDir)\$(TargetName) PMDK_UTF8_API;SDS_ENABLED;NTDDI_VERSION=NTDDI_WIN10_RS1;_CRT_SECURE_NO_WARNINGS;_WINDLL;NDEBUG;%(PreprocessorDefinitions) CompileAsC true platform.h Level3 true true false Neither true shlwapi.lib;ntdll.lib;%(AdditionalDependencies) $(TargetName).def DebugFastLink false false $(SolutionDir)\common;$(SolutionDir)\windows\include pmdk-1.8/src/windows/win_mmap.c0000664000000000000000000006747713615011243015272 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * Copyright (c) 2015-2017, Microsoft Corporation. All rights reserved. * Copyright (c) 2016, Hewlett Packard Enterprise Development LP * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * win_mmap.c -- memory-mapped files for Windows */ /* * XXX - The initial approach to PMDK for Windows port was to minimize the * amount of changes required in the core part of the library, and to avoid * preprocessor conditionals, if possible. For that reason, some of the * Linux system calls that have no equivalents on Windows have been emulated * using Windows API. * Note that it was not a goal to fully emulate POSIX-compliant behavior * of mentioned functions. They are used only internally, so current * implementation is just good enough to satisfy PMDK needs and to make it * work on Windows. * * This is a subject for change in the future. Likely, all these functions * will be replaced with "util_xxx" wrappers with OS-specific implementation * for Linux and Windows. * * Known issues: * - on Windows, mapping granularity/alignment is 64KB, not 4KB; * - mprotect() behavior and protection flag handling in mmap() is slightly * different than on Linux (see comments below). */ #include #include "mmap.h" #include "util.h" #include "out.h" #include "win_mmap.h" /* uncomment for more debug information on mmap trackers */ /* #define MMAP_DEBUG_INFO */ NTSTATUS NtFreeVirtualMemory(_In_ HANDLE ProcessHandle, _Inout_ PVOID *BaseAddress, _Inout_ PSIZE_T RegionSize, _In_ ULONG FreeType); /* * XXX Unify the Linux and Windows code and replace this structure with * the map tracking list defined in mmap.h. */ SRWLOCK FileMappingQLock = SRWLOCK_INIT; struct FMLHead FileMappingQHead = PMDK_SORTEDQ_HEAD_INITIALIZER(FileMappingQHead); /* * mmap_file_mapping_comparer -- (internal) compares the two file mapping * trackers */ static LONG_PTR mmap_file_mapping_comparer(PFILE_MAPPING_TRACKER a, PFILE_MAPPING_TRACKER b) { return ((LONG_PTR)a->BaseAddress - (LONG_PTR)b->BaseAddress); } #ifdef MMAP_DEBUG_INFO /* * mmap_info -- (internal) dump info about all the mapping trackers */ static void mmap_info(void) { LOG(4, NULL); AcquireSRWLockShared(&FileMappingQLock); PFILE_MAPPING_TRACKER mt; for (mt = PMDK_SORTEDQ_FIRST(&FileMappingQHead); mt != (void *)&FileMappingQHead; mt = PMDK_SORTEDQ_NEXT(mt, ListEntry)) { LOG(4, "FH %08x FMH %08x AD %p-%p (%zu) " "OF %08x FL %zu AC %d F %d", mt->FileHandle, mt->FileMappingHandle, mt->BaseAddress, mt->EndAddress, (char *)mt->EndAddress - (char *)mt->BaseAddress, mt->Offset, mt->FileLen, mt->Access, mt->Flags); } ReleaseSRWLockShared(&FileMappingQLock); } #endif /* * mmap_reserve -- (internal) reserve virtual address range */ static void * mmap_reserve(void *addr, size_t len) { LOG(4, "addr %p len %zu", addr, len); ASSERTeq((uintptr_t)addr % Mmap_align, 0); ASSERTeq(len % Mmap_align, 0); void *reserved_addr = VirtualAlloc(addr, len, MEM_RESERVE, PAGE_NOACCESS); if (reserved_addr == NULL) { ERR("cannot find a contiguous region - " "addr: %p, len: %lx, gle: 0x%08x", addr, len, GetLastError()); errno = ENOMEM; return MAP_FAILED; } return reserved_addr; } /* * mmap_unreserve -- (internal) frees the range that's previously reserved */ static int mmap_unreserve(void *addr, size_t len) { LOG(4, "addr %p len %zu", addr, len); ASSERTeq((uintptr_t)addr % Mmap_align, 0); ASSERTeq(len % Mmap_align, 0); size_t bytes_returned; MEMORY_BASIC_INFORMATION basic_info; bytes_returned = VirtualQuery(addr, &basic_info, sizeof(basic_info)); if (bytes_returned != sizeof(basic_info)) { ERR("cannot query the virtual address properties of the range " "- addr: %p, len: %d", addr, len); errno = EINVAL; return -1; } if (basic_info.State == MEM_RESERVE) { DWORD nt_status; void *release_addr = addr; size_t release_size = len; nt_status = NtFreeVirtualMemory(GetCurrentProcess(), &release_addr, &release_size, MEM_RELEASE); if (nt_status != 0) { ERR("cannot release the reserved virtual space - " "addr: %p, len: %d, nt_status: 0x%08x", addr, len, nt_status); errno = EINVAL; return -1; } ASSERTeq(release_addr, addr); ASSERTeq(release_size, len); LOG(4, "freed reservation - addr: %p, size: %d", release_addr, release_size); } else { LOG(4, "range not reserved - addr: %p, size: %d", addr, len); } return 0; } /* * win_mmap_init -- initialization of file mapping tracker */ void win_mmap_init(void) { AcquireSRWLockExclusive(&FileMappingQLock); PMDK_SORTEDQ_INIT(&FileMappingQHead); ReleaseSRWLockExclusive(&FileMappingQLock); } /* * win_mmap_fini -- file mapping tracker cleanup routine */ void win_mmap_fini(void) { /* * Let's make sure that no one is in the middle of updating the * list by grabbing the lock. */ AcquireSRWLockExclusive(&FileMappingQLock); while (!PMDK_SORTEDQ_EMPTY(&FileMappingQHead)) { PFILE_MAPPING_TRACKER mt; mt = (PFILE_MAPPING_TRACKER)PMDK_SORTEDQ_FIRST( &FileMappingQHead); PMDK_SORTEDQ_REMOVE(&FileMappingQHead, mt, ListEntry); if (mt->BaseAddress != NULL) UnmapViewOfFile(mt->BaseAddress); size_t release_size = (char *)mt->EndAddress - (char *)mt->BaseAddress; /* * Free reservation after file mapping (if reservation was * bigger than length of mapped file) */ void *release_addr = (char *)mt->BaseAddress + mt->FileLen; mmap_unreserve(release_addr, release_size - mt->FileLen); if (mt->FileMappingHandle != NULL) CloseHandle(mt->FileMappingHandle); if (mt->FileHandle != NULL) CloseHandle(mt->FileHandle); free(mt); } ReleaseSRWLockExclusive(&FileMappingQLock); } #define PROT_ALL (PROT_READ|PROT_WRITE|PROT_EXEC) /* * mmap -- map file into memory * * XXX - If read-only mapping was created initially, it is not possible * to change protection to R/W, even if the file itself was open in R/W mode. * To workaround that, we could modify mmap() to create R/W mapping first, * then change the protection to R/O. This way, it should be possible * to elevate permissions later. */ void * mmap(void *addr, size_t len, int prot, int flags, int fd, os_off_t offset) { LOG(4, "addr %p len %zu prot %d flags %d fd %d offset %ju", addr, len, prot, flags, fd, offset); if (len == 0) { ERR("invalid length: %zu", len); errno = EINVAL; return MAP_FAILED; } if ((prot & ~PROT_ALL) != 0) { ERR("invalid flags: 0x%08x", flags); /* invalid protection flags */ errno = EINVAL; return MAP_FAILED; } if (((flags & MAP_PRIVATE) && (flags & MAP_SHARED)) || ((flags & (MAP_PRIVATE | MAP_SHARED)) == 0)) { ERR("neither MAP_PRIVATE or MAP_SHARED is set, or both: 0x%08x", flags); errno = EINVAL; return MAP_FAILED; } /* XXX shall we use SEC_LARGE_PAGES flag? */ DWORD protect = 0; DWORD access = 0; /* on x86, PROT_WRITE implies PROT_READ */ if (prot & PROT_WRITE) { if (flags & MAP_PRIVATE) { access = FILE_MAP_COPY; if (prot & PROT_EXEC) protect = PAGE_EXECUTE_WRITECOPY; else protect = PAGE_WRITECOPY; } else { /* FILE_MAP_ALL_ACCESS == FILE_MAP_WRITE */ access = FILE_MAP_ALL_ACCESS; if (prot & PROT_EXEC) protect = PAGE_EXECUTE_READWRITE; else protect = PAGE_READWRITE; } } else if (prot & PROT_READ) { access = FILE_MAP_READ; if (prot & PROT_EXEC) protect = PAGE_EXECUTE_READ; else protect = PAGE_READONLY; } else { /* XXX - PAGE_NOACCESS is not supported by CreateFileMapping */ ERR("PAGE_NOACCESS is not supported"); errno = ENOTSUP; return MAP_FAILED; } if (((uintptr_t)addr % Mmap_align) != 0) { if ((flags & MAP_FIXED) == 0) { /* ignore invalid hint if no MAP_FIXED flag is set */ addr = NULL; } else { ERR("hint address is not well-aligned: %p", addr); errno = EINVAL; return MAP_FAILED; } } if ((offset % Mmap_align) != 0) { ERR("offset is not well-aligned: %ju", offset); errno = EINVAL; return MAP_FAILED; } if ((flags & MAP_FIXED) != 0) { /* * Free any reservations that the caller might have, also we * have to unmap any existing mappings in this region as per * mmap's manual. * XXX - Ideally we should unmap only if the prot and flags * are similar, we are deferring it as we don't rely on it * yet. */ int ret = munmap(addr, len); if (ret != 0) { ERR("!munmap: addr %p len %zu", addr, len); return MAP_FAILED; } } size_t len_align = roundup(len, Mmap_align); size_t filelen; size_t filelen_align; HANDLE fh; if (flags & MAP_ANON) { /* * In our implementation we are choosing to ignore fd when * MAP_ANON is set, instead of failing. */ fh = INVALID_HANDLE_VALUE; /* ignore/override offset */ offset = 0; filelen = len; filelen_align = len_align; if ((flags & MAP_NORESERVE) != 0) { /* * For anonymous mappings the meaning of MAP_NORESERVE * flag is pretty much the same as SEC_RESERVE. */ protect |= SEC_RESERVE; } } else { LARGE_INTEGER filesize; if (fd == -1) { ERR("invalid file descriptor: %d", fd); errno = EBADF; return MAP_FAILED; } /* * We need to keep file handle open for proper * implementation of msync() and to hold the file lock. */ if (!DuplicateHandle(GetCurrentProcess(), (HANDLE)_get_osfhandle(fd), GetCurrentProcess(), &fh, 0, FALSE, DUPLICATE_SAME_ACCESS)) { ERR("cannot duplicate handle - fd: %d, gle: 0x%08x", fd, GetLastError()); errno = ENOMEM; return MAP_FAILED; } /* * If we are asked to map more than the file size, map till the * file size and reserve the following. */ if (!GetFileSizeEx(fh, &filesize)) { ERR("cannot query the file size - fh: %d, gle: 0x%08x", fd, GetLastError()); CloseHandle(fh); return MAP_FAILED; } if (offset >= (os_off_t)filesize.QuadPart) { errno = EINVAL; ERR("offset is beyond the file size"); CloseHandle(fh); return MAP_FAILED; } /* calculate length of the mapped portion of the file */ filelen = filesize.QuadPart - offset; if (filelen > len) filelen = len; filelen_align = roundup(filelen, Mmap_align); if ((offset + len) > (size_t)filesize.QuadPart) { /* * Reserve virtual address for the rest of range we need * to map, and free a portion in the beginning for this * allocation. */ void *reserved_addr = mmap_reserve(addr, len_align); if (reserved_addr == MAP_FAILED) { ERR("cannot reserve region"); CloseHandle(fh); return MAP_FAILED; } if (addr != reserved_addr && (flags & MAP_FIXED) != 0) { ERR("cannot find a contiguous region - " "addr: %p, len: %lx, gle: 0x%08x", addr, len, GetLastError()); if (mmap_unreserve(reserved_addr, len_align) != 0) { ASSERT(FALSE); ERR("cannot free reserved region"); } errno = ENOMEM; CloseHandle(fh); return MAP_FAILED; } addr = reserved_addr; if (mmap_unreserve(reserved_addr, filelen_align) != 0) { ASSERT(FALSE); ERR("cannot free reserved region"); CloseHandle(fh); return MAP_FAILED; } } } HANDLE fmh = CreateFileMapping(fh, NULL, /* security attributes */ protect, (DWORD) ((filelen + offset) >> 32), (DWORD) ((filelen + offset) & 0xFFFFFFFF), NULL); if (fmh == NULL) { DWORD gle = GetLastError(); ERR("CreateFileMapping, gle: 0x%08x", gle); if (gle == ERROR_ACCESS_DENIED) errno = EACCES; else errno = EINVAL; /* XXX */ CloseHandle(fh); return MAP_FAILED; } void *base = MapViewOfFileEx(fmh, access, (DWORD) (offset >> 32), (DWORD) (offset & 0xFFFFFFFF), filelen, addr); /* hint address */ if (base == NULL) { if (addr == NULL || (flags & MAP_FIXED) != 0) { ERR("MapViewOfFileEx, gle: 0x%08x", GetLastError()); errno = EINVAL; CloseHandle(fh); CloseHandle(fmh); return MAP_FAILED; } /* try again w/o hint */ base = MapViewOfFileEx(fmh, access, (DWORD) (offset >> 32), (DWORD) (offset & 0xFFFFFFFF), filelen, NULL); /* no hint address */ } if (base == NULL) { ERR("MapViewOfFileEx, gle: 0x%08x", GetLastError()); errno = ENOMEM; CloseHandle(fh); CloseHandle(fmh); return MAP_FAILED; } /* * We will track the file mapping handle on a lookaside list so that * we don't have to modify the fact that we only return back the base * address rather than a more elaborate structure. */ PFILE_MAPPING_TRACKER mt = malloc(sizeof(struct FILE_MAPPING_TRACKER)); if (mt == NULL) { ERR("!malloc"); CloseHandle(fh); CloseHandle(fmh); return MAP_FAILED; } mt->Flags = 0; mt->FileHandle = fh; mt->FileMappingHandle = fmh; mt->BaseAddress = base; mt->EndAddress = (void *)((char *)base + len_align); mt->Access = access; mt->Offset = offset; mt->FileLen = filelen_align; /* * XXX: Use the QueryVirtualMemoryInformation when available in the new * SDK. If the file is DAX mapped say so in the FILE_MAPPING_TRACKER * Flags. */ DWORD filesystemFlags; if (fh == INVALID_HANDLE_VALUE) { LOG(4, "anonymous mapping - not DAX mapped - handle: %p", fh); } else if (GetVolumeInformationByHandleW(fh, NULL, 0, NULL, NULL, &filesystemFlags, NULL, 0)) { if (filesystemFlags & FILE_DAX_VOLUME) { mt->Flags |= FILE_MAPPING_TRACKER_FLAG_DIRECT_MAPPED; } else { LOG(4, "file is not DAX mapped - handle: %p", fh); } } else { ERR("failed to query volume information : %08x", GetLastError()); } AcquireSRWLockExclusive(&FileMappingQLock); PMDK_SORTEDQ_INSERT(&FileMappingQHead, mt, ListEntry, FILE_MAPPING_TRACKER, mmap_file_mapping_comparer); ReleaseSRWLockExclusive(&FileMappingQLock); #ifdef MMAP_DEBUG_INFO mmap_info(); #endif return base; } /* * mmap_split -- (internal) replace existing mapping with another one(s) * * Unmaps the region between [begin,end]. If it's in a middle of the existing * mapping, it results in two new mappings and duplicated file/mapping handles. */ static int mmap_split(PFILE_MAPPING_TRACKER mt, void *begin, void *end) { LOG(4, "begin %p end %p", begin, end); ASSERTeq((uintptr_t)begin % Mmap_align, 0); ASSERTeq((uintptr_t)end % Mmap_align, 0); PFILE_MAPPING_TRACKER mtb = NULL; PFILE_MAPPING_TRACKER mte = NULL; HANDLE fh = mt->FileHandle; HANDLE fmh = mt->FileMappingHandle; size_t len; /* * In this routine we copy flags from mt to the two subsets that we * create. All flags may not be appropriate to propagate so let's * assert about the flags we know, if some one adds a new flag in the * future they would know about this copy and take appropricate action. */ C_ASSERT(FILE_MAPPING_TRACKER_FLAGS_MASK == 1); /* * 1) b e b e * xxxxxxxxxxxxx => xxx.......xxxx - mtb+mte * 2) b e b e * xxxxxxxxxxxxx => xxxxxxx....... - mtb * 3) b e b e * xxxxxxxxxxxxx => ........xxxxxx - mte * 4) b e b e * xxxxxxxxxxxxx => .............. - */ if (begin > mt->BaseAddress) { /* case #1/2 */ /* new mapping at the beginning */ mtb = malloc(sizeof(struct FILE_MAPPING_TRACKER)); if (mtb == NULL) { ERR("!malloc"); goto err; } mtb->Flags = mt->Flags; mtb->FileHandle = fh; mtb->FileMappingHandle = fmh; mtb->BaseAddress = mt->BaseAddress; mtb->EndAddress = begin; mtb->Access = mt->Access; mtb->Offset = mt->Offset; len = (char *)begin - (char *)mt->BaseAddress; mtb->FileLen = len >= mt->FileLen ? mt->FileLen : len; } if (end < mt->EndAddress) { /* case #1/3 */ /* new mapping at the end */ mte = malloc(sizeof(struct FILE_MAPPING_TRACKER)); if (mte == NULL) { ERR("!malloc"); goto err; } if (!mtb) { /* case #3 */ mte->FileHandle = fh; mte->FileMappingHandle = fmh; } else { /* case #1 - need to duplicate handles */ mte->FileHandle = NULL; mte->FileMappingHandle = NULL; if (!DuplicateHandle(GetCurrentProcess(), fh, GetCurrentProcess(), &mte->FileHandle, 0, FALSE, DUPLICATE_SAME_ACCESS)) { ERR("DuplicateHandle, gle: 0x%08x", GetLastError()); goto err; } if (!DuplicateHandle(GetCurrentProcess(), fmh, GetCurrentProcess(), &mte->FileMappingHandle, 0, FALSE, DUPLICATE_SAME_ACCESS)) { ERR("DuplicateHandle, gle: 0x%08x", GetLastError()); goto err; } } mte->Flags = mt->Flags; mte->BaseAddress = end; mte->EndAddress = mt->EndAddress; mte->Access = mt->Access; mte->Offset = mt->Offset + ((char *)mte->BaseAddress - (char *)mt->BaseAddress); len = (char *)end - (char *)mt->BaseAddress; mte->FileLen = len >= mt->FileLen ? 0 : mt->FileLen - len; } if (mt->FileLen > 0 && UnmapViewOfFile(mt->BaseAddress) == FALSE) { ERR("UnmapViewOfFile, gle: 0x%08x", GetLastError()); goto err; } len = (char *)mt->EndAddress - (char *)mt->BaseAddress; if (len > mt->FileLen) { void *addr = (char *)mt->BaseAddress + mt->FileLen; mmap_unreserve(addr, len - mt->FileLen); } if (!mtb && !mte) { /* case #4 */ CloseHandle(fmh); CloseHandle(fh); } /* * free entry for the original mapping */ PMDK_SORTEDQ_REMOVE(&FileMappingQHead, mt, ListEntry); free(mt); if (mtb) { len = (char *)mtb->EndAddress - (char *)mtb->BaseAddress; if (len > mtb->FileLen) { void *addr = (char *)mtb->BaseAddress + mtb->FileLen; void *raddr = mmap_reserve(addr, len - mtb->FileLen); if (raddr == MAP_FAILED) { ERR("cannot find a contiguous region - " "addr: %p, len: %lx, gle: 0x%08x", addr, len, GetLastError()); goto err; } } if (mtb->FileLen > 0) { void *base = MapViewOfFileEx(mtb->FileMappingHandle, mtb->Access, (DWORD) (mtb->Offset >> 32), (DWORD) (mtb->Offset & 0xFFFFFFFF), mtb->FileLen, mtb->BaseAddress); /* hint address */ if (base == NULL) { ERR("MapViewOfFileEx, gle: 0x%08x", GetLastError()); goto err; } } PMDK_SORTEDQ_INSERT(&FileMappingQHead, mtb, ListEntry, FILE_MAPPING_TRACKER, mmap_file_mapping_comparer); } if (mte) { len = (char *)mte->EndAddress - (char *)mte->BaseAddress; if (len > mte->FileLen) { void *addr = (char *)mte->BaseAddress + mte->FileLen; void *raddr = mmap_reserve(addr, len - mte->FileLen); if (raddr == MAP_FAILED) { ERR("cannot find a contiguous region - " "addr: %p, len: %lx, gle: 0x%08x", addr, len, GetLastError()); goto err; } } if (mte->FileLen > 0) { void *base = MapViewOfFileEx(mte->FileMappingHandle, mte->Access, (DWORD) (mte->Offset >> 32), (DWORD) (mte->Offset & 0xFFFFFFFF), mte->FileLen, mte->BaseAddress); /* hint address */ if (base == NULL) { ERR("MapViewOfFileEx, gle: 0x%08x", GetLastError()); goto err_mte; } } PMDK_SORTEDQ_INSERT(&FileMappingQHead, mte, ListEntry, FILE_MAPPING_TRACKER, mmap_file_mapping_comparer); } return 0; err: if (mtb) { ASSERTeq(mtb->FileMappingHandle, fmh); ASSERTeq(mtb->FileHandle, fh); CloseHandle(mtb->FileMappingHandle); CloseHandle(mtb->FileHandle); len = (char *)mtb->EndAddress - (char *)mtb->BaseAddress; if (len > mtb->FileLen) { void *addr = (char *)mtb->BaseAddress + mtb->FileLen; mmap_unreserve(addr, len - mtb->FileLen); } } err_mte: if (mte) { if (mte->FileMappingHandle) CloseHandle(mte->FileMappingHandle); if (mte->FileHandle) CloseHandle(mte->FileHandle); len = (char *)mte->EndAddress - (char *)mte->BaseAddress; if (len > mte->FileLen) { void *addr = (char *)mte->BaseAddress + mte->FileLen; mmap_unreserve(addr, len - mte->FileLen); } } free(mtb); free(mte); return -1; } /* * munmap -- delete mapping */ int munmap(void *addr, size_t len) { LOG(4, "addr %p len %zu", addr, len); if (((uintptr_t)addr % Mmap_align) != 0) { ERR("address is not well-aligned: %p", addr); errno = EINVAL; return -1; } if (len == 0) { ERR("invalid length: %zu", len); errno = EINVAL; return -1; } int retval = -1; if (len > UINTPTR_MAX - (uintptr_t)addr) { /* limit len to not get beyond address space */ len = UINTPTR_MAX - (uintptr_t)addr; } void *begin = addr; void *end = (void *)((char *)addr + len); AcquireSRWLockExclusive(&FileMappingQLock); PFILE_MAPPING_TRACKER mt; PFILE_MAPPING_TRACKER next; for (mt = PMDK_SORTEDQ_FIRST(&FileMappingQHead); mt != (void *)&FileMappingQHead; mt = next) { /* * Pick the next entry before we split there by delete the * this one (NOTE: mmap_spilt could delete this entry). */ next = PMDK_SORTEDQ_NEXT(mt, ListEntry); if (mt->BaseAddress >= end) { LOG(4, "ignoring all mapped ranges beyond given range"); break; } if (mt->EndAddress <= begin) { LOG(4, "skipping a mapped range before given range"); continue; } void *begin2 = begin > mt->BaseAddress ? begin : mt->BaseAddress; void *end2 = end < mt->EndAddress ? end : mt->EndAddress; size_t len2 = (char *)end2 - (char *)begin2; void *align_end = (void *)roundup((uintptr_t)end2, Mmap_align); if (mmap_split(mt, begin2, align_end) != 0) { LOG(2, "mapping split failed"); goto err; } if (len > len2) { len -= len2; } else { len = 0; break; } } /* * If we didn't find any mapped regions in our list attempt to free * as if the entire range is reserved. * * XXX: We don't handle a range having few mapped regions and few * reserved regions. */ if (len > 0) mmap_unreserve(addr, roundup(len, Mmap_align)); retval = 0; err: ReleaseSRWLockExclusive(&FileMappingQLock); if (retval == -1) errno = EINVAL; #ifdef MMAP_DEBUG_INFO mmap_info(); #endif return retval; } #define MS_ALL (MS_SYNC|MS_ASYNC|MS_INVALIDATE) /* * msync -- synchronize a file with a memory map */ int msync(void *addr, size_t len, int flags) { LOG(4, "addr %p len %zu flags %d", addr, len, flags); if ((flags & ~MS_ALL) != 0) { ERR("invalid flags: 0x%08x", flags); errno = EINVAL; return -1; } /* * XXX - On Linux it is allowed to call msync() without MS_SYNC * nor MS_ASYNC. */ if (((flags & MS_SYNC) && (flags & MS_ASYNC)) || ((flags & (MS_SYNC | MS_ASYNC)) == 0)) { ERR("neither MS_SYNC or MS_ASYNC is set, or both: 0x%08x", flags); errno = EINVAL; return -1; } if (((uintptr_t)addr % Pagesize) != 0) { ERR("address is not page-aligned: %p", addr); errno = EINVAL; return -1; } if (len == 0) { LOG(4, "zero-length region - do nothing"); return 0; /* do nothing */ } if (len > UINTPTR_MAX - (uintptr_t)addr) { /* limit len to not get beyond address space */ len = UINTPTR_MAX - (uintptr_t)addr; } int retval = -1; void *begin = addr; void *end = (void *)((char *)addr + len); AcquireSRWLockShared(&FileMappingQLock); PFILE_MAPPING_TRACKER mt; PMDK_SORTEDQ_FOREACH(mt, &FileMappingQHead, ListEntry) { if (mt->BaseAddress >= end) { LOG(4, "ignoring all mapped ranges beyond given range"); break; } if (mt->EndAddress <= begin) { LOG(4, "skipping a mapped range before given range"); continue; } void *begin2 = begin > mt->BaseAddress ? begin : mt->BaseAddress; void *end2 = end < mt->EndAddress ? end : mt->EndAddress; size_t len2 = (char *)end2 - (char *)begin2; /* do nothing for anonymous mappings */ if (mt->FileHandle != INVALID_HANDLE_VALUE) { if (FlushViewOfFile(begin2, len2) == FALSE) { ERR("FlushViewOfFile, gle: 0x%08x", GetLastError()); errno = ENOMEM; goto err; } if (FlushFileBuffers(mt->FileHandle) == FALSE) { ERR("FlushFileBuffers, gle: 0x%08x", GetLastError()); errno = EINVAL; goto err; } } if (len > len2) { len -= len2; } else { len = 0; break; } } if (len > 0) { ERR("indicated memory (or part of it) was not mapped"); errno = ENOMEM; } else { retval = 0; } err: ReleaseSRWLockShared(&FileMappingQLock); return retval; } #define PROT_ALL (PROT_READ|PROT_WRITE|PROT_EXEC) /* * mprotect -- set protection on a region of memory * * XXX - If the memory range passed to mprotect() includes invalid pages, * returned status will indicate error, and errno is set to ENOMEM. * However, the protection change is actually applied to all the valid pages, * ignoring the rest. * This is different than on Linux, where it stops on the first invalid page. */ int mprotect(void *addr, size_t len, int prot) { LOG(4, "addr %p len %zu prot %d", addr, len, prot); if (((uintptr_t)addr % Pagesize) != 0) { ERR("address is not page-aligned: %p", addr); errno = EINVAL; return -1; } if (len == 0) { LOG(4, "zero-length region - do nothing"); return 0; /* do nothing */ } if (len > UINTPTR_MAX - (uintptr_t)addr) { len = UINTPTR_MAX - (uintptr_t)addr; LOG(4, "limit len to %zu to not get beyond address space", len); } DWORD protect = 0; if ((prot & PROT_READ) && (prot & PROT_WRITE)) { protect |= PAGE_READWRITE; if (prot & PROT_EXEC) protect |= PAGE_EXECUTE_READWRITE; } else if (prot & PROT_READ) { protect |= PAGE_READONLY; if (prot & PROT_EXEC) protect |= PAGE_EXECUTE_READ; } else { protect |= PAGE_NOACCESS; } int retval = -1; void *begin = addr; void *end = (void *)((char *)addr + len); AcquireSRWLockShared(&FileMappingQLock); PFILE_MAPPING_TRACKER mt; PMDK_SORTEDQ_FOREACH(mt, &FileMappingQHead, ListEntry) { if (mt->BaseAddress >= end) { LOG(4, "ignoring all mapped ranges beyond given range"); break; } if (mt->EndAddress <= begin) { LOG(4, "skipping a mapped range before given range"); continue; } void *begin2 = begin > mt->BaseAddress ? begin : mt->BaseAddress; void *end2 = end < mt->EndAddress ? end : mt->EndAddress; /* * protect of region to VirtualProtection must be compatible * with the access protection specified for this region * when the view was mapped using MapViewOfFileEx */ if (mt->Access == FILE_MAP_COPY) { if (protect & PAGE_READWRITE) { protect &= ~PAGE_READWRITE; protect |= PAGE_WRITECOPY; } else if (protect & PAGE_EXECUTE_READWRITE) { protect &= ~PAGE_EXECUTE_READWRITE; protect |= PAGE_EXECUTE_WRITECOPY; } } size_t len2 = (char *)end2 - (char *)begin2; DWORD oldprot = 0; BOOL ret; ret = VirtualProtect(begin2, len2, protect, &oldprot); if (ret == FALSE) { DWORD gle = GetLastError(); ERR("VirtualProtect, gle: 0x%08x", gle); /* translate error code */ switch (gle) { case ERROR_INVALID_PARAMETER: errno = EACCES; break; case ERROR_INVALID_ADDRESS: errno = ENOMEM; break; default: errno = EINVAL; break; } goto err; } if (len > len2) { len -= len2; } else { len = 0; break; } } if (len > 0) { ERR("indicated memory (or part of it) was not mapped"); errno = ENOMEM; } else { retval = 0; } err: ReleaseSRWLockShared(&FileMappingQLock); return retval; } pmdk-1.8/src/windows/srcversion/0000775000000000000000000000000013615011243015467 5ustar rootrootpmdk-1.8/src/windows/srcversion/srcversion.vcxproj0000664000000000000000000001157613615011243021313 0ustar rootroot Debug x64 Release x64 {901F04DB-E1A5-4A41-8B81-9D31C19ACD59} Win32Proj srcversion 10.0.16299.0 Application true v140 NotSet Application true v140 NotSet true true NotUsing Level3 _DEBUG;_CONSOLE;WINAPI_PARTITION_SYSTEM;%(PreprocessorDefinitions) platform.h 4996 CompileAsC MultiThreadedDebugDLL Console true powershell.exe -ExecutionPolicy Bypass -file "$(SolutionDir)..\utils\SRCVERSION.ps1" $(SRCVERSION) __NON_EXISTENT_FILE__ generate srcversion.h NotUsing Level3 NDEBUG;_CONSOLE;WINAPI_PARTITION_SYSTEM;%(PreprocessorDefinitions) platform.h 4996 CompileAsC MaxSpeed MultiThreadedDLL Default Console true powershell.exe -ExecutionPolicy Bypass -file "$(SolutionDir)..\utils\SRCVERSION.ps1" $(SRCVERSION) __NON_EXISTENT_FILE__ generate srcversion.h pmdk-1.8/src/librpmem/0000775000000000000000000000000013615011243013407 5ustar rootrootpmdk-1.8/src/librpmem/rpmem_util.h0000664000000000000000000000513413615011243015740 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_util.h -- util functions for librpmem header file */ #ifndef RPMEM_UTIL_H #define RPMEM_UTIL_H 1 #ifdef __cplusplus extern "C" { #endif enum { LERR = 1, LWARN = 2, LNOTICE = 3, LINFO = 4, _LDBG = 10, }; #define RPMEM_LOG(level, fmt, args...) LOG(L##level, fmt, ## args) #define RPMEM_DBG(fmt, args...) LOG(_LDBG, fmt, ## args) #define RPMEM_FATAL(fmt, args...) FATAL(fmt, ## args) #define RPMEM_ASSERT(cond) ASSERT(cond) #define RPMEM_PERSIST_FLAGS_ALL RPMEM_PERSIST_RELAXED #define RPMEM_PERSIST_FLAGS_MASK ((unsigned)(~RPMEM_PERSIST_FLAGS_ALL)) #define RPMEM_FLUSH_FLAGS_ALL RPMEM_FLUSH_RELAXED #define RPMEM_FLUSH_FLAGS_MASK ((unsigned)(~RPMEM_FLUSH_FLAGS_ALL)) const char *rpmem_util_proto_errstr(enum rpmem_err err); int rpmem_util_proto_errno(enum rpmem_err err); void rpmem_util_cmds_init(void); void rpmem_util_cmds_fini(void); const char *rpmem_util_cmd_get(void); void rpmem_util_get_env_max_nlanes(unsigned *max_nlanes); void rpmem_util_get_env_wq_size(unsigned *wq_size); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/librpmem/librpmem.c0000664000000000000000000000624613615011243015372 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * librpmem.c -- entry points for librpmem */ #include #include #include "librpmem.h" #include "rpmem.h" #include "rpmem_common.h" #include "rpmem_util.h" #include "rpmem_fip.h" #include "util.h" #include "out.h" /* * librpmem_init -- load-time initialization for librpmem * * Called automatically by the run-time loader. */ ATTR_CONSTRUCTOR void librpmem_init(void) { util_init(); out_init(RPMEM_LOG_PREFIX, RPMEM_LOG_LEVEL_VAR, RPMEM_LOG_FILE_VAR, RPMEM_MAJOR_VERSION, RPMEM_MINOR_VERSION); LOG(3, NULL); rpmem_util_cmds_init(); rpmem_util_get_env_max_nlanes(&Rpmem_max_nlanes); rpmem_util_get_env_wq_size(&Rpmem_wq_size); } /* * librpmem_fini -- librpmem cleanup routine * * Called automatically when the process terminates. */ ATTR_DESTRUCTOR void librpmem_fini(void) { LOG(3, NULL); rpmem_util_cmds_fini(); out_fini(); } /* * rpmem_check_version -- see if library meets application version requirements */ const char * rpmem_check_version(unsigned major_required, unsigned minor_required) { LOG(3, "major_required %u minor_required %u", major_required, minor_required); if (major_required != RPMEM_MAJOR_VERSION) { ERR("librpmem major version mismatch (need %u, found %u)", major_required, RPMEM_MAJOR_VERSION); return out_get_errormsg(); } if (minor_required > RPMEM_MINOR_VERSION) { ERR("librpmem minor version mismatch (need %u, found %u)", minor_required, RPMEM_MINOR_VERSION); return out_get_errormsg(); } return NULL; } /* * rpmem_errormsg -- return the last error message */ const char * rpmem_errormsg(void) { return out_get_errormsg(); } pmdk-1.8/src/librpmem/rpmem.h0000664000000000000000000000421313615011243014700 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem.h -- internal definitions for librpmem */ #include "alloc.h" #include "fault_injection.h" #define RPMEM_LOG_PREFIX "librpmem" #define RPMEM_LOG_LEVEL_VAR "RPMEM_LOG_LEVEL" #define RPMEM_LOG_FILE_VAR "RPMEM_LOG_FILE" #if FAULT_INJECTION void rpmem_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at); int rpmem_fault_injection_enabled(void); #else static inline void rpmem_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { abort(); } static inline int rpmem_fault_injection_enabled(void) { return 0; } #endif pmdk-1.8/src/librpmem/rpmem_fip.c0000664000000000000000000013676513615011243015553 0ustar rootroot/* * Copyright 2016-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_fip.c -- rpmem libfabric provider module source file */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "out.h" #include "util.h" #include "os_thread.h" #include "os.h" #include "rpmem_common.h" #include "rpmem_fip_common.h" #include "rpmem_proto.h" #include "rpmem_util.h" #include "rpmem_fip_msg.h" #include "rpmem_fip.h" #include "valgrind_internal.h" #define RPMEM_FI_ERR(e, fmt, args...)\ ERR(fmt ": %s", ## args, fi_strerror((e))) #define RPMEM_FI_CLOSE(f, fmt, args...) (\ {\ int oerrno = errno;\ int ret = fi_close(&(f)->fid);\ if (ret)\ RPMEM_FI_ERR(ret, fmt, ## args);\ errno = oerrno;\ ret;\ }) #define LANE_ALIGN_SIZE 64 #define LANE_ALIGN __attribute__((aligned(LANE_ALIGN_SIZE))) #define RPMEM_RAW_BUFF_SIZE 4096 #define RPMEM_RAW_SIZE 8 typedef ssize_t (*rpmem_fip_flush_fn)(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags); typedef int (*rpmem_fip_drain_fn)(struct rpmem_fip *fip, unsigned lane); typedef ssize_t (*rpmem_fip_persist_fn)(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags); typedef int (*rpmem_fip_init_fn)(struct rpmem_fip *fip); typedef void (*rpmem_fip_fini_fn)(struct rpmem_fip *fip); typedef ssize_t (*cq_read_fn)(struct fid_cq *cq, void *buf, size_t count); static ssize_t cq_read_infinite(struct fid_cq *cq, void *buf, size_t count) { return fi_cq_sread(cq, buf, count, NULL, -1); } /* * rpmem_fip_ops -- operations specific for persistency method */ struct rpmem_fip_ops { rpmem_fip_flush_fn flush; rpmem_fip_drain_fn drain; rpmem_fip_persist_fn persist; rpmem_fip_init_fn lanes_init; rpmem_fip_init_fn lanes_init_mem; rpmem_fip_fini_fn lanes_fini; rpmem_fip_init_fn lanes_post; }; /* * rpmem_fip_lane -- base lane structure */ struct rpmem_fip_lane { struct fid_ep *ep; /* endpoint */ struct fid_cq *cq; /* completion queue */ uint64_t event; size_t wq_elems; /* # of elements in work queue */ int wq_is_flushing; /* work queue is during flush */ }; /* * rpmem_fip_plane -- persist operation's lane */ struct rpmem_fip_plane { struct rpmem_fip_lane base; /* base lane structure */ struct rpmem_fip_rma write; /* WRITE message */ struct rpmem_fip_rma write_cq; /* WRITE message with completion */ struct rpmem_fip_rma read; /* READ message */ struct rpmem_fip_msg send; /* SEND message */ struct rpmem_fip_msg recv; /* RECV message */ } LANE_ALIGN; /* * rpmem_fip_rlane -- read operation's lane */ struct rpmem_fip_rlane { struct rpmem_fip_lane base; /* base lane structure */ struct rpmem_fip_rma read; /* READ message */ }; struct rpmem_fip { struct fi_info *fi; /* fabric interface information */ struct fid_fabric *fabric; /* fabric domain */ struct fid_domain *domain; /* fabric protection domain */ struct fid_eq *eq; /* event queue */ int closing; /* closing connections in progress */ size_t cq_size; /* completion queue size */ uint64_t raddr; /* remote memory base address */ uint64_t rkey; /* remote memory protection key */ void *laddr; /* local memory base address */ size_t size; /* memory size */ struct fid_mr *mr; /* local memory region */ void *mr_desc; /* local memory descriptor */ enum rpmem_persist_method persist_method; const struct rpmem_fip_ops *ops; unsigned nlanes; size_t buff_size; struct rpmem_fip_plane *lanes; os_thread_t monitor; void *pmsg; /* persist message buffer */ size_t pmsg_size; struct fid_mr *pmsg_mr; /* persist message memory region */ void *pmsg_mr_desc; /* persist message memory descriptor */ struct rpmem_msg_persist_resp *pres; /* persist response buffer */ struct fid_mr *pres_mr; /* persist response memory region */ void *pres_mr_desc; /* persist response memory descriptor */ void *raw_buff; /* READ-after-WRITE buffer */ struct fid_mr *raw_mr; /* RAW memory region */ void *raw_mr_desc; /* RAW memory descriptor */ cq_read_fn cq_read; /* CQ read function */ }; /* * rpmem_fip_is_closing -- (internal) atomically reads and returns the * closing flag */ static inline int rpmem_fip_is_closing(struct rpmem_fip *fip) { int ret; util_atomic_load_explicit32(&fip->closing, &ret, memory_order_acquire); return ret; } /* * rpmem_fip_set_closing -- (internal) atomically set the closing flag */ static inline void rpmem_fip_set_closing(struct rpmem_fip *fip) { /* * load and store without barriers should be good enough here. * fetch_and_or are used as workaround for helgrind issue. */ util_fetch_and_or32(&fip->closing, 1); } /* * rpmem_fip_lane_begin -- (internal) initialize list of events for lane */ static inline void rpmem_fip_lane_begin(struct rpmem_fip_lane *lanep, uint64_t event) { lanep->event = event; } /* * rpmem_fip_lane_init -- (internal) initialize single lane */ static int rpmem_fip_lane_init(struct rpmem_fip *fip, struct rpmem_fip_lane *lanep) { int ret; struct fi_cq_attr cq_attr = { .size = fip->cq_size, .flags = 0, .format = FI_CQ_FORMAT_MSG, .wait_obj = FI_WAIT_UNSPEC, .signaling_vector = 0, .wait_cond = FI_CQ_COND_NONE, .wait_set = NULL, }; /* create a completion queue */ ret = fi_cq_open(fip->domain, &cq_attr, &lanep->cq, NULL); if (ret) { RPMEM_FI_ERR(ret, "opening completion queue"); goto err_cq_open; } /* create an endpoint */ ret = fi_endpoint(fip->domain, fip->fi, &lanep->ep, NULL); if (ret) { RPMEM_FI_ERR(ret, "allocating endpoint"); goto err_endpoint; } /* * Bind an event queue to an endpoint to get * connection-related events for the endpoint. */ ret = fi_ep_bind(lanep->ep, &fip->eq->fid, 0); if (ret) { RPMEM_FI_ERR(ret, "binding event queue to endpoint"); goto err_ep_bind_eq; } /* * Bind a completion queue to an endpoint to get completion * events of specified inbound/outbound operations. * * FI_SELECTIVE_COMPLETION means all inbound/outbound operations * must explicitly specify if the completion event should be * generated or not using FI_COMPLETION flag. * * The completion events received are highly related to the * persistency method used and are configured in lanes * initialization specified for persistency method utilized. */ ret = fi_ep_bind(lanep->ep, &lanep->cq->fid, FI_RECV | FI_TRANSMIT | FI_SELECTIVE_COMPLETION); if (ret) { RPMEM_FI_ERR(ret, "binding completion queue to endpoint"); goto err_ep_bind_cq; } /* * Enable endpoint so it is possible to post inbound/outbound * operations if required. */ ret = fi_enable(lanep->ep); if (ret) { RPMEM_FI_ERR(ret, "activating endpoint"); goto err_fi_enable; } return 0; err_fi_enable: err_ep_bind_cq: err_ep_bind_eq: err_endpoint: RPMEM_FI_CLOSE(lanep->cq, "closing completion queue"); err_cq_open: return -1; } /* * rpmem_fip_lane_fini -- (internal) deinitialize single lane */ static int rpmem_fip_lane_fini(struct rpmem_fip_lane *lanep) { int ret; int lret = 0; ret = RPMEM_FI_CLOSE(lanep->ep, "closing endpoint"); if (ret) lret = ret; ret = RPMEM_FI_CLOSE(lanep->cq, "closing completion queue"); if (ret) lret = ret; return lret; } /* * rpmem_fip_lane_wait -- (internal) wait for specific event on completion queue */ static int rpmem_fip_lane_wait(struct rpmem_fip *fip, struct rpmem_fip_lane *lanep, uint64_t e) { ssize_t sret = 0; struct fi_cq_err_entry err; const char *str_err; int ret = 0; struct fi_cq_msg_entry cq_entry; while (lanep->event & e) { if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; sret = fip->cq_read(lanep->cq, &cq_entry, 1); if (unlikely(sret == -FI_EAGAIN) || sret == 0) continue; if (unlikely(sret < 0)) { ret = (int)sret; goto err_cq_read; } lanep->event &= ~cq_entry.flags; } return 0; err_cq_read: sret = fi_cq_readerr(lanep->cq, &err, 0); if (sret < 0) { RPMEM_FI_ERR((int)sret, "error reading from completion queue: " "cannot read error from event queue"); goto err; } str_err = fi_cq_strerror(lanep->cq, err.prov_errno, NULL, NULL, 0); RPMEM_LOG(ERR, "error reading from completion queue: %s", str_err); err: if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ return ret; } /* * rpmem_fip_set_nlanes -- (internal) set maximum number of lanes supported */ static void rpmem_fip_set_nlanes(struct rpmem_fip *fip, unsigned nlanes) { size_t max_nlanes = rpmem_fip_max_nlanes(fip->fi); RPMEM_ASSERT(max_nlanes < UINT_MAX); fip->nlanes = min((unsigned)max_nlanes, nlanes); } /* * rpmem_fip_getinfo -- (internal) get fabric interface information */ static int rpmem_fip_getinfo(struct rpmem_fip *fip, const char *node, const char *service, enum rpmem_provider provider, size_t max_wq_size, enum rpmem_persist_method pm) { int ret = -1; struct fi_info *hints = rpmem_fip_get_hints(provider); if (!hints) { RPMEM_LOG(ERR, "!getting fabric interface information hints"); goto err_hints; } /* * WQ size is: * - >= size required by persist method (pm_wq_size) * - >= size forced by environment variable (Rpmem_wq_size) * - but it has to be <= max_wq_size reported by provider */ size_t pm_wq_size = rpmem_fip_wq_size(pm, RPMEM_FIP_NODE_CLIENT); hints->tx_attr->size = min( max(pm_wq_size, Rpmem_wq_size), max_wq_size); hints->rx_attr->size = rpmem_fip_rx_size(pm, RPMEM_FIP_NODE_CLIENT); /* get maximum available */ ret = fi_getinfo(RPMEM_FIVERSION, node, service, 0, hints, &fip->fi); if (ret) { RPMEM_FI_ERR(ret, "getting fabric interface information"); goto err_fi_getinfo; } rpmem_fip_print_info(fip->fi); /* fallback to free the hints */ err_fi_getinfo: fi_freeinfo(hints); err_hints: return ret; } /* * rpmem_fip_init_fabric_res -- (internal) initialize common fabric resources */ static int rpmem_fip_init_fabric_res(struct rpmem_fip *fip) { int ret; ret = fi_fabric(fip->fi->fabric_attr, &fip->fabric, NULL); if (ret) { RPMEM_FI_ERR(ret, "opening fabric domain"); goto err_fi_fabric; } ret = fi_domain(fip->fabric, fip->fi, &fip->domain, NULL); if (ret) { RPMEM_FI_ERR(ret, "opening fabric access domain"); goto err_fi_domain; } struct fi_eq_attr eq_attr = { .size = 0, /* use default value */ .flags = 0, .wait_obj = FI_WAIT_UNSPEC, .signaling_vector = 0, .wait_set = NULL, }; ret = fi_eq_open(fip->fabric, &eq_attr, &fip->eq, NULL); if (ret) { RPMEM_FI_ERR(ret, "opening event queue"); goto err_eq_open; } return 0; err_eq_open: RPMEM_FI_CLOSE(fip->domain, "closing fabric access domain"); err_fi_domain: RPMEM_FI_CLOSE(fip->fabric, "closing fabric domain"); err_fi_fabric: return ret; } /* * rpmem_fip_fini_fabric_res -- (internal) deinitialize common fabric resources */ static void rpmem_fip_fini_fabric_res(struct rpmem_fip *fip) { RPMEM_FI_CLOSE(fip->eq, "closing event queue"); RPMEM_FI_CLOSE(fip->domain, "closing fabric access domain"); RPMEM_FI_CLOSE(fip->fabric, "closing fabric domain"); } /* * rpmem_fip_init_memory -- (internal) initialize common memory resources */ static int rpmem_fip_init_memory(struct rpmem_fip *fip) { ASSERTne(Pagesize, 0); int ret; /* * Register local memory space. The local memory will be used * with WRITE operation in rpmem_fip_persist function thus * the FI_WRITE access flag. */ ret = fi_mr_reg(fip->domain, fip->laddr, fip->size, FI_WRITE, 0, 0, 0, &fip->mr, NULL); if (ret) { RPMEM_FI_ERR(ret, "registrating memory"); return ret; } /* get local memory descriptor */ fip->mr_desc = fi_mr_desc(fip->mr); return 0; } /* * rpmem_fip_fini_memory -- (internal) deinitialize common memory resources */ static void rpmem_fip_fini_memory(struct rpmem_fip *fip) { RPMEM_FI_CLOSE(fip->mr, "unregistering memory"); } /* * rpmem_fip_lanes_init_common -- (internal) initialize common lanes resources */ static int rpmem_fip_lanes_init_common(struct rpmem_fip *fip) { int ret; ret = posix_memalign((void **)&fip->lanes, LANE_ALIGN_SIZE, fip->nlanes * sizeof(*fip->lanes)); if (ret) { RPMEM_LOG(ERR, "!allocating lanes"); goto err_alloc_lanes; } memset(fip->lanes, 0, fip->nlanes * sizeof(*fip->lanes)); unsigned i; for (i = 0; i < fip->nlanes; i++) { ret = rpmem_fip_lane_init(fip, &fip->lanes[i].base); if (ret) goto err_lane_init; } return 0; err_lane_init: for (unsigned j = 0; j < i; j++) rpmem_fip_lane_fini(&fip->lanes[i].base); free(fip->lanes); err_alloc_lanes: return -1; } /* * rpmem_fip_lanes_fini_common -- (internal) deinitialize common lanes * resrouces */ static int rpmem_fip_lanes_fini_common(struct rpmem_fip *fip) { int lret = 0; int ret; for (unsigned i = 0; i < fip->nlanes; i++) { ret = rpmem_fip_lane_fini(&fip->lanes[i].base); if (ret) lret = ret; } free(fip->lanes); return lret; } /* * rpmem_fip_lanes_init -- (internal) initialize lanes */ static int rpmem_fip_lanes_init(struct rpmem_fip *fip) { int ret; ret = rpmem_fip_lanes_init_common(fip); if (ret) return ret; ret = fip->ops->lanes_init(fip); if (ret) goto err_init_lanes; return 0; err_init_lanes: rpmem_fip_lanes_fini_common(fip); return ret; } /* * rpmem_fip_lane_connect -- (internal) connect on a single lane */ static int rpmem_fip_lane_connect(struct rpmem_fip *fip, struct rpmem_fip_lane *lanep) { struct fi_eq_cm_entry entry; int ret; ret = fi_connect(lanep->ep, fip->fi->dest_addr, NULL, 0); if (ret) { RPMEM_FI_ERR(ret, "initiating connection request"); return ret; } return rpmem_fip_read_eq_check(fip->eq, &entry, FI_CONNECTED, &lanep->ep->fid, RPMEM_CONNECT_TIMEOUT); } /* * rpmem_fip_lanes_connect -- (internal) establish connections on all lanes */ static int rpmem_fip_lanes_connect(struct rpmem_fip *fip) { int ret; for (unsigned i = 0; i < fip->nlanes; i++) { struct rpmem_fip_lane *lanep = &fip->lanes[i].base; ret = rpmem_fip_lane_connect(fip, lanep); if (ret) return ret; } return 0; } /* * rpmem_fip_lanes_shutdown -- shutdown all endpoints */ static int rpmem_fip_lanes_shutdown(struct rpmem_fip *fip) { int ret; int lret = 0; for (unsigned i = 0; i < fip->nlanes; i++) { ret = fi_shutdown(fip->lanes[i].base.ep, 0); if (ret) { RPMEM_FI_ERR(ret, "disconnecting endpoint"); lret = ret; } } return lret; } /* * rpmem_fip_lane_prep_write -- (internal) choose right WRITE structure * according to flags and prepare for collecting its completion */ static inline struct rpmem_fip_rma * rpmem_fip_lane_prep_write(struct rpmem_fip_plane *lanep, unsigned flags) { if (flags & RPMEM_COMPLETION) { rpmem_fip_lane_begin(&lanep->base, FI_WRITE); return &lanep->write_cq; } return &lanep->write; } /* * rpmem_fip_monitor_thread -- (internal) monitor in-band connection */ static void * rpmem_fip_monitor_thread(void *arg) { struct rpmem_fip *fip = (struct rpmem_fip *)arg; struct fi_eq_cm_entry entry; uint32_t event; int ret; while (!rpmem_fip_is_closing(fip)) { ret = rpmem_fip_read_eq(fip->eq, &entry, &event, RPMEM_MONITOR_TIMEOUT); if (unlikely(ret == 0) && event == FI_SHUTDOWN) { RPMEM_LOG(ERR, "event queue got FI_SHUTDOWN"); /* mark in-band connection as closing */ rpmem_fip_set_closing(fip); for (unsigned i = 0; i < fip->nlanes; i++) { fi_cq_signal(fip->lanes[i].base.cq); } } } return NULL; } /* * rpmem_fip_monitor_init -- (internal) initialize in-band monitor */ static int rpmem_fip_monitor_init(struct rpmem_fip *fip) { errno = os_thread_create(&fip->monitor, NULL, rpmem_fip_monitor_thread, fip); if (errno) { RPMEM_LOG(ERR, "!connenction monitor thread"); return -1; } return 0; } /* * rpmem_fip_monitor_fini -- (internal) finalize in-band monitor */ static int rpmem_fip_monitor_fini(struct rpmem_fip *fip) { rpmem_fip_set_closing(fip); int ret = os_thread_join(&fip->monitor, NULL); if (ret) { RPMEM_LOG(ERR, "joining monitor thread failed"); } return ret; } /* * rpmem_fip_init_lanes_common -- (internal) initialize lanes */ static int rpmem_fip_init_lanes_common(struct rpmem_fip *fip) { ASSERTne(Pagesize, 0); int ret = 0; /* allocate persist messages buffer */ fip->pmsg_size = roundup(sizeof(struct rpmem_msg_persist) + fip->buff_size, (size_t)64); size_t msg_size = fip->nlanes * fip->pmsg_size; msg_size = PAGE_ALIGNED_UP_SIZE(msg_size); errno = posix_memalign((void **)&fip->pmsg, Pagesize, msg_size); if (errno) { RPMEM_LOG(ERR, "!allocating messages buffer"); ret = -1; goto err_malloc_pmsg; } /* * Register persist messages buffer. The persist messages * are sent to daemon thus the FI_SEND access flag. */ ret = fi_mr_reg(fip->domain, fip->pmsg, msg_size, FI_SEND, 0, 0, 0, &fip->pmsg_mr, NULL); if (ret) { RPMEM_FI_ERR(ret, "registering messages buffer"); goto err_fi_mr_reg_pmsg; } /* get persist messages buffer local descriptor */ fip->pmsg_mr_desc = fi_mr_desc(fip->pmsg_mr); /* allocate persist response messages buffer */ size_t msg_resp_size = fip->nlanes * sizeof(struct rpmem_msg_persist_resp); msg_resp_size = PAGE_ALIGNED_UP_SIZE(msg_resp_size); errno = posix_memalign((void **)&fip->pres, Pagesize, msg_resp_size); if (errno) { RPMEM_LOG(ERR, "!allocating messages response buffer"); ret = -1; goto err_malloc_pres; } /* * Register persist messages response buffer. The persist response * messages are received from daemon thus the FI_RECV access flag. */ ret = fi_mr_reg(fip->domain, fip->pres, msg_resp_size, FI_RECV, 0, 0, 0, &fip->pres_mr, NULL); if (ret) { RPMEM_FI_ERR(ret, "registering messages response buffer"); goto err_fi_mr_reg_pres; } /* get persist response messages buffer local descriptor */ fip->pres_mr_desc = fi_mr_desc(fip->pres_mr); return 0; err_fi_mr_reg_pres: free(fip->pres); err_malloc_pres: RPMEM_FI_CLOSE(fip->pmsg_mr, "unregistering messages buffer"); err_fi_mr_reg_pmsg: free(fip->pmsg); err_malloc_pmsg: return ret; } /* * rpmem_fip_get_pmsg -- return persist message buffer */ static inline struct rpmem_msg_persist * rpmem_fip_get_pmsg(struct rpmem_fip *fip, size_t idx) { return (struct rpmem_msg_persist *) ((uintptr_t)fip->pmsg + idx * fip->pmsg_size); } /* * rpmem_fip_init_mem_lanes_gpspm -- initialize lanes rma structures */ static int rpmem_fip_init_mem_lanes_gpspm(struct rpmem_fip *fip) { /* * Initialize all required structures for: * WRITE, SEND and RECV operations. * * If the completion is required the FI_COMPLETION flag and * appropriate context should be used. * * In GPSPM only the RECV and SEND completions are required. * * For RECV the context is RECV operation structure used for * fi_recvmsg(3) function call. * * For SEND the context is lane structure. * * The received buffer contains a lane id which is used * to obtain a lane which must be signaled that operation * has been completed. */ unsigned i; for (i = 0; i < fip->nlanes; i++) { /* WRITE */ rpmem_fip_rma_init(&fip->lanes[i].write, fip->mr_desc, 0, fip->rkey, &fip->lanes[i], 0); /* SEND */ rpmem_fip_msg_init(&fip->lanes[i].send, fip->pmsg_mr_desc, 0, &fip->lanes[i], rpmem_fip_get_pmsg(fip, i), 0 /* size must be provided when sending msg */, FI_COMPLETION); /* RECV */ rpmem_fip_msg_init(&fip->lanes[i].recv, fip->pres_mr_desc, 0, &fip->lanes[i].recv, &fip->pres[i], sizeof(fip->pres[i]), FI_COMPLETION); } return 0; } /* * rpmem_fip_fini_lanes_common -- (internal) deinitialize lanes for GPSPM */ static void rpmem_fip_fini_lanes_common(struct rpmem_fip *fip) { RPMEM_FI_CLOSE(fip->pmsg_mr, "unregistering messages buffer"); RPMEM_FI_CLOSE(fip->pres_mr, "unregistering messages " "response buffer"); free(fip->pmsg); free(fip->pres); } /* * rpmem_fip_init_lanes_apm -- (internal) initialize lanes for APM */ static int rpmem_fip_init_lanes_apm(struct rpmem_fip *fip) { ASSERTne(Pagesize, 0); int ret; ret = rpmem_fip_init_lanes_common(fip); if (ret) goto err_init_lanes_common; ASSERT(IS_PAGE_ALIGNED(RPMEM_RAW_BUFF_SIZE)); errno = posix_memalign((void **)&fip->raw_buff, Pagesize, RPMEM_RAW_BUFF_SIZE); if (errno) { RPMEM_LOG(ERR, "!allocating APM RAW buffer"); goto err_malloc_raw; } /* register read-after-write buffer */ ret = fi_mr_reg(fip->domain, fip->raw_buff, RPMEM_RAW_BUFF_SIZE, FI_REMOTE_WRITE, 0, 0, 0, &fip->raw_mr, NULL); if (ret) { RPMEM_FI_ERR(ret, "registering APM read buffer"); goto err_fi_raw_mr; } /* get read-after-write buffer local descriptor */ fip->raw_mr_desc = fi_mr_desc(fip->raw_mr); return 0; err_fi_raw_mr: free(fip->raw_buff); err_malloc_raw: rpmem_fip_fini_lanes_common(fip); err_init_lanes_common: return -1; } /* * rpmem_fip_init_mem_lanes_apm -- initialize lanes rma structures */ static int rpmem_fip_init_mem_lanes_apm(struct rpmem_fip *fip) { /* * Initialize all required structures for: * WRITE and READ operations. * * If the completion is required the FI_COMPLETION flag and * appropriate context should be used. * * In APM only the READ completion is required. * The context is a lane structure. */ for (unsigned i = 0; i < fip->nlanes; i++) { /* WRITE */ rpmem_fip_rma_init(&fip->lanes[i].write, fip->mr_desc, 0, fip->rkey, &fip->lanes[i], 0); /* WRITE + FI_COMPLETION */ rpmem_fip_rma_init(&fip->lanes[i].write_cq, fip->mr_desc, 0, fip->rkey, &fip->lanes[i], FI_COMPLETION); /* READ */ rpmem_fip_rma_init(&fip->lanes[i].read, fip->raw_mr_desc, 0, fip->rkey, &fip->lanes[i], FI_COMPLETION); /* SEND */ rpmem_fip_msg_init(&fip->lanes[i].send, fip->pmsg_mr_desc, 0, &fip->lanes[i], rpmem_fip_get_pmsg(fip, i), fip->pmsg_size, FI_COMPLETION); /* RECV */ rpmem_fip_msg_init(&fip->lanes[i].recv, fip->pres_mr_desc, 0, &fip->lanes[i].recv, &fip->pres[i], sizeof(fip->pres[i]), FI_COMPLETION); } return 0; } /* * rpmem_fip_fini_lanes_apm -- (internal) deinitialize lanes for APM */ static void rpmem_fip_fini_lanes_apm(struct rpmem_fip *fip) { RPMEM_FI_CLOSE(fip->raw_mr, "unregistering APM read buffer"); free(fip->raw_buff); rpmem_fip_fini_lanes_common(fip); } /* * rpmem_fip_wq_inc -- (internal) increment number of elements in WQ */ static inline void rpmem_fip_wq_inc(struct rpmem_fip_plane *lanep) { ++lanep->base.wq_elems; } /* * rpmem_fip_wq_set_empty -- (internal) zero number of elements in WQ */ static inline void rpmem_fip_wq_set_empty(struct rpmem_fip_plane *lanep) { RPMEM_ASSERT(!lanep->base.wq_is_flushing); lanep->base.wq_elems = 0; } /* * rpmem_fip_wq_require_flush -- (internal) is WQ almost full */ static inline int rpmem_fip_wq_require_flush(struct rpmem_fip *fip, struct rpmem_fip_plane *lanep) { RPMEM_ASSERT(lanep->base.wq_elems < fip->fi->tx_attr->size); return lanep->base.wq_elems + 1 == fip->fi->tx_attr->size; } /* * rpmem_fip_wq_is_flushing -- (internal) is WQ flush started */ static inline int rpmem_fip_wq_is_flushing(struct rpmem_fip_plane *lanep) { return lanep->base.wq_is_flushing; } /* * rpmem_fip_wq_set_flushing -- (internal) mark WQ flush start */ static inline void rpmem_fip_wq_set_flushing(struct rpmem_fip_plane *lanep) { lanep->base.wq_is_flushing = 1; } /* * if WQ is almost full last WRITE has to report its completion * otherwise it is unknown when subsequent commands can be posted */ #define RPMEM_FIP_WQ_FLUSH_REQ RPMEM_COMPLETION /* * rpmem_fip_wq_flush_wait -- (internal) wait for WRITE completion * to make sure WQ can accept subsequent commands */ static inline int rpmem_fip_wq_flush_wait(struct rpmem_fip *fip, struct rpmem_fip_plane *lanep) { RPMEM_ASSERT(lanep->base.wq_elems == fip->fi->tx_attr->size); RPMEM_ASSERT(lanep->base.wq_is_flushing); /* wait for WRITE completion */ int ret = rpmem_fip_lane_wait(fip, &lanep->base, FI_WRITE); if (unlikely(ret)) { LOG(2, "waiting for WRITE completion failed"); return ret; } /* when WRITE completion is reaped WQ is empty */ lanep->base.wq_is_flushing = 0; rpmem_fip_wq_set_empty(lanep); return 0; } /* * rpmem_fip_wq_inc_and_flush -- (internal) increment number of elements in WQ * and flush it */ static inline int rpmem_fip_wq_inc_and_flush(struct rpmem_fip *fip, struct rpmem_fip_plane *lanep) { rpmem_fip_wq_inc(lanep); rpmem_fip_wq_set_flushing(lanep); return rpmem_fip_wq_flush_wait(fip, lanep); } /* * rpmem_fip_wq_flush_check -- (internal) check if WQ requires flush or it is * during flushing and handle each case */ static inline int rpmem_fip_wq_flush_check(struct rpmem_fip *fip, struct rpmem_fip_plane *lanep, unsigned *flags) { if (rpmem_fip_wq_is_flushing(lanep)) return rpmem_fip_wq_flush_wait(fip, lanep); if (rpmem_fip_wq_require_flush(fip, lanep)) *flags |= RPMEM_FIP_WQ_FLUSH_REQ; return 0; } /* * rpmem_fip_get_wq_size -- get WQ size (for validation purposes only) */ inline size_t rpmem_fip_get_wq_size(struct rpmem_fip *fip) { RPMEM_ASSERT(fip); RPMEM_ASSERT(fip->fi); RPMEM_ASSERT(fip->fi->tx_attr); return fip->fi->tx_attr->size; } /* * rpmem_fip_flush_raw -- (internal) perform flush operation using rma WRITE */ static int rpmem_fip_flush_raw(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { struct rpmem_fip_plane *lanep = &fip->lanes[lane]; int ret; void *laddr = (void *)((uintptr_t)fip->laddr + offset); uint64_t raddr = fip->raddr + offset; struct rpmem_fip_rma *write = rpmem_fip_lane_prep_write(lanep, flags); /* WRITE for requested memory region */ ret = rpmem_fip_writemsg(lanep->base.ep, write, laddr, len, raddr); if (unlikely(ret)) { RPMEM_FI_ERR(ret, "RMA write"); return ret; } if (flags & RPMEM_FIP_WQ_FLUSH_REQ) rpmem_fip_wq_set_flushing(lanep); return 0; } /* * rpmem_fip_drain_raw -- (internal) perform drain operation using rma READ */ static int rpmem_fip_drain_raw(struct rpmem_fip *fip, unsigned lane) { struct rpmem_fip_plane *lanep = &fip->lanes[lane]; int ret; rpmem_fip_lane_begin(&lanep->base, FI_READ); /* READ to read-after-write buffer */ ret = rpmem_fip_readmsg(lanep->base.ep, &lanep->read, fip->raw_buff, RPMEM_RAW_SIZE, fip->raddr); if (unlikely(ret)) { RPMEM_FI_ERR(ret, "RMA read"); return ret; } /* wait for READ completion */ ret = rpmem_fip_lane_wait(fip, &lanep->base, FI_READ); if (unlikely(ret)) { ERR("waiting for READ completion failed"); return ret; } return 0; } /* * rpmem_fip_persist_raw -- (internal) perform persist operation using * READ after WRITE mechanism */ static int rpmem_fip_persist_raw(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { int ret; ret = rpmem_fip_flush_raw(fip, offset, len, lane, flags); if (unlikely(ret)) return ret; /* flush WQ prior to posting subsequent message */ if (flags & RPMEM_FIP_WQ_FLUSH_REQ) { struct rpmem_fip_plane *lanep = &fip->lanes[lane]; ret = rpmem_fip_wq_inc_and_flush(fip, lanep); if (unlikely(ret)) return ret; } return rpmem_fip_drain_raw(fip, lane); } /* * rpmem_fip_post_resp -- (internal) post persist response message buffer */ static inline int rpmem_fip_post_resp(struct rpmem_fip *fip, struct rpmem_fip_plane *lanep) { int ret = rpmem_fip_recvmsg(lanep->base.ep, &lanep->recv); if (unlikely(ret)) { RPMEM_FI_ERR(ret, "posting recv buffer"); return ret; } return 0; } /* * rpmem_fip_persist_saw -- (internal) perform persist operation using * SEND after WRITE mechanism */ static int rpmem_fip_persist_saw(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { struct rpmem_fip_plane *lanep = &fip->lanes[lane]; void *laddr = (void *)((uintptr_t)fip->laddr + offset); uint64_t raddr = fip->raddr + offset; struct rpmem_msg_persist *msg; int ret; ret = rpmem_fip_lane_wait(fip, &lanep->base, FI_SEND); if (unlikely(ret)) { ERR("waiting for SEND completion failed"); return ret; } struct rpmem_fip_rma *write = rpmem_fip_lane_prep_write(lanep, flags); /* WRITE for requested memory region */ ret = rpmem_fip_writemsg(lanep->base.ep, write, laddr, len, raddr); if (unlikely(ret)) { RPMEM_FI_ERR((int)ret, "RMA write"); return ret; } /* flush WQ prior to posting subsequent message */ if (flags & RPMEM_FIP_WQ_FLUSH_REQ) { ret = rpmem_fip_wq_inc_and_flush(fip, lanep); if (unlikely(ret)) return ret; } rpmem_fip_lane_begin(&lanep->base, FI_RECV | FI_SEND); /* SEND persist message */ msg = rpmem_fip_msg_get_pmsg(&lanep->send); msg->flags = (flags & RPMEM_FLUSH_PERSIST_MASK); msg->lane = lane; msg->addr = raddr; msg->size = len; ret = rpmem_fip_sendmsg(lanep->base.ep, &lanep->send, sizeof(*msg)); if (unlikely(ret)) { RPMEM_FI_ERR(ret, "MSG send"); return ret; } /* wait for persist operation completion */ ret = rpmem_fip_lane_wait(fip, &lanep->base, FI_RECV); if (unlikely(ret)) { ERR("waiting for RECV completion failed"); return ret; } ret = rpmem_fip_post_resp(fip, lanep); if (unlikely(ret)) { ERR("posting RECV buffer failed"); return ret; } return 0; } /* * rpmem_fip_persist_send -- (internal) perform persist operation using * RDMA SEND operation with data inlined in the message buffer. */ static int rpmem_fip_persist_send(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { RPMEM_ASSERT(len <= fip->buff_size); struct rpmem_fip_plane *lanep = &fip->lanes[lane]; void *laddr = (void *)((uintptr_t)fip->laddr + offset); uint64_t raddr = fip->raddr + offset; struct rpmem_msg_persist *msg; int ret; ret = rpmem_fip_lane_wait(fip, &lanep->base, FI_SEND); if (unlikely(ret)) { ERR("waiting for SEND completion failed"); return ret; } rpmem_fip_lane_begin(&lanep->base, FI_RECV | FI_SEND); /* SEND persist message */ msg = rpmem_fip_msg_get_pmsg(&lanep->send); msg->flags = flags; msg->lane = lane; msg->addr = raddr; msg->size = len; memcpy(msg->data, laddr, len); ret = rpmem_fip_sendmsg(lanep->base.ep, &lanep->send, sizeof(*msg) + len); if (unlikely(ret)) { RPMEM_FI_ERR(ret, "MSG send"); return ret; } /* wait for persist operation completion */ ret = rpmem_fip_lane_wait(fip, &lanep->base, FI_RECV); if (unlikely(ret)) { ERR("waiting for RECV completion failed"); return ret; } ret = rpmem_fip_post_resp(fip, lanep); if (unlikely(ret)) { ERR("posting RECV buffer failed"); return ret; } return 0; } /* * rpmem_fip_persist_gpspm_sockets -- (internal) perform persist operation * for GPSPM - sockets provider implementation which doesn't use the * inline persist operation */ static ssize_t rpmem_fip_persist_gpspm_sockets(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { unsigned mode = flags & RPMEM_FLUSH_PERSIST_MASK; if (mode == RPMEM_PERSIST_SEND) flags = (flags & ~RPMEM_FLUSH_PERSIST_MASK) | RPMEM_FLUSH_WRITE; int ret = rpmem_fip_wq_flush_check(fip, &fip->lanes[lane], &flags); if (unlikely(ret)) return -abs(ret); /* Limit len to the max value of the return type. */ len = min(len, SSIZE_MAX); ret = rpmem_fip_persist_saw(fip, offset, len, lane, flags); if (ret) return -abs(ret); rpmem_fip_wq_set_empty(&fip->lanes[lane]); return (ssize_t)len; } /* * rpmem_fip_persist_apm_sockets -- (internal) perform persist operation * for APM - sockets provider implementation which doesn't use the * inline persist operation */ static ssize_t rpmem_fip_persist_apm_sockets(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { /* Limit len to the max value of the return type. */ len = min(len, SSIZE_MAX); int ret = rpmem_fip_wq_flush_check(fip, &fip->lanes[lane], &flags); if (unlikely(ret)) return -abs(ret); ret = rpmem_fip_persist_raw(fip, offset, len, lane, flags); if (unlikely(ret)) return -abs(ret); rpmem_fip_wq_set_empty(&fip->lanes[lane]); return (ssize_t)len; } /* * rpmem_fip_persist_gpspm -- (internal) perform persist operation for GPSPM */ static ssize_t rpmem_fip_persist_gpspm(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { /* Limit len to the max value of the return type. */ len = min(len, SSIZE_MAX); unsigned mode = flags & RPMEM_FLUSH_PERSIST_MASK; int ret = rpmem_fip_wq_flush_check(fip, &fip->lanes[lane], &flags); if (unlikely(ret)) return -abs(ret); if (mode == RPMEM_PERSIST_SEND) { len = min(len, fip->buff_size); ret = rpmem_fip_persist_send(fip, offset, len, lane, flags); } else { ret = rpmem_fip_persist_saw(fip, offset, len, lane, flags); } if (ret) return -abs(ret); rpmem_fip_wq_set_empty(&fip->lanes[lane]); return (ssize_t)len; } /* * rpmem_fip_drain_nop -- (internal) perform drain operation as NOP */ static int rpmem_fip_drain_nop(struct rpmem_fip *fip, unsigned lane) { (void) fip; (void) lane; return 0; } /* * rpmem_fip_flush_apm -- (internal) perform flush operation for APM */ static ssize_t rpmem_fip_flush_apm(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { struct rpmem_fip_plane *lanep = &fip->lanes[lane]; int ret; /* Limit len to the max value of the return type. */ len = min(len, SSIZE_MAX); unsigned mode = flags & RPMEM_FLUSH_PERSIST_MASK; ret = rpmem_fip_wq_flush_check(fip, lanep, &flags); if (unlikely(ret)) return ret; if (mode == RPMEM_PERSIST_SEND) { /* * XXX: Probably posting Send in the flush and waiting for the * response in the drain will give some performance gains. */ len = min(len, fip->buff_size); ret = rpmem_fip_persist_send(fip, offset, len, lane, flags); } else { ret = rpmem_fip_flush_raw(fip, offset, len, lane, flags); } if (ret) return -abs(ret); rpmem_fip_wq_inc(lanep); return (ssize_t)len; } /* * rpmem_fip_drain_apm -- (internal) perform drain operation for APM */ static int rpmem_fip_drain_apm(struct rpmem_fip *fip, unsigned lane) { struct rpmem_fip_plane *lanep = &fip->lanes[lane]; int ret; if (unlikely(rpmem_fip_wq_is_flushing(lanep))) { ret = rpmem_fip_wq_flush_wait(fip, lanep); if (unlikely(ret)) return ret; } ret = rpmem_fip_drain_raw(fip, lane); /* successful drain means WQ is empty */ if (likely(!ret)) rpmem_fip_wq_set_empty(lanep); return ret; } /* * rpmem_fip_persist_apm -- (internal) perform persist operation for APM */ static ssize_t rpmem_fip_persist_apm(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { /* Limit len to the max value of the return type. */ len = min(len, SSIZE_MAX); unsigned mode = flags & RPMEM_FLUSH_PERSIST_MASK; int ret = rpmem_fip_wq_flush_check(fip, &fip->lanes[lane], &flags); if (unlikely(ret)) return -abs(ret); if (unlikely(mode == RPMEM_DEEP_PERSIST)) ret = rpmem_fip_persist_saw(fip, offset, len, lane, flags); else if (mode == RPMEM_PERSIST_SEND) { len = min(len, fip->buff_size); ret = rpmem_fip_persist_send(fip, offset, len, lane, flags); } else { ret = rpmem_fip_persist_raw(fip, offset, len, lane, flags); } if (unlikely(ret)) return -abs(ret); rpmem_fip_wq_set_empty(&fip->lanes[lane]); return (ssize_t)len; } /* * rpmem_fip_post_lanes_common -- (internal) post all persist response message * buffers */ static int rpmem_fip_post_lanes_common(struct rpmem_fip *fip) { int ret = 0; for (unsigned i = 0; i < fip->nlanes; i++) { ret = rpmem_fip_post_resp(fip, &fip->lanes[i]); if (ret) break; } return ret; } /* * rpmem_fip_ops -- some operations specific for persistency method used * * Note: GPSPM flush is emulated by persist whereas drain is a nop. * * Probably splitting Send-after-Write into two stages (flush + drain) * will give some performance gains for GPSPM mode. */ static const struct rpmem_fip_ops rpmem_fip_ops[MAX_RPMEM_PROV][MAX_RPMEM_PM] = { [RPMEM_PROV_LIBFABRIC_VERBS] = { [RPMEM_PM_GPSPM] = { .flush = rpmem_fip_persist_gpspm, .drain = rpmem_fip_drain_nop, .persist = rpmem_fip_persist_gpspm, .lanes_init = rpmem_fip_init_lanes_common, .lanes_init_mem = rpmem_fip_init_mem_lanes_gpspm, .lanes_fini = rpmem_fip_fini_lanes_common, .lanes_post = rpmem_fip_post_lanes_common, }, [RPMEM_PM_APM] = { .flush = rpmem_fip_flush_apm, .drain = rpmem_fip_drain_apm, .persist = rpmem_fip_persist_apm, .lanes_init = rpmem_fip_init_lanes_apm, .lanes_init_mem = rpmem_fip_init_mem_lanes_apm, .lanes_fini = rpmem_fip_fini_lanes_apm, .lanes_post = rpmem_fip_post_lanes_common, }, }, [RPMEM_PROV_LIBFABRIC_SOCKETS] = { [RPMEM_PM_GPSPM] = { .flush = rpmem_fip_persist_gpspm_sockets, .drain = rpmem_fip_drain_nop, .persist = rpmem_fip_persist_gpspm_sockets, .lanes_init = rpmem_fip_init_lanes_common, .lanes_init_mem = rpmem_fip_init_mem_lanes_gpspm, .lanes_fini = rpmem_fip_fini_lanes_common, .lanes_post = rpmem_fip_post_lanes_common, }, [RPMEM_PM_APM] = { .flush = rpmem_fip_flush_apm, .drain = rpmem_fip_drain_apm, .persist = rpmem_fip_persist_apm_sockets, .lanes_init = rpmem_fip_init_lanes_apm, .lanes_init_mem = rpmem_fip_init_mem_lanes_apm, .lanes_fini = rpmem_fip_fini_lanes_apm, .lanes_post = rpmem_fip_post_lanes_common, }, } }; /* * rpmem_fip_set_attr -- (internal) set required attributes */ static void rpmem_fip_set_attr(struct rpmem_fip *fip, struct rpmem_fip_attr *attr) { fip->raddr = (uint64_t)attr->raddr; fip->rkey = attr->rkey; fip->laddr = attr->laddr; fip->size = attr->size; fip->buff_size = attr->buff_size; fip->persist_method = attr->persist_method; rpmem_fip_set_nlanes(fip, attr->nlanes); /* one for read operation */ fip->cq_size = rpmem_fip_cq_size(fip->persist_method, RPMEM_FIP_NODE_CLIENT); fip->ops = &rpmem_fip_ops[attr->provider][fip->persist_method]; } /* * rpmem_fip_init -- initialize fabric provider */ struct rpmem_fip * rpmem_fip_init(const char *node, const char *service, struct rpmem_fip_attr *attr, unsigned *nlanes) { int ret; struct rpmem_fip *fip = calloc(1, sizeof(*fip)); if (!fip) { RPMEM_LOG(ERR, "!allocating fabric handle"); return NULL; } ret = rpmem_fip_getinfo(fip, node, service, attr->provider, attr->max_wq_size, attr->persist_method); if (ret) goto err_getinfo; fip->cq_read = attr->provider == RPMEM_PROV_LIBFABRIC_VERBS ? fi_cq_read : cq_read_infinite; rpmem_fip_set_attr(fip, attr); *nlanes = fip->nlanes; ret = rpmem_fip_init_fabric_res(fip); if (ret) goto err_init_fabric_res; ret = rpmem_fip_lanes_init(fip); if (ret) goto err_init_lanes; return fip; err_init_lanes: rpmem_fip_fini_fabric_res(fip); err_init_fabric_res: fi_freeinfo(fip->fi); err_getinfo: free(fip); return NULL; } /* * rpmem_fip_fini -- deinitialize fabric provider */ void rpmem_fip_fini(struct rpmem_fip *fip) { fip->ops->lanes_fini(fip); rpmem_fip_lanes_fini_common(fip); rpmem_fip_fini_fabric_res(fip); fi_freeinfo(fip->fi); free(fip); } /* * rpmem_fip_connect -- connect to remote peer */ int rpmem_fip_connect(struct rpmem_fip *fip) { int ret; ret = rpmem_fip_lanes_connect(fip); if (ret) goto err_lanes_connect; ret = rpmem_fip_monitor_init(fip); if (ret) goto err_monitor; ret = rpmem_fip_init_memory(fip); if (ret) goto err_init_memory; ret = fip->ops->lanes_init_mem(fip); if (ret) goto err_init_lanes_mem; ret = fip->ops->lanes_post(fip); if (ret) goto err_lanes_post; return 0; err_lanes_post: err_init_lanes_mem: rpmem_fip_fini_memory(fip); err_init_memory: rpmem_fip_monitor_fini(fip); err_monitor: rpmem_fip_lanes_shutdown(fip); err_lanes_connect: return ret; } /* * rpmem_fip_close -- close connection to remote peer */ int rpmem_fip_close(struct rpmem_fip *fip) { int ret; int lret = 0; if (unlikely(rpmem_fip_is_closing(fip))) goto close_monitor; rpmem_fip_fini_memory(fip); ret = rpmem_fip_lanes_shutdown(fip); if (ret) lret = ret; close_monitor: /* close fip monitor */ ret = rpmem_fip_monitor_fini(fip); if (ret) lret = ret; return lret; } /* * rpmem_fip_flush -- perform remote flush operation */ int rpmem_fip_flush(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { RPMEM_ASSERT((flags & RPMEM_FLUSH_PERSIST_MASK) <= RPMEM_PERSIST_MAX); RPMEM_ASSERT(flags != RPMEM_DEEP_PERSIST); if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ RPMEM_ASSERT(lane < fip->nlanes); if (unlikely(lane >= fip->nlanes)) return EINVAL; /* it will be passed to errno */ if (unlikely(offset >= fip->size || offset + len > fip->size)) return EINVAL; /* it will be passed to errno */ if (unlikely(len == 0)) return 0; int ret = 0; while (len > 0) { size_t tmplen = min(len, fip->fi->ep_attr->max_msg_size); ssize_t r = fip->ops->flush(fip, offset, tmplen, lane, flags); if (r < 0) { RPMEM_LOG(ERR, "flush operation failed"); ret = (int)r; goto err; } tmplen = (size_t)r; offset += tmplen; len -= tmplen; } err: if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ return ret; } /* * rpmem_fip_drain -- perform remote drain operation */ int rpmem_fip_drain(struct rpmem_fip *fip, unsigned lane) { if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ RPMEM_ASSERT(lane < fip->nlanes); if (unlikely(lane >= fip->nlanes)) return EINVAL; /* it will be passed to errno */ int ret = fip->ops->drain(fip, lane); if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ return ret; } /* * rpmem_fip_persist -- perform remote persist operation */ int rpmem_fip_persist(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags) { RPMEM_ASSERT((flags & RPMEM_FLUSH_PERSIST_MASK) <= RPMEM_PERSIST_MAX); if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ RPMEM_ASSERT(lane < fip->nlanes); if (unlikely(lane >= fip->nlanes)) return EINVAL; /* it will be passed to errno */ if (unlikely(offset >= fip->size || offset + len > fip->size)) return EINVAL; /* it will be passed to errno */ if (unlikely(len == 0)) return 0; int ret = 0; while (len > 0) { size_t tmplen = min(len, fip->fi->ep_attr->max_msg_size); ssize_t r = fip->ops->persist(fip, offset, tmplen, lane, flags); if (r < 0) { RPMEM_LOG(ERR, "persist operation failed"); ret = (int)r; goto err; } tmplen = (size_t)r; offset += tmplen; len -= tmplen; } err: if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ return ret; } /* * rpmem_fip_read -- perform read operation */ int rpmem_fip_read(struct rpmem_fip *fip, void *buff, size_t len, size_t off, unsigned lane) { int ret; if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ RPMEM_ASSERT(lane < fip->nlanes); if (unlikely(lane >= fip->nlanes)) return EINVAL; /* it will be passed to errno */ if (unlikely(len == 0)) { return 0; } size_t rd_buff_len = len < fip->fi->ep_attr->max_msg_size ? len : fip->fi->ep_attr->max_msg_size; void *rd_buff; /* buffer for read operation */ struct fid_mr *rd_mr; /* read buffer memory region */ void *rd_mr_desc; /* read buffer memory descriptor */ struct rpmem_fip_rlane rd_lane; /* allocate buffer for read operation */ errno = posix_memalign((void **)&rd_buff, Pagesize, rd_buff_len); if (errno) { RPMEM_LOG(ERR, "!allocating read buffer"); ret = errno; goto err_malloc_rd_buff; } /* * Register buffer for read operation. * The read operation utilizes READ operation thus * the FI_REMOTE_WRITE flag. */ ret = fi_mr_reg(fip->domain, rd_buff, rd_buff_len, FI_REMOTE_WRITE, 0, 0, 0, &rd_mr, NULL); if (ret) { RPMEM_FI_ERR(ret, "registrating read buffer"); goto err_rd_mr; } /* get read buffer local memory descriptor */ rd_mr_desc = fi_mr_desc(rd_mr); /* * Initialize READ message. The completion is required in order * to signal thread that READ operation has been completed. */ rpmem_fip_rma_init(&rd_lane.read, rd_mr_desc, 0, fip->rkey, &rd_lane, FI_COMPLETION); size_t rd = 0; uint8_t *cbuff = buff; struct rpmem_fip_lane *lanep = &fip->lanes[lane].base; while (rd < len) { size_t rd_len = len - rd < rd_buff_len ? len - rd : rd_buff_len; size_t rd_off = off + rd; uint64_t raddr = fip->raddr + rd_off; rpmem_fip_lane_begin(lanep, FI_READ); ret = rpmem_fip_readmsg(lanep->ep, &rd_lane.read, rd_buff, rd_len, raddr); if (ret) { RPMEM_FI_ERR(ret, "RMA read"); goto err_readmsg; } VALGRIND_DO_MAKE_MEM_DEFINED(rd_buff, rd_len); ret = rpmem_fip_lane_wait(fip, lanep, FI_READ); if (ret) { ERR("error when processing read request"); goto err_lane_wait; } memcpy(&cbuff[rd], rd_buff, rd_len); rd += rd_len; } ret = 0; err_lane_wait: err_readmsg: RPMEM_FI_CLOSE(rd_mr, "unregistering memory"); err_rd_mr: free(rd_buff); err_malloc_rd_buff: if (unlikely(rpmem_fip_is_closing(fip))) return ECONNRESET; /* it will be passed to errno */ return ret; } /* * parse_bool -- convert string value to boolean */ static int parse_bool(const char *str_value) { if (strcmp(str_value, "0") == 0 || strcasecmp(str_value, "false") == 0 || strcasecmp(str_value, "no") == 0 || strcasecmp(str_value, "off") == 0) { return 0; } if (strcmp(str_value, "1") == 0 || strcasecmp(str_value, "true") == 0 || strcasecmp(str_value, "yes") == 0 || strcasecmp(str_value, "on") == 0) { return 1; } return -1; } /* * rpmem_fip_param_get -- read environment variable in the libfabric way * * - If parameter does not exist the output value is not changed. * - If the environment variable is not set the output value is not changed. * - If the environment variable is set and its value is not correct the output * value is set to error value. * - If the environment variable is set and its value is correct the output * value is set according to the environment variable value. */ static void rpmem_fip_param_get(const char *var_name, int *value) { struct fi_param *params; int count; int ret = fi_getparams(¶ms, &count); if (ret != FI_SUCCESS) { RPMEM_FI_ERR(ret, "getting fabric parameters list"); return; } for (int i = 0; i < count; ++i) { if (strcmp(params[i].name, var_name) != 0) continue; if (!params[i].value) { break; } *value = parse_bool(params[i].value); break; } fi_freeparams(params); } #define LIBFABRIC_FORK_UNSAFE_VAR "FI_FORK_UNSAFE" /* * rpmem_fip_probe_fork_safety -- probe if libfabric is fork safe */ void rpmem_fip_probe_fork_safety(void) { int *fork_unsafe = &Rpmem_fork_unsafe; /* false by default */ rpmem_fip_param_get(LIBFABRIC_FORK_UNSAFE_VAR, fork_unsafe); } pmdk-1.8/src/librpmem/rpmem_cmd.c0000664000000000000000000001402013615011243015513 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_cmd.c -- simple interface for running an executable in child process */ #include #include #include #include #include #include #include #include #include #include #include "util.h" #include "out.h" #include "os.h" #include "rpmem_common.h" #include "rpmem_util.h" #include "rpmem_cmd.h" /* * rpmem_cmd_init -- initialize command */ struct rpmem_cmd * rpmem_cmd_init(void) { struct rpmem_cmd *cmd = calloc(1, sizeof(*cmd)); if (!cmd) { RPMEM_LOG(ERR, "allocating command buffer"); goto err_alloc_cmd; } return cmd; err_alloc_cmd: return NULL; } /* * rpmem_cmd_fini -- deinitialize command */ void rpmem_cmd_fini(struct rpmem_cmd *cmd) { for (int i = 0; i < cmd->args.argc; i++) free(cmd->args.argv[i]); free(cmd->args.argv); free(cmd); } /* * rpmem_cmd_push -- push back command's argument */ int rpmem_cmd_push(struct rpmem_cmd *cmd, const char *arg) { size_t argv_count = (size_t)cmd->args.argc + 2; char **argv = realloc(cmd->args.argv, argv_count * sizeof(char *)); if (!argv) { RPMEM_LOG(ERR, "reallocating command argv"); goto err_realloc; } cmd->args.argv = argv; char *arg_dup = strdup(arg); if (!arg_dup) { RPMEM_LOG(ERR, "allocating argument"); goto err_strdup; } cmd->args.argv[cmd->args.argc] = arg_dup; cmd->args.argc++; cmd->args.argv[cmd->args.argc] = NULL; return 0; err_strdup: err_realloc: return -1; } /* * rpmem_cmd_log -- print executing command */ static void rpmem_cmd_log(struct rpmem_cmd *cmd) { RPMEM_ASSERT(cmd->args.argc > 0); size_t size = 0; for (int i = 0; i < cmd->args.argc; i++) { size += strlen(cmd->args.argv[i]) + 1; } char *buff = malloc(size); if (!buff) { RPMEM_LOG(ERR, "allocating log buffer for command"); return; } size_t pos = 0; for (int i = 0; pos < size && i < cmd->args.argc; i++) { int ret = snprintf(&buff[pos], size - pos, "%s%s", cmd->args.argv[i], i == cmd->args.argc - 1 ? "" : " "); if (ret < 0) { RPMEM_LOG(ERR, "printing command's argument failed"); goto out; } pos += (size_t)ret; } RPMEM_LOG(INFO, "executing command '%s'", buff); out: free(buff); } /* * rpmem_cmd_run -- run command and connect with stdin, stdout and stderr * using unix sockets. * * The communication with child process is done via socketpairs on * stdin, stdout and stderr. The socketpairs are used instead of pipes * because reading from disconnected pipe causes a SIGPIPE signal. * When using socketpair it is possible to read data using recv(3) * function with MSG_NOSIGNAL flag, which doesn't send a signal. */ int rpmem_cmd_run(struct rpmem_cmd *cmd) { int fd_in[2]; int fd_out[2]; int fd_err[2]; rpmem_cmd_log(cmd); /* socketpair for stdin */ int ret = socketpair(AF_UNIX, SOCK_STREAM, 0, fd_in); if (ret < 0) { RPMEM_LOG(ERR, "creating pipe for stdin"); goto err_pipe_in; } /* parent process stdin socket */ cmd->fd_in = fd_in[1]; /* socketpair for stdout */ ret = socketpair(AF_UNIX, SOCK_STREAM, 0, fd_out); if (ret < 0) { RPMEM_LOG(ERR, "creating pipe for stdout"); goto err_pipe_out; } /* parent process stdout socket */ cmd->fd_out = fd_out[0]; /* socketpair for stderr */ ret = socketpair(AF_UNIX, SOCK_STREAM, 0, fd_err); if (ret < 0) { RPMEM_LOG(ERR, "creating pipe for stderr"); goto err_pipe_err; } /* socketpair for stderr */ cmd->fd_err = fd_err[0]; cmd->pid = fork(); if (cmd->pid == -1) { RPMEM_LOG(ERR, "forking command"); goto err_fork; } if (!cmd->pid) { dup2(fd_in[0], 0); dup2(fd_out[1], 1); dup2(fd_err[1], 2); execvp(cmd->args.argv[0], cmd->args.argv); exit(EXIT_FAILURE); } os_close(fd_in[0]); os_close(fd_out[1]); os_close(fd_err[1]); return 0; err_fork: os_close(fd_err[0]); os_close(fd_err[1]); err_pipe_err: os_close(fd_out[0]); os_close(fd_out[1]); err_pipe_out: os_close(fd_in[0]); os_close(fd_in[1]); err_pipe_in: return -1; } /* * rpmem_cmd_wait -- wait for process to change state */ int rpmem_cmd_wait(struct rpmem_cmd *cmd, int *status) { if (cmd->pid <= 0) { RPMEM_LOG(ERR, "wrong PID: %i", cmd->pid); errno = EINVAL; return -1; } if (waitpid(cmd->pid, status, 0) != cmd->pid) { RPMEM_LOG(ERR, "!waitpid failed"); return -1; } return 0; } /* * rpmem_cmd_term -- close child process's unix sockets */ void rpmem_cmd_term(struct rpmem_cmd *cmd) { os_close(cmd->fd_in); os_close(cmd->fd_out); os_close(cmd->fd_err); RPMEM_ASSERT(cmd->pid > 0); } pmdk-1.8/src/librpmem/rpmem_obc.h0000664000000000000000000000506713615011243015533 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_obc.h -- rpmem out-of-band connection client header file */ #ifndef RPMEM_OBC_H #define RPMEM_OBC_H 1 #include #include #include "librpmem.h" #ifdef __cplusplus extern "C" { #endif struct rpmem_obc; struct rpmem_obc *rpmem_obc_init(void); void rpmem_obc_fini(struct rpmem_obc *rpc); int rpmem_obc_connect(struct rpmem_obc *rpc, const struct rpmem_target_info *info); int rpmem_obc_disconnect(struct rpmem_obc *rpc); int rpmem_obc_monitor(struct rpmem_obc *rpc, int nonblock); int rpmem_obc_create(struct rpmem_obc *rpc, const struct rpmem_req_attr *req, struct rpmem_resp_attr *res, const struct rpmem_pool_attr *pool_attr); int rpmem_obc_open(struct rpmem_obc *rpc, const struct rpmem_req_attr *req, struct rpmem_resp_attr *res, struct rpmem_pool_attr *pool_attr); int rpmem_obc_set_attr(struct rpmem_obc *rpc, const struct rpmem_pool_attr *pool_attr); int rpmem_obc_close(struct rpmem_obc *rpc, int flags); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/librpmem/librpmem.link.in0000664000000000000000000000355113615011243016506 0ustar rootroot# # Copyright 2016-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # src/librpmem/librpmem.link -- linker link file for librpmem # LIBRPMEM_1.0 { global: rpmem_create; rpmem_open; rpmem_set_attr; rpmem_close; rpmem_remove; rpmem_flush; rpmem_drain; rpmem_persist; rpmem_deep_persist; rpmem_read; rpmem_check_version; rpmem_errormsg; fault_injection; local: *; }; pmdk-1.8/src/librpmem/README0000664000000000000000000000034713615011243014273 0ustar rootrootThis directory contains a librpmem library which provides remote access to persistent memory over RDMA. ** DEPENDENCIES: ** The librpmem library depends on libfabric (version >= 1.4.2) library: https://github.com/ofiwg/libfabric pmdk-1.8/src/librpmem/rpmem_cmd.h0000664000000000000000000000440113615011243015522 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_cmd.h -- helper module for invoking separate process */ #ifndef RPMEM_CMD_H #define RPMEM_CMD_H 1 #include #ifdef __cplusplus extern "C" { #endif struct rpmem_cmd { int fd_in; /* stdin */ int fd_out; /* stdout */ int fd_err; /* stderr */ struct { char **argv; int argc; } args; /* command arguments */ pid_t pid; /* pid of process */ }; struct rpmem_cmd *rpmem_cmd_init(void); int rpmem_cmd_push(struct rpmem_cmd *cmd, const char *arg); int rpmem_cmd_run(struct rpmem_cmd *cmd); void rpmem_cmd_term(struct rpmem_cmd *cmd); int rpmem_cmd_wait(struct rpmem_cmd *cmd, int *status); void rpmem_cmd_fini(struct rpmem_cmd *cmd); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/librpmem/rpmem_util.c0000664000000000000000000001434413615011243015736 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_util.c -- util functions for librpmem source file */ #include #include #include #include #include "out.h" #include "os.h" #include "librpmem.h" #include "rpmem_proto.h" #include "rpmem_common.h" #include "rpmem_util.h" static const struct rpmem_err_str_errno { int err; const char *str; } rpmem_err_str_errno[MAX_RPMEM_ERR] = { [RPMEM_SUCCESS] = { .err = 0, .str = "Success", }, [RPMEM_ERR_BADPROTO] = { .err = EPROTONOSUPPORT, .str = "Protocol version number mismatch", }, [RPMEM_ERR_BADNAME] = { .err = EINVAL, .str = "Invalid pool descriptor", }, [RPMEM_ERR_BADSIZE] = { .err = EFBIG, .str = "Invalid pool size", }, [RPMEM_ERR_BADNLANES] = { .err = EINVAL, .str = "Invalid number of lanes", }, [RPMEM_ERR_BADPROVIDER] = { .err = EINVAL, .str = "Invalid provider", }, [RPMEM_ERR_FATAL] = { .err = EREMOTEIO, .str = "Fatal error", }, [RPMEM_ERR_FATAL_CONN] = { .err = ECONNABORTED, .str = "Fatal in-band connection error", }, [RPMEM_ERR_BUSY] = { .err = EBUSY, .str = "Pool already in use", }, [RPMEM_ERR_EXISTS] = { .err = EEXIST, .str = "Pool already exists", }, [RPMEM_ERR_PROVNOSUP] = { .err = EMEDIUMTYPE, .str = "Provider not supported", }, [RPMEM_ERR_NOEXIST] = { .err = ENOENT, .str = "Pool set or its part doesn't exist or it is " "unavailable", }, [RPMEM_ERR_NOACCESS] = { .err = EACCES, .str = "Pool set permission denied", }, [RPMEM_ERR_POOL_CFG] = { .err = EINVAL, .str = "Invalid pool set configuration", }, }; static char *Rpmem_cmds; static char **Rpmem_cmd_arr; static size_t Rpmem_current_cmd; static size_t Rpmem_ncmds; #define RPMEM_CMD_SEPARATOR '|' /* * rpmem_util_proto_errstr -- return error string for error code */ const char * rpmem_util_proto_errstr(enum rpmem_err err) { RPMEM_ASSERT(err < MAX_RPMEM_ERR); const char *ret = rpmem_err_str_errno[err].str; RPMEM_ASSERT(ret); return ret; } /* * rpmem_util_proto_errno -- return appropriate errno value for error code */ int rpmem_util_proto_errno(enum rpmem_err err) { RPMEM_ASSERT(err < MAX_RPMEM_ERR); return rpmem_err_str_errno[err].err; } /* * rpmem_util_cmds_inc -- increase size of array for rpmem commands */ static void rpmem_util_cmds_inc(void) { Rpmem_ncmds++; Rpmem_cmd_arr = realloc(Rpmem_cmd_arr, Rpmem_ncmds * sizeof(*Rpmem_cmd_arr)); if (!Rpmem_cmd_arr) RPMEM_FATAL("!realloc"); } /* * rpmem_util_cmds_init -- read a RPMEM_CMD from the environment variable */ void rpmem_util_cmds_init(void) { char *cmd = os_getenv(RPMEM_CMD_ENV); if (!cmd) cmd = RPMEM_DEF_CMD; Rpmem_cmds = strdup(cmd); if (!Rpmem_cmds) RPMEM_FATAL("!strdup"); char *next = Rpmem_cmds; while (next) { rpmem_util_cmds_inc(); Rpmem_cmd_arr[Rpmem_ncmds - 1] = next; next = strchr(next, RPMEM_CMD_SEPARATOR); if (next) { *next = '\0'; next++; } } } /* * rpmem_util_env_fini -- release RPMEM_CMD copy */ void rpmem_util_cmds_fini(void) { RPMEM_ASSERT(Rpmem_cmds); RPMEM_ASSERT(Rpmem_cmd_arr); RPMEM_ASSERT(Rpmem_current_cmd < Rpmem_ncmds); free(Rpmem_cmds); Rpmem_cmds = NULL; free(Rpmem_cmd_arr); Rpmem_cmd_arr = NULL; Rpmem_ncmds = 0; Rpmem_current_cmd = 0; } /* * rpmem_util_cmd_get -- get a next command from RPMEM_CMD * * RPMEM_CMD can contain multiple commands separated by RPMEM_CMD_SEPARATOR. * Commands from RPMEM_CMD are read sequentially and used to establish out of * band connections to remote nodes in the order read from a poolset file. * */ const char * rpmem_util_cmd_get(void) { RPMEM_ASSERT(Rpmem_cmds); RPMEM_ASSERT(Rpmem_cmd_arr); RPMEM_ASSERT(Rpmem_current_cmd < Rpmem_ncmds); char *ret = Rpmem_cmd_arr[Rpmem_current_cmd]; Rpmem_current_cmd = (Rpmem_current_cmd + 1) % Rpmem_ncmds; return ret; } /* * rpmem_util_get_env_uint -- read the unsigned value from environment */ static void rpmem_util_get_env_uint(const char *env, unsigned *pval) { char *env_val = os_getenv(env); if (env_val && env_val[0] != '\0') { char *endptr; errno = 0; long val = strtol(env_val, &endptr, 10); if (endptr[0] != '\0' || val <= 0 || (errno == ERANGE && (val == LONG_MAX || val == LONG_MIN))) { RPMEM_LOG(ERR, "%s variable must be a positive integer", env); } else { *pval = val < UINT_MAX ? (unsigned)val: UINT_MAX; } } } /* * rpmem_util_get_env_max_nlanes -- read the maximum number of lanes from * RPMEM_MAX_NLANES */ void rpmem_util_get_env_max_nlanes(unsigned *max_nlanes) { rpmem_util_get_env_uint(RPMEM_MAX_NLANES_ENV, max_nlanes); } /* * rpmem_util_get_env_wq_size -- read the required WQ size from env */ void rpmem_util_get_env_wq_size(unsigned *wq_size) { rpmem_util_get_env_uint(RPMEM_WQ_SIZE_ENV, wq_size); } pmdk-1.8/src/librpmem/rpmem.c0000664000000000000000000005306113615011243014700 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem.c -- main source file for librpmem */ #include #include #include #include #include #include #include "librpmem.h" #include "out.h" #include "os.h" #include "os_thread.h" #include "util.h" #include "rpmem.h" #include "rpmem_common.h" #include "rpmem_util.h" #include "rpmem_obc.h" #include "rpmem_fip.h" #include "rpmem_fip_common.h" #include "rpmem_ssh.h" #include "rpmem_proto.h" #define RPMEM_REMOVE_FLAGS_ALL (\ RPMEM_REMOVE_FORCE | \ RPMEM_REMOVE_POOL_SET \ ) #define RPMEM_CHECK_FORK() do {\ if (Rpmem_fork_unsafe) {\ ERR("libfabric is initialized without fork() support");\ return NULL;\ }\ } while (0) static os_once_t Rpmem_fork_unsafe_key_once = OS_ONCE_INIT; /* * rpmem_pool -- remote pool context */ struct rpmem_pool { struct rpmem_obc *obc; /* out-of-band connection handle */ struct rpmem_fip *fip; /* fabric provider handle */ struct rpmem_target_info *info; char fip_service[NI_MAXSERV]; enum rpmem_provider provider; size_t max_wq_size; /* max WQ size supported by provider */ os_thread_t monitor; int closing; int no_headers; /* * Last error code, need to be volatile because it can * be accessed by multiple threads. */ volatile int error; }; /* * env_get_bool -- parse value of specified environment variable as a bool * * Return values: * 0 - defined, valp has value * 1 - not defined * -1 - parsing error */ static int env_get_bool(const char *name, int *valp) { LOG(3, "name %s, valp %p", name, valp); const char *env = os_getenv(name); if (!env) return 1; char *endptr; errno = 0; long val = strtol(env, &endptr, 10); if (*endptr != '\0' || errno) goto err; if (val < INT_MIN || val > INT_MAX) goto err; *valp = (int)val; return 0; err: RPMEM_LOG(ERR, "!parsing '%s' environment variable failed", name); return -1; } /* * rpmem_get_provider -- set provider based on node address and environment */ static int rpmem_set_provider(RPMEMpool *rpp, const char *node) { LOG(3, "rpp %p, node %s", rpp, node); struct rpmem_fip_probe probe; enum rpmem_provider prov = RPMEM_PROV_UNKNOWN; int ret = rpmem_fip_probe_get(node, &probe); if (ret) return -1; /* * The sockets provider can be used only if specified environment * variable is set to 1. */ if (rpmem_fip_probe(probe, RPMEM_PROV_LIBFABRIC_SOCKETS)) { int enable; ret = env_get_bool(RPMEM_PROV_SOCKET_ENV, &enable); if (!ret && enable) { prov = RPMEM_PROV_LIBFABRIC_SOCKETS; } } /* * The verbs provider is enabled by default. If appropriate * environment variable is set to 0, the verbs provider is disabled. * * The verbs provider has higher priority than sockets provider. */ if (rpmem_fip_probe(probe, RPMEM_PROV_LIBFABRIC_VERBS)) { int enable; ret = env_get_bool(RPMEM_PROV_VERBS_ENV, &enable); if (ret == 1 || (!ret && enable)) prov = RPMEM_PROV_LIBFABRIC_VERBS; } if (prov == RPMEM_PROV_UNKNOWN) return -1; RPMEM_ASSERT(prov < MAX_RPMEM_PROV); rpp->max_wq_size = probe.max_wq_size[prov]; rpp->provider = prov; return 0; } /* * rpmem_monitor_thread -- connection monitor background thread */ static void * rpmem_monitor_thread(void *arg) { LOG(3, "arg %p", arg); RPMEMpool *rpp = arg; int ret = rpmem_obc_monitor(rpp->obc, 0); if (ret && !rpp->closing) { RPMEM_LOG(ERR, "unexpected data received"); rpp->error = errno; } return NULL; } /* * rpmem_common_init -- common routine for initialization */ static RPMEMpool * rpmem_common_init(const char *target) { LOG(3, "target %s", target); int ret; RPMEMpool *rpp = calloc(1, sizeof(*rpp)); if (!rpp) { ERR("!calloc"); goto err_malloc_rpmem; } rpp->info = rpmem_target_parse(target); if (!rpp->info) { ERR("!parsing target node address failed"); goto err_target_split; } ret = rpmem_set_provider(rpp, rpp->info->node); if (ret) { errno = ENOMEDIUM; ERR("cannot find provider"); goto err_provider; } RPMEM_LOG(NOTICE, "provider: %s", rpmem_provider_to_str(rpp->provider)); if (rpp->provider == RPMEM_PROV_LIBFABRIC_SOCKETS) { /* libfabric's sockets provider does not support IPv6 */ RPMEM_LOG(NOTICE, "forcing using IPv4"); rpp->info->flags |= RPMEM_FLAGS_USE_IPV4; } rpp->obc = rpmem_obc_init(); if (!rpp->obc) { ERR("!out-of-band connection initialization failed"); goto err_obc_init; } RPMEM_LOG(INFO, "establishing out-of-band connection"); ret = rpmem_obc_connect(rpp->obc, rpp->info); if (ret) { ERR("!out-of-band connection failed"); goto err_obc_connect; } RPMEM_LOG(NOTICE, "out-of-band connection established"); return rpp; err_obc_connect: rpmem_obc_fini(rpp->obc); err_obc_init: err_provider: rpmem_target_free(rpp->info); err_target_split: free(rpp); err_malloc_rpmem: return NULL; } /* * rpmem_common_fini -- common routing for deinitialization */ static void rpmem_common_fini(RPMEMpool *rpp, int join) { LOG(3, "rpp %p, join %d", rpp, join); rpmem_obc_disconnect(rpp->obc); if (join) { int ret = os_thread_join(&rpp->monitor, NULL); if (ret) { errno = ret; ERR("joining monitor thread failed"); } } rpmem_obc_fini(rpp->obc); rpmem_target_free(rpp->info); free(rpp); } /* * rpmem_common_fip_init -- common routine for initializing fabric provider */ static int rpmem_common_fip_init(RPMEMpool *rpp, struct rpmem_req_attr *req, struct rpmem_resp_attr *resp, void *pool_addr, size_t pool_size, unsigned *nlanes, size_t buff_size) { LOG(3, "rpp %p, req %p, resp %p, pool_addr %p, pool_size %zu, nlanes " "%p", rpp, req, resp, pool_addr, pool_size, nlanes); int ret; struct rpmem_fip_attr fip_attr = { .provider = req->provider, .max_wq_size = rpp->max_wq_size, .persist_method = resp->persist_method, .laddr = pool_addr, .size = pool_size, .buff_size = buff_size, .nlanes = min(*nlanes, resp->nlanes), .raddr = (void *)resp->raddr, .rkey = resp->rkey, }; ret = snprintf(rpp->fip_service, sizeof(rpp->fip_service), "%u", resp->port); if (ret <= 0) { ERR("snprintf: %d", ret); goto err_port; } rpp->fip = rpmem_fip_init(rpp->info->node, rpp->fip_service, &fip_attr, nlanes); if (!rpp->fip) { ERR("!in-band connection initialization failed"); ret = -1; goto err_fip_init; } RPMEM_LOG(NOTICE, "final nlanes: %u", *nlanes); RPMEM_LOG(INFO, "establishing in-band connection"); ret = rpmem_fip_connect(rpp->fip); if (ret) { ERR("!establishing in-band connection failed"); goto err_fip_connect; } RPMEM_LOG(NOTICE, "in-band connection established"); return 0; err_fip_connect: rpmem_fip_fini(rpp->fip); err_fip_init: err_port: return ret; } /* * rpmem_common_fip_fini -- common routine for deinitializing fabric provider */ static void rpmem_common_fip_fini(RPMEMpool *rpp) { LOG(3, "rpp %p", rpp); RPMEM_LOG(INFO, "closing in-band connection"); rpmem_fip_fini(rpp->fip); RPMEM_LOG(NOTICE, "in-band connection closed"); } /* * rpmem_log_args -- log input arguments for rpmem_create and rpmem_open */ static void rpmem_log_args(const char *req, const char *target, const char *pool_set_name, void *pool_addr, size_t pool_size, unsigned nlanes) { LOG(3, "req %s, target %s, pool_set_name %s, pool_addr %p, pool_size " "%zu, nlanes %d", req, target, pool_set_name, pool_addr, pool_size, nlanes); RPMEM_LOG(NOTICE, "%s request:", req); RPMEM_LOG(NOTICE, "\ttarget: %s", target); RPMEM_LOG(NOTICE, "\tpool set: %s", pool_set_name); RPMEM_LOG(INFO, "\tpool addr: %p", pool_addr); RPMEM_LOG(INFO, "\tpool size: %lu", pool_size); RPMEM_LOG(NOTICE, "\tnlanes: %u", nlanes); } /* * rpmem_log_resp -- log response attributes */ static void rpmem_log_resp(const char *req, const struct rpmem_resp_attr *resp) { LOG(3, "req %s, resp %p", req, resp); RPMEM_LOG(NOTICE, "%s request response:", req); RPMEM_LOG(NOTICE, "\tnlanes: %u", resp->nlanes); RPMEM_LOG(NOTICE, "\tport: %u", resp->port); RPMEM_LOG(NOTICE, "\tpersist method: %s", rpmem_persist_method_to_str(resp->persist_method)); RPMEM_LOG(NOTICE, "\tremote addr: 0x%" PRIx64, resp->raddr); } /* * rpmem_check_args -- validate user's arguments */ static int rpmem_check_args(void *pool_addr, size_t pool_size, unsigned *nlanes) { LOG(3, "pool_addr %p, pool_size %zu, nlanes %p", pool_addr, pool_size, nlanes); if (!pool_addr) { errno = EINVAL; ERR("invalid pool address"); return -1; } if (!IS_PAGE_ALIGNED((uintptr_t)pool_addr)) { errno = EINVAL; ERR("Pool address must be aligned to page size (%llu)", Pagesize); return -1; } if (!IS_PAGE_ALIGNED(pool_size)) { errno = EINVAL; ERR("Pool size must be aligned to page size (%llu)", Pagesize); return -1; } if (!pool_size) { errno = EINVAL; ERR("invalid pool size"); return -1; } if (!nlanes) { errno = EINVAL; ERR("lanes pointer cannot be NULL"); return -1; } if (!(*nlanes)) { errno = EINVAL; ERR("number of lanes must be positive"); return -1; } return 0; } /* * rpmem_create -- create remote pool on target node * * target -- target node in format [@][:] * pool_set_name -- remote pool set name * pool_addr -- local pool memory address which will be replicated * pool_size -- required pool size * nlanes -- number of lanes * create_attr -- pool attributes used for creating the pool on remote node */ RPMEMpool * rpmem_create(const char *target, const char *pool_set_name, void *pool_addr, size_t pool_size, unsigned *nlanes, const struct rpmem_pool_attr *create_attr) { LOG(3, "target %s, pool_set_name %s, pool_addr %p, pool_size %zu, " "nlanes %p, create_attr %p", target, pool_set_name, pool_addr, pool_size, nlanes, create_attr); os_once(&Rpmem_fork_unsafe_key_once, &rpmem_fip_probe_fork_safety); RPMEM_CHECK_FORK(); rpmem_log_args("create", target, pool_set_name, pool_addr, pool_size, *nlanes); if (rpmem_check_args(pool_addr, pool_size, nlanes)) return NULL; RPMEMpool *rpp = rpmem_common_init(target); if (!rpp) goto err_common_init; size_t buff_size = RPMEM_DEF_BUFF_SIZE; struct rpmem_req_attr req = { .pool_size = pool_size, .nlanes = min(*nlanes, Rpmem_max_nlanes), .provider = rpp->provider, .pool_desc = pool_set_name, .buff_size = buff_size, }; struct rpmem_resp_attr resp; int ret = rpmem_obc_create(rpp->obc, &req, &resp, create_attr); if (ret) { RPMEM_LOG(ERR, "!create request failed"); goto err_obc_create; } if (create_attr == NULL || util_is_zeroed(create_attr, sizeof(*create_attr))) rpp->no_headers = 1; rpmem_log_resp("create", &resp); ret = rpmem_common_fip_init(rpp, &req, &resp, pool_addr, pool_size, nlanes, buff_size); if (ret) goto err_fip_init; ret = os_thread_create(&rpp->monitor, NULL, rpmem_monitor_thread, rpp); if (ret) { errno = ret; ERR("!starting monitor thread"); goto err_monitor; } return rpp; err_monitor: rpmem_common_fip_fini(rpp); err_fip_init: rpmem_obc_close(rpp->obc, RPMEM_CLOSE_FLAGS_REMOVE); err_obc_create: rpmem_common_fini(rpp, 0); err_common_init: return NULL; } /* * rpmem_open -- open remote pool on target node * * target -- target node in format [@][:] * pool_set_name -- remote pool set name * pool_addr -- local pool memory address which will be replicated * pool_size -- required pool size * nlanes -- number of lanes * open_attr -- pool attributes, received from remote host */ RPMEMpool * rpmem_open(const char *target, const char *pool_set_name, void *pool_addr, size_t pool_size, unsigned *nlanes, struct rpmem_pool_attr *open_attr) { LOG(3, "target %s, pool_set_name %s, pool_addr %p, pool_size %zu, " "nlanes %p, create_attr %p", target, pool_set_name, pool_addr, pool_size, nlanes, open_attr); os_once(&Rpmem_fork_unsafe_key_once, &rpmem_fip_probe_fork_safety); RPMEM_CHECK_FORK(); rpmem_log_args("open", target, pool_set_name, pool_addr, pool_size, *nlanes); if (rpmem_check_args(pool_addr, pool_size, nlanes)) return NULL; RPMEMpool *rpp = rpmem_common_init(target); if (!rpp) goto err_common_init; size_t buff_size = RPMEM_DEF_BUFF_SIZE; struct rpmem_req_attr req = { .pool_size = pool_size, .nlanes = min(*nlanes, Rpmem_max_nlanes), .provider = rpp->provider, .pool_desc = pool_set_name, .buff_size = buff_size, }; struct rpmem_resp_attr resp; int ret = rpmem_obc_open(rpp->obc, &req, &resp, open_attr); if (ret) { RPMEM_LOG(ERR, "!open request failed"); goto err_obc_create; } if (open_attr == NULL || util_is_zeroed(open_attr, sizeof(*open_attr))) rpp->no_headers = 1; rpmem_log_resp("open", &resp); ret = rpmem_common_fip_init(rpp, &req, &resp, pool_addr, pool_size, nlanes, buff_size); if (ret) goto err_fip_init; ret = os_thread_create(&rpp->monitor, NULL, rpmem_monitor_thread, rpp); if (ret) { errno = ret; ERR("!starting monitor thread"); goto err_monitor; } return rpp; err_monitor: rpmem_common_fip_fini(rpp); err_fip_init: rpmem_obc_close(rpp->obc, 0); err_obc_create: rpmem_common_fini(rpp, 0); err_common_init: return NULL; } /* * rpmem_close -- close remote pool on target node */ int rpmem_close(RPMEMpool *rpp) { LOG(3, "rpp %p", rpp); RPMEM_LOG(INFO, "closing out-of-band connection"); util_fetch_and_or32(&rpp->closing, 1); rpmem_fip_close(rpp->fip); int ret = rpmem_obc_close(rpp->obc, 0); if (ret) ERR("!close request failed"); RPMEM_LOG(NOTICE, "out-of-band connection closed"); rpmem_common_fip_fini(rpp); rpmem_common_fini(rpp, 1); return ret; } /* * rpmem_flush -- flush to target node operation * * rpp -- remote pool handle * offset -- offset in pool * length -- length of flush operation * lane -- lane number * flags -- additional flags */ int rpmem_flush(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane, unsigned flags) { LOG(3, "rpp %p, offset %zu, length %zu, lane %d, flags 0x%x", rpp, offset, length, lane, flags); if (unlikely(rpp->error)) { errno = rpp->error; return -1; } if (flags & RPMEM_FLUSH_FLAGS_MASK) { ERR("invalid flags (0x%x)", flags); errno = EINVAL; return -1; } if (rpp->no_headers == 0 && offset < RPMEM_HDR_SIZE) { ERR("offset (%zu) in pool is less than %d bytes", offset, RPMEM_HDR_SIZE); errno = EINVAL; return -1; } /* * By default use RDMA SEND flush mode which has atomicity * guarantees. For relaxed flush use RDMA WRITE. */ unsigned mode = RPMEM_PERSIST_SEND; if (flags & RPMEM_FLUSH_RELAXED) mode = RPMEM_FLUSH_WRITE; int ret = rpmem_fip_flush(rpp->fip, offset, length, lane, mode); if (unlikely(ret)) { LOG(2, "flush operation failed"); rpp->error = ret; errno = rpp->error; return -1; } return 0; } /* * rpmem_drain -- drain on target node operation * * rpp -- remote pool handle * lane -- lane number * flags -- additional flags */ int rpmem_drain(RPMEMpool *rpp, unsigned lane, unsigned flags) { LOG(3, "rpp %p, lane %d, flags 0x%x", rpp, lane, flags); if (unlikely(rpp->error)) { errno = rpp->error; return -1; } if (flags != 0) { ERR("invalid flags (0x%x)", flags); errno = EINVAL; return -1; } int ret = rpmem_fip_drain(rpp->fip, lane); if (unlikely(ret)) { LOG(2, "drain operation failed"); rpp->error = ret; errno = rpp->error; return -1; } return 0; } /* * rpmem_persist -- persist operation on target node * * rpp -- remote pool handle * offset -- offset in pool * length -- length of persist operation * lane -- lane number */ int rpmem_persist(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane, unsigned flags) { LOG(3, "rpp %p, offset %zu, length %zu, lane %d, flags 0x%x", rpp, offset, length, lane, flags); if (unlikely(rpp->error)) { errno = rpp->error; return -1; } if (flags & RPMEM_PERSIST_FLAGS_MASK) { ERR("invalid flags (0x%x)", flags); errno = EINVAL; return -1; } if (rpp->no_headers == 0 && offset < RPMEM_HDR_SIZE) { ERR("offset (%zu) in pool is less than %d bytes", offset, RPMEM_HDR_SIZE); errno = EINVAL; return -1; } /* * By default use RDMA SEND persist mode which has atomicity * guarantees. For relaxed persist use RDMA WRITE. */ unsigned mode = RPMEM_PERSIST_SEND; if (flags & RPMEM_PERSIST_RELAXED) mode = RPMEM_FLUSH_WRITE; int ret = rpmem_fip_persist(rpp->fip, offset, length, lane, mode); if (unlikely(ret)) { LOG(2, "persist operation failed"); rpp->error = ret; errno = rpp->error; return -1; } return 0; } /* * rpmem_deep_persist -- deep flush operation on target node * * rpp -- remote pool handle * offset -- offset in pool * length -- length of deep flush operation * lane -- lane number */ int rpmem_deep_persist(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane) { LOG(3, "rpp %p, offset %zu, length %zu, lane %d", rpp, offset, length, lane); if (unlikely(rpp->error)) { errno = rpp->error; return -1; } if (offset < RPMEM_HDR_SIZE) { ERR("offset (%zu) in pool is less than %d bytes", offset, RPMEM_HDR_SIZE); errno = EINVAL; return -1; } int ret = rpmem_fip_persist(rpp->fip, offset, length, lane, RPMEM_DEEP_PERSIST); if (unlikely(ret)) { ERR("persist operation failed"); rpp->error = ret; errno = rpp->error; return -1; } return 0; } /* * rpmem_read -- read data from remote pool: * * rpp -- remote pool handle * buff -- output buffer * offset -- offset in pool * length -- length of read operation */ int rpmem_read(RPMEMpool *rpp, void *buff, size_t offset, size_t length, unsigned lane) { LOG(3, "rpp %p, buff %p, offset %zu, length %zu, lane %d", rpp, buff, offset, length, lane); if (unlikely(rpp->error)) { errno = rpp->error; return -1; } if (rpp->no_headers == 0 && offset < RPMEM_HDR_SIZE) LOG(1, "reading from pool at offset (%zu) less than %d bytes", offset, RPMEM_HDR_SIZE); int ret = rpmem_fip_read(rpp->fip, buff, length, offset, lane); if (unlikely(ret)) { errno = ret; ERR("!read operation failed"); rpp->error = ret; return -1; } return 0; } /* * rpmem_set_attr -- overwrite pool attributes on the remote node * * rpp -- remote pool handle * attr -- new pool attributes for the pool on remote node */ int rpmem_set_attr(RPMEMpool *rpp, const struct rpmem_pool_attr *attr) { LOG(3, "rpp %p, attr %p", rpp, attr); if (unlikely(rpp->error)) { errno = rpp->error; return -1; } int ret = rpmem_obc_set_attr(rpp->obc, attr); if (ret) { RPMEM_LOG(ERR, "!set attributes request failed"); } return ret; } /* * rpmem_remove -- remove pool from remote node * * target -- target node in format [@][:] * pool_set_name -- remote pool set name * flags -- bitwise OR of one or more of the following flags: * - RPMEM_REMOVE_FORCE * - RPMEM_REMOVE_POOL_SET */ int rpmem_remove(const char *target, const char *pool_set, int flags) { LOG(3, "target %s, pool_set %s, flags %d", target, pool_set, flags); if (flags & ~(RPMEM_REMOVE_FLAGS_ALL)) { ERR("invalid flags specified"); errno = EINVAL; return -1; } struct rpmem_target_info *info = rpmem_target_parse(target); if (!info) { ERR("!parsing target node address failed"); goto err_target; } const char *argv[5]; argv[0] = "--remove"; argv[1] = pool_set; const char **cur = &argv[2]; if (flags & RPMEM_REMOVE_FORCE) *cur++ = "--force"; if (flags & RPMEM_REMOVE_POOL_SET) *cur++ = "--pool-set"; *cur = NULL; struct rpmem_ssh *ssh = rpmem_ssh_execv(info, argv); if (!ssh) { ERR("!executing ssh command failed"); goto err_ssh_exec; } int ret; ret = rpmem_ssh_monitor(ssh, 0); if (ret) { ERR("!waiting for remote command failed"); goto err_ssh_monitor; } ret = rpmem_ssh_close(ssh); if (ret) { errno = ret; ERR("remote command failed"); goto err_ssh_close; } rpmem_target_free(info); return 0; err_ssh_monitor: rpmem_ssh_close(ssh); err_ssh_close: err_ssh_exec: rpmem_target_free(info); err_target: return -1; } #if FAULT_INJECTION void rpmem_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { return common_inject_fault_at(type, nth, at); } int rpmem_fault_injection_enabled(void) { return common_fault_injection_enabled(); } #endif pmdk-1.8/src/librpmem/rpmem_fip.h0000664000000000000000000000557613615011243015553 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_fip.h -- rpmem libfabric provider module header file */ #ifndef RPMEM_FIP_H #define RPMEM_FIP_H #include #include #include #include #ifdef __cplusplus extern "C" { #endif struct rpmem_fip; struct rpmem_fip_attr { enum rpmem_provider provider; size_t max_wq_size; enum rpmem_persist_method persist_method; void *laddr; size_t size; size_t buff_size; unsigned nlanes; void *raddr; uint64_t rkey; }; struct rpmem_fip *rpmem_fip_init(const char *node, const char *service, struct rpmem_fip_attr *attr, unsigned *nlanes); void rpmem_fip_fini(struct rpmem_fip *fip); int rpmem_fip_connect(struct rpmem_fip *fip); int rpmem_fip_close(struct rpmem_fip *fip); int rpmem_fip_process_start(struct rpmem_fip *fip); int rpmem_fip_process_stop(struct rpmem_fip *fip); int rpmem_fip_flush(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags); int rpmem_fip_drain(struct rpmem_fip *fip, unsigned lane); int rpmem_fip_persist(struct rpmem_fip *fip, size_t offset, size_t len, unsigned lane, unsigned flags); int rpmem_fip_read(struct rpmem_fip *fip, void *buff, size_t len, size_t off, unsigned lane); void rpmem_fip_probe_fork_safety(void); size_t rpmem_fip_get_wq_size(struct rpmem_fip *fip); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/librpmem/rpmem_ssh.c0000664000000000000000000002272613615011243015561 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_ssh.c -- rpmem ssh transport layer source file */ #include #include #include #include #include #include #include #include #include "util.h" #include "os.h" #include "out.h" #include "rpmem_common.h" #include "rpmem_ssh.h" #include "rpmem_cmd.h" #include "rpmem_util.h" #define ERR_BUFF_LEN 4095 /* +1 in order to be sure it is always null-terminated */ static char error_str[ERR_BUFF_LEN + 1]; struct rpmem_ssh { struct rpmem_cmd *cmd; }; /* * get_ssh -- return ssh command name */ static const char * get_ssh(void) { char *cmd = os_getenv(RPMEM_SSH_ENV); if (!cmd) cmd = RPMEM_DEF_SSH; return cmd; } /* * get_user_at_node -- returns string containing user@node */ static char * get_user_at_node(const struct rpmem_target_info *info) { char *user_at_node = NULL; if (info->flags & RPMEM_HAS_USER) { size_t ulen = strlen(info->user); size_t nlen = strlen(info->node); size_t len = ulen + 1 + nlen + 1; user_at_node = malloc(len); if (!user_at_node) goto err_malloc; int ret = snprintf(user_at_node, len, "%s@%s", info->user, info->node); if (ret < 0 || (size_t)ret + 1 != len) goto err_printf; } else { user_at_node = strdup(info->node); if (!user_at_node) goto err_malloc; } return user_at_node; err_printf: free(user_at_node); err_malloc: return NULL; } /* * get_cmd -- return an RPMEM_CMD with appended list of arguments */ static char * get_cmd(const char **argv) { const char *env_cmd = rpmem_util_cmd_get(); char *cmd = strdup(env_cmd); if (!cmd) return NULL; size_t cmd_len = strlen(cmd) + 1; const char *arg; while ((arg = *argv++) != NULL) { size_t len = strlen(arg); size_t new_cmd_len = cmd_len + len + 1; char *tmp = realloc(cmd, new_cmd_len); if (!tmp) goto err; cmd = tmp; /* append the argument to the command */ cmd[cmd_len - 1] = ' '; memcpy(&cmd[cmd_len], arg, len); cmd[cmd_len + len] = '\0'; cmd_len = new_cmd_len; } return cmd; err: free(cmd); return NULL; } /* * valist_to_argv -- convert va_list to argv array */ static const char ** valist_to_argv(va_list args) { const char **argv = malloc(sizeof(const char *)); if (!argv) return NULL; argv[0] = NULL; size_t nargs = 0; const char *arg; while ((arg = va_arg(args, const char *)) != NULL) { nargs++; const char **tmp = realloc(argv, (nargs + 1) * sizeof(const char *)); if (!tmp) goto err; argv = tmp; argv[nargs - 1] = arg; argv[nargs] = NULL; } return argv; err: free(argv); return NULL; } /* * rpmem_ssh_execv -- open ssh connection and run $RPMEMD_CMD with * additional NULL-terminated list of arguments. */ struct rpmem_ssh * rpmem_ssh_execv(const struct rpmem_target_info *info, const char **argv) { struct rpmem_ssh *rps = calloc(1, sizeof(*rps)); if (!rps) goto err_zalloc; char *user_at_node = get_user_at_node(info); if (!user_at_node) goto err_user_node; rps->cmd = rpmem_cmd_init(); if (!rps->cmd) goto err_cmd_init; char *cmd = get_cmd(argv); if (!cmd) goto err_cmd; int ret = rpmem_cmd_push(rps->cmd, get_ssh()); if (ret) goto err_push; if (info->flags & RPMEM_HAS_SERVICE) { /* port number is optional */ ret = rpmem_cmd_push(rps->cmd, "-p"); if (ret) goto err_push; ret = rpmem_cmd_push(rps->cmd, info->service); if (ret) goto err_push; } /* * Disable allocating pseudo-terminal in order to transfer binary * data safely. */ ret = rpmem_cmd_push(rps->cmd, "-T"); if (ret) goto err_push; if (info->flags & RPMEM_FLAGS_USE_IPV4) { ret = rpmem_cmd_push(rps->cmd, "-4"); if (ret) goto err_push; } /* fail if password required for authentication */ ret = rpmem_cmd_push(rps->cmd, "-oBatchMode=yes"); if (ret) goto err_push; ret = rpmem_cmd_push(rps->cmd, user_at_node); if (ret) goto err_push; ret = rpmem_cmd_push(rps->cmd, cmd); if (ret) goto err_push; ret = rpmem_cmd_run(rps->cmd); if (ret) goto err_run; free(user_at_node); free(cmd); return rps; err_run: err_push: free(cmd); err_cmd: rpmem_cmd_fini(rps->cmd); err_cmd_init: free(user_at_node); err_user_node: free(rps); err_zalloc: return NULL; } /* * rpmem_ssh_exec -- open ssh connection and run $RPMEMD_CMD with * additional NULL-terminated list of arguments. */ struct rpmem_ssh * rpmem_ssh_exec(const struct rpmem_target_info *info, ...) { struct rpmem_ssh *ssh; va_list args; va_start(args, info); const char **argv = valist_to_argv(args); if (argv) ssh = rpmem_ssh_execv(info, argv); else ssh = NULL; va_end(args); free(argv); return ssh; } /* * rpmem_ssh_open -- open ssh connection with specified node and wait for status */ struct rpmem_ssh * rpmem_ssh_open(const struct rpmem_target_info *info) { struct rpmem_ssh *ssh = rpmem_ssh_exec(info, NULL); if (!ssh) return NULL; /* * Read initial status from invoked command. * This is for synchronization purposes and to make it possible * to inform client that command's initialization failed. */ int32_t status; int ret = rpmem_ssh_recv(ssh, &status, sizeof(status)); if (ret) { if (ret == 1 || errno == ECONNRESET) ERR("%s", rpmem_ssh_strerror(ssh, errno)); else ERR("!%s", info->node); goto err_recv_status; } if (status) { ERR("%s: unexpected status received -- '%d'", info->node, status); errno = status; goto err_status; } RPMEM_LOG(INFO, "received status: %u", status); return ssh; err_recv_status: err_status: rpmem_ssh_close(ssh); return NULL; } /* * rpmem_ssh_close -- close ssh connection */ int rpmem_ssh_close(struct rpmem_ssh *rps) { int ret, rv; rpmem_cmd_term(rps->cmd); rv = rpmem_cmd_wait(rps->cmd, &ret); if (rv) return rv; rpmem_cmd_fini(rps->cmd); free(rps); if (WIFEXITED(ret)) return WEXITSTATUS(ret); if (WIFSIGNALED(ret)) { ERR("signal received -- %d", WTERMSIG(ret)); return -1; } ERR("exit status -- %d", WEXITSTATUS(ret)); return -1; } /* * rpmem_ssh_send -- send data using ssh transport layer * * The data is encoded using base64. */ int rpmem_ssh_send(struct rpmem_ssh *rps, const void *buff, size_t len) { int ret = rpmem_xwrite(rps->cmd->fd_in, buff, len, MSG_NOSIGNAL); if (ret == 1) { errno = ECONNRESET; } else if (ret < 0) { if (errno == EPIPE) errno = ECONNRESET; } return ret; } /* * rpmem_ssh_recv -- receive data using ssh transport layer * * The received data is decoded using base64. */ int rpmem_ssh_recv(struct rpmem_ssh *rps, void *buff, size_t len) { int ret = rpmem_xread(rps->cmd->fd_out, buff, len, MSG_NOSIGNAL); if (ret == 1) { errno = ECONNRESET; } else if (ret < 0) { if (errno == EPIPE) errno = ECONNRESET; } return ret; } /* * rpmem_ssh_monitor -- check connection state of ssh * * Return value: * 0 - disconnected * 1 - connected * <0 - error */ int rpmem_ssh_monitor(struct rpmem_ssh *rps, int nonblock) { uint32_t buff; int flags = MSG_PEEK; if (nonblock) flags |= MSG_DONTWAIT; int ret = rpmem_xread(rps->cmd->fd_out, &buff, sizeof(buff), flags); if (!ret) { errno = EPROTO; return -1; } if (ret < 0) { if (errno == EAGAIN || errno == EWOULDBLOCK) return 1; else return ret; } return 0; } /* * rpmem_ssh_strerror -- read error using stderr channel */ const char * rpmem_ssh_strerror(struct rpmem_ssh *rps, int oerrno) { size_t len = 0; ssize_t ret; while ((ret = read(rps->cmd->fd_err, error_str + len, ERR_BUFF_LEN - len))) { if (ret < 0) return "reading error string failed"; len += (size_t)ret; } error_str[len] = '\0'; if (len == 0) { if (oerrno) { char buff[UTIL_MAX_ERR_MSG]; util_strerror(oerrno, buff, UTIL_MAX_ERR_MSG); snprintf(error_str, ERR_BUFF_LEN, "%s", buff); } else { snprintf(error_str, ERR_BUFF_LEN, "unknown error"); } } else { /* get rid of new line and carriage return chars */ char *cr = strchr(error_str, '\r'); if (cr) *cr = '\0'; char *nl = strchr(error_str, '\n'); if (nl) *nl = '\0'; } return error_str; } pmdk-1.8/src/librpmem/rpmem_obc.c0000664000000000000000000004103513615011243015521 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_obc.c -- rpmem out-of-band connection client source file */ #include #include #include #include #include #include #include #include "librpmem.h" #include "rpmem.h" #include "rpmem_common.h" #include "rpmem_obc.h" #include "rpmem_proto.h" #include "rpmem_util.h" #include "rpmem_ssh.h" #include "out.h" #include "sys_util.h" #include "util.h" /* * rpmem_obc -- rpmem out-of-band client connection handle */ struct rpmem_obc { struct rpmem_ssh *ssh; }; /* * rpmem_obc_is_connected -- (internal) return non-zero value if client is * connected */ static inline int rpmem_obc_is_connected(struct rpmem_obc *rpc) { return rpc->ssh != NULL; } /* * rpmem_obc_check_ibc_attr -- (internal) check in-band connection * attributes */ static int rpmem_obc_check_ibc_attr(struct rpmem_msg_ibc_attr *ibc) { if (ibc->port == 0 || ibc->port > UINT16_MAX) { ERR("invalid port number received -- %u", ibc->port); errno = EPROTO; return -1; } if (ibc->persist_method != RPMEM_PM_GPSPM && ibc->persist_method != RPMEM_PM_APM) { ERR("invalid persistency method received -- %u", ibc->persist_method); errno = EPROTO; return -1; } return 0; } /* * rpmem_obc_check_port -- (internal) verify target node port number */ static int rpmem_obc_check_port(const struct rpmem_target_info *info) { if (!(info->flags & RPMEM_HAS_SERVICE)) return 0; if (*info->service == '\0') { ERR("invalid port number -- '%s'", info->service); goto err; } errno = 0; char *endptr; long port = strtol(info->service, &endptr, 10); if (errno || *endptr != '\0') { ERR("invalid port number -- '%s'", info->service); goto err; } if (port < 1) { ERR("port number must be positive -- '%s'", info->service); goto err; } if (port > UINT16_MAX) { ERR("port number too large -- '%s'", info->service); goto err; } return 0; err: errno = EINVAL; return -1; } /* * rpmem_obc_close_conn -- (internal) close connection */ static void rpmem_obc_close_conn(struct rpmem_obc *rpc) { rpmem_ssh_close(rpc->ssh); (void) util_fetch_and_and64(&rpc->ssh, 0); } /* * rpmem_obc_init_msg_hdr -- (internal) initialize message header */ static void rpmem_obc_set_msg_hdr(struct rpmem_msg_hdr *hdrp, enum rpmem_msg_type type, size_t size) { hdrp->type = type; hdrp->size = size; } /* * rpmem_obc_set_pool_desc -- (internal) fill the pool descriptor field */ static void rpmem_obc_set_pool_desc(struct rpmem_msg_pool_desc *pool_desc, const char *desc, size_t size) { RPMEM_ASSERT(size <= UINT32_MAX); RPMEM_ASSERT(size > 0); pool_desc->size = (uint32_t)size; memcpy(pool_desc->desc, desc, size); pool_desc->desc[size - 1] = '\0'; } /* * rpmem_obc_alloc_create_msg -- (internal) allocate and fill create request * message */ static struct rpmem_msg_create * rpmem_obc_alloc_create_msg(const struct rpmem_req_attr *req, const struct rpmem_pool_attr *pool_attr, size_t *msg_sizep) { size_t pool_desc_size = strlen(req->pool_desc) + 1; size_t msg_size = sizeof(struct rpmem_msg_create) + pool_desc_size; struct rpmem_msg_create *msg = malloc(msg_size); if (!msg) { ERR("!cannot allocate create request message"); return NULL; } rpmem_obc_set_msg_hdr(&msg->hdr, RPMEM_MSG_TYPE_CREATE, msg_size); msg->c.major = RPMEM_PROTO_MAJOR; msg->c.minor = RPMEM_PROTO_MINOR; msg->c.pool_size = req->pool_size; msg->c.nlanes = req->nlanes; msg->c.provider = req->provider; msg->c.buff_size = req->buff_size; rpmem_obc_set_pool_desc(&msg->pool_desc, req->pool_desc, pool_desc_size); if (pool_attr) { pack_rpmem_pool_attr(pool_attr, &msg->pool_attr); } else { RPMEM_LOG(INFO, "using zeroed pool attributes"); memset(&msg->pool_attr, 0, sizeof(msg->pool_attr)); } *msg_sizep = msg_size; return msg; } /* * rpmem_obc_check_req -- (internal) check request attributes */ static int rpmem_obc_check_req(const struct rpmem_req_attr *req) { if (req->provider >= MAX_RPMEM_PROV) { ERR("invalid provider specified -- %u", req->provider); errno = EINVAL; return -1; } return 0; } /* * rpmem_obj_check_hdr_resp -- (internal) check response message header */ static int rpmem_obc_check_hdr_resp(struct rpmem_msg_hdr_resp *resp, enum rpmem_msg_type type, size_t size) { if (resp->type != type) { ERR("invalid message type received -- %u", resp->type); errno = EPROTO; return -1; } if (resp->size != size) { ERR("invalid message size received -- %lu", resp->size); errno = EPROTO; return -1; } if (resp->status >= MAX_RPMEM_ERR) { ERR("invalid status received -- %u", resp->status); errno = EPROTO; return -1; } if (resp->status) { enum rpmem_err status = (enum rpmem_err)resp->status; ERR("%s", rpmem_util_proto_errstr(status)); errno = rpmem_util_proto_errno(status); return -1; } return 0; } /* * rpmem_obc_check_create_resp -- (internal) check create response message */ static int rpmem_obc_check_create_resp(struct rpmem_msg_create_resp *resp) { if (rpmem_obc_check_hdr_resp(&resp->hdr, RPMEM_MSG_TYPE_CREATE_RESP, sizeof(struct rpmem_msg_create_resp))) return -1; if (rpmem_obc_check_ibc_attr(&resp->ibc)) return -1; return 0; } /* * rpmem_obc_get_res -- (internal) read response attributes */ static void rpmem_obc_get_res(struct rpmem_resp_attr *res, struct rpmem_msg_ibc_attr *ibc) { res->port = (unsigned short)ibc->port; res->rkey = ibc->rkey; res->raddr = ibc->raddr; res->persist_method = (enum rpmem_persist_method)ibc->persist_method; res->nlanes = ibc->nlanes; } /* * rpmem_obc_alloc_open_msg -- (internal) allocate and fill open request message */ static struct rpmem_msg_open * rpmem_obc_alloc_open_msg(const struct rpmem_req_attr *req, const struct rpmem_pool_attr *pool_attr, size_t *msg_sizep) { size_t pool_desc_size = strlen(req->pool_desc) + 1; size_t msg_size = sizeof(struct rpmem_msg_open) + pool_desc_size; struct rpmem_msg_open *msg = malloc(msg_size); if (!msg) { ERR("!cannot allocate open request message"); return NULL; } rpmem_obc_set_msg_hdr(&msg->hdr, RPMEM_MSG_TYPE_OPEN, msg_size); msg->c.major = RPMEM_PROTO_MAJOR; msg->c.minor = RPMEM_PROTO_MINOR; msg->c.pool_size = req->pool_size; msg->c.nlanes = req->nlanes; msg->c.provider = req->provider; msg->c.buff_size = req->buff_size; rpmem_obc_set_pool_desc(&msg->pool_desc, req->pool_desc, pool_desc_size); *msg_sizep = msg_size; return msg; } /* * rpmem_obc_check_open_resp -- (internal) check open response message */ static int rpmem_obc_check_open_resp(struct rpmem_msg_open_resp *resp) { if (rpmem_obc_check_hdr_resp(&resp->hdr, RPMEM_MSG_TYPE_OPEN_RESP, sizeof(struct rpmem_msg_open_resp))) return -1; if (rpmem_obc_check_ibc_attr(&resp->ibc)) return -1; return 0; } /* * rpmem_obc_check_close_resp -- (internal) check close response message */ static int rpmem_obc_check_close_resp(struct rpmem_msg_close_resp *resp) { if (rpmem_obc_check_hdr_resp(&resp->hdr, RPMEM_MSG_TYPE_CLOSE_RESP, sizeof(struct rpmem_msg_close_resp))) return -1; return 0; } /* * rpmem_obc_check_set_attr_resp -- (internal) check set attributes response * message */ static int rpmem_obc_check_set_attr_resp(struct rpmem_msg_set_attr_resp *resp) { if (rpmem_obc_check_hdr_resp(&resp->hdr, RPMEM_MSG_TYPE_SET_ATTR_RESP, sizeof(struct rpmem_msg_set_attr_resp))) return -1; return 0; } /* * rpmem_obc_init -- initialize rpmem obc handle */ struct rpmem_obc * rpmem_obc_init(void) { struct rpmem_obc *rpc = calloc(1, sizeof(*rpc)); if (!rpc) { RPMEM_LOG(ERR, "!allocation of rpmem obc failed"); return NULL; } return rpc; } /* * rpmem_obc_fini -- destroy rpmem obc handle * * This function must be called with connection already closed - after calling * the rpmem_obc_disconnect or after receiving relevant value from * rpmem_obc_monitor. */ void rpmem_obc_fini(struct rpmem_obc *rpc) { free(rpc); } /* * rpmem_obc_connect -- connect to target node * * Connects to target node, the target must be in the following format: * [:]. If the port number is not specified the default * ssh port will be used. The is translated into IP address. * * Returns an error if connection is already established. */ int rpmem_obc_connect(struct rpmem_obc *rpc, const struct rpmem_target_info *info) { if (rpmem_obc_is_connected(rpc)) { errno = EALREADY; goto err_notconnected; } if (rpmem_obc_check_port(info)) goto err_port; rpc->ssh = rpmem_ssh_open(info); if (!rpc->ssh) goto err_ssh_open; return 0; err_ssh_open: err_port: err_notconnected: return -1; } /* * rpmem_obc_disconnect -- close the connection to target node * * Returns error if socket is not connected. */ int rpmem_obc_disconnect(struct rpmem_obc *rpc) { if (rpmem_obc_is_connected(rpc)) { rpmem_obc_close_conn(rpc); return 0; } errno = ENOTCONN; return -1; } /* * rpmem_obc_monitor -- monitor connection with target node * * The nonblock variable indicates whether this function should return * immediately (= 1) or may block (= 0). * * If the function detects that socket was closed by remote peer it is * closed on local side and set to -1, so there is no need to call * rpmem_obc_disconnect function. Please take a look at functions' * descriptions to see which functions cannot be used if the connection * has been already closed. * * This function expects there is no data pending on socket, if any data * is pending this function returns an error and sets errno to EPROTO. * * Return values: * 0 - not connected * 1 - connected * < 0 - error */ int rpmem_obc_monitor(struct rpmem_obc *rpc, int nonblock) { if (!rpmem_obc_is_connected(rpc)) return 0; return rpmem_ssh_monitor(rpc->ssh, nonblock); } /* * rpmem_obc_create -- perform create request operation * * Returns error if connection has not been established yet. */ int rpmem_obc_create(struct rpmem_obc *rpc, const struct rpmem_req_attr *req, struct rpmem_resp_attr *res, const struct rpmem_pool_attr *pool_attr) { if (!rpmem_obc_is_connected(rpc)) { ERR("out-of-band connection not established"); errno = ENOTCONN; goto err_notconnected; } if (rpmem_obc_check_req(req)) goto err_req; size_t msg_size; struct rpmem_msg_create *msg = rpmem_obc_alloc_create_msg(req, pool_attr, &msg_size); if (!msg) goto err_alloc_msg; RPMEM_LOG(INFO, "sending create request message"); rpmem_hton_msg_create(msg); if (rpmem_ssh_send(rpc->ssh, msg, msg_size)) { ERR("!sending create request message failed"); goto err_msg_send; } RPMEM_LOG(NOTICE, "create request message sent"); RPMEM_LOG(INFO, "receiving create request response"); struct rpmem_msg_create_resp resp; if (rpmem_ssh_recv(rpc->ssh, &resp, sizeof(resp))) { ERR("!receiving create request response failed"); goto err_msg_recv; } RPMEM_LOG(NOTICE, "create request response received"); rpmem_ntoh_msg_create_resp(&resp); if (rpmem_obc_check_create_resp(&resp)) goto err_msg_resp; rpmem_obc_get_res(res, &resp.ibc); free(msg); return 0; err_msg_resp: err_msg_recv: err_msg_send: free(msg); err_alloc_msg: err_req: err_notconnected: return -1; } /* * rpmem_obc_open -- perform open request operation * * Returns error if connection is not already established. */ int rpmem_obc_open(struct rpmem_obc *rpc, const struct rpmem_req_attr *req, struct rpmem_resp_attr *res, struct rpmem_pool_attr *pool_attr) { if (!rpmem_obc_is_connected(rpc)) { ERR("out-of-band connection not established"); errno = ENOTCONN; goto err_notconnected; } if (rpmem_obc_check_req(req)) goto err_req; size_t msg_size; struct rpmem_msg_open *msg = rpmem_obc_alloc_open_msg(req, pool_attr, &msg_size); if (!msg) goto err_alloc_msg; RPMEM_LOG(INFO, "sending open request message"); rpmem_hton_msg_open(msg); if (rpmem_ssh_send(rpc->ssh, msg, msg_size)) { ERR("!sending open request message failed"); goto err_msg_send; } RPMEM_LOG(NOTICE, "open request message sent"); RPMEM_LOG(INFO, "receiving open request response"); struct rpmem_msg_open_resp resp; if (rpmem_ssh_recv(rpc->ssh, &resp, sizeof(resp))) { ERR("!receiving open request response failed"); goto err_msg_recv; } RPMEM_LOG(NOTICE, "open request response received"); rpmem_ntoh_msg_open_resp(&resp); if (rpmem_obc_check_open_resp(&resp)) goto err_msg_resp; rpmem_obc_get_res(res, &resp.ibc); if (pool_attr) unpack_rpmem_pool_attr(&resp.pool_attr, pool_attr); free(msg); return 0; err_msg_resp: err_msg_recv: err_msg_send: free(msg); err_alloc_msg: err_req: err_notconnected: return -1; } /* * rpmem_obc_set_attr -- perform set attributes request operation * * Returns error if connection is not already established. */ int rpmem_obc_set_attr(struct rpmem_obc *rpc, const struct rpmem_pool_attr *pool_attr) { if (!rpmem_obc_is_connected(rpc)) { ERR("out-of-band connection not established"); errno = ENOTCONN; goto err_notconnected; } struct rpmem_msg_set_attr msg; rpmem_obc_set_msg_hdr(&msg.hdr, RPMEM_MSG_TYPE_SET_ATTR, sizeof(msg)); if (pool_attr) { memcpy(&msg.pool_attr, pool_attr, sizeof(msg.pool_attr)); } else { RPMEM_LOG(INFO, "using zeroed pool attributes"); memset(&msg.pool_attr, 0, sizeof(msg.pool_attr)); } RPMEM_LOG(INFO, "sending set attributes request message"); rpmem_hton_msg_set_attr(&msg); if (rpmem_ssh_send(rpc->ssh, &msg, sizeof(msg))) { ERR("!sending set attributes request message failed"); goto err_msg_send; } RPMEM_LOG(NOTICE, "set attributes request message sent"); RPMEM_LOG(INFO, "receiving set attributes request response"); struct rpmem_msg_set_attr_resp resp; if (rpmem_ssh_recv(rpc->ssh, &resp, sizeof(resp))) { ERR("!receiving set attributes request response failed"); goto err_msg_recv; } RPMEM_LOG(NOTICE, "set attributes request response received"); rpmem_ntoh_msg_set_attr_resp(&resp); if (rpmem_obc_check_set_attr_resp(&resp)) goto err_msg_resp; return 0; err_msg_resp: err_msg_recv: err_msg_send: err_notconnected: return -1; } /* * rpmem_obc_close -- perform close request operation * * Returns error if connection is not already established. * * NOTE: this function does not close the connection, but sends close request * message to remote node and receives a response. The connection must be * closed using rpmem_obc_disconnect function. */ int rpmem_obc_close(struct rpmem_obc *rpc, int flags) { if (!rpmem_obc_is_connected(rpc)) { errno = ENOTCONN; return -1; } struct rpmem_msg_close msg; rpmem_obc_set_msg_hdr(&msg.hdr, RPMEM_MSG_TYPE_CLOSE, sizeof(msg)); msg.flags = (uint32_t)flags; RPMEM_LOG(INFO, "sending close request message"); rpmem_hton_msg_close(&msg); if (rpmem_ssh_send(rpc->ssh, &msg, sizeof(msg))) { RPMEM_LOG(ERR, "!sending close request failed"); return -1; } RPMEM_LOG(NOTICE, "close request message sent"); RPMEM_LOG(INFO, "receiving close request response"); struct rpmem_msg_close_resp resp; if (rpmem_ssh_recv(rpc->ssh, &resp, sizeof(resp))) { RPMEM_LOG(ERR, "!receiving close request response failed"); return -1; } RPMEM_LOG(NOTICE, "close request response received"); rpmem_ntoh_msg_close_resp(&resp); if (rpmem_obc_check_close_resp(&resp)) return -1; return 0; } pmdk-1.8/src/librpmem/Makefile0000664000000000000000000000444313615011243015054 0ustar rootroot# Copyright 2016-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/librpmem/Makefile -- Makefile for librpmem # include ../common.inc vpath %.c ../rpmem_common ifeq ($(BUILD_RPMEM),y) LIBRARY_NAME = rpmem LIBRARY_SO_VERSION = 1 LIBRARY_VERSION = 0.0 SOURCE = $(COMMON)/alloc.c\ $(COMMON)/os_posix.c\ $(COMMON)/os_thread_posix.c\ $(COMMON)/out.c\ $(COMMON)/util.c\ $(COMMON)/util_posix.c\ librpmem.c\ rpmem.c\ rpmem_obc.c\ rpmem_cmd.c\ rpmem_ssh.c\ rpmem_common.c\ rpmem_util.c\ rpmem_fip_common.c\ rpmem_fip.c else $(info NOTE: Skipping librpmem because $(BUILD_RPMEM_INFO)) endif include ../Makefile.inc ifeq ($(BUILD_RPMEM),y) LIBS += -pthread LIBS += $(shell $(PKG_CONFIG) --libs libfabric) CFLAGS += $(shell $(PKG_CONFIG) --cflags libfabric) CFLAGS += -I. -I../rpmem_common CFLAGS += -DRPMEMC_LOG_RPMEM endif pmdk-1.8/src/librpmem/rpmem_ssh.h0000664000000000000000000000451513615011243015562 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem_ssh.h -- rpmem ssh transport layer header file */ #ifndef RPMEM_SSH_H #define RPMEM_SSH_H 1 #include #ifdef __cplusplus extern "C" { #endif struct rpmem_ssh; struct rpmem_ssh *rpmem_ssh_open(const struct rpmem_target_info *info); struct rpmem_ssh *rpmem_ssh_exec(const struct rpmem_target_info *info, ...); struct rpmem_ssh *rpmem_ssh_execv(const struct rpmem_target_info *info, const char **argv); int rpmem_ssh_close(struct rpmem_ssh *rps); int rpmem_ssh_send(struct rpmem_ssh *rps, const void *buff, size_t len); int rpmem_ssh_recv(struct rpmem_ssh *rps, void *buff, size_t len); int rpmem_ssh_monitor(struct rpmem_ssh *rps, int nonblock); const char *rpmem_ssh_strerror(struct rpmem_ssh *rps, int oerrno); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmem2/0000775000000000000000000000000013615011243013307 5ustar rootrootpmdk-1.8/src/libpmem2/libpmem2.c0000664000000000000000000000526213615011243015167 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmem2.c -- pmem2 entry points for libpmem2 */ #include #include #include "libpmem2.h" #include "pmem2.h" #include "pmemcommon.h" /* * libpmem2_init -- load-time initialization for libpmem2 * * Called automatically by the run-time loader. */ ATTR_CONSTRUCTOR void libpmem2_init(void) { util_init(); out_init(PMEM2_LOG_PREFIX, PMEM2_LOG_LEVEL_VAR, PMEM2_LOG_FILE_VAR, PMEM2_MAJOR_VERSION, PMEM2_MINOR_VERSION); LOG(3, NULL); /* XXX possible pmem2_init placeholder */ } /* * libpmem2_fini -- libpmem2 cleanup routine * * Called automatically when the process terminates. */ ATTR_DESTRUCTOR void libpmem2_fini(void) { LOG(3, NULL); out_fini(); } /* * pmem2_errormsgU -- return last error message */ #ifndef _WIN32 static inline #endif const char * pmem2_errormsgU(void) { return out_get_errormsg(); } #ifndef _WIN32 /* * pmem2_errormsg -- return last error message */ const char * pmem2_errormsg(void) { return pmem2_errormsgU(); } #else /* * pmem2_errormsgW -- return last error message as wchar_t */ const wchar_t * pmem2_errormsgW(void) { return out_get_errormsgW(); } #endif pmdk-1.8/src/libpmem2/config_windows.c0000664000000000000000000000556013615011243016500 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * config_windows.c -- windows specific pmem2_config implementation */ #include #include "libpmem2.h" #include "out.h" #include "config.h" /* * pmem2_config_set_fd -- sets fd in config struct */ int pmem2_config_set_fd(struct pmem2_config *cfg, int fd) { if (fd < 0) { cfg->handle = INVALID_HANDLE_VALUE; return 0; } HANDLE handle = (HANDLE)_get_osfhandle(fd); if (handle == INVALID_HANDLE_VALUE) { /* * _get_osfhandle aborts in an error case, so technically * this is dead code. But according to MSDN it is * setting an errno on failure, so we can return it in case of * "windows magic" happen and this function "accidentally" * will not abort. */ ERR("!_get_osfhandle"); return -errno; } return pmem2_config_set_handle(cfg, handle); } /* * pmem2_config_set_handle -- convert fd to handle */ int pmem2_config_set_handle(struct pmem2_config *cfg, HANDLE handle) { if (handle == INVALID_HANDLE_VALUE) { cfg->handle = INVALID_HANDLE_VALUE; return 0; } BY_HANDLE_FILE_INFORMATION not_used; if (!GetFileInformationByHandle(handle, ¬_used)) { ERR("!!GetFileInformationByHandle"); /* XXX: convert last error to errno */ return PMEM2_E_INVALID_ARG; } /* XXX: winapi doesn't provide option to get open flags from HANDLE */ cfg->handle = handle; return 0; } pmdk-1.8/src/libpmem2/libpmem2_main.c0000664000000000000000000000371513615011243016174 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmem2_main.c -- entry point for libpmem2.dll */ void libpmem2_init(void); void libpmem2_fini(void); int APIENTRY DllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID lpReserved) { switch (dwReason) { case DLL_PROCESS_ATTACH: libpmem2_init(); break; case DLL_THREAD_ATTACH: case DLL_THREAD_DETACH: break; case DLL_PROCESS_DETACH: libpmem2_fini(); break; } return TRUE; } pmdk-1.8/src/libpmem2/libpmem2.link.in0000664000000000000000000000451213615011243016304 0ustar rootroot# # Copyright 2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # src/libpmem2.link -- linker link file for libpmem2 # LIBPMEM2_1.0 { global: pmem2_config_new; pmem2_config_set_fd; pmem2_config_delete; pmem2_config_set_offset; pmem2_config_set_length; pmem2_config_set_sharing; pmem2_config_set_protection; pmem2_config_use_anonymous_mapping; pmem2_config_set_address; pmem2_config_set_required_store_granularity; pmem2_map; pmem2_unmap; pmem2_map_get_address; pmem2_map_get_size; pmem2_map_get_store_granularity; pmem2_get_persist_fn; pmem2_get_flush_fn; pmem2_get_drain_fn; pmem2_get_memmove_fn; pmem2_get_memcpy_fn; pmem2_get_memset_fn; pmem2_get_device_id; pmem2_get_device_usc; pmem2_badblock_iterator_new; pmem2_badblock_next; pmem2_badblock_iterator_delete; pmem2_badblock_clear; pmem2_errormsg; local: *; }; pmdk-1.8/src/libpmem2/pmem2_utils.h0000664000000000000000000000335413615011243015725 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmem2_utils.h -- libpmem2 utilities functions */ #ifndef PMEM2_UTILS_H #define PMEM2_UTILS_H 1 void *pmem2_malloc(size_t size, int *err); #endif /* PMEM2_UTILS_H */ pmdk-1.8/src/libpmem2/pmem2_utils.c0000664000000000000000000000370113615011243015714 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmem2_utils.c -- libpmem2 utilities functions */ #include #include "alloc.h" #include "libpmem2.h" #include "out.h" #include "pmem2_utils.h" /* * pmem2_malloc -- allocate buffer and handle error */ void * pmem2_malloc(size_t size, int *err) { void *ptr = Malloc(size); *err = 0; if (ptr == NULL) { ERR("!malloc(%zu)", size); *err = -errno; } return ptr; } pmdk-1.8/src/libpmem2/libpmem2.rc0000664000000000000000000000344313615011243015350 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmem2.rc -- libpmem2 resource file */ #include #define FILE_NAME "libpmem2.dll" #define DESCRIPTION "libpmem2 - persistent memory support library v2 (EXPERIMENTAL)" #define TYPE VFT_DLL #include pmdk-1.8/src/libpmem2/pmem2.h0000664000000000000000000000366413615011243014511 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmem2.h -- internal definitions for libpmem2 */ #ifndef PMEM2_H #define PMEM2_H #include "libpmem2.h" #ifdef __cplusplus extern "C" { #endif #define PMEM2_MAJOR_VERSION 0 #define PMEM2_MINOR_VERSION 0 #define PMEM2_LOG_PREFIX "libpmem2" #define PMEM2_LOG_LEVEL_VAR "PMEM2_LOG_LEVEL" #define PMEM2_LOG_FILE_VAR "PMEM2_LOG_FILE" #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmem2/config_posix.c0000664000000000000000000000416413615011243016147 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * config_linux.c -- linux specific pmem2_config implementation */ #include #include #include "out.h" #include "config.h" /* * pmem2_config_set_fd -- sets fd in config struct */ int pmem2_config_set_fd(struct pmem2_config *cfg, int fd) { if (fd < 0) { cfg->fd = INVALID_FD; return 0; } int flags = fcntl(fd, F_GETFL); if (flags == -1) { ERR("!fcntl"); return -errno; } if ((flags & O_ACCMODE) == O_WRONLY) { ERR("fd must be open with O_RDONLY or O_RDRW"); return PMEM2_E_INVALID_HANDLE; } cfg->fd = fd; return 0; } pmdk-1.8/src/libpmem2/pmem2.c0000664000000000000000000001023113615011243014470 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmem2.c -- pmem2 entry points for libpmem2 */ #include "libpmem2.h" #include "pmem2.h" int pmem2_config_set_offset(struct pmem2_config *cfg, size_t offset) { return PMEM2_E_NOSUPP; } int pmem2_config_set_length(struct pmem2_config *cfg, size_t length) { return PMEM2_E_NOSUPP; } int pmem2_config_set_sharing(struct pmem2_config *cfg, unsigned type) { return PMEM2_E_NOSUPP; } int pmem2_config_set_protection(struct pmem2_config *cfg, unsigned flag) { return PMEM2_E_NOSUPP; } int pmem2_config_use_anonymous_mapping(struct pmem2_config *cfg, unsigned on) { return PMEM2_E_NOSUPP; } int pmem2_config_set_address(struct pmem2_config *cfg, unsigned type, void *addr) { return PMEM2_E_NOSUPP; } int pmem2_config_set_required_store_granularity(struct pmem2_config *cfg, enum pmem2_granularity g) { return PMEM2_E_NOSUPP; } int pmem2_map(const struct pmem2_config *cfg, struct pmem2_map **map) { return PMEM2_E_NOSUPP; } int pmem2_unmap(struct pmem2_map **map) { return PMEM2_E_NOSUPP; } void * pmem2_map_get_address(struct pmem2_map *map) { return NULL; } size_t pmem2_map_get_size(struct pmem2_map *map) { return 0; } enum pmem2_granularity pmem2_map_get_store_granularity(struct pmem2_map *map) { return PMEM2_GRANULARITY_PAGE; } pmem2_persist_fn * pmem2_get_persist_fn(struct pmem2_map *map) { return NULL; } pmem2_flush_fn * pmem2_get_flush_fn(struct pmem2_map *map) { return NULL; } pmem2_drain_fn * pmem2_get_drain_fn(struct pmem2_map *map) { return NULL; } pmem2_memmove_fn * pmem2_get_memmove_fn(struct pmem2_map *map) { return NULL; } pmem2_memcpy_fn * pmem2_get_memcpy_fn(struct pmem2_map *map) { return NULL; } pmem2_memset_fn * pmem2_get_memset_fn(struct pmem2_map *map) { return NULL; } #ifndef _WIN32 int pmem2_get_device_id(const struct pmem2_config *cfg, char *id, size_t *len) { return PMEM2_E_NOSUPP; } #else int pmem2_get_device_idW(const struct pmem2_config *cfg, wchar_t *id, size_t *len) { return PMEM2_E_NOSUPP; } int pmem2_get_device_idU(const struct pmem2_config *cfg, char *id, size_t *len) { return PMEM2_E_NOSUPP; } #endif int pmem2_get_device_usc(const struct pmem2_config *cfg, uint64_t *usc) { return PMEM2_E_NOSUPP; } int pmem2_badblock_iterator_new(const struct pmem2_config *cfg, struct pmem2_badblock_iterator **pbb) { return PMEM2_E_NOSUPP; } int pmem2_badblock_next(struct pmem2_badblock_iterator *pbb, struct pmem2_badblock *bb) { return PMEM2_E_NOSUPP; } void pmem2_badblock_iterator_delete( struct pmem2_badblock_iterator **pbb) { } int pmem2_badblock_clear(const struct pmem2_config *cfg, const struct pmem2_badblock *bb) { return PMEM2_E_NOSUPP; } pmdk-1.8/src/libpmem2/libpmem2.def0000664000000000000000000000453413615011243015504 0ustar rootroot;;;; Begin Copyright Notice ; ; Copyright 2019, Intel Corporation ; ; Redistribution and use in source and binary forms, with or without ; modification, are permitted provided that the following conditions ; are met: ; ; * Redistributions of source code must retain the above copyright ; notice, this list of conditions and the following disclaimer. ; ; * Redistributions in binary form must reproduce the above copyright ; notice, this list of conditions and the following disclaimer in ; the documentation and/or other materials provided with the ; distribution. ; ; * Neither the name of the copyright holder nor the names of its ; contributors may be used to endorse or promote products derived ; from this software without specific prior written permission. ; ; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ; "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT ; LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ; A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT ; OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, ; SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT ; LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, ; DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY ; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT ; (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE ; OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ; ;;;; End Copyright Notice LIBRARY libpmem2 VERSION 1.0 EXPORTS pmem2_config_new pmem2_config_set_fd pmem2_config_set_handle pmem2_config_delete pmem2_config_set_offset pmem2_config_set_length pmem2_config_set_sharing pmem2_config_set_protection pmem2_config_use_anonymous_mapping pmem2_config_set_address pmem2_config_set_required_store_granularity pmem2_map pmem2_unmap pmem2_map_get_address pmem2_map_get_size pmem2_map_get_store_granularity pmem2_get_persist_fn pmem2_get_flush_fn pmem2_get_drain_fn pmem2_get_memmove_fn pmem2_get_memcpy_fn pmem2_get_memset_fn pmem2_get_device_idW pmem2_get_device_idU pmem2_get_device_usc pmem2_badblock_iterator_new pmem2_badblock_next pmem2_badblock_iterator_delete pmem2_badblock_clear pmem2_errormsgU pmem2_errormsgW DllMain pmdk-1.8/src/libpmem2/libpmem2.vcxproj0000664000000000000000000000756413615011243016447 0ustar rootroot Debug x64 Release x64 {901f04db-e1a5-4a41-8b81-9d31c19acd59} {F596C36C-5C96-4F08-B420-8908AF500954} DynamicLibrary libpmem2 libpmem2 en-US 14.0 10.0.10240.0 10.0.16299.0 DynamicLibrary true v140 DynamicLibrary false false v140 pmdk-1.8/src/libpmem2/libpmem2.vcxproj.filters0000664000000000000000000000503613615011243020106 0ustar rootroot {16473205-8f12-4d4c-b1e9-e14ea3013e70} h {17275273-f923-45ff-9b7e-b2ea76561168} c;def {6c8580b3-4650-42ca-9589-ec45a8f4278c} Source Files Source Files Source Files Source Files Source Files\common Source Files\common Source Files\common Source Files\common Source Files\common Source Files\common Source Files\common Source Files Header Files Header Files Header Files Header Files Source Files Source Files pmdk-1.8/src/libpmem2/config.h0000664000000000000000000000411313615011243014724 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * config.h -- internal definitions for pmem2_config */ #ifndef PMEM2_CONFIG_H #define PMEM2_CONFIG_H #include "libpmem2.h" #define INVALID_FD (-1) struct pmem2_config { #ifdef _WIN32 HANDLE handle; #else int fd; /* a source file descriptor for the designed mapping */ #endif /* offset from the beginning of the file */ size_t offset; size_t length; /* length of the mapping */ size_t alignment; /* required alignment of the mapping */ }; void config_init(struct pmem2_config *cfg); #endif /* PMEM2_CONFIG_H */ pmdk-1.8/src/libpmem2/map.h0000664000000000000000000000356513615011243014246 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map.h -- internal definitions for libpmem2 */ #ifndef PMEM2_MAP_H #define PMEM2_MAP_H #include #ifdef __cplusplus extern "C" { #endif struct pmem2_map { void *addr; /* base address */ size_t length; /* effective length of the mapping */ }; #ifdef __cplusplus } #endif #endif /* map.h */ pmdk-1.8/src/libpmem2/Makefile0000664000000000000000000000366413615011243014760 0ustar rootroot# Copyright 2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/libpmem2/Makefile -- Makefile for libpmem2 # include ../common.inc LIBRARY_NAME = pmem2 LIBRARY_SO_VERSION = 1 LIBRARY_VERSION = 0.0 SOURCE =\ $(COMMON)/alloc.c\ $(COMMON)/os_posix.c\ $(COMMON)/os_thread_posix.c\ $(COMMON)/out.c\ $(COMMON)/util.c\ $(COMMON)/util_posix.c\ libpmem2.c\ config.c\ config_posix.c\ pmem2.c\ pmem2_utils.c include ../Makefile.inc CFLAGS += -I. LIBS += -pthread pmdk-1.8/src/libpmem2/config.c0000664000000000000000000000463213615011243014725 0ustar rootroot/* * Copyright 2019-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * config.c -- pmem2_config implementation */ #include #include "alloc.h" #include "config.h" #include "libpmem2.h" #include "out.h" #include "pmem2.h" #include "pmem2_utils.h" /* * config_init -- (internal) initialize cfg structure. */ void config_init(struct pmem2_config *cfg) { #ifdef _WIN32 cfg->handle = INVALID_HANDLE_VALUE; #else cfg->fd = INVALID_FD; #endif cfg->offset = 0; cfg->length = 0; cfg->alignment = 0; } /* * pmem2_config_new -- allocates and initialize cfg structure. */ int pmem2_config_new(struct pmem2_config **cfg) { int ret; *cfg = pmem2_malloc(sizeof(**cfg), &ret); if (ret) return ret; ASSERTne(cfg, NULL); config_init(*cfg); return 0; } /* * pmem2_config_delete -- deallocate cfg structure. */ int pmem2_config_delete(struct pmem2_config **cfg) { Free(*cfg); *cfg = NULL; return 0; } pmdk-1.8/src/libpmemlog/0000775000000000000000000000000013615011243013727 5ustar rootrootpmdk-1.8/src/libpmemlog/libpmemlog_main.c0000664000000000000000000000421013615011243017223 0ustar rootroot/* * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmemlog_main.c -- entry point for libpmemlog.dll * * XXX - This is a placeholder. All the library initialization/cleanup * that is done in library ctors/dtors, as well as TLS initialization * should be moved here. */ void libpmemlog_init(void); void libpmemlog_fini(void); int APIENTRY DllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID lpReserved) { switch (dwReason) { case DLL_PROCESS_ATTACH: libpmemlog_init(); break; case DLL_THREAD_ATTACH: case DLL_THREAD_DETACH: break; case DLL_PROCESS_DETACH: libpmemlog_fini(); break; } return TRUE; } pmdk-1.8/src/libpmemlog/libpmemlog.def0000664000000000000000000000406513615011243016543 0ustar rootroot;;;; Begin Copyright Notice ; ; Copyright (c) 2016-2018, Intel Corporation ; ; Redistribution and use in source and binary forms, with or without ; modification, are permitted provided that the following conditions ; are met: ; ; * Redistributions of source code must retain the above copyright ; notice, this list of conditions and the following disclaimer. ; ; * Redistributions in binary form must reproduce the above copyright ; notice, this list of conditions and the following disclaimer in ; the documentation and/or other materials provided with the ; distribution. ; ; * Neither the name of Intel Corporation nor the names of its ; contributors may be used to endorse or promote products derived ; from this software without specific prior written permission. ; ; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ; "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT ; LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ; A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT ; OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, ; SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT ; LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, ; DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY ; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT ; (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE ; OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ; ;;;; End Copyright Notice LIBRARY libpmemlog VERSION 1.0 EXPORTS pmemlog_check_versionU pmemlog_check_versionW pmemlog_ctl_execU; pmemlog_ctl_execW; pmemlog_ctl_getU; pmemlog_ctl_getW; pmemlog_ctl_setU; pmemlog_ctl_setW; pmemlog_set_funcs pmemlog_errormsgU pmemlog_errormsgW pmemlog_createU pmemlog_createW pmemlog_openU pmemlog_openW pmemlog_close pmemlog_checkU pmemlog_checkW pmemlog_nbyte pmemlog_append pmemlog_appendv pmemlog_rewind pmemlog_tell pmemlog_walk DllMain pmdk-1.8/src/libpmemlog/libpmemlog.link.in0000664000000000000000000000370313615011243017345 0ustar rootroot# # Copyright 2014-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # src/libpmemlog.link -- linker link file for libpmemlog # LIBPMEMLOG_1.0 { global: pmemlog_check_version; pmemlog_ctl_exec; pmemlog_ctl_get; pmemlog_ctl_set; pmemlog_set_funcs; pmemlog_errormsg; pmemlog_create; pmemlog_open; pmemlog_close; pmemlog_check; pmemlog_nbyte; pmemlog_append; pmemlog_appendv; pmemlog_tell; pmemlog_rewind; pmemlog_walk; fault_injection; local: *; }; pmdk-1.8/src/libpmemlog/libpmemlog.c0000664000000000000000000001327013615011243016225 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmemlog.c -- pmem entry points for libpmemlog */ #include #include #include "libpmemlog.h" #include "ctl_global.h" #include "pmemcommon.h" #include "log.h" /* * The variable from which the config is directly loaded. The string * cannot contain any comments or extraneous white characters. */ #define LOG_CONFIG_ENV_VARIABLE "PMEMLOG_CONF" /* * The variable that points to a config file from which the config is loaded. */ #define LOG_CONFIG_FILE_ENV_VARIABLE "PMEMLOG_CONF_FILE" /* * log_ctl_init_and_load -- (static) initializes CTL and loads configuration * from env variable and file */ static int log_ctl_init_and_load(PMEMlogpool *plp) { LOG(3, "plp %p", plp); if (plp != NULL && (plp->ctl = ctl_new()) == NULL) { LOG(2, "!ctl_new"); return -1; } char *env_config = os_getenv(LOG_CONFIG_ENV_VARIABLE); if (env_config != NULL) { if (ctl_load_config_from_string(plp ? plp->ctl : NULL, plp, env_config) != 0) { LOG(2, "unable to parse config stored in %s " "environment variable", LOG_CONFIG_ENV_VARIABLE); goto err; } } char *env_config_file = os_getenv(LOG_CONFIG_FILE_ENV_VARIABLE); if (env_config_file != NULL && env_config_file[0] != '\0') { if (ctl_load_config_from_file(plp ? plp->ctl : NULL, plp, env_config_file) != 0) { LOG(2, "unable to parse config stored in %s " "file (from %s environment variable)", env_config_file, LOG_CONFIG_FILE_ENV_VARIABLE); goto err; } } return 0; err: if (plp) ctl_delete(plp->ctl); return -1; } /* * log_init -- load-time initialization for log * * Called automatically by the run-time loader. */ ATTR_CONSTRUCTOR void libpmemlog_init(void) { ctl_global_register(); if (log_ctl_init_and_load(NULL)) FATAL("error: %s", pmemlog_errormsg()); common_init(PMEMLOG_LOG_PREFIX, PMEMLOG_LOG_LEVEL_VAR, PMEMLOG_LOG_FILE_VAR, PMEMLOG_MAJOR_VERSION, PMEMLOG_MINOR_VERSION); LOG(3, NULL); } /* * libpmemlog_fini -- libpmemlog cleanup routine * * Called automatically when the process terminates. */ ATTR_DESTRUCTOR void libpmemlog_fini(void) { LOG(3, NULL); common_fini(); } /* * pmemlog_check_versionU -- see if lib meets application version requirements */ #ifndef _WIN32 static inline #endif const char * pmemlog_check_versionU(unsigned major_required, unsigned minor_required) { LOG(3, "major_required %u minor_required %u", major_required, minor_required); if (major_required != PMEMLOG_MAJOR_VERSION) { ERR("libpmemlog major version mismatch (need %u, found %u)", major_required, PMEMLOG_MAJOR_VERSION); return out_get_errormsg(); } if (minor_required > PMEMLOG_MINOR_VERSION) { ERR("libpmemlog minor version mismatch (need %u, found %u)", minor_required, PMEMLOG_MINOR_VERSION); return out_get_errormsg(); } return NULL; } #ifndef _WIN32 /* * pmemlog_check_version -- see if lib meets application version requirements */ const char * pmemlog_check_version(unsigned major_required, unsigned minor_required) { return pmemlog_check_versionU(major_required, minor_required); } #else /* * pmemlog_check_versionW -- see if lib meets application version requirements */ const wchar_t * pmemlog_check_versionW(unsigned major_required, unsigned minor_required) { if (pmemlog_check_versionU(major_required, minor_required) != NULL) return out_get_errormsgW(); else return NULL; } #endif /* * pmemlog_set_funcs -- allow overriding libpmemlog's call to malloc, etc. */ void pmemlog_set_funcs( void *(*malloc_func)(size_t size), void (*free_func)(void *ptr), void *(*realloc_func)(void *ptr, size_t size), char *(*strdup_func)(const char *s)) { LOG(3, NULL); util_set_alloc_funcs(malloc_func, free_func, realloc_func, strdup_func); } /* * pmemlog_errormsgU -- return last error message */ #ifndef _WIN32 static inline #endif const char * pmemlog_errormsgU(void) { return out_get_errormsg(); } #ifndef _WIN32 /* * pmemlog_errormsg -- return last error message */ const char * pmemlog_errormsg(void) { return pmemlog_errormsgU(); } #else /* * pmemlog_errormsgW -- return last error message as wchar_t */ const wchar_t * pmemlog_errormsgW(void) { return out_get_errormsgW(); } #endif pmdk-1.8/src/libpmemlog/libpmemlog.vcxproj.filters0000664000000000000000000001342513615011243021147 0ustar rootroot Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files {49cfa2b4-cfcb-4c02-928a-c04d1cceffb8} {ac09c2fe-a24b-4a86-8763-d4e06d996ef3} Source Files Source Files pmdk-1.8/src/libpmemlog/log.c0000664000000000000000000005133613615011243014664 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * log.c -- log memory pool entry points for libpmem */ #include #include #include #include #include #include #include #include #include #include #include "libpmem.h" #include "libpmemlog.h" #include "ctl_global.h" #include "os.h" #include "set.h" #include "out.h" #include "log.h" #include "mmap.h" #include "sys_util.h" #include "util_pmem.h" #include "valgrind_internal.h" static const struct pool_attr Log_create_attr = { LOG_HDR_SIG, LOG_FORMAT_MAJOR, LOG_FORMAT_FEAT_DEFAULT, {0}, {0}, {0}, {0}, {0} }; static const struct pool_attr Log_open_attr = { LOG_HDR_SIG, LOG_FORMAT_MAJOR, LOG_FORMAT_FEAT_CHECK, {0}, {0}, {0}, {0}, {0} }; /* * log_descr_create -- (internal) create log memory pool descriptor */ static void log_descr_create(PMEMlogpool *plp, size_t poolsize) { LOG(3, "plp %p poolsize %zu", plp, poolsize); ASSERTeq(poolsize % Pagesize, 0); /* create required metadata */ plp->start_offset = htole64(roundup(sizeof(*plp), LOG_FORMAT_DATA_ALIGN)); plp->end_offset = htole64(poolsize); plp->write_offset = plp->start_offset; /* store non-volatile part of pool's descriptor */ util_persist(plp->is_pmem, &plp->start_offset, 3 * sizeof(uint64_t)); } /* * log_descr_check -- (internal) validate log memory pool descriptor */ static int log_descr_check(PMEMlogpool *plp, size_t poolsize) { LOG(3, "plp %p poolsize %zu", plp, poolsize); struct pmemlog hdr = *plp; log_convert2h(&hdr); if ((hdr.start_offset != roundup(sizeof(*plp), LOG_FORMAT_DATA_ALIGN)) || (hdr.end_offset != poolsize) || (hdr.start_offset > hdr.end_offset)) { ERR("wrong start/end offsets " "(start: %" PRIu64 " end: %" PRIu64 "), " "pool size %zu", hdr.start_offset, hdr.end_offset, poolsize); errno = EINVAL; return -1; } if ((hdr.write_offset > hdr.end_offset) || (hdr.write_offset < hdr.start_offset)) { ERR("wrong write offset (start: %" PRIu64 " end: %" PRIu64 " write: %" PRIu64 ")", hdr.start_offset, hdr.end_offset, hdr.write_offset); errno = EINVAL; return -1; } LOG(3, "start: %" PRIu64 ", end: %" PRIu64 ", write: %" PRIu64 "", hdr.start_offset, hdr.end_offset, hdr.write_offset); return 0; } /* * log_runtime_init -- (internal) initialize log memory pool runtime data */ static int log_runtime_init(PMEMlogpool *plp, int rdonly) { LOG(3, "plp %p rdonly %d", plp, rdonly); /* remove volatile part of header */ VALGRIND_REMOVE_PMEM_MAPPING(&plp->addr, sizeof(struct pmemlog) - sizeof(struct pool_hdr) - 3 * sizeof(uint64_t)); /* * Use some of the memory pool area for run-time info. This * run-time state is never loaded from the file, it is always * created here, so no need to worry about byte-order. */ plp->rdonly = rdonly; if ((plp->rwlockp = Malloc(sizeof(*plp->rwlockp))) == NULL) { ERR("!Malloc for a RW lock"); return -1; } util_rwlock_init(plp->rwlockp); /* * If possible, turn off all permissions on the pool header page. * * The prototype PMFS doesn't allow this when large pages are in * use. It is not considered an error if this fails. */ RANGE_NONE(plp->addr, sizeof(struct pool_hdr), plp->is_dev_dax); /* the rest should be kept read-only (debug version only) */ RANGE_RO((char *)plp->addr + sizeof(struct pool_hdr), plp->size - sizeof(struct pool_hdr), plp->is_dev_dax); return 0; } /* * pmemlog_createU -- create a log memory pool */ #ifndef _WIN32 static inline #endif PMEMlogpool * pmemlog_createU(const char *path, size_t poolsize, mode_t mode) { LOG(3, "path %s poolsize %zu mode %d", path, poolsize, mode); struct pool_set *set; struct pool_attr adj_pool_attr = Log_create_attr; /* force set SDS feature */ if (SDS_at_create) adj_pool_attr.features.incompat |= POOL_FEAT_SDS; else adj_pool_attr.features.incompat &= ~POOL_FEAT_SDS; if (util_pool_create(&set, path, poolsize, PMEMLOG_MIN_POOL, PMEMLOG_MIN_PART, &adj_pool_attr, NULL, REPLICAS_DISABLED) != 0) { LOG(2, "cannot create pool or pool set"); return NULL; } ASSERT(set->nreplicas > 0); struct pool_replica *rep = set->replica[0]; PMEMlogpool *plp = rep->part[0].addr; VALGRIND_REMOVE_PMEM_MAPPING(&plp->addr, sizeof(struct pmemlog) - ((uintptr_t)&plp->addr - (uintptr_t)&plp->hdr)); plp->addr = plp; plp->size = rep->repsize; plp->set = set; plp->is_pmem = rep->is_pmem; plp->is_dev_dax = rep->part[0].is_dev_dax; /* is_dev_dax implies is_pmem */ ASSERT(!plp->is_dev_dax || plp->is_pmem); /* create pool descriptor */ log_descr_create(plp, rep->repsize); /* initialize runtime parts */ if (log_runtime_init(plp, 0) != 0) { ERR("pool initialization failed"); goto err; } if (util_poolset_chmod(set, mode)) goto err; util_poolset_fdclose(set); LOG(3, "plp %p", plp); return plp; err: LOG(4, "error clean up"); int oerrno = errno; util_poolset_close(set, DELETE_CREATED_PARTS); errno = oerrno; return NULL; } #ifndef _WIN32 /* * pmemlog_create -- create a log memory pool */ PMEMlogpool * pmemlog_create(const char *path, size_t poolsize, mode_t mode) { return pmemlog_createU(path, poolsize, mode); } #else /* * pmemlog_createW -- create a log memory pool */ PMEMlogpool * pmemlog_createW(const wchar_t *path, size_t poolsize, mode_t mode) { char *upath = util_toUTF8(path); if (upath == NULL) return NULL; PMEMlogpool *ret = pmemlog_createU(upath, poolsize, mode); util_free_UTF8(upath); return ret; } #endif /* * log_open_common -- (internal) open a log memory pool * * This routine does all the work, but takes a cow flag so internal * calls can map a read-only pool if required. */ static PMEMlogpool * log_open_common(const char *path, unsigned flags) { LOG(3, "path %s flags 0x%x", path, flags); struct pool_set *set; if (util_pool_open(&set, path, PMEMLOG_MIN_PART, &Log_open_attr, NULL, NULL, flags) != 0) { LOG(2, "cannot open pool or pool set"); return NULL; } ASSERT(set->nreplicas > 0); struct pool_replica *rep = set->replica[0]; PMEMlogpool *plp = rep->part[0].addr; VALGRIND_REMOVE_PMEM_MAPPING(&plp->addr, sizeof(struct pmemlog) - ((uintptr_t)&plp->addr - (uintptr_t)&plp->hdr)); plp->addr = plp; plp->size = rep->repsize; plp->set = set; plp->is_pmem = rep->is_pmem; plp->is_dev_dax = rep->part[0].is_dev_dax; /* is_dev_dax implies is_pmem */ ASSERT(!plp->is_dev_dax || plp->is_pmem); if (set->nreplicas > 1) { errno = ENOTSUP; ERR("!replicas not supported"); goto err; } /* validate pool descriptor */ if (log_descr_check(plp, rep->repsize) != 0) { LOG(2, "descriptor check failed"); goto err; } /* initialize runtime parts */ if (log_runtime_init(plp, set->rdonly) != 0) { ERR("pool initialization failed"); goto err; } util_poolset_fdclose(set); LOG(3, "plp %p", plp); return plp; err: LOG(4, "error clean up"); int oerrno = errno; util_poolset_close(set, DO_NOT_DELETE_PARTS); errno = oerrno; return NULL; } /* * pmemlog_openU -- open an existing log memory pool */ #ifndef _WIN32 static inline #endif PMEMlogpool * pmemlog_openU(const char *path) { LOG(3, "path %s", path); return log_open_common(path, COW_at_open ? POOL_OPEN_COW : 0); } #ifndef _WIN32 /* * pmemlog_open -- open an existing log memory pool */ PMEMlogpool * pmemlog_open(const char *path) { return pmemlog_openU(path); } #else /* * pmemlog_openW -- open an existing log memory pool */ PMEMlogpool * pmemlog_openW(const wchar_t *path) { char *upath = util_toUTF8(path); if (upath == NULL) return NULL; PMEMlogpool *ret = pmemlog_openU(upath); util_free_UTF8(upath); return ret; } #endif /* * pmemlog_close -- close a log memory pool */ void pmemlog_close(PMEMlogpool *plp) { LOG(3, "plp %p", plp); util_rwlock_destroy(plp->rwlockp); Free((void *)plp->rwlockp); util_poolset_close(plp->set, DO_NOT_DELETE_PARTS); } /* * pmemlog_nbyte -- return usable size of a log memory pool */ size_t pmemlog_nbyte(PMEMlogpool *plp) { LOG(3, "plp %p", plp); util_rwlock_rdlock(plp->rwlockp); size_t size = le64toh(plp->end_offset) - le64toh(plp->start_offset); LOG(4, "plp %p nbyte %zu", plp, size); util_rwlock_unlock(plp->rwlockp); return size; } /* * log_persist -- (internal) persist data, then metadata * * On entry, the write lock should be held. */ static void log_persist(PMEMlogpool *plp, uint64_t new_write_offset) { uint64_t old_write_offset = le64toh(plp->write_offset); size_t length = new_write_offset - old_write_offset; /* unprotect the log space range (debug version only) */ RANGE_RW((char *)plp->addr + old_write_offset, length, plp->is_dev_dax); /* persist the data */ if (plp->is_pmem) pmem_drain(); /* data already flushed */ else pmem_msync((char *)plp->addr + old_write_offset, length); /* protect the log space range (debug version only) */ RANGE_RO((char *)plp->addr + old_write_offset, length, plp->is_dev_dax); /* unprotect the pool descriptor (debug version only) */ RANGE_RW((char *)plp->addr + sizeof(struct pool_hdr), LOG_FORMAT_DATA_ALIGN, plp->is_dev_dax); /* write the metadata */ plp->write_offset = htole64(new_write_offset); /* persist the metadata */ if (plp->is_pmem) pmem_persist(&plp->write_offset, sizeof(plp->write_offset)); else pmem_msync(&plp->write_offset, sizeof(plp->write_offset)); /* set the write-protection again (debug version only) */ RANGE_RO((char *)plp->addr + sizeof(struct pool_hdr), LOG_FORMAT_DATA_ALIGN, plp->is_dev_dax); } /* * pmemlog_append -- add data to a log memory pool */ int pmemlog_append(PMEMlogpool *plp, const void *buf, size_t count) { int ret = 0; LOG(3, "plp %p buf %p count %zu", plp, buf, count); if (plp->rdonly) { ERR("can't append to read-only log"); errno = EROFS; return -1; } util_rwlock_wrlock(plp->rwlockp); /* get the current values */ uint64_t end_offset = le64toh(plp->end_offset); uint64_t write_offset = le64toh(plp->write_offset); if (write_offset >= end_offset) { /* no space left */ errno = ENOSPC; ERR("!pmemlog_append"); ret = -1; goto end; } /* make sure we don't write past the available space */ if (count > (end_offset - write_offset)) { errno = ENOSPC; ERR("!pmemlog_append"); ret = -1; goto end; } char *data = plp->addr; /* * unprotect the log space range, where the new data will be stored * (debug version only) */ RANGE_RW(&data[write_offset], count, plp->is_dev_dax); if (plp->is_pmem) pmem_memcpy_nodrain(&data[write_offset], buf, count); else memcpy(&data[write_offset], buf, count); /* protect the log space range (debug version only) */ RANGE_RO(&data[write_offset], count, plp->is_dev_dax); write_offset += count; /* persist the data and the metadata */ log_persist(plp, write_offset); end: util_rwlock_unlock(plp->rwlockp); return ret; } /* * pmemlog_appendv -- add gathered data to a log memory pool */ int pmemlog_appendv(PMEMlogpool *plp, const struct iovec *iov, int iovcnt) { LOG(3, "plp %p iovec %p iovcnt %d", plp, iov, iovcnt); int ret = 0; int i; if (iovcnt < 0) { errno = EINVAL; ERR("iovcnt is less than zero: %d", iovcnt); return -1; } if (plp->rdonly) { ERR("can't append to read-only log"); errno = EROFS; return -1; } util_rwlock_wrlock(plp->rwlockp); /* get the current values */ uint64_t end_offset = le64toh(plp->end_offset); uint64_t write_offset = le64toh(plp->write_offset); if (write_offset >= end_offset) { /* no space left */ errno = ENOSPC; ERR("!pmemlog_appendv"); ret = -1; goto end; } char *data = plp->addr; uint64_t count = 0; char *buf; /* calculate required space */ for (i = 0; i < iovcnt; ++i) count += iov[i].iov_len; /* check if there is enough free space */ if (count > (end_offset - write_offset)) { errno = ENOSPC; ret = -1; goto end; } /* append the data */ for (i = 0; i < iovcnt; ++i) { buf = iov[i].iov_base; count = iov[i].iov_len; /* * unprotect the log space range, where the new data will be * stored (debug version only) */ RANGE_RW(&data[write_offset], count, plp->is_dev_dax); if (plp->is_pmem) pmem_memcpy_nodrain(&data[write_offset], buf, count); else memcpy(&data[write_offset], buf, count); /* * protect the log space range (debug version only) */ RANGE_RO(&data[write_offset], count, plp->is_dev_dax); write_offset += count; } /* persist the data and the metadata */ log_persist(plp, write_offset); end: util_rwlock_unlock(plp->rwlockp); return ret; } /* * pmemlog_tell -- return current write point in a log memory pool */ long long pmemlog_tell(PMEMlogpool *plp) { LOG(3, "plp %p", plp); util_rwlock_rdlock(plp->rwlockp); ASSERT(le64toh(plp->write_offset) >= le64toh(plp->start_offset)); long long wp = (long long)(le64toh(plp->write_offset) - le64toh(plp->start_offset)); LOG(4, "write offset %lld", wp); util_rwlock_unlock(plp->rwlockp); return wp; } /* * pmemlog_rewind -- discard all data, resetting a log memory pool to empty */ void pmemlog_rewind(PMEMlogpool *plp) { LOG(3, "plp %p", plp); if (plp->rdonly) { ERR("can't rewind read-only log"); errno = EROFS; return; } util_rwlock_wrlock(plp->rwlockp); /* unprotect the pool descriptor (debug version only) */ RANGE_RW((char *)plp->addr + sizeof(struct pool_hdr), LOG_FORMAT_DATA_ALIGN, plp->is_dev_dax); plp->write_offset = plp->start_offset; if (plp->is_pmem) pmem_persist(&plp->write_offset, sizeof(uint64_t)); else pmem_msync(&plp->write_offset, sizeof(uint64_t)); /* set the write-protection again (debug version only) */ RANGE_RO((char *)plp->addr + sizeof(struct pool_hdr), LOG_FORMAT_DATA_ALIGN, plp->is_dev_dax); util_rwlock_unlock(plp->rwlockp); } /* * pmemlog_walk -- walk through all data in a log memory pool * * chunksize of 0 means process_chunk gets called once for all data * as a single chunk. */ void pmemlog_walk(PMEMlogpool *plp, size_t chunksize, int (*process_chunk)(const void *buf, size_t len, void *arg), void *arg) { LOG(3, "plp %p chunksize %zu", plp, chunksize); /* * We are assuming that the walker doesn't change the data it's reading * in place. We prevent everyone from changing the data behind our back * until we are done with processing it. */ util_rwlock_rdlock(plp->rwlockp); char *data = plp->addr; uint64_t write_offset = le64toh(plp->write_offset); uint64_t data_offset = le64toh(plp->start_offset); size_t len; if (chunksize == 0) { /* most common case: process everything at once */ len = write_offset - data_offset; LOG(3, "length %zu", len); (*process_chunk)(&data[data_offset], len, arg); } else { /* * Walk through the complete record, chunk by chunk. * The callback returns 0 to terminate the walk. */ while (data_offset < write_offset) { len = MIN(chunksize, write_offset - data_offset); if (!(*process_chunk)(&data[data_offset], len, arg)) break; data_offset += chunksize; } } util_rwlock_unlock(plp->rwlockp); } /* * pmemlog_checkU -- log memory pool consistency check * * Returns true if consistent, zero if inconsistent, -1/error if checking * cannot happen due to other errors. */ #ifndef _WIN32 static inline #endif int pmemlog_checkU(const char *path) { LOG(3, "path \"%s\"", path); PMEMlogpool *plp = log_open_common(path, POOL_OPEN_COW); if (plp == NULL) return -1; /* errno set by log_open_common() */ int consistent = 1; /* validate pool descriptor */ uint64_t hdr_start = le64toh(plp->start_offset); uint64_t hdr_end = le64toh(plp->end_offset); uint64_t hdr_write = le64toh(plp->write_offset); if (hdr_start != roundup(sizeof(*plp), LOG_FORMAT_DATA_ALIGN)) { ERR("wrong value of start_offset"); consistent = 0; } if (hdr_end != plp->size) { ERR("wrong value of end_offset"); consistent = 0; } if (hdr_start > hdr_end) { ERR("start_offset greater than end_offset"); consistent = 0; } if (hdr_start > hdr_write) { ERR("start_offset greater than write_offset"); consistent = 0; } if (hdr_write > hdr_end) { ERR("write_offset greater than end_offset"); consistent = 0; } pmemlog_close(plp); if (consistent) LOG(4, "pool consistency check OK"); return consistent; } #ifndef _WIN32 /* * pmemlog_check -- log memory pool consistency check * * Returns true if consistent, zero if inconsistent, -1/error if checking * cannot happen due to other errors. */ int pmemlog_check(const char *path) { return pmemlog_checkU(path); } #else /* * pmemlog_checkW -- log memory pool consistency check */ int pmemlog_checkW(const wchar_t *path) { char *upath = util_toUTF8(path); if (upath == NULL) return -1; int ret = pmemlog_checkU(upath); util_free_UTF8(upath); return ret; } #endif /* * pmemlog_ctl_getU -- programmatically executes a read ctl query */ #ifndef _WIN32 static inline #endif int pmemlog_ctl_getU(PMEMlogpool *plp, const char *name, void *arg) { LOG(3, "plp %p name %s arg %p", plp, name, arg); return ctl_query(plp == NULL ? NULL : plp->ctl, plp, CTL_QUERY_PROGRAMMATIC, name, CTL_QUERY_READ, arg); } /* * pmemblk_ctl_setU -- programmatically executes a write ctl query */ #ifndef _WIN32 static inline #endif int pmemlog_ctl_setU(PMEMlogpool *plp, const char *name, void *arg) { LOG(3, "plp %p name %s arg %p", plp, name, arg); return ctl_query(plp == NULL ? NULL : plp->ctl, plp, CTL_QUERY_PROGRAMMATIC, name, CTL_QUERY_WRITE, arg); } /* * pmemlog_ctl_execU -- programmatically executes a runnable ctl query */ #ifndef _WIN32 static inline #endif int pmemlog_ctl_execU(PMEMlogpool *plp, const char *name, void *arg) { LOG(3, "plp %p name %s arg %p", plp, name, arg); return ctl_query(plp == NULL ? NULL : plp->ctl, plp, CTL_QUERY_PROGRAMMATIC, name, CTL_QUERY_RUNNABLE, arg); } #ifndef _WIN32 /* * pmemlog_ctl_get -- programmatically executes a read ctl query */ int pmemlog_ctl_get(PMEMlogpool *plp, const char *name, void *arg) { return pmemlog_ctl_getU(plp, name, arg); } /* * pmemlog_ctl_set -- programmatically executes a write ctl query */ int pmemlog_ctl_set(PMEMlogpool *plp, const char *name, void *arg) { return pmemlog_ctl_setU(plp, name, arg); } /* * pmemlog_ctl_exec -- programmatically executes a runnable ctl query */ int pmemlog_ctl_exec(PMEMlogpool *plp, const char *name, void *arg) { return pmemlog_ctl_execU(plp, name, arg); } #else /* * pmemlog_ctl_getW -- programmatically executes a read ctl query */ int pmemlog_ctl_getW(PMEMlogpool *plp, const wchar_t *name, void *arg) { char *uname = util_toUTF8(name); if (uname == NULL) return -1; int ret = pmemlog_ctl_getU(plp, uname, arg); util_free_UTF8(uname); return ret; } /* * pmemlog_ctl_setW -- programmatically executes a write ctl query */ int pmemlog_ctl_setW(PMEMlogpool *plp, const wchar_t *name, void *arg) { char *uname = util_toUTF8(name); if (uname == NULL) return -1; int ret = pmemlog_ctl_setU(plp, uname, arg); util_free_UTF8(uname); return ret; } /* * pmemlog_ctl_execW -- programmatically executes a runnable ctl query */ int pmemlog_ctl_execW(PMEMlogpool *plp, const wchar_t *name, void *arg) { char *uname = util_toUTF8(name); if (uname == NULL) return -1; int ret = pmemlog_ctl_execU(plp, uname, arg); util_free_UTF8(uname); return ret; } #endif #if FAULT_INJECTION void pmemlog_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { common_inject_fault_at(type, nth, at); } int pmemlog_fault_injection_enabled(void) { return common_fault_injection_enabled(); } #endif pmdk-1.8/src/libpmemlog/log.h0000664000000000000000000001037313615011243014665 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * log.h -- internal definitions for libpmem log module */ #ifndef LOG_H #define LOG_H 1 #include #include #include #include "ctl.h" #include "util.h" #include "os_thread.h" #include "pool_hdr.h" #include "page_size.h" #ifdef __cplusplus extern "C" { #endif #include "alloc.h" #include "fault_injection.h" #define PMEMLOG_LOG_PREFIX "libpmemlog" #define PMEMLOG_LOG_LEVEL_VAR "PMEMLOG_LOG_LEVEL" #define PMEMLOG_LOG_FILE_VAR "PMEMLOG_LOG_FILE" /* attributes of the log memory pool format for the pool header */ #define LOG_HDR_SIG "PMEMLOG" /* must be 8 bytes including '\0' */ #define LOG_FORMAT_MAJOR 1 #define LOG_FORMAT_FEAT_DEFAULT \ {POOL_FEAT_COMPAT_DEFAULT, POOL_FEAT_INCOMPAT_DEFAULT, 0x0000} #define LOG_FORMAT_FEAT_CHECK \ {POOL_FEAT_COMPAT_VALID, POOL_FEAT_INCOMPAT_VALID, 0x0000} static const features_t log_format_feat_default = LOG_FORMAT_FEAT_DEFAULT; struct pmemlog { struct pool_hdr hdr; /* memory pool header */ /* root info for on-media format... */ uint64_t start_offset; /* start offset of the usable log space */ uint64_t end_offset; /* maximum offset of the usable log space */ uint64_t write_offset; /* current write point for the log */ /* some run-time state, allocated out of memory pool... */ void *addr; /* mapped region */ size_t size; /* size of mapped region */ int is_pmem; /* true if pool is PMEM */ int rdonly; /* true if pool is opened read-only */ os_rwlock_t *rwlockp; /* pointer to RW lock */ int is_dev_dax; /* true if mapped on device dax */ struct ctl *ctl; /* top level node of the ctl tree structure */ struct pool_set *set; /* pool set info */ }; /* data area starts at this alignment after the struct pmemlog above */ #define LOG_FORMAT_DATA_ALIGN ((uintptr_t)PMEM_PAGESIZE) /* * log_convert2h -- convert pmemlog structure to host byte order */ static inline void log_convert2h(struct pmemlog *plp) { plp->start_offset = le64toh(plp->start_offset); plp->end_offset = le64toh(plp->end_offset); plp->write_offset = le64toh(plp->write_offset); } /* * log_convert2le -- convert pmemlog structure to LE byte order */ static inline void log_convert2le(struct pmemlog *plp) { plp->start_offset = htole64(plp->start_offset); plp->end_offset = htole64(plp->end_offset); plp->write_offset = htole64(plp->write_offset); } #if FAULT_INJECTION void pmemlog_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at); int pmemlog_fault_injection_enabled(void); #else static inline void pmemlog_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { abort(); } static inline int pmemlog_fault_injection_enabled(void) { return 0; } #endif #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmemlog/libpmemlog.vcxproj0000664000000000000000000001357713615011243017510 0ustar rootroot Debug x64 Release x64 {9e9e3d25-2139-4a5d-9200-18148ddead45} {901f04db-e1a5-4a41-8b81-9d31c19acd59} {0B1818EB-BDC8-4865-964F-DB8BF05CFD86} DynamicLibrary libpmemlog libpmemlog en-US 14.0 10.0.16299.0 10.0.10240.0 DynamicLibrary true v140 DynamicLibrary false false v140 pmdk-1.8/src/libpmemlog/Makefile0000664000000000000000000000350113615011243015366 0ustar rootroot# Copyright 2014-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/libpmemlog/Makefile -- Makefile for libpmemlog # LIBRARY_NAME = pmemlog LIBRARY_SO_VERSION = 1 LIBRARY_VERSION = 0.0 include ../common/pmemcommon.inc SOURCE +=\ libpmemlog.c\ log.c include ../Makefile.inc CFLAGS += $(LIBNDCTL_CFLAGS) LIBS += -pthread -lpmem $(LIBNDCTL_LIBS) pmdk-1.8/src/libpmemlog/libpmemlog.rc0000664000000000000000000000343213615011243016406 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmemlog.rc -- libpmemlog resource file */ #include #define FILE_NAME "libpmemlog.dll" #define DESCRIPTION "libpmemlog - persistent memory resident log file" #define TYPE VFT_DLL #include pmdk-1.8/src/.clang-format0000664000000000000000000000163013615011243014153 0ustar rootrootBasedOnStyle: LLVM IndentWidth: 8 UseTab: Always BreakBeforeBraces: Custom BraceWrapping: AfterClass: false AfterControlStatement: false AfterEnum: false AfterFunction: true AfterNamespace: true AfterObjCDeclaration: false AfterStruct: false AfterUnion: false BeforeCatch: false BeforeElse: false IndentBraces: false AllowShortIfStatementsOnASingleLine: false IndentCaseLabels: false AlwaysBreakAfterDefinitionReturnType: true SpaceBeforeParens: ControlStatements SpacesBeforeTrailingComments: 1 SpacesInCStyleCastParentheses: false SpacesInContainerLiterals: false PointerAlignment: Right ContinuationIndentWidth: 8 AlignOperands: false IndentCaseLabels: true ConstructorInitializerAllOnOneLineOrOnePerLine: true AlwaysBreakTemplateDeclarations: true AccessModifierOffset: -8 AllowShortBlocksOnASingleLine: false AllowShortFunctionsOnASingleLine: false BreakStringLiterals: false pmdk-1.8/src/libvmem/0000775000000000000000000000000013615011243013233 5ustar rootrootpmdk-1.8/src/libvmem/README.md0000664000000000000000000000012613615011243014511 0ustar rootrootThis library has been moved to a [separate repository](https://github.com/pmem/vmem). pmdk-1.8/src/tools/0000775000000000000000000000000013615011243012740 5ustar rootrootpmdk-1.8/src/tools/pmempool/0000775000000000000000000000000013615011243014570 5ustar rootrootpmdk-1.8/src/tools/pmempool/info_log.c0000664000000000000000000001255713615011243016542 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * info_log.c -- pmempool info command source file for log pool */ #include #include #include #include #include "common.h" #include "output.h" #include "info.h" /* * info_log_data -- print used data from log pool */ static int info_log_data(struct pmem_info *pip, int v, struct pmemlog *plp) { if (!outv_check(v)) return 0; uint64_t size_used = plp->write_offset - plp->start_offset; if (size_used == 0) return 0; uint8_t *addr = pool_set_file_map(pip->pfile, plp->start_offset); if (addr == MAP_FAILED) { warn("%s", pip->file_name); outv_err("cannot read pmem log data\n"); return -1; } if (pip->args.log.walk == 0) { outv_title(v, "PMEMLOG data"); struct range *curp = NULL; PMDK_LIST_FOREACH(curp, &pip->args.ranges.head, next) { uint8_t *ptr = addr + curp->first; if (curp->last >= size_used) curp->last = size_used - 1; uint64_t count = curp->last - curp->first + 1; outv_hexdump(v, ptr, count, curp->first + plp->start_offset, 1); size_used -= count; if (!size_used) break; } } else { /* * Walk through used data with fixed chunk size * passed by user. */ uint64_t nchunks = size_used / pip->args.log.walk; outv_title(v, "PMEMLOG data [chunks: total = %lu size = %ld]", nchunks, pip->args.log.walk); struct range *curp = NULL; PMDK_LIST_FOREACH(curp, &pip->args.ranges.head, next) { uint64_t i; for (i = curp->first; i <= curp->last && i < nchunks; i++) { outv(v, "Chunk %10lu:\n", i); outv_hexdump(v, addr + i * pip->args.log.walk, pip->args.log.walk, plp->start_offset + i * pip->args.log.walk, 1); } } } return 0; } /* * info_logs_stats -- print log type pool statistics */ static void info_log_stats(struct pmem_info *pip, int v, struct pmemlog *plp) { uint64_t size_total = plp->end_offset - plp->start_offset; uint64_t size_used = plp->write_offset - plp->start_offset; uint64_t size_avail = size_total - size_used; if (size_total == 0) return; double perc_used = (double)size_used / (double)size_total * 100.0; double perc_avail = 100.0 - perc_used; outv_title(v, "PMEM LOG Statistics"); outv_field(v, "Total", "%s", out_get_size_str(size_total, pip->args.human)); outv_field(v, "Available", "%s [%s]", out_get_size_str(size_avail, pip->args.human), out_get_percentage(perc_avail)); outv_field(v, "Used", "%s [%s]", out_get_size_str(size_used, pip->args.human), out_get_percentage(perc_used)); } /* * info_log_descriptor -- print pmemlog descriptor and return 1 if * write offset is valid */ static int info_log_descriptor(struct pmem_info *pip, int v, struct pmemlog *plp) { outv_title(v, "PMEM LOG Header"); /* dump pmemlog header without pool_hdr */ outv_hexdump(pip->args.vhdrdump, (uint8_t *)plp + sizeof(plp->hdr), sizeof(*plp) - sizeof(plp->hdr), sizeof(plp->hdr), 1); log_convert2h(plp); int write_offset_valid = plp->write_offset >= plp->start_offset && plp->write_offset <= plp->end_offset; outv_field(v, "Start offset", "0x%lx", plp->start_offset); outv_field(v, "Write offset", "0x%lx [%s]", plp->write_offset, write_offset_valid ? "OK":"ERROR"); outv_field(v, "End offset", "0x%lx", plp->end_offset); return write_offset_valid; } /* * pmempool_info_log -- print information about log type pool */ int pmempool_info_log(struct pmem_info *pip) { int ret = 0; struct pmemlog *plp = malloc(sizeof(struct pmemlog)); if (!plp) err(1, "Cannot allocate memory for pmemlog structure"); if (pmempool_info_read(pip, plp, sizeof(struct pmemlog), 0)) { outv_err("cannot read pmemlog header\n"); free(plp); return -1; } if (info_log_descriptor(pip, VERBOSE_DEFAULT, plp)) { info_log_stats(pip, pip->args.vstats, plp); ret = info_log_data(pip, pip->args.vdata, plp); } free(plp); return ret; } pmdk-1.8/src/tools/pmempool/convert.c0000664000000000000000000000704313615011243016420 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * convert.c -- pmempool convert command source file */ #include #include #include #include #include #include #include "convert.h" #include "os.h" #ifdef _WIN32 static const char *delimiter = ";"; static const char *convert_bin = "\\pmdk-convert.exe"; #else static const char *delimiter = ":"; static const char *convert_bin = "/pmdk-convert"; #endif // _WIN32 static int pmempool_convert_get_path(char *p, size_t max_len) { char *path = strdup(os_getenv("PATH")); if (!path) { perror("strdup"); return -1; } char *dir = strtok(path, delimiter); while (dir) { size_t length = strlen(dir) + strlen(convert_bin) + 1; if (length > max_len) { fprintf(stderr, "very long dir in PATH, ignoring\n"); continue; } strcpy(p, dir); strcat(p, convert_bin); if (os_access(p, F_OK) == 0) { free(path); return 0; } dir = strtok(NULL, delimiter); } free(path); return -1; } /* * pmempool_convert_help -- print help message for convert command. This is * help message from pmdk-convert tool. */ void pmempool_convert_help(const char *appname) { char path[4096]; if (pmempool_convert_get_path(path, sizeof(path))) { fprintf(stderr, "pmdk-convert is not installed. Please install it.\n"); exit(1); } char *args[] = { path, "-h", NULL }; os_execv(path, args); perror("execv"); exit(1); } /* * pmempool_convert_func -- main function for convert command. * It invokes pmdk-convert tool. */ int pmempool_convert_func(const char *appname, int argc, char *argv[]) { char path[4096]; if (pmempool_convert_get_path(path, sizeof(path))) { fprintf(stderr, "pmdk-convert is not installed. Please install it.\n"); exit(1); } char **args = malloc(((size_t)argc + 1) * sizeof(*args)); if (!args) { perror("malloc"); exit(1); } args[0] = path; for (int i = 1; i < argc; ++i) args[i] = argv[i]; args[argc] = NULL; os_execv(args[0], args); perror("execv"); free(args); exit(1); } pmdk-1.8/src/tools/pmempool/pmempool.rc0000664000000000000000000000721213615011243016750 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmempool.rc -- pmempool resource file */ #include <windows.h> #define FILE_NAME "pmempool.exe" #define DESCRIPTION "pmempool - Persistent Memory Pool Management Tool" #define TYPE VFT_APP MAINICON ICON "../../../res/PMDK.ico" #include <common.rc>pmdk-1.8/src/tools/pmempool/dump.c0000664000000000000000000002257413615011243015713 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * create.c -- pmempool create command source file */ #include #include #include #include #include #include #include #include #include "common.h" #include "dump.h" #include "output.h" #include "os.h" #include "libpmemblk.h" #include "libpmemlog.h" #define VERBOSE_DEFAULT 1 /* * pmempool_dump -- context and arguments for dump command */ struct pmempool_dump { char *fname; char *ofname; char *range; FILE *ofh; int hex; uint64_t bsize; struct ranges ranges; size_t chunksize; uint64_t chunkcnt; }; /* * pmempool_dump_default -- default arguments and context values */ static const struct pmempool_dump pmempool_dump_default = { .fname = NULL, .ofname = NULL, .range = NULL, .ofh = NULL, .hex = 1, .bsize = 0, .chunksize = 0, .chunkcnt = 0, }; /* * long_options -- command line options */ static const struct option long_options[] = { {"output", required_argument, NULL, 'o' | OPT_ALL}, {"binary", no_argument, NULL, 'b' | OPT_ALL}, {"range", required_argument, NULL, 'r' | OPT_ALL}, {"chunk", required_argument, NULL, 'c' | OPT_LOG}, {"help", no_argument, NULL, 'h' | OPT_ALL}, {NULL, 0, NULL, 0 }, }; /* * help_str -- string for help message */ static const char * const help_str = "Dump user data from pool\n" "\n" "Available options:\n" " -o, --output output file name\n" " -b, --binary dump data in binary format\n" " -r, --range range of bytes/blocks/data chunks\n" " -c, --chunk size of chunk for PMEMLOG pool\n" " -h, --help display this help and exit\n" "\n" "For complete documentation see %s-dump(1) manual page.\n" ; /* * print_usage -- print application usage short description */ static void print_usage(const char *appname) { printf("Usage: %s dump [] \n", appname); } /* * print_version -- print version string */ static void print_version(const char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * pmempool_dump_help -- print help message for dump command */ void pmempool_dump_help(const char *appname) { print_usage(appname); print_version(appname); printf(help_str, appname); } /* * pmempool_dump_log_process_chunk -- callback for pmemlog_walk */ static int pmempool_dump_log_process_chunk(const void *buf, size_t len, void *arg) { struct pmempool_dump *pdp = (struct pmempool_dump *)arg; if (len == 0) return 0; struct range *curp = NULL; if (pdp->chunksize) { PMDK_LIST_FOREACH(curp, &pdp->ranges.head, next) { if (pdp->chunkcnt >= curp->first && pdp->chunkcnt <= curp->last && pdp->chunksize <= len) { if (pdp->hex) { outv_hexdump(VERBOSE_DEFAULT, buf, pdp->chunksize, pdp->chunksize * pdp->chunkcnt, 0); } else { if (fwrite(buf, pdp->chunksize, 1, pdp->ofh) != 1) err(1, "%s", pdp->ofname); } } } pdp->chunkcnt++; } else { PMDK_LIST_FOREACH(curp, &pdp->ranges.head, next) { if (curp->first >= len) continue; uint8_t *ptr = (uint8_t *)buf + curp->first; if (curp->last >= len) curp->last = len - 1; uint64_t count = curp->last - curp->first + 1; if (pdp->hex) { outv_hexdump(VERBOSE_DEFAULT, ptr, count, curp->first, 0); } else { if (fwrite(ptr, count, 1, pdp->ofh) != 1) err(1, "%s", pdp->ofname); } } } return 1; } /* * pmempool_dump_parse_range -- parse range passed by arguments */ static int pmempool_dump_parse_range(struct pmempool_dump *pdp, size_t max) { struct range entire; memset(&entire, 0, sizeof(entire)); entire.last = max; if (util_parse_ranges(pdp->range, &pdp->ranges, entire)) { outv_err("invalid range value specified" " -- '%s'\n", pdp->range); return -1; } if (PMDK_LIST_EMPTY(&pdp->ranges.head)) util_ranges_add(&pdp->ranges, entire); return 0; } /* * pmempool_dump_log -- dump data from pmem log pool */ static int pmempool_dump_log(struct pmempool_dump *pdp) { PMEMlogpool *plp = pmemlog_open(pdp->fname); if (!plp) { warn("%s", pdp->fname); return -1; } os_off_t off = pmemlog_tell(plp); if (off < 0) { warn("%s", pdp->fname); pmemlog_close(plp); return -1; } if (off == 0) goto end; size_t max = (size_t)off - 1; if (pdp->chunksize) max /= pdp->chunksize; if (pmempool_dump_parse_range(pdp, max)) return -1; pdp->chunkcnt = 0; pmemlog_walk(plp, pdp->chunksize, pmempool_dump_log_process_chunk, pdp); end: pmemlog_close(plp); return 0; } /* * pmempool_dump_blk -- dump data from pmem blk pool */ static int pmempool_dump_blk(struct pmempool_dump *pdp) { PMEMblkpool *pbp = pmemblk_open(pdp->fname, pdp->bsize); if (!pbp) { warn("%s", pdp->fname); return -1; } if (pmempool_dump_parse_range(pdp, pmemblk_nblock(pbp) - 1)) return -1; uint8_t *buff = malloc(pdp->bsize); if (!buff) err(1, "Cannot allocate memory for pmemblk block buffer"); int ret = 0; uint64_t i; struct range *curp = NULL; PMDK_LIST_FOREACH(curp, &pdp->ranges.head, next) { assert((os_off_t)curp->last >= 0); for (i = curp->first; i <= curp->last; i++) { if (pmemblk_read(pbp, buff, (os_off_t)i)) { ret = -1; outv_err("reading block number %lu " "failed\n", i); break; } if (pdp->hex) { uint64_t offset = i * pdp->bsize; outv_hexdump(VERBOSE_DEFAULT, buff, pdp->bsize, offset, 0); } else { if (fwrite(buff, pdp->bsize, 1, pdp->ofh) != 1) { warn("write"); ret = -1; break; } } } } free(buff); pmemblk_close(pbp); return ret; } static const struct option_requirement option_requirements[] = { { 0, 0, 0} }; /* * pmempool_dump_func -- dump command main function */ int pmempool_dump_func(const char *appname, int argc, char *argv[]) { struct pmempool_dump pd = pmempool_dump_default; PMDK_LIST_INIT(&pd.ranges.head); out_set_vlevel(VERBOSE_DEFAULT); struct options *opts = util_options_alloc(long_options, sizeof(long_options) / sizeof(long_options[0]), option_requirements); int ret = 0; long long chunksize; int opt; while ((opt = util_options_getopt(argc, argv, "ho:br:c:", opts)) != -1) { switch (opt) { case 'o': pd.ofname = optarg; break; case 'b': pd.hex = 0; break; case 'r': pd.range = optarg; break; case 'c': chunksize = atoll(optarg); if (chunksize <= 0) { outv_err("invalid chunk size specified '%s'\n", optarg); exit(EXIT_FAILURE); } pd.chunksize = (size_t)chunksize; break; case 'h': pmempool_dump_help(appname); exit(EXIT_SUCCESS); default: print_usage(appname); exit(EXIT_FAILURE); } } if (optind < argc) { pd.fname = argv[optind]; } else { print_usage(appname); exit(EXIT_FAILURE); } if (pd.ofname == NULL) { /* use standard output by default */ pd.ofh = stdout; } else { pd.ofh = os_fopen(pd.ofname, "wb"); if (!pd.ofh) { warn("%s", pd.ofname); exit(EXIT_FAILURE); } } /* set output stream - stdout or file passed by -o option */ out_set_stream(pd.ofh); struct pmem_pool_params params; /* parse pool type and block size for pmem blk pool */ pmem_pool_parse_params(pd.fname, ¶ms, 1); ret = util_options_verify(opts, params.type); if (ret) goto out; switch (params.type) { case PMEM_POOL_TYPE_LOG: ret = pmempool_dump_log(&pd); break; case PMEM_POOL_TYPE_BLK: pd.bsize = params.blk.bsize; ret = pmempool_dump_blk(&pd); break; case PMEM_POOL_TYPE_OBJ: outv_err("%s: PMEMOBJ pool not supported\n", pd.fname); ret = -1; goto out; case PMEM_POOL_TYPE_UNKNOWN: outv_err("%s: unknown pool type -- '%s'\n", pd.fname, params.signature); ret = -1; goto out; default: outv_err("%s: cannot determine type of pool\n", pd.fname); ret = -1; goto out; } if (ret) outv_err("%s: dumping pool file failed\n", pd.fname); out: if (pd.ofh != stdout) fclose(pd.ofh); util_ranges_clear(&pd.ranges); util_options_free(opts); return ret; } pmdk-1.8/src/tools/pmempool/info.c0000664000000000000000000006656013615011243015704 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * info.c -- pmempool info command main source file */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include "common.h" #include "output.h" #include "out.h" #include "info.h" #include "set.h" #include "file.h" #include "os_badblock.h" #include "badblock.h" #define DEFAULT_CHUNK_TYPES\ ((1<> (ALIGNMENT_DESC_BITS * (x))) & ((1 << ALIGNMENT_DESC_BITS) - 1))) #define UNDEF_REPLICA UINT_MAX #define UNDEF_PART UINT_MAX /* * Default arguments */ static const struct pmempool_info_args pmempool_info_args_default = { /* * Picked experimentally based on used fields names. * This should be at least the number of characters of * the longest field name. */ .col_width = 24, .human = false, .force = false, .badblocks = PRINT_BAD_BLOCKS_NOT_SET, .type = PMEM_POOL_TYPE_UNKNOWN, .vlevel = VERBOSE_DEFAULT, .vdata = VERBOSE_SILENT, .vhdrdump = VERBOSE_SILENT, .vstats = VERBOSE_SILENT, .log = { .walk = 0, }, .blk = { .vmap = VERBOSE_SILENT, .vflog = VERBOSE_SILENT, .vbackup = VERBOSE_SILENT, .skip_zeros = false, .skip_error = false, .skip_no_flag = false, }, .obj = { .vlanes = VERBOSE_SILENT, .vroot = VERBOSE_SILENT, .vobjects = VERBOSE_SILENT, .valloc = VERBOSE_SILENT, .voobhdr = VERBOSE_SILENT, .vheap = VERBOSE_SILENT, .vzonehdr = VERBOSE_SILENT, .vchunkhdr = VERBOSE_SILENT, .vbitmap = VERBOSE_SILENT, .lanes_recovery = false, .ignore_empty_obj = false, .chunk_types = DEFAULT_CHUNK_TYPES, .replica = 0, }, }; /* * long-options -- structure holding long options. */ static const struct option long_options[] = { {"version", no_argument, NULL, 'V' | OPT_ALL}, {"verbose", no_argument, NULL, 'v' | OPT_ALL}, {"help", no_argument, NULL, 'h' | OPT_ALL}, {"human", no_argument, NULL, 'n' | OPT_ALL}, {"force", required_argument, NULL, 'f' | OPT_ALL}, {"data", no_argument, NULL, 'd' | OPT_ALL}, {"headers-hex", no_argument, NULL, 'x' | OPT_ALL}, {"stats", no_argument, NULL, 's' | OPT_ALL}, {"range", required_argument, NULL, 'r' | OPT_ALL}, {"bad-blocks", required_argument, NULL, 'k' | OPT_ALL}, {"walk", required_argument, NULL, 'w' | OPT_LOG}, {"skip-zeros", no_argument, NULL, 'z' | OPT_BLK | OPT_BTT}, {"skip-error", no_argument, NULL, 'e' | OPT_BLK | OPT_BTT}, {"skip-no-flag", no_argument, NULL, 'u' | OPT_BLK | OPT_BTT}, {"map", no_argument, NULL, 'm' | OPT_BLK | OPT_BTT}, {"flog", no_argument, NULL, 'g' | OPT_BLK | OPT_BTT}, {"backup", no_argument, NULL, 'B' | OPT_BLK | OPT_BTT}, {"lanes", no_argument, NULL, 'l' | OPT_OBJ}, {"recovery", no_argument, NULL, 'R' | OPT_OBJ}, {"section", required_argument, NULL, 'S' | OPT_OBJ}, {"object-store", no_argument, NULL, 'O' | OPT_OBJ}, {"types", required_argument, NULL, 't' | OPT_OBJ}, {"no-empty", no_argument, NULL, 'E' | OPT_OBJ}, {"alloc-header", no_argument, NULL, 'A' | OPT_OBJ}, {"oob-header", no_argument, NULL, 'a' | OPT_OBJ}, {"root", no_argument, NULL, 'o' | OPT_OBJ}, {"heap", no_argument, NULL, 'H' | OPT_OBJ}, {"zones", no_argument, NULL, 'Z' | OPT_OBJ}, {"chunks", no_argument, NULL, 'C' | OPT_OBJ}, {"chunk-type", required_argument, NULL, 'T' | OPT_OBJ}, {"bitmap", no_argument, NULL, 'b' | OPT_OBJ}, {"replica", required_argument, NULL, 'p' | OPT_OBJ}, {NULL, 0, NULL, 0 }, }; static const struct option_requirement option_requirements[] = { { .opt = 'r', .type = PMEM_POOL_TYPE_LOG, .req = OPT_REQ0('d') }, { .opt = 'r', .type = PMEM_POOL_TYPE_BLK | PMEM_POOL_TYPE_BTT, .req = OPT_REQ0('d') | OPT_REQ1('m') }, { .opt = 'z', .type = PMEM_POOL_TYPE_BLK | PMEM_POOL_TYPE_BTT, .req = OPT_REQ0('d') | OPT_REQ1('m') }, { .opt = 'e', .type = PMEM_POOL_TYPE_BLK | PMEM_POOL_TYPE_BTT, .req = OPT_REQ0('d') | OPT_REQ1('m') }, { .opt = 'u', .type = PMEM_POOL_TYPE_BLK | PMEM_POOL_TYPE_BTT, .req = OPT_REQ0('d') | OPT_REQ1('m') }, { .opt = 'r', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('O') | OPT_REQ1('Z') | OPT_REQ2('C') | OPT_REQ3('l'), }, { .opt = 'R', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('l') }, { .opt = 'S', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('l') }, { .opt = 'E', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('O') }, { .opt = 'T', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('C') }, { .opt = 'b', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('H') }, { .opt = 'b', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('C') }, { .opt = 'A', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('O') | OPT_REQ1('l') | OPT_REQ2('o') }, { .opt = 'a', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('O') | OPT_REQ1('l') | OPT_REQ2('o') }, { .opt = 't', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('O') | OPT_REQ1('s'), }, { .opt = 'C', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('O') | OPT_REQ1('H') | OPT_REQ2('s'), }, { .opt = 'Z', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('O') | OPT_REQ1('H') | OPT_REQ2('s'), }, { .opt = 'd', .type = PMEM_POOL_TYPE_OBJ, .req = OPT_REQ0('O') | OPT_REQ1('o'), }, { 0, 0, 0} }; /* * help_str -- string for help message */ static const char * const help_str = "Show information about pmem pool from specified file.\n" "\n" "Common options:\n" " -h, --help Print this help and exit.\n" " -V, --version Print version and exit.\n" " -v, --verbose Increase verbisity level.\n" " -f, --force blk|log|obj|btt Force parsing a pool of specified type.\n" " -n, --human Print sizes in human readable format.\n" " -x, --headers-hex Hexdump all headers.\n" " -d, --data Dump log data and blocks.\n" " -s, --stats Print statistics.\n" " -r, --range Range of blocks/chunks/objects.\n" " -k, --bad-blocks= Print bad blocks.\n" "\n" "Options for PMEMLOG:\n" " -w, --walk Chunk size.\n" "\n" "Options for PMEMBLK:\n" " -m, --map Print BTT Map entries.\n" " -g, --flog Print BTT FLOG entries.\n" " -B, --backup Print BTT Info header backup.\n" " -z, --skip-zeros Skip blocks marked with zero flag.\n" " -e, --skip-error Skip blocks marked with error flag.\n" " -u, --skip-no-flag Skip blocks not marked with any flag.\n" "\n" "Options for PMEMOBJ:\n" " -l, --lanes [] Print lanes from specified range.\n" " -R, --recovery Print only lanes which need recovery.\n" " -S, --section tx,allocator,list Print only specified sections.\n" " -O, --object-store Print object store.\n" " -t, --types Specify objects' type numbers range.\n" " -E, --no-empty Print only non-empty object store lists.\n" " -o, --root Print root object information\n" " -A, --alloc-header Print allocation header for objects in\n" " object store.\n" " -a, --oob-header Print OOB header\n" " -H, --heap Print heap header.\n" " -Z, --zones [] Print zones header. If range is specified\n" " and --object|-O option is specified prints\n" " objects from specified zones only.\n" " -C, --chunks [] Print zones header. If range is specified\n" " and --object|-O option is specified prints\n" " objects from specified zones only.\n" " -T, --chunk-type used,free,run,footer\n" " Print only specified type(s) of chunk.\n" " [requires --chunks|-C]\n" " -b, --bitmap Print chunk run's bitmap in graphical\n" " format. [requires --chunks|-C]\n" " -p, --replica Print info from specified replica\n" "For complete documentation see %s-info(1) manual page.\n" ; /* * print_usage -- print application usage short description */ static void print_usage(const char *appname) { printf("Usage: %s info [] \n", appname); } /* * print_version -- print version string */ static void print_version(const char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * pmempool_info_help -- print application usage detailed description */ void pmempool_info_help(const char *appname) { print_usage(appname); print_version(appname); printf(help_str, appname); } /* * parse_args -- parse command line arguments * * Parse command line arguments and store them in pmempool_info_args * structure. * Terminates process if invalid arguments passed. */ static int parse_args(const char *appname, int argc, char *argv[], struct pmempool_info_args *argsp, struct options *opts) { int opt; if (argc == 1) { print_usage(appname); return -1; } struct ranges *rangesp = &argsp->ranges; while ((opt = util_options_getopt(argc, argv, "vhnf:ezuF:L:c:dmxVw:gBsr:lRS:OECZHT:bot:aAp:k:", opts)) != -1) { switch (opt) { case 'v': argsp->vlevel = VERBOSE_MAX; break; case 'V': print_version(appname); exit(EXIT_SUCCESS); case 'h': pmempool_info_help(appname); exit(EXIT_SUCCESS); case 'n': argsp->human = true; break; case 'f': argsp->type = pmem_pool_type_parse_str(optarg); if (argsp->type == PMEM_POOL_TYPE_UNKNOWN) { outv_err("'%s' -- unknown pool type\n", optarg); return -1; } argsp->force = true; break; case 'k': if (strcmp(optarg, "no") == 0) { argsp->badblocks = PRINT_BAD_BLOCKS_NO; } else if (strcmp(optarg, "yes") == 0) { argsp->badblocks = PRINT_BAD_BLOCKS_YES; } else { outv_err( "'%s' -- invalid argument of the '-k/--bad-blocks' option\n", optarg); return -1; } break; case 'e': argsp->blk.skip_error = true; break; case 'z': argsp->blk.skip_zeros = true; break; case 'u': argsp->blk.skip_no_flag = true; break; case 'r': if (util_parse_ranges(optarg, rangesp, ENTIRE_UINT64)) { outv_err("'%s' -- cannot parse range(s)\n", optarg); return -1; } if (rangesp == &argsp->ranges) argsp->use_range = 1; break; case 'd': argsp->vdata = VERBOSE_DEFAULT; break; case 'm': argsp->blk.vmap = VERBOSE_DEFAULT; break; case 'g': argsp->blk.vflog = VERBOSE_DEFAULT; break; case 'B': argsp->blk.vbackup = VERBOSE_DEFAULT; break; case 'x': argsp->vhdrdump = VERBOSE_DEFAULT; break; case 's': argsp->vstats = VERBOSE_DEFAULT; break; case 'w': argsp->log.walk = (size_t)atoll(optarg); if (argsp->log.walk == 0) { outv_err("'%s' -- invalid chunk size\n", optarg); return -1; } break; case 'l': argsp->obj.vlanes = VERBOSE_DEFAULT; rangesp = &argsp->obj.lane_ranges; break; case 'R': argsp->obj.lanes_recovery = true; break; case 'O': argsp->obj.vobjects = VERBOSE_DEFAULT; rangesp = &argsp->ranges; break; case 'a': argsp->obj.voobhdr = VERBOSE_DEFAULT; break; case 'A': argsp->obj.valloc = VERBOSE_DEFAULT; break; case 'E': argsp->obj.ignore_empty_obj = true; break; case 'Z': argsp->obj.vzonehdr = VERBOSE_DEFAULT; rangesp = &argsp->obj.zone_ranges; break; case 'C': argsp->obj.vchunkhdr = VERBOSE_DEFAULT; rangesp = &argsp->obj.chunk_ranges; break; case 'H': argsp->obj.vheap = VERBOSE_DEFAULT; break; case 'T': argsp->obj.chunk_types = 0; if (util_parse_chunk_types(optarg, &argsp->obj.chunk_types) || (argsp->obj.chunk_types & (1 << CHUNK_TYPE_UNKNOWN))) { outv_err("'%s' -- cannot parse chunk type(s)\n", optarg); return -1; } break; case 'o': argsp->obj.vroot = VERBOSE_DEFAULT; break; case 't': if (util_parse_ranges(optarg, &argsp->obj.type_ranges, ENTIRE_UINT64)) { outv_err("'%s' -- cannot parse range(s)\n", optarg); return -1; } break; case 'b': argsp->obj.vbitmap = VERBOSE_DEFAULT; break; case 'p': { char *endptr; int olderrno = errno; errno = 0; long long ll = strtoll(optarg, &endptr, 10); if ((endptr && *endptr != '\0') || errno) { outv_err("'%s' -- invalid replica number", optarg); return -1; } errno = olderrno; argsp->obj.replica = (size_t)ll; break; } default: print_usage(appname); return -1; } } if (optind < argc) { argsp->file = argv[optind]; } else { print_usage(appname); return -1; } if (!argsp->use_range) util_ranges_add(&argsp->ranges, ENTIRE_UINT64); if (util_ranges_empty(&argsp->obj.type_ranges)) util_ranges_add(&argsp->obj.type_ranges, ENTIRE_UINT64); if (util_ranges_empty(&argsp->obj.lane_ranges)) util_ranges_add(&argsp->obj.lane_ranges, ENTIRE_UINT64); if (util_ranges_empty(&argsp->obj.zone_ranges)) util_ranges_add(&argsp->obj.zone_ranges, ENTIRE_UINT64); if (util_ranges_empty(&argsp->obj.chunk_ranges)) util_ranges_add(&argsp->obj.chunk_ranges, ENTIRE_UINT64); return 0; } /* * pmempool_info_read -- read data from file */ int pmempool_info_read(struct pmem_info *pip, void *buff, size_t nbytes, uint64_t off) { return pool_set_file_read(pip->pfile, buff, nbytes, off); } /* * pmempool_info_badblocks -- (internal) prints info about file badblocks */ static int pmempool_info_badblocks(struct pmem_info *pip, const char *file_name, int v) { int ret; if (pip->args.badblocks != PRINT_BAD_BLOCKS_YES) return 0; struct badblocks *bbs = badblocks_new(); if (bbs == NULL) return -1; ret = os_badblocks_get(file_name, bbs); if (ret) { if (errno == ENOTSUP) { outv(v, BB_NOT_SUPP "\n"); ret = -1; goto exit_free; } outv_err("checking bad blocks failed -- '%s'", file_name); goto exit_free; } if (bbs->bb_cnt == 0 || bbs->bbv == NULL) goto exit_free; outv(v, "bad blocks:\n"); outv(v, "\toffset\t\tlength\n"); unsigned b; for (b = 0; b < bbs->bb_cnt; b++) { outv(v, "\t%llu\t\t%u\n", B2SEC(bbs->bbv[b].offset), B2SEC(bbs->bbv[b].length)); } exit_free: badblocks_delete(bbs); return ret; } /* * pmempool_info_part -- (internal) print info about poolset part */ static int pmempool_info_part(struct pmem_info *pip, unsigned repn, unsigned partn, int v) { /* get path of the part file */ const char *path = NULL; if (repn != UNDEF_REPLICA && partn != UNDEF_PART) { outv(v, "part %u:\n", partn); struct pool_set_part *part = &pip->pfile->poolset->replica[repn]->part[partn]; path = part->path; } else { outv(v, "Part file:\n"); path = pip->file_name; } outv_field(v, "path", "%s", path); enum file_type type = util_file_get_type(path); if (type < 0) return -1; const char *type_str = type == TYPE_DEVDAX ? "device dax" : "regular file"; outv_field(v, "type", "%s", type_str); /* get size of the part file */ ssize_t size = util_file_get_size(path); if (size < 0) { outv_err("couldn't get size of %s", path); return -1; } outv_field(v, "size", "%s", out_get_size_str((size_t)size, pip->args.human)); /* get alignment of device dax */ if (type == TYPE_DEVDAX) { size_t alignment = util_file_device_dax_alignment(path); outv_field(v, "alignment", "%s", out_get_size_str(alignment, pip->args.human)); } /* look for bad blocks */ if (pmempool_info_badblocks(pip, path, VERBOSE_DEFAULT)) { outv_err("Unable to retrieve badblock info"); return -1; } return 0; } /* * pmempool_info_directory -- (internal) print information about directory */ static void pmempool_info_directory(struct pool_set_directory *d, int v) { outv(v, "Directory %s:\n", d->path); outv_field(v, "reservation size", "%lu", d->resvsize); } /* * pmempool_info_replica -- (internal) print info about replica */ static int pmempool_info_replica(struct pmem_info *pip, unsigned repn, int v) { struct pool_replica *rep = pip->pfile->poolset->replica[repn]; outv(v, "Replica %u%s - %s", repn, repn == 0 ? " (master)" : "", rep->remote == NULL ? "local" : "remote"); if (rep->remote) { outv(v, ":\n"); outv_field(v, "node", "%s", rep->remote->node_addr); outv_field(v, "pool set", "%s", rep->remote->pool_desc); return 0; } outv(v, ", %u part(s):\n", rep->nparts); for (unsigned p = 0; p < rep->nparts; ++p) { if (pmempool_info_part(pip, repn, p, v)) return -1; } if (pip->pfile->poolset->directory_based) { size_t nd = VEC_SIZE(&rep->directory); outv(v, "%lu %s:\n", nd, nd == 1 ? "Directory" : "Directories"); struct pool_set_directory *d; VEC_FOREACH_BY_PTR(d, &rep->directory) { pmempool_info_directory(d, v); } } return 0; } /* * pmempool_info_poolset -- (internal) print info about poolset structure */ static int pmempool_info_poolset(struct pmem_info *pip, int v) { ASSERTeq(pip->params.is_poolset, 1); if (pip->pfile->poolset->directory_based) outv(v, "Directory-based Poolset structure:\n"); else outv(v, "Poolset structure:\n"); outv_field(v, "Number of replicas", "%u", pip->pfile->poolset->nreplicas); for (unsigned r = 0; r < pip->pfile->poolset->nreplicas; ++r) { if (pmempool_info_replica(pip, r, v)) return -1; } if (pip->pfile->poolset->options > 0) { outv_title(v, "Poolset options"); if (pip->pfile->poolset->options & OPTION_SINGLEHDR) outv(v, "%s", "SINGLEHDR\n"); } return 0; } /* * pmempool_info_pool_hdr -- (internal) print pool header information */ static int pmempool_info_pool_hdr(struct pmem_info *pip, int v) { static const char *alignment_desc_str[] = { " char", " short", " int", " long", " long long", " size_t", " os_off_t", " float", " double", " long double", " void *", }; static const size_t alignment_desc_n = sizeof(alignment_desc_str) / sizeof(alignment_desc_str[0]); int ret = 0; struct pool_hdr *hdr = malloc(sizeof(struct pool_hdr)); if (!hdr) err(1, "Cannot allocate memory for pool_hdr"); if (pmempool_info_read(pip, hdr, sizeof(*hdr), 0)) { outv_err("cannot read pool header\n"); free(hdr); return -1; } struct arch_flags arch_flags; util_get_arch_flags(&arch_flags); outv_title(v, "POOL Header"); outv_hexdump(pip->args.vhdrdump, hdr, sizeof(*hdr), 0, 1); util_convert2h_hdr_nocheck(hdr); outv_field(v, "Signature", "%.*s%s", POOL_HDR_SIG_LEN, hdr->signature, pip->params.is_part ? " [part file]" : ""); outv_field(v, "Major", "%d", hdr->major); outv_field(v, "Mandatory features", "%s", out_get_incompat_features_str(hdr->features.incompat)); outv_field(v, "Not mandatory features", "0x%x", hdr->features.compat); outv_field(v, "Forced RO", "0x%x", hdr->features.ro_compat); outv_field(v, "Pool set UUID", "%s", out_get_uuid_str(hdr->poolset_uuid)); outv_field(v, "UUID", "%s", out_get_uuid_str(hdr->uuid)); outv_field(v, "Previous part UUID", "%s", out_get_uuid_str(hdr->prev_part_uuid)); outv_field(v, "Next part UUID", "%s", out_get_uuid_str(hdr->next_part_uuid)); outv_field(v, "Previous replica UUID", "%s", out_get_uuid_str(hdr->prev_repl_uuid)); outv_field(v, "Next replica UUID", "%s", out_get_uuid_str(hdr->next_repl_uuid)); outv_field(v, "Creation Time", "%s", out_get_time_str((time_t)hdr->crtime)); uint64_t ad = hdr->arch_flags.alignment_desc; uint64_t cur_ad = arch_flags.alignment_desc; outv_field(v, "Alignment Descriptor", "%s", out_get_alignment_desc_str(ad, cur_ad)); for (size_t i = 0; i < alignment_desc_n; i++) { uint64_t a = GET_ALIGNMENT(ad, i); if (ad == cur_ad) { outv_field(v + 1, alignment_desc_str[i], "%2lu", a); } else { uint64_t av = GET_ALIGNMENT(cur_ad, i); if (a == av) { outv_field(v + 1, alignment_desc_str[i], "%2lu [OK]", a); } else { outv_field(v + 1, alignment_desc_str[i], "%2lu [wrong! should be %2lu]", a, av); } } } outv_field(v, "Class", "%s", out_get_arch_machine_class_str( hdr->arch_flags.machine_class)); outv_field(v, "Data", "%s", out_get_arch_data_str(hdr->arch_flags.data)); outv_field(v, "Machine", "%s", out_get_arch_machine_str(hdr->arch_flags.machine)); outv_field(v, "Last shutdown", "%s", out_get_last_shutdown_str(hdr->sds.dirty)); outv_field(v, "Checksum", "%s", out_get_checksum(hdr, sizeof(*hdr), &hdr->checksum, POOL_HDR_CSUM_END_OFF(hdr))); free(hdr); return ret; } /* * pmempool_info_file -- print info about single file */ static int pmempool_info_file(struct pmem_info *pip, const char *file_name) { int ret = 0; pip->file_name = file_name; /* * If force flag is set 'types' fields _must_ hold * single pool type - this is validated when processing * command line arguments. */ if (pip->args.force) { pip->type = pip->args.type; } else { if (pmem_pool_parse_params(file_name, &pip->params, 1)) { if (errno) perror(file_name); else outv_err("%s: cannot determine type of pool\n", file_name); return -1; } pip->type = pip->params.type; } if (PMEM_POOL_TYPE_UNKNOWN == pip->type) { outv_err("%s: unknown pool type -- '%s'\n", file_name, pip->params.signature); return -1; } else if (!pip->args.force && !pip->params.is_checksum_ok) { outv_err("%s: invalid checksum\n", file_name); return -1; } else { if (util_options_verify(pip->opts, pip->type)) return -1; pip->pfile = pool_set_file_open(file_name, 0, !pip->args.force); if (!pip->pfile) { perror(file_name); return -1; } /* check if we should check and print bad blocks */ if (pip->args.badblocks == PRINT_BAD_BLOCKS_NOT_SET) { struct pool_hdr hdr; if (pmempool_info_read(pip, &hdr, sizeof(hdr), 0)) { outv_err("cannot read pool header\n"); goto out_close; } util_convert2h_hdr_nocheck(&hdr); if (hdr.features.compat & POOL_FEAT_CHECK_BAD_BLOCKS) pip->args.badblocks = PRINT_BAD_BLOCKS_YES; else pip->args.badblocks = PRINT_BAD_BLOCKS_NO; } if (pip->type != PMEM_POOL_TYPE_BTT) { struct pool_set *ps = pip->pfile->poolset; for (unsigned r = 0; r < ps->nreplicas; ++r) { if (ps->replica[r]->remote == NULL && mprotect(ps->replica[r]->part[0].addr, ps->replica[r]->repsize, PROT_READ) < 0) { outv_err( "%s: failed to change pool protection", pip->pfile->fname); ret = -1; goto out_close; } } } if (pip->args.obj.replica) { size_t nreplicas = pool_set_file_nreplicas(pip->pfile); if (nreplicas == 1) { outv_err("only master replica available"); ret = -1; goto out_close; } if (pip->args.obj.replica >= nreplicas) { outv_err("replica number out of range" " (valid range is: 0-%" PRIu64 ")", nreplicas - 1); ret = -1; goto out_close; } if (pool_set_file_set_replica(pip->pfile, pip->args.obj.replica)) { outv_err("setting replica number failed"); ret = -1; goto out_close; } } /* hdr info is not present in btt device */ if (pip->type != PMEM_POOL_TYPE_BTT) { if (pip->params.is_poolset && pmempool_info_poolset(pip, VERBOSE_DEFAULT)) { ret = -1; goto out_close; } if (!pip->params.is_poolset && pmempool_info_part(pip, UNDEF_REPLICA, UNDEF_PART, VERBOSE_DEFAULT)) { ret = -1; goto out_close; } if (pmempool_info_pool_hdr(pip, VERBOSE_DEFAULT)) { ret = -1; goto out_close; } } if (pip->params.is_part) { ret = 0; goto out_close; } switch (pip->type) { case PMEM_POOL_TYPE_LOG: ret = pmempool_info_log(pip); break; case PMEM_POOL_TYPE_BLK: ret = pmempool_info_blk(pip); break; case PMEM_POOL_TYPE_OBJ: ret = pmempool_info_obj(pip); break; case PMEM_POOL_TYPE_BTT: ret = pmempool_info_btt(pip); break; case PMEM_POOL_TYPE_UNKNOWN: default: ret = -1; break; } out_close: pool_set_file_close(pip->pfile); } return ret; } /* * pmempool_info_alloc -- allocate pmem info context */ static struct pmem_info * pmempool_info_alloc(void) { struct pmem_info *pip = malloc(sizeof(struct pmem_info)); if (!pip) err(1, "Cannot allocate memory for pmempool info context"); if (pip) { memset(pip, 0, sizeof(*pip)); /* set default command line parameters */ memcpy(&pip->args, &pmempool_info_args_default, sizeof(pip->args)); pip->opts = util_options_alloc(long_options, sizeof(long_options) / sizeof(long_options[0]), option_requirements); PMDK_LIST_INIT(&pip->args.ranges.head); PMDK_LIST_INIT(&pip->args.obj.type_ranges.head); PMDK_LIST_INIT(&pip->args.obj.lane_ranges.head); PMDK_LIST_INIT(&pip->args.obj.zone_ranges.head); PMDK_LIST_INIT(&pip->args.obj.chunk_ranges.head); PMDK_TAILQ_INIT(&pip->obj.stats.type_stats); } return pip; } /* * pmempool_info_free -- free pmem info context */ static void pmempool_info_free(struct pmem_info *pip) { if (pip->obj.stats.zone_stats) { for (uint64_t i = 0; i < pip->obj.stats.n_zones; ++i) VEC_DELETE(&pip->obj.stats.zone_stats[i].class_stats); free(pip->obj.stats.zone_stats); } util_options_free(pip->opts); util_ranges_clear(&pip->args.ranges); util_ranges_clear(&pip->args.obj.type_ranges); util_ranges_clear(&pip->args.obj.zone_ranges); util_ranges_clear(&pip->args.obj.chunk_ranges); util_ranges_clear(&pip->args.obj.lane_ranges); while (!PMDK_TAILQ_EMPTY(&pip->obj.stats.type_stats)) { struct pmem_obj_type_stats *type = PMDK_TAILQ_FIRST(&pip->obj.stats.type_stats); PMDK_TAILQ_REMOVE(&pip->obj.stats.type_stats, type, next); free(type); } free(pip); } int pmempool_info_func(const char *appname, int argc, char *argv[]) { int ret = 0; struct pmem_info *pip = pmempool_info_alloc(); /* read command line arguments */ if ((ret = parse_args(appname, argc, argv, &pip->args, pip->opts)) == 0) { /* set some output format values */ out_set_vlevel(pip->args.vlevel); out_set_col_width(pip->args.col_width); ret = pmempool_info_file(pip, pip->args.file); } pmempool_info_free(pip); return ret; } pmdk-1.8/src/tools/pmempool/info_obj.c0000664000000000000000000006213413615011243016527 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * info_obj.c -- pmempool info command source file for obj pool */ #include #include #include #include #include #include #include #include #include "alloc_class.h" #include "set.h" #include "common.h" #include "output.h" #include "info.h" #include "util.h" #define BITMAP_BUFF_SIZE 1024 #define OFF_TO_PTR(pop, off) ((void *)((uintptr_t)(pop) + (off))) #define PTR_TO_OFF(pop, ptr) ((uintptr_t)(ptr) - (uintptr_t)(pop)) /* * lane_need_recovery -- return 1 if lane section needs recovery */ static int lane_need_recovery(struct pmem_info *pip, struct lane_layout *lane) { return ulog_recovery_needed((struct ulog *)&lane->external, 1) || ulog_recovery_needed((struct ulog *)&lane->internal, 1) || ulog_recovery_needed((struct ulog *)&lane->undo, 0); } #define RUN_BITMAP_SEPARATOR_DISTANCE 8 /* * get_bitmap_str -- get bitmap single value string */ static const char * get_bitmap_str(uint64_t val, unsigned values) { static char buff[BITMAP_BUFF_SIZE]; unsigned j = 0; for (unsigned i = 0; i < values && j < BITMAP_BUFF_SIZE - 3; i++) { buff[j++] = ((val & ((uint64_t)1 << i)) ? 'x' : '.'); if ((i + 1) % RUN_BITMAP_SEPARATOR_DISTANCE == 0) buff[j++] = ' '; } buff[j] = '\0'; return buff; } /* * pmem_obj_stats_get_type -- get stats for specified type number */ static struct pmem_obj_type_stats * pmem_obj_stats_get_type(struct pmem_obj_stats *stats, uint64_t type_num) { struct pmem_obj_type_stats *type; struct pmem_obj_type_stats *type_dest = NULL; PMDK_TAILQ_FOREACH(type, &stats->type_stats, next) { if (type->type_num == type_num) return type; if (!type_dest && type->type_num > type_num) type_dest = type; } type = calloc(1, sizeof(*type)); if (!type) { outv_err("cannot allocate memory for type stats\n"); exit(EXIT_FAILURE); } type->type_num = type_num; if (type_dest) PMDK_TAILQ_INSERT_BEFORE(type_dest, type, next); else PMDK_TAILQ_INSERT_TAIL(&stats->type_stats, type, next); return type; } struct info_obj_redo_args { int v; size_t i; struct pmem_info *pip; }; /* * info_obj_redo_entry - print redo log entry info */ static int info_obj_redo_entry(struct ulog_entry_base *e, void *arg, const struct pmem_ops *p_ops) { struct info_obj_redo_args *a = arg; struct ulog_entry_val *ev; struct ulog_entry_buf *eb; switch (ulog_entry_type(e)) { case ULOG_OPERATION_AND: case ULOG_OPERATION_OR: case ULOG_OPERATION_SET: ev = (struct ulog_entry_val *)e; outv(a->v, "%010zu: " "Offset: 0x%016jx " "Value: 0x%016jx ", a->i++, ulog_entry_offset(e), ev->value); break; case ULOG_OPERATION_BUF_CPY: case ULOG_OPERATION_BUF_SET: eb = (struct ulog_entry_buf *)e; outv(a->v, "%010zu: " "Offset: 0x%016jx " "Size: %s ", a->i++, ulog_entry_offset(e), out_get_size_str(eb->size, a->pip->args.human)); break; default: ASSERT(0); /* unreachable */ } return 0; } /* * info_obj_redo -- print ulog log entries */ static void info_obj_ulog(struct pmem_info *pip, int v, struct ulog *ulog, const struct pmem_ops *ops) { outv_title(v, "Log entries"); struct info_obj_redo_args args = {v, 0, pip}; ulog_foreach_entry(ulog, info_obj_redo_entry, &args, ops); } /* * info_obj_alloc_hdr -- print allocation header */ static void info_obj_alloc_hdr(struct pmem_info *pip, int v, const struct memory_block *m) { outv_title(v, "Allocation Header"); outv_field(v, "Size", "%s", out_get_size_str(m->m_ops->get_user_size(m), pip->args.human)); outv_field(v, "Extra", "%lu", m->m_ops->get_extra(m)); outv_field(v, "Flags", "0x%x", m->m_ops->get_flags(m)); } /* * info_obj_object_hdr -- print object headers and data */ static void info_obj_object_hdr(struct pmem_info *pip, int v, int vid, const struct memory_block *m, uint64_t id) { struct pmemobjpool *pop = pip->obj.pop; void *data = m->m_ops->get_user_data(m); outv_nl(vid); outv_field(vid, "Object", "%lu", id); outv_field(vid, "Offset", "0x%016lx", PTR_TO_OFF(pop, data)); int vahdr = v && pip->args.obj.valloc; int voobh = v && pip->args.obj.voobhdr; outv_indent(vahdr || voobh, 1); info_obj_alloc_hdr(pip, vahdr, m); outv_hexdump(v && pip->args.vdata, data, m->m_ops->get_real_size(m), PTR_TO_OFF(pip->obj.pop, data), 1); outv_indent(vahdr || voobh, -1); } /* * info_obj_lane_section -- print lane's section */ static void info_obj_lane(struct pmem_info *pip, int v, struct lane_layout *lane) { struct pmem_ops p_ops; p_ops.base = pip->obj.pop; outv_title(v, "Undo Log"); outv_indent(v, 1); info_obj_ulog(pip, v, (struct ulog *)&lane->undo, &p_ops); outv_indent(v, -1); outv_nl(v); outv_title(v, "Internal Undo Log"); outv_indent(v, 1); info_obj_ulog(pip, v, (struct ulog *)&lane->internal, &p_ops); outv_indent(v, -1); outv_title(v, "External Undo Log"); outv_indent(v, 1); info_obj_ulog(pip, v, (struct ulog *)&lane->external, &p_ops); outv_indent(v, -1); } /* * info_obj_lanes -- print lanes structures */ static void info_obj_lanes(struct pmem_info *pip) { int v = pip->args.obj.vlanes; if (!outv_check(v)) return; struct pmemobjpool *pop = pip->obj.pop; /* * Iterate through all lanes from specified range and print * specified sections. */ struct lane_layout *lanes = (void *)((char *)pip->obj.pop + pop->lanes_offset); struct range *curp = NULL; FOREACH_RANGE(curp, &pip->args.obj.lane_ranges) { for (uint64_t i = curp->first; i <= curp->last && i < pop->nlanes; i++) { /* For -R check print lane only if needs recovery */ if (pip->args.obj.lanes_recovery && !lane_need_recovery(pip, &lanes[i])) continue; outv_title(v, "Lane %" PRIu64, i); outv_indent(v, 1); info_obj_lane(pip, v, &lanes[i]); outv_indent(v, -1); } } } /* * info_obj_heap -- print pmemobj heap headers */ static void info_obj_heap(struct pmem_info *pip) { int v = pip->args.obj.vheap; struct pmemobjpool *pop = pip->obj.pop; struct heap_layout *layout = OFF_TO_PTR(pop, pop->heap_offset); struct heap_header *heap = &layout->header; outv(v, "\nPMEMOBJ Heap Header:\n"); outv_hexdump(v && pip->args.vhdrdump, heap, sizeof(*heap), pop->heap_offset, 1); outv_field(v, "Signature", "%s", heap->signature); outv_field(v, "Major", "%ld", heap->major); outv_field(v, "Minor", "%ld", heap->minor); outv_field(v, "Chunk size", "%s", out_get_size_str(heap->chunksize, pip->args.human)); outv_field(v, "Chunks per zone", "%ld", heap->chunks_per_zone); outv_field(v, "Checksum", "%s", out_get_checksum(heap, sizeof(*heap), &heap->checksum, 0)); } /* * info_obj_zone -- print information about zone */ static void info_obj_zone_hdr(struct pmem_info *pip, int v, struct zone_header *zone) { outv_hexdump(v && pip->args.vhdrdump, zone, sizeof(*zone), PTR_TO_OFF(pip->obj.pop, zone), 1); outv_field(v, "Magic", "%s", out_get_zone_magic_str(zone->magic)); outv_field(v, "Size idx", "%u", zone->size_idx); } /* * info_obj_object -- print information about object */ static void info_obj_object(struct pmem_info *pip, const struct memory_block *m, uint64_t objid) { if (!util_ranges_contain(&pip->args.ranges, objid)) return; uint64_t type_num = m->m_ops->get_extra(m); if (!util_ranges_contain(&pip->args.obj.type_ranges, type_num)) return; uint64_t real_size = m->m_ops->get_real_size(m); pip->obj.stats.n_total_objects++; pip->obj.stats.n_total_bytes += real_size; struct pmem_obj_type_stats *type_stats = pmem_obj_stats_get_type(&pip->obj.stats, type_num); type_stats->n_objects++; type_stats->n_bytes += real_size; int vid = pip->args.obj.vobjects; int v = pip->args.obj.vobjects; outv_indent(v, 1); info_obj_object_hdr(pip, v, vid, m, objid); outv_indent(v, -1); } /* * info_obj_run_bitmap -- print chunk run's bitmap */ static void info_obj_run_bitmap(int v, struct run_bitmap *b) { /* print only used values for lower verbosity */ uint32_t i; for (i = 0; i < b->nbits / RUN_BITS_PER_VALUE; i++) outv(v, "%s\n", get_bitmap_str(b->values[i], RUN_BITS_PER_VALUE)); unsigned mod = b->nbits % RUN_BITS_PER_VALUE; if (mod != 0) { outv(v, "%s\n", get_bitmap_str(b->values[i], mod)); } } /* * info_obj_memblock_is_root -- (internal) checks whether the object is root */ static int info_obj_memblock_is_root(struct pmem_info *pip, const struct memory_block *m) { uint64_t roff = pip->obj.pop->root_offset; if (roff == 0) return 0; struct memory_block rm = memblock_from_offset(pip->obj.heap, roff); return MEMORY_BLOCK_EQUALS(*m, rm); } /* * info_obj_run_cb -- (internal) run object callback */ static int info_obj_run_cb(const struct memory_block *m, void *arg) { struct pmem_info *pip = arg; if (info_obj_memblock_is_root(pip, m)) return 0; info_obj_object(pip, m, pip->obj.objid++); return 0; } static struct pmem_obj_class_stats * info_obj_class_stats_get_or_insert(struct pmem_obj_zone_stats *stats, uint64_t unit_size, uint64_t alignment, uint32_t nallocs, uint16_t flags) { struct pmem_obj_class_stats *cstats; VEC_FOREACH_BY_PTR(cstats, &stats->class_stats) { if (cstats->alignment == alignment && cstats->flags == flags && cstats->nallocs == nallocs && cstats->unit_size == unit_size) return cstats; } struct pmem_obj_class_stats s = {0, 0, unit_size, alignment, nallocs, flags}; if (VEC_PUSH_BACK(&stats->class_stats, s) != 0) return NULL; return &VEC_BACK(&stats->class_stats); } /* * info_obj_chunk -- print chunk info */ static void info_obj_chunk(struct pmem_info *pip, uint64_t c, uint64_t z, struct chunk_header *chunk_hdr, struct chunk *chunk, struct pmem_obj_zone_stats *stats) { int v = pip->args.obj.vchunkhdr; outv(v, "\n"); outv_field(v, "Chunk", "%lu", c); struct pmemobjpool *pop = pip->obj.pop; outv_hexdump(v && pip->args.vhdrdump, chunk_hdr, sizeof(*chunk_hdr), PTR_TO_OFF(pop, chunk_hdr), 1); outv_field(v, "Type", "%s", out_get_chunk_type_str(chunk_hdr->type)); outv_field(v, "Flags", "0x%x %s", chunk_hdr->flags, out_get_chunk_flags(chunk_hdr->flags)); outv_field(v, "Size idx", "%u", chunk_hdr->size_idx); struct memory_block m = MEMORY_BLOCK_NONE; m.zone_id = (uint32_t)z; m.chunk_id = (uint32_t)c; m.size_idx = (uint32_t)chunk_hdr->size_idx; memblock_rebuild_state(pip->obj.heap, &m); if (chunk_hdr->type == CHUNK_TYPE_USED || chunk_hdr->type == CHUNK_TYPE_FREE) { VEC_FRONT(&stats->class_stats).n_units += chunk_hdr->size_idx; if (chunk_hdr->type == CHUNK_TYPE_USED) { VEC_FRONT(&stats->class_stats).n_used += chunk_hdr->size_idx; /* skip root object */ if (!info_obj_memblock_is_root(pip, &m)) { info_obj_object(pip, &m, pip->obj.objid++); } } } else if (chunk_hdr->type == CHUNK_TYPE_RUN) { struct chunk_run *run = (struct chunk_run *)chunk; outv_hexdump(v && pip->args.vhdrdump, run, sizeof(run->hdr.block_size) + sizeof(run->hdr.alignment), PTR_TO_OFF(pop, run), 1); struct run_bitmap bitmap; m.m_ops->get_bitmap(&m, &bitmap); struct pmem_obj_class_stats *cstats = info_obj_class_stats_get_or_insert(stats, run->hdr.block_size, run->hdr.alignment, bitmap.nbits, chunk_hdr->flags); if (cstats == NULL) { outv_err("out of memory, can't allocate statistics"); return; } outv_field(v, "Block size", "%s", out_get_size_str(run->hdr.block_size, pip->args.human)); uint32_t units = bitmap.nbits; uint32_t free_space = 0; uint32_t max_free_block = 0; m.m_ops->calc_free(&m, &free_space, &max_free_block); uint32_t used = units - free_space; cstats->n_units += units; cstats->n_used += used; outv_field(v, "Bitmap", "%u / %u", used, units); info_obj_run_bitmap(v && pip->args.obj.vbitmap, &bitmap); m.m_ops->iterate_used(&m, info_obj_run_cb, pip); } } /* * info_obj_zone_chunks -- print chunk headers from specified zone */ static void info_obj_zone_chunks(struct pmem_info *pip, struct zone *zone, uint64_t z, struct pmem_obj_zone_stats *stats) { VEC_INIT(&stats->class_stats); struct pmem_obj_class_stats default_class_stats = {0, 0, CHUNKSIZE, 0, 0, 0}; VEC_PUSH_BACK(&stats->class_stats, default_class_stats); uint64_t c = 0; while (c < zone->header.size_idx) { enum chunk_type type = zone->chunk_headers[c].type; uint64_t size_idx = zone->chunk_headers[c].size_idx; if (util_ranges_contain(&pip->args.obj.chunk_ranges, c)) { if (pip->args.obj.chunk_types & (1ULL << type)) { stats->n_chunks++; stats->n_chunks_type[type]++; stats->size_chunks += size_idx; stats->size_chunks_type[type] += size_idx; info_obj_chunk(pip, c, z, &zone->chunk_headers[c], &zone->chunks[c], stats); } if (size_idx > 1 && type != CHUNK_TYPE_RUN && pip->args.obj.chunk_types & (1 << CHUNK_TYPE_FOOTER)) { size_t f = c + size_idx - 1; info_obj_chunk(pip, f, z, &zone->chunk_headers[f], &zone->chunks[f], stats); } } c += size_idx; } } /* * info_obj_root_obj -- print root object */ static void info_obj_root_obj(struct pmem_info *pip) { int v = pip->args.obj.vroot; struct pmemobjpool *pop = pip->obj.pop; if (!pop->root_offset) { outv(v, "\nNo root object...\n"); return; } outv_title(v, "Root object"); outv_field(v, "Offset", "0x%016zx", pop->root_offset); uint64_t root_size = pop->root_size; outv_field(v, "Size", "%s", out_get_size_str(root_size, pip->args.human)); struct memory_block m = memblock_from_offset( pip->obj.heap, pop->root_offset); /* do not print object id and offset for root object */ info_obj_object_hdr(pip, v, VERBOSE_SILENT, &m, 0); } /* * info_obj_zones -- print zones and chunks */ static void info_obj_zones_chunks(struct pmem_info *pip) { if (!outv_check(pip->args.obj.vheap) && !outv_check(pip->args.vstats) && !outv_check(pip->args.obj.vobjects)) return; struct pmemobjpool *pop = pip->obj.pop; struct heap_layout *layout = OFF_TO_PTR(pop, pop->heap_offset); size_t maxzone = util_heap_max_zone(pop->heap_size); pip->obj.stats.n_zones = maxzone; pip->obj.stats.zone_stats = calloc(maxzone, sizeof(struct pmem_obj_zone_stats)); if (!pip->obj.stats.zone_stats) err(1, "Cannot allocate memory for zone stats"); for (size_t i = 0; i < maxzone; i++) { struct zone *zone = ZID_TO_ZONE(layout, i); if (util_ranges_contain(&pip->args.obj.zone_ranges, i)) { int vvv = pip->args.obj.vheap && (pip->args.obj.vzonehdr || pip->args.obj.vchunkhdr); outv_title(vvv, "Zone %zu", i); if (zone->header.magic == ZONE_HEADER_MAGIC) pip->obj.stats.n_zones_used++; info_obj_zone_hdr(pip, pip->args.obj.vheap && pip->args.obj.vzonehdr, &zone->header); outv_indent(vvv, 1); info_obj_zone_chunks(pip, zone, i, &pip->obj.stats.zone_stats[i]); outv_indent(vvv, -1); } } } /* * info_obj_descriptor -- print pmemobj descriptor */ static void info_obj_descriptor(struct pmem_info *pip) { int v = VERBOSE_DEFAULT; if (!outv_check(v)) return; outv(v, "\nPMEM OBJ Header:\n"); struct pmemobjpool *pop = pip->obj.pop; uint8_t *hdrptr = (uint8_t *)pop + sizeof(pop->hdr); size_t hdrsize = sizeof(*pop) - sizeof(pop->hdr); size_t hdroff = sizeof(pop->hdr); outv_hexdump(pip->args.vhdrdump, hdrptr, hdrsize, hdroff, 1); /* check if layout is zeroed */ char *layout = util_check_memory((uint8_t *)pop->layout, sizeof(pop->layout), 0) ? pop->layout : "(null)"; /* address for checksum */ void *dscp = (void *)((uintptr_t)(pop) + sizeof(struct pool_hdr)); outv_field(v, "Layout", "%s", layout); outv_field(v, "Lanes offset", "0x%lx", pop->lanes_offset); outv_field(v, "Number of lanes", "%lu", pop->nlanes); outv_field(v, "Heap offset", "0x%lx", pop->heap_offset); outv_field(v, "Heap size", "%lu", pop->heap_size); outv_field(v, "Checksum", "%s", out_get_checksum(dscp, OBJ_DSC_P_SIZE, &pop->checksum, 0)); outv_field(v, "Root offset", "0x%lx", pop->root_offset); /* run id with -v option */ outv_field(v + 1, "Run id", "%lu", pop->run_id); } /* * info_obj_stats_objjects -- print objects' statistics */ static void info_obj_stats_objects(struct pmem_info *pip, int v, struct pmem_obj_stats *stats) { outv_field(v, "Number of objects", "%lu", stats->n_total_objects); outv_field(v, "Number of bytes", "%s", out_get_size_str( stats->n_total_bytes, pip->args.human)); outv_title(v, "Objects by type"); outv_indent(v, 1); struct pmem_obj_type_stats *type_stats; PMDK_TAILQ_FOREACH(type_stats, &pip->obj.stats.type_stats, next) { if (!type_stats->n_objects) continue; double n_objects_perc = 100.0 * (double)type_stats->n_objects / (double)stats->n_total_objects; double n_bytes_perc = 100.0 * (double)type_stats->n_bytes / (double)stats->n_total_bytes; outv_nl(v); outv_field(v, "Type number", "%lu", type_stats->type_num); outv_field(v, "Number of objects", "%lu [%s]", type_stats->n_objects, out_get_percentage(n_objects_perc)); outv_field(v, "Number of bytes", "%s [%s]", out_get_size_str( type_stats->n_bytes, pip->args.human), out_get_percentage(n_bytes_perc)); } outv_indent(v, -1); } /* * info_boj_stats_alloc_classes -- print allocation classes' statistics */ static void info_obj_stats_alloc_classes(struct pmem_info *pip, int v, struct pmem_obj_zone_stats *stats) { uint64_t total_bytes = 0; uint64_t total_used = 0; outv_indent(v, 1); struct pmem_obj_class_stats *cstats; VEC_FOREACH_BY_PTR(cstats, &stats->class_stats) { if (cstats->n_units == 0) continue; double used_perc = 100.0 * (double)cstats->n_used / (double)cstats->n_units; outv_nl(v); outv_field(v, "Unit size", "%s", out_get_size_str( cstats->unit_size, pip->args.human)); outv_field(v, "Units", "%lu", cstats->n_units); outv_field(v, "Used units", "%lu [%s]", cstats->n_used, out_get_percentage(used_perc)); uint64_t bytes = cstats->unit_size * cstats->n_units; uint64_t used = cstats->unit_size * cstats->n_used; total_bytes += bytes; total_used += used; double used_bytes_perc = 100.0 * (double)used / (double)bytes; outv_field(v, "Bytes", "%s", out_get_size_str(bytes, pip->args.human)); outv_field(v, "Used bytes", "%s [%s]", out_get_size_str(used, pip->args.human), out_get_percentage(used_bytes_perc)); } outv_indent(v, -1); double used_bytes_perc = total_bytes ? 100.0 * (double)total_used / (double)total_bytes : 0.0; outv_nl(v); outv_field(v, "Total bytes", "%s", out_get_size_str(total_bytes, pip->args.human)); outv_field(v, "Total used bytes", "%s [%s]", out_get_size_str(total_used, pip->args.human), out_get_percentage(used_bytes_perc)); } /* * info_obj_stats_chunks -- print chunks' statistics */ static void info_obj_stats_chunks(struct pmem_info *pip, int v, struct pmem_obj_zone_stats *stats) { outv_field(v, "Number of chunks", "%lu", stats->n_chunks); outv_indent(v, 1); for (unsigned type = 0; type < MAX_CHUNK_TYPE; type++) { double type_perc = 100.0 * (double)stats->n_chunks_type[type] / (double)stats->n_chunks; if (stats->n_chunks_type[type]) { outv_field(v, out_get_chunk_type_str(type), "%lu [%s]", stats->n_chunks_type[type], out_get_percentage(type_perc)); } } outv_indent(v, -1); outv_nl(v); outv_field(v, "Total chunks size", "%s", out_get_size_str( stats->size_chunks, pip->args.human)); outv_indent(v, 1); for (unsigned type = 0; type < MAX_CHUNK_TYPE; type++) { double type_perc = 100.0 * (double)stats->size_chunks_type[type] / (double)stats->size_chunks; if (stats->size_chunks_type[type]) { outv_field(v, out_get_chunk_type_str(type), "%lu [%s]", stats->size_chunks_type[type], out_get_percentage(type_perc)); } } outv_indent(v, -1); } /* * info_obj_add_zone_stats -- add stats to total */ static void info_obj_add_zone_stats(struct pmem_obj_zone_stats *total, struct pmem_obj_zone_stats *stats) { total->n_chunks += stats->n_chunks; total->size_chunks += stats->size_chunks; for (int type = 0; type < MAX_CHUNK_TYPE; type++) { total->n_chunks_type[type] += stats->n_chunks_type[type]; total->size_chunks_type[type] += stats->size_chunks_type[type]; } struct pmem_obj_class_stats *cstats; VEC_FOREACH_BY_PTR(cstats, &stats->class_stats) { struct pmem_obj_class_stats *ctotal = info_obj_class_stats_get_or_insert(total, cstats->unit_size, cstats->alignment, cstats->nallocs, cstats->flags); if (ctotal == NULL) { outv_err("out of memory, can't allocate statistics"); return; } ctotal->n_units += cstats->n_units; ctotal->n_used += cstats->n_used; } } /* * info_obj_stats_zones -- print zones' statistics */ static void info_obj_stats_zones(struct pmem_info *pip, int v, struct pmem_obj_stats *stats, struct pmem_obj_zone_stats *total) { double used_zones_perc = 100.0 * (double)stats->n_zones_used / (double)stats->n_zones; outv_field(v, "Number of zones", "%lu", stats->n_zones); outv_field(v, "Number of used zones", "%lu [%s]", stats->n_zones_used, out_get_percentage(used_zones_perc)); outv_indent(v, 1); for (uint64_t i = 0; i < stats->n_zones_used; i++) { outv_title(v, "Zone %" PRIu64, i); struct pmem_obj_zone_stats *zstats = &stats->zone_stats[i]; info_obj_stats_chunks(pip, v, zstats); outv_title(v, "Zone's allocation classes"); info_obj_stats_alloc_classes(pip, v, zstats); info_obj_add_zone_stats(total, zstats); } outv_indent(v, -1); } /* * info_obj_stats -- print statistics */ static void info_obj_stats(struct pmem_info *pip) { int v = pip->args.vstats; if (!outv_check(v)) return; struct pmem_obj_stats *stats = &pip->obj.stats; struct pmem_obj_zone_stats total; memset(&total, 0, sizeof(total)); outv_title(v, "Statistics"); outv_title(v, "Objects"); info_obj_stats_objects(pip, v, stats); outv_title(v, "Heap"); info_obj_stats_zones(pip, v, stats, &total); if (stats->n_zones_used > 1) { outv_title(v, "Total zone's statistics"); outv_title(v, "Chunks statistics"); info_obj_stats_chunks(pip, v, &total); outv_title(v, "Allocation classes"); info_obj_stats_alloc_classes(pip, v, &total); } VEC_DELETE(&total.class_stats); } static struct pmem_info *Pip; #ifndef _WIN32 static void info_obj_sa_sigaction(int signum, siginfo_t *info, void *context) { uintptr_t offset = (uintptr_t)info->si_addr - (uintptr_t)Pip->obj.pop; outv_err("Invalid offset 0x%lx\n", offset); exit(EXIT_FAILURE); } static struct sigaction info_obj_sigaction = { .sa_sigaction = info_obj_sa_sigaction, .sa_flags = SA_SIGINFO }; #else #define CALL_FIRST 1 static LONG CALLBACK exception_handler(_In_ PEXCEPTION_POINTERS ExceptionInfo) { PEXCEPTION_RECORD record = ExceptionInfo->ExceptionRecord; if (record->ExceptionCode != EXCEPTION_ACCESS_VIOLATION) { return EXCEPTION_CONTINUE_SEARCH; } uintptr_t offset = (uintptr_t)record->ExceptionInformation[1] - (uintptr_t)Pip->obj.pop; outv_err("Invalid offset 0x%lx\n", offset); exit(EXIT_FAILURE); } #endif /* * info_obj -- print information about obj pool type */ int pmempool_info_obj(struct pmem_info *pip) { pip->obj.pop = pool_set_file_map(pip->pfile, 0); if (pip->obj.pop == NULL) return -1; pip->obj.size = pip->pfile->size; struct palloc_heap *heap = calloc(1, sizeof(*heap)); if (heap == NULL) err(1, "Cannot allocate memory for heap data"); heap->layout = OFF_TO_PTR(pip->obj.pop, pip->obj.pop->heap_offset); heap->base = pip->obj.pop; pip->obj.alloc_classes = alloc_class_collection_new(); pip->obj.heap = heap; Pip = pip; #ifndef _WIN32 if (sigaction(SIGSEGV, &info_obj_sigaction, NULL)) { #else if (AddVectoredExceptionHandler(CALL_FIRST, exception_handler) == NULL) { #endif perror("sigaction"); return -1; } pip->obj.uuid_lo = pmemobj_get_uuid_lo(pip->obj.pop); info_obj_descriptor(pip); info_obj_lanes(pip); info_obj_root_obj(pip); info_obj_heap(pip); info_obj_zones_chunks(pip); info_obj_stats(pip); free(heap); alloc_class_collection_delete(pip->obj.alloc_classes); return 0; } pmdk-1.8/src/tools/pmempool/common.h0000664000000000000000000001646013615011243016240 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * common.h -- declarations of common functions */ #include #include #include #include #include "queue.h" #include "log.h" #include "blk.h" #include "libpmemobj.h" #include "lane.h" #include "ulog.h" #include "memops.h" #include "pmalloc.h" #include "list.h" #include "obj.h" #include "memblock.h" #include "heap_layout.h" #include "tx.h" #include "heap.h" #include "btt_layout.h" #include "page_size.h" /* XXX - modify Linux makefiles to generate srcversion.h and remove #ifdef */ #ifdef _WIN32 #include "srcversion.h" #endif #define COUNT_OF(x) (sizeof(x) / sizeof(0[x])) #define OPT_SHIFT 12 #define OPT_MASK (~((1 << OPT_SHIFT) - 1)) #define OPT_LOG (1 << (PMEM_POOL_TYPE_LOG + OPT_SHIFT)) #define OPT_BLK (1 << (PMEM_POOL_TYPE_BLK + OPT_SHIFT)) #define OPT_OBJ (1 << (PMEM_POOL_TYPE_OBJ + OPT_SHIFT)) #define OPT_BTT (1 << (PMEM_POOL_TYPE_BTT + OPT_SHIFT)) #define OPT_ALL (OPT_LOG | OPT_BLK | OPT_OBJ | OPT_BTT) #define OPT_REQ_SHIFT 8 #define OPT_REQ_MASK ((1 << OPT_REQ_SHIFT) - 1) #define _OPT_REQ(c, n) ((c) << (OPT_REQ_SHIFT * (n))) #define OPT_REQ0(c) _OPT_REQ(c, 0) #define OPT_REQ1(c) _OPT_REQ(c, 1) #define OPT_REQ2(c) _OPT_REQ(c, 2) #define OPT_REQ3(c) _OPT_REQ(c, 3) #define OPT_REQ4(c) _OPT_REQ(c, 4) #define OPT_REQ5(c) _OPT_REQ(c, 5) #define OPT_REQ6(c) _OPT_REQ(c, 6) #define OPT_REQ7(c) _OPT_REQ(c, 7) #ifndef min #define min(a, b) ((a) < (b) ? (a) : (b)) #endif #define FOREACH_RANGE(range, ranges)\ PMDK_LIST_FOREACH(range, &(ranges)->head, next) #define PLIST_OFF_TO_PTR(pop, off)\ ((off) == 0 ? NULL : (void *)((uintptr_t)(pop) + (off) - OBJ_OOB_SIZE)) #define ENTRY_TO_ALLOC_HDR(entry)\ ((void *)((uintptr_t)(entry) - sizeof(struct allocation_header))) #define OBJH_FROM_PTR(ptr)\ ((void *)((uintptr_t)(ptr) - sizeof(struct legacy_object_header))) #define DEFAULT_HDR_SIZE PMEM_PAGESIZE #define DEFAULT_DESC_SIZE PMEM_PAGESIZE #define POOL_HDR_DESC_SIZE (DEFAULT_HDR_SIZE + DEFAULT_DESC_SIZE) #define PTR_TO_ALLOC_HDR(ptr)\ ((void *)((uintptr_t)(ptr) -\ sizeof(struct legacy_object_header))) #define OBJH_TO_PTR(objh)\ ((void *)((uintptr_t)(objh) + sizeof(struct legacy_object_header))) /* invalid answer for ask_* functions */ #define INV_ANS '\0' #define FORMAT_PRINTF(a, b) __attribute__((__format__(__printf__, (a), (b)))) /* * pmem_pool_type_t -- pool types */ typedef enum { PMEM_POOL_TYPE_LOG = 0x01, PMEM_POOL_TYPE_BLK = 0x02, PMEM_POOL_TYPE_OBJ = 0x04, PMEM_POOL_TYPE_BTT = 0x08, PMEM_POOL_TYPE_ALL = 0x0f, PMEM_POOL_TYPE_UNKNOWN = 0x80, } pmem_pool_type_t; struct option_requirement { int opt; pmem_pool_type_t type; uint64_t req; }; struct options { const struct option *opts; size_t noptions; char *bitmap; const struct option_requirement *req; }; struct pmem_pool_params { pmem_pool_type_t type; char signature[POOL_HDR_SIG_LEN]; uint64_t size; mode_t mode; int is_poolset; int is_part; int is_checksum_ok; union { struct { uint64_t bsize; } blk; struct { char layout[PMEMOBJ_MAX_LAYOUT]; } obj; }; }; struct pool_set_file { int fd; char *fname; void *addr; size_t size; struct pool_set *poolset; size_t replica; time_t mtime; mode_t mode; bool fileio; }; struct pool_set_file *pool_set_file_open(const char *fname, int rdonly, int check); void pool_set_file_close(struct pool_set_file *file); int pool_set_file_read(struct pool_set_file *file, void *buff, size_t nbytes, uint64_t off); int pool_set_file_write(struct pool_set_file *file, void *buff, size_t nbytes, uint64_t off); int pool_set_file_set_replica(struct pool_set_file *file, size_t replica); size_t pool_set_file_nreplicas(struct pool_set_file *file); void *pool_set_file_map(struct pool_set_file *file, uint64_t offset); void pool_set_file_persist(struct pool_set_file *file, const void *addr, size_t len); struct range { PMDK_LIST_ENTRY(range) next; uint64_t first; uint64_t last; }; struct ranges { PMDK_LIST_HEAD(rangeshead, range) head; }; pmem_pool_type_t pmem_pool_type_parse_hdr(const struct pool_hdr *hdrp); pmem_pool_type_t pmem_pool_type(const void *base_pool_addr); int pmem_pool_checksum(const void *base_pool_addr); pmem_pool_type_t pmem_pool_type_parse_str(const char *str); uint64_t pmem_pool_get_min_size(pmem_pool_type_t type); int pmem_pool_parse_params(const char *fname, struct pmem_pool_params *paramsp, int check); int util_poolset_map(const char *fname, struct pool_set **poolset, int rdonly); struct options *util_options_alloc(const struct option *options, size_t nopts, const struct option_requirement *req); void util_options_free(struct options *opts); int util_options_verify(const struct options *opts, pmem_pool_type_t type); int util_options_getopt(int argc, char *argv[], const char *optstr, const struct options *opts); pmem_pool_type_t util_get_pool_type_second_page(const void *pool_base_addr); int util_parse_mode(const char *str, mode_t *mode); int util_parse_ranges(const char *str, struct ranges *rangesp, struct range entire); int util_ranges_add(struct ranges *rangesp, struct range range); void util_ranges_clear(struct ranges *rangesp); int util_ranges_contain(const struct ranges *rangesp, uint64_t n); int util_ranges_empty(const struct ranges *rangesp); int util_check_memory(const uint8_t *buff, size_t len, uint8_t val); int util_parse_chunk_types(const char *str, uint64_t *types); int util_parse_lane_sections(const char *str, uint64_t *types); char ask(char op, char *answers, char def_ans, const char *fmt, va_list ap); char ask_Yn(char op, const char *fmt, ...) FORMAT_PRINTF(2, 3); char ask_yN(char op, const char *fmt, ...) FORMAT_PRINTF(2, 3); unsigned util_heap_max_zone(size_t size); int util_pool_clear_badblocks(const char *path, int create); static const struct range ENTIRE_UINT64 = { { NULL, NULL }, /* range */ 0, /* first */ UINT64_MAX /* last */ }; pmdk-1.8/src/tools/pmempool/feature.c0000664000000000000000000001270413615011243016373 0ustar rootroot/* * Copyright 2018-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * feature.c -- pmempool feature command source file */ #include #include #include "common.h" #include "feature.h" #include "output.h" #include "libpmempool.h" /* operations over features */ enum feature_op { undefined, enable, disable, query }; /* * feature_ctx -- context and arguments for feature command */ struct feature_ctx { int verbose; const char *fname; enum feature_op op; enum pmempool_feature feature; unsigned flags; }; /* * pmempool_feature_default -- default arguments for feature command */ static const struct feature_ctx pmempool_feature_default = { .verbose = 0, .fname = NULL, .op = undefined, .feature = UINT32_MAX, .flags = 0 }; /* * help_str -- string for help message */ static const char * const help_str = "Toggle or query a pool feature\n" "\n" "For complete documentation see %s-feature(1) manual page.\n" ; /* * long_options -- command line options */ static const struct option long_options[] = { {"enable", required_argument, NULL, 'e'}, {"disable", required_argument, NULL, 'd'}, {"query", required_argument, NULL, 'q'}, {"verbose", no_argument, NULL, 'v'}, {"help", no_argument, NULL, 'h'}, {NULL, 0, NULL, 0 }, }; /* * print_usage -- print short description of application's usage */ static void print_usage(const char *appname) { printf("Usage: %s feature [] \n", appname); printf( "feature: SINGLEHDR, CKSUM_2K, SHUTDOWN_STATE, CHECK_BAD_BLOCKS\n"); } /* * print_version -- print version string */ static void print_version(const char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * pmempool_feature_help -- print help message for feature command */ void pmempool_feature_help(const char *appname) { print_usage(appname); print_version(appname); printf(help_str, appname); } /* * feature_perform -- perform operation over function */ static int feature_perform(struct feature_ctx *pfp) { int ret; switch (pfp->op) { case enable: return pmempool_feature_enable(pfp->fname, pfp->feature, pfp->flags); case disable: return pmempool_feature_disable(pfp->fname, pfp->feature, pfp->flags); case query: ret = pmempool_feature_query(pfp->fname, pfp->feature, pfp->flags); if (ret < 0) return 1; printf("%d", ret); return 0; default: outv_err("Invalid option."); return -1; } } /* * set_op -- set operation */ static void set_op(const char *appname, struct feature_ctx *pfp, enum feature_op op, const char *feature) { /* only one operation allowed */ if (pfp->op != undefined) goto misuse; pfp->op = op; /* parse feature name */ uint32_t fval = util_str2pmempool_feature(feature); if (fval == UINT32_MAX) goto misuse; pfp->feature = (enum pmempool_feature)fval; return; misuse: print_usage(appname); exit(EXIT_FAILURE); } /* * parse_args -- parse command line arguments */ static int parse_args(struct feature_ctx *pfp, const char *appname, int argc, char *argv[]) { int opt; while ((opt = getopt_long(argc, argv, "vhe:d:q:h", long_options, NULL)) != -1) { switch (opt) { case 'e': set_op(appname, pfp, enable, optarg); break; case 'd': set_op(appname, pfp, disable, optarg); break; case 'q': set_op(appname, pfp, query, optarg); break; case 'v': pfp->verbose = 2; break; case 'h': pmempool_feature_help(appname); exit(EXIT_SUCCESS); default: print_usage(appname); exit(EXIT_FAILURE); } } if (optind >= argc) { print_usage(appname); exit(EXIT_FAILURE); } pfp->fname = argv[optind]; return 0; } /* * pmempool_feature_func -- main function for feature command */ int pmempool_feature_func(const char *appname, int argc, char *argv[]) { struct feature_ctx pf = pmempool_feature_default; int ret = 0; /* parse command line arguments */ ret = parse_args(&pf, appname, argc, argv); if (ret) return ret; /* set verbosity level */ out_set_vlevel(pf.verbose); return feature_perform(&pf); } pmdk-1.8/src/tools/pmempool/info_blk.c0000664000000000000000000003427113615011243016526 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * info_blk.c -- pmempool info command source file for blk pool */ #include #include #include #include #include #include #include "os.h" #include "common.h" #include "output.h" #include "info.h" #include "btt.h" /* * pmempool_info_get_range -- get blocks/data chunk range * * Get range based on command line arguments and maximum value. * Return value: * 0 - range is empty * 1 - range is not empty */ static int pmempool_info_get_range(struct pmem_info *pip, struct range *rangep, struct range *curp, uint32_t max, uint64_t offset) { /* not using range */ if (!pip->args.use_range) { rangep->first = 0; rangep->last = max; return 1; } if (curp->first > offset + max) return 0; if (curp->first >= offset) rangep->first = curp->first - offset; else rangep->first = 0; if (curp->last < offset) return 0; if (curp->last <= offset + max) rangep->last = curp->last - offset; else rangep->last = max; return 1; } /* * info_blk_skip_block -- get action type for block/data chunk * * Return value indicating whether processing block/data chunk * should be skipped. * * Return values: * 0 - continue processing * 1 - skip current block */ static int info_blk_skip_block(struct pmem_info *pip, int is_zero, int is_error) { if (pip->args.blk.skip_no_flag && !is_zero && !is_error) return 1; if (is_zero && pip->args.blk.skip_zeros) return 1; if (is_error && pip->args.blk.skip_error) return 1; return 0; } /* * info_btt_data -- print block data and corresponding flags from map */ static int info_btt_data(struct pmem_info *pip, int v, struct btt_info *infop, uint64_t arena_off, uint64_t offset, uint64_t *countp) { if (!outv_check(v)) return 0; int ret = 0; size_t mapsize = infop->external_nlba * BTT_MAP_ENTRY_SIZE; uint32_t *map = malloc(mapsize); if (!map) err(1, "Cannot allocate memory for BTT map"); uint8_t *block_buff = malloc(infop->external_lbasize); if (!block_buff) err(1, "Cannot allocate memory for pmemblk block buffer"); /* read btt map area */ if (pmempool_info_read(pip, (uint8_t *)map, mapsize, arena_off + infop->mapoff)) { outv_err("wrong BTT Map size or offset\n"); ret = -1; goto error; } uint64_t i; struct range *curp = NULL; struct range range; FOREACH_RANGE(curp, &pip->args.ranges) { if (pmempool_info_get_range(pip, &range, curp, infop->external_nlba - 1, offset) == 0) continue; for (i = range.first; i <= range.last; i++) { uint32_t map_entry = le32toh(map[i]); int is_init = (map_entry & ~BTT_MAP_ENTRY_LBA_MASK) == 0; int is_zero = (map_entry & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_ZERO || is_init; int is_error = (map_entry & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_ERROR; uint64_t blockno = is_init ? i : map_entry & BTT_MAP_ENTRY_LBA_MASK; if (info_blk_skip_block(pip, is_zero, is_error)) continue; /* compute block's data address */ uint64_t block_off = arena_off + infop->dataoff + blockno * infop->internal_lbasize; if (pmempool_info_read(pip, block_buff, infop->external_lbasize, block_off)) { outv_err("cannot read %lu block\n", i); ret = -1; goto error; } if (*countp == 0) outv_title(v, "PMEM BLK blocks data"); /* * Print block number, offset and flags * from map entry. */ outv(v, "Block %10lu: offset: %s\n", offset + i, out_get_btt_map_entry(map_entry)); /* dump block's data */ outv_hexdump(v, block_buff, infop->external_lbasize, block_off, 1); *countp = *countp + 1; } } error: free(map); free(block_buff); return ret; } /* * info_btt_map -- print all map entries */ static int info_btt_map(struct pmem_info *pip, int v, struct btt_info *infop, uint64_t arena_off, uint64_t offset, uint64_t *count) { if (!outv_check(v) && !outv_check(pip->args.vstats)) return 0; int ret = 0; size_t mapsize = infop->external_nlba * BTT_MAP_ENTRY_SIZE; uint32_t *map = malloc(mapsize); if (!map) err(1, "Cannot allocate memory for BTT map"); /* read btt map area */ if (pmempool_info_read(pip, (uint8_t *)map, mapsize, arena_off + infop->mapoff)) { outv_err("wrong BTT Map size or offset\n"); ret = -1; goto error; } uint32_t arena_count = 0; uint64_t i; struct range *curp = NULL; struct range range; FOREACH_RANGE(curp, &pip->args.ranges) { if (pmempool_info_get_range(pip, &range, curp, infop->external_nlba - 1, offset) == 0) continue; for (i = range.first; i <= range.last; i++) { uint32_t entry = le32toh(map[i]); int is_zero = (entry & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_ZERO || (entry & ~BTT_MAP_ENTRY_LBA_MASK) == 0; int is_error = (entry & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_ERROR; if (info_blk_skip_block(pip, is_zero, is_error) == 0) { if (arena_count == 0) outv_title(v, "PMEM BLK BTT Map"); if (is_zero) pip->blk.stats.zeros++; if (is_error) pip->blk.stats.errors++; if (!is_zero && !is_error) pip->blk.stats.noflag++; pip->blk.stats.total++; arena_count++; (*count)++; outv(v, "%010lu: %s\n", offset + i, out_get_btt_map_entry(entry)); } } } error: free(map); return ret; } /* * info_btt_flog -- print all flog entries */ static int info_btt_flog(struct pmem_info *pip, int v, struct btt_info *infop, uint64_t arena_off) { if (!outv_check(v)) return 0; int ret = 0; struct btt_flog *flogp = NULL; struct btt_flog *flogpp = NULL; uint64_t flog_size = infop->nfree * roundup(2 * sizeof(struct btt_flog), BTT_FLOG_PAIR_ALIGN); flog_size = roundup(flog_size, BTT_ALIGNMENT); uint8_t *buff = malloc(flog_size); if (!buff) err(1, "Cannot allocate memory for FLOG entries"); if (pmempool_info_read(pip, buff, flog_size, arena_off + infop->flogoff)) { outv_err("cannot read BTT FLOG"); ret = -1; goto error; } outv_title(v, "PMEM BLK BTT FLOG"); uint8_t *ptr = buff; uint32_t i; for (i = 0; i < infop->nfree; i++) { flogp = (struct btt_flog *)ptr; flogpp = flogp + 1; btt_flog_convert2h(flogp); btt_flog_convert2h(flogpp); outv(v, "%010d:\n", i); outv_field(v, "LBA", "0x%08x", flogp->lba); outv_field(v, "Old map", "0x%08x: %s", flogp->old_map, out_get_btt_map_entry(flogp->old_map)); outv_field(v, "New map", "0x%08x: %s", flogp->new_map, out_get_btt_map_entry(flogp->new_map)); outv_field(v, "Seq", "0x%x", flogp->seq); outv_field(v, "LBA'", "0x%08x", flogpp->lba); outv_field(v, "Old map'", "0x%08x: %s", flogpp->old_map, out_get_btt_map_entry(flogpp->old_map)); outv_field(v, "New map'", "0x%08x: %s", flogpp->new_map, out_get_btt_map_entry(flogpp->new_map)); outv_field(v, "Seq'", "0x%x", flogpp->seq); ptr += BTT_FLOG_PAIR_ALIGN; } error: free(buff); return ret; } /* * info_btt_stats -- print btt related statistics */ static void info_btt_stats(struct pmem_info *pip, int v) { if (pip->blk.stats.total > 0) { outv_title(v, "PMEM BLK Statistics"); double perc_zeros = (double)pip->blk.stats.zeros / (double)pip->blk.stats.total * 100.0; double perc_errors = (double)pip->blk.stats.errors / (double)pip->blk.stats.total * 100.0; double perc_noflag = (double)pip->blk.stats.noflag / (double)pip->blk.stats.total * 100.0; outv_field(v, "Total blocks", "%u", pip->blk.stats.total); outv_field(v, "Zeroed blocks", "%u [%s]", pip->blk.stats.zeros, out_get_percentage(perc_zeros)); outv_field(v, "Error blocks", "%u [%s]", pip->blk.stats.errors, out_get_percentage(perc_errors)); outv_field(v, "Blocks without flag", "%u [%s]", pip->blk.stats.noflag, out_get_percentage(perc_noflag)); } } /* * info_btt_info -- print btt_info structure fields */ static int info_btt_info(struct pmem_info *pip, int v, struct btt_info *infop) { outv_field(v, "Signature", "%.*s", BTTINFO_SIG_LEN, infop->sig); outv_field(v, "UUID of container", "%s", out_get_uuid_str(infop->parent_uuid)); outv_field(v, "Flags", "0x%x", infop->flags); outv_field(v, "Major", "%d", infop->major); outv_field(v, "Minor", "%d", infop->minor); outv_field(v, "External LBA size", "%s", out_get_size_str(infop->external_lbasize, pip->args.human)); outv_field(v, "External LBA count", "%u", infop->external_nlba); outv_field(v, "Internal LBA size", "%s", out_get_size_str(infop->internal_lbasize, pip->args.human)); outv_field(v, "Internal LBA count", "%u", infop->internal_nlba); outv_field(v, "Free blocks", "%u", infop->nfree); outv_field(v, "Info block size", "%s", out_get_size_str(infop->infosize, pip->args.human)); outv_field(v, "Next arena offset", "0x%lx", infop->nextoff); outv_field(v, "Arena data offset", "0x%lx", infop->dataoff); outv_field(v, "Area map offset", "0x%lx", infop->mapoff); outv_field(v, "Area flog offset", "0x%lx", infop->flogoff); outv_field(v, "Info block backup offset", "0x%lx", infop->infooff); outv_field(v, "Checksum", "%s", out_get_checksum(infop, sizeof(*infop), &infop->checksum, 0)); return 0; } /* * info_btt_layout -- print information about BTT layout */ static int info_btt_layout(struct pmem_info *pip, os_off_t btt_off) { int ret = 0; if (btt_off <= 0) { outv_err("wrong BTT layout offset\n"); return -1; } struct btt_info *infop = NULL; infop = malloc(sizeof(struct btt_info)); if (!infop) err(1, "Cannot allocate memory for BTT Info structure"); int narena = 0; uint64_t cur_lba = 0; uint64_t count_data = 0; uint64_t count_map = 0; uint64_t offset = (uint64_t)btt_off; uint64_t nextoff = 0; do { /* read btt info area */ if (pmempool_info_read(pip, infop, sizeof(*infop), offset)) { ret = -1; outv_err("cannot read BTT Info header\n"); goto err; } if (util_check_memory((uint8_t *)infop, sizeof(*infop), 0) == 0) { outv(1, "\n\n"); break; } outv(1, "\n[ARENA %d]", narena); outv_title(1, "PMEM BLK BTT Info Header"); outv_hexdump(pip->args.vhdrdump, infop, sizeof(*infop), offset, 1); btt_info_convert2h(infop); nextoff = infop->nextoff; /* print btt info fields */ if (info_btt_info(pip, 1, infop)) { ret = -1; goto err; } /* dump blocks data */ if (info_btt_data(pip, pip->args.vdata, infop, offset, cur_lba, &count_data)) { ret = -1; goto err; } /* print btt map entries and get statistics */ if (info_btt_map(pip, pip->args.blk.vmap, infop, offset, cur_lba, &count_map)) { ret = -1; goto err; } /* print flog entries */ if (info_btt_flog(pip, pip->args.blk.vflog, infop, offset)) { ret = -1; goto err; } /* increment LBA's counter before reading info backup */ cur_lba += infop->external_nlba; /* read btt info backup area */ if (pmempool_info_read(pip, infop, sizeof(*infop), offset + infop->infooff)) { outv_err("wrong BTT Info Backup size or offset\n"); ret = -1; goto err; } outv_title(pip->args.blk.vbackup, "PMEM BLK BTT Info Header Backup"); if (outv_check(pip->args.blk.vbackup)) outv_hexdump(pip->args.vhdrdump, infop, sizeof(*infop), offset + infop->infooff, 1); btt_info_convert2h(infop); info_btt_info(pip, pip->args.blk.vbackup, infop); offset += nextoff; narena++; } while (nextoff > 0); info_btt_stats(pip, pip->args.vstats); err: if (infop) free(infop); return ret; } /* * info_blk_descriptor -- print pmemblk descriptor */ static void info_blk_descriptor(struct pmem_info *pip, int v, struct pmemblk *pbp) { size_t pmemblk_size; #ifdef DEBUG pmemblk_size = offsetof(struct pmemblk, write_lock); #else pmemblk_size = sizeof(*pbp); #endif outv_title(v, "PMEM BLK Header"); /* dump pmemblk header without pool_hdr */ outv_hexdump(pip->args.vhdrdump, (uint8_t *)pbp + sizeof(pbp->hdr), pmemblk_size - sizeof(pbp->hdr), sizeof(pbp->hdr), 1); outv_field(v, "Block size", "%s", out_get_size_str(pbp->bsize, pip->args.human)); outv_field(v, "Is zeroed", pbp->is_zeroed ? "true" : "false"); } /* * pmempool_info_blk -- print information about block type pool */ int pmempool_info_blk(struct pmem_info *pip) { int ret; struct pmemblk *pbp = malloc(sizeof(struct pmemblk)); if (!pbp) err(1, "Cannot allocate memory for pmemblk structure"); if (pmempool_info_read(pip, pbp, sizeof(struct pmemblk), 0)) { outv_err("cannot read pmemblk header\n"); free(pbp); return -1; } info_blk_descriptor(pip, VERBOSE_DEFAULT, pbp); ssize_t btt_off = (char *)pbp->data - (char *)pbp->addr; ret = info_btt_layout(pip, btt_off); free(pbp); return ret; } /* * pmempool_info_btt -- print information about btt device */ int pmempool_info_btt(struct pmem_info *pip) { int ret; outv(1, "\nBTT Device"); ret = info_btt_layout(pip, DEFAULT_HDR_SIZE); return ret; } pmdk-1.8/src/tools/pmempool/output.h0000664000000000000000000000700113615011243016277 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * output.h -- declarations of output printing related functions */ #include #include #include void out_set_vlevel(int vlevel); void out_set_stream(FILE *stream); void out_set_prefix(const char *prefix); void out_set_col_width(unsigned col_width); void outv_err(const char *fmt, ...) FORMAT_PRINTF(1, 2); void out_err(const char *file, int line, const char *func, const char *fmt, ...) FORMAT_PRINTF(4, 5); void outv_err_vargs(const char *fmt, va_list ap); void outv_indent(int vlevel, int i); void outv(int vlevel, const char *fmt, ...) FORMAT_PRINTF(2, 3); void outv_nl(int vlevel); int outv_check(int vlevel); void outv_title(int vlevel, const char *fmt, ...) FORMAT_PRINTF(2, 3); void outv_field(int vlevel, const char *field, const char *fmt, ...) FORMAT_PRINTF(3, 4); void outv_hexdump(int vlevel, const void *addr, size_t len, size_t offset, int sep); const char *out_get_uuid_str(uuid_t uuid); const char *out_get_time_str(time_t time); const char *out_get_size_str(uint64_t size, int human); const char *out_get_percentage(double percentage); const char *out_get_checksum(void *addr, size_t len, uint64_t *csump, uint64_t skip_off); const char *out_get_btt_map_entry(uint32_t map); const char *out_get_pool_type_str(pmem_pool_type_t type); const char *out_get_pool_signature(pmem_pool_type_t type); const char *out_get_tx_state_str(uint64_t state); const char *out_get_chunk_type_str(enum chunk_type type); const char *out_get_chunk_flags(uint16_t flags); const char *out_get_zone_magic_str(uint32_t magic); const char *out_get_pmemoid_str(PMEMoid oid, uint64_t uuid_lo); const char *out_get_arch_machine_class_str(uint8_t machine_class); const char *out_get_arch_data_str(uint8_t data); const char *out_get_arch_machine_str(uint16_t machine); const char *out_get_last_shutdown_str(uint8_t dirty); const char *out_get_alignment_desc_str(uint64_t ad, uint64_t cur_ad); const char *out_get_incompat_features_str(uint32_t incompat); pmdk-1.8/src/tools/pmempool/pmempool.vcxproj0000664000000000000000000002070713615011243020043 0ustar rootroot Debug x64 Release x64 {492baa3d-0d5d-478e-9765-500463ae69aa} {f7c6c6b6-4142-4c82-8699-4a9d8183181b} {0b1818eb-bdc8-4865-964f-db8bf05cfd86} {1baa1617-93ae-4196-8a1a-bd492fb18aef} {cf9a0883-6334-44c7-ac29-349468c78e27} {9e9e3d25-2139-4a5d-9200-18148ddead45} {9186eac4-2f34-4f17-b940-6585d7869bcd} {7DC3B3DD-73ED-4602-9AF3-8D7053620DEA} Win32Proj pmempool 10.0.16299.0 Application true v140 NotSet Application false v140 false NotSet $(SolutionDir)\common;$(SolutionDir)\test\unittest;$(SolutionDir)\windows\include;$(SolutionDir)\include;$(SolutionDir)\windows\getopt;$(SolutionDir)\libpmemlog;$(SolutionDir)\libpmemblk;$(SolutionDir)\libpmemobj;$(IncludePath) $(SolutionDir)$(Platform)\$(Configuration)\libs\ $(SolutionDir)\common;$(SolutionDir)\test\unittest;$(SolutionDir)\windows\include;$(SolutionDir)\include;$(SolutionDir)\windows\getopt;$(SolutionDir)\libpmemlog;$(SolutionDir)\libpmemblk;$(SolutionDir)\libpmemobj;$(IncludePath) $(SolutionDir)$(Platform)\$(Configuration)\libs\ NotUsing Level3 PMDK_UTF8_API;SDS_ENABLED; NTDDI_VERSION=NTDDI_WIN10_RS1;_DEBUG;_CONSOLE;%(PreprocessorDefinitions) platform.h CompileAsC false true Console Shlwapi.lib;%(AdditionalDependencies) true Debug _DEBUG Level3 NotUsing true PMDK_UTF8_API;SDS_ENABLED; NTDDI_VERSION=NTDDI_WIN10_RS1;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) CompileAsC platform.h false true Console Shlwapi.lib;%(AdditionalDependencies) true DebugFastLink pmdk-1.8/src/tools/pmempool/dump.h0000664000000000000000000000335413615011243015713 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * dump.h -- pmempool dump command header file */ int pmempool_dump_func(const char *appname, int argc, char *argv[]); void pmempool_dump_help(const char *appname); pmdk-1.8/src/tools/pmempool/README0000664000000000000000000002600613615011243015454 0ustar rootrootPersistent Memory Development Kit This is src/tools/pmempool/README. This file contains the high-level description of pmempool utility. 1. Introduction 2. Subcommands 2.1. info 2.2. check 2.3. create 2.4. dump 2.5. rm 2.6. sync 2.7. transform 2.8. convert 3. Source code 4. Packaging 5. Versioning 1. Introduction --------------- The main purpose of pmempool is to provide a user with set of utilities for off-line analysis and manipulation of pools created by pmem libraries. The pmempool is a generic command which consists of subcommands for specific purposes. Some of commands are required to work without any impact on processed pool, but some of them may create new or modify existing one. The pmempool may be useful for troubleshooting and may be used by system administrators and by software developers work on applications based on Persistent Memory Development Kit. The latter may find these tools useful for testing and debugging purposes also. Currently there is a following set of commands available: * info - Prints information and statistics in human-readable format about specified pool. * check - Checks pool's consistency and repairs pool if it is not consistent. * create - Creates a pool of specified type with additional properties specific for this type of pool. * dump - Dumps usable data from pool in hexadecimal or binary format. * rm - Removes pool file or all pool files listed in poolset configuration file. * sync - Synchronizes replicas within a poolset. * transform - Modifies internal structure of a poolset. * convert - Updates the pool to the latest available layout version. This file contains high-level description of available commands and their features. For details about usage and available command line arguments please refer to specific manual pages. There is one common manual page with description of all commands - pmempool(1) and manual pages which describe all commands in detail: pmempool-info(1) pmempool-check(1) pmempool-create(1) pmempool-dump(1) pmempool-rm(1) pmempool-sync(1) pmempool-transform(1) pmempool-convert(1) Subsequent sections contain detailed description of each command, information about the source code, packaging and versioning scheme. 2. Subcommands -------------- The pmempool application contains number of commands which perform specific operations on pool. The following subsections contain detailed description of existing commands. 2.1. info --------- The pmempool invoked with *info* command analyzes the existing pool created by PMDK libraries. The main task of this command is to print all usable information from pool headers and user data in human readable format. It automatically recognizes pool type by parsing and analyzing pool header. The recognition is done by checking the signature in pool header. The main intention of *info* command is to present internal data structures as they are stored in file - not for checking consistency. For this purpose there is *check* command available. The pmempool with *info* command analyzes a pool file as long as it is possible regarding correctness of internal meta-data (correct offsets, sizes etc.). If it is not possible to analyze rest of file, pmempool exits with an error code and prints an appropriate error message. Currently there is lack of interprocess synchronization for pool files, so the pmempool with *info* command should be invoked off-line. Using pmempool on pool file which may be modified by another process may lead to stopping processing the file. There is a set of common features for all pool types and a set of features specific for particular pool type. All features are described below. ** Common features * The basic function of *info* command is to print information about the most important internal data structures from specific pool. By default this is done by invoking pmempool with *info* command and one or more files. * It is possible to print basic statistics about the pool by passing appropriate command line argument. * The type of pool is recognized automatically. The exact list of headers and internal meta-data depends on pool's type. All information is displayed in human-readable format. * The pool header may be corrupted and automatic recognition of pool's type will fail. In order to analyze a pool file as a pool of specific type it is possible to force that by specifying the desired pool type using appropriate command line argument. * Headers and internal meta-data are displayed in human-readable format by default. However it is possible to display them in mixed format which consists of hexadecimal dump of headers and parsed data in human-readable format. * By default only non-volatile fields from internal structures are displayed. In order to display volatile fields you should increase the verbosity level. * By default all sizes are displayed in bytes unit. It is possible to print them in more human-readable formats with appropriate units (e.g. 4k, 8M, 16G). ** Features for *log* pool type * By default pmempool with *info* command displays the pool header, log pool type specific header and statistics. It is possible to print data in hexadecimal format by passing appropriate command line option. * It is possible to walk through the usable data using fixed data chunk size. This feature uses similar approach as pmemlog_walk() function. For details please refer to libpmemlog(7). ** Features for *blk* pool type * By default pmempool with *info* command displays the pool header, blk pool type specific header, BTT Info header and statistics. * It is possible to print more headers and internal data by passing specific command line options. It is possible to print the following sections: BTT Map entries, BTT FLOG, BTT Info backup and data blocks. * It is possible to print specific range of blocks in both absolute or relative manner (e.g. display blocks from 10 to 1000, display 10 blocks starting from block number 1000) * By default when displaying data blocks all blocks are displayed. However it is possible to skip blocks marked with zero or error flags, or to skip blocks which are not marked by any flag. Skipping blocks has impact on blocks ranges (e.g. display 10 blocks marked with error flag in the range from 0 to 10000) 2.2. check ---------- The pmempool invoked with *check* command checks existing pool's consistency. If the pool is consistent pmempool exits with exit code 0. Otherwise nonzero error code is returned and appropriate message is displayed. In addition it may also try to fix some common known errors upon explicit demand of user. In this case a pool file will be opened in read-write mode so the user should be aware of modifications made by pmempool application. Below is the description of available features: * By default pmempool with *check* command prints brief description about encountered error(s) and proper error value is returned. If there is no error nothing is printed and exit code is 0. * If an error is encountered while checking a consistency of a pool it is possible to try to fix all errors. In this case the pool file will be opened in read-write mode. * User may request to _not_ modify pool's file when trying to repair it but just to report what would be done if the repair was performed. * When repairing a pool user may request to create backup before any modification is made. If it is not possible to create full backup of existing pool, the process will terminate. 2.3. create ----------- The pmempool invoked with *create* command creates a pool file of specific type and size. Depending on pool's type it is possible to provide more desired properties of a pool. Below is the description of available features: * The main feature is to create pool of specified type and size. Therefore it is required to pass at least two command line arguments. * User may want to create a pool file with size of the whole partition. This is possible by passing proper command line argument. * It is possible to create a pool with the same parameters as another pool passed in command line arguments - it may be considered as cloning the pool. 2.4. dump --------- The pmempool invoked with *dump* command dumps usable data from specified pool. This may be dumped either in hexadecimal or binary format. Below is the description of available features: * The main feature is to dump all data from a pool file. In case of dumping data to terminal by default data is dumped in hexadecimal format. In case of redirecting standard output to a file data will be dumped in binary format. * It is possible to specify the format of dumped data to either hexadecimal or binary. * By default data is dumped to standard output. However it is possible to specify a file name to dump data into. * In case of pmem blk pool type it is possible to set range of blocks in either absolute or relative manner. * In case of pmem log pool type it is possible to set size of chunk and range of chunks to dump in either absolute or relative manner. 2.5. rm ------- The pmempool *rm* command is a simple helper utility which removes pool files created using either PMDK libraries or pmempool *create* command. * The main feature is to parse the poolset configuration file and remove all listed pool files. * It is possible to run the pmempool *rm* command in interactive mode, where before removing each file the user must confirm the removal operation. * The command line interface is similar to interface provided by standard, system *rm* command. 2.6. sync --------- The pmempool *sync* synchronize data between replicas within a poolset. The command has the following features: * Metadata in a poolset are checked for consistency. * Missing or damaged parts are recreated. 2.7. transform -------------- The pmempool *transform* command modifies internal structure of a poolset. Available features of the command: * Adding replicas. * Removing replicas. 2.8. convert -------------- The pmempool invoked with the *convert* command performs a conversion of the specified pool to the newest layout supported by this tool. Currently only libpmemobj pools are supported. It is advised to have a backup of the pool before conversion. 3. Source code -------------- The source code of pmempool is located in pmempool directory. By default pmempool is installed in $(DESTDIR)/usr/bin directory. You can change it by passing $(TOOLSDIR) variable to "make install". For example, the following command will install pmempool in ~/bin directory: $ make install DESTDIR=~ TOOLSDIR=/bin See the top-level README file for detailed information about building and installation. 4. Packaging ------------ The pmempool application is provided in separate packages. Both rpm and dpkg packages are built automatically with other packages. See the top-level README file for detailed information about building packages. 5. Versioning ------------- The versioning of pmempool application is the same as all PMDK libraries. pmdk-1.8/src/tools/pmempool/synchronize.h0000664000000000000000000000336313615011243017321 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * synchronize.h -- pmempool sync command header file */ int pmempool_sync_func(const char *appname, int argc, char *argv[]); void pmempool_sync_help(const char *appname); pmdk-1.8/src/tools/pmempool/transform.h0000664000000000000000000000340013615011243016751 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * transform.h -- pmempool transform command header file */ int pmempool_transform_func(const char *appname, int argc, char *argv[]); void pmempool_transform_help(const char *appname); pmdk-1.8/src/tools/pmempool/create.c0000664000000000000000000004016613615011243016206 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * create.c -- pmempool create command source file */ #include #include #include #include #include #include #include #include #include #include #include #include #include "common.h" #include "file.h" #include "create.h" #include "os.h" #include "set.h" #include "output.h" #include "libpmemblk.h" #include "libpmemlog.h" #include "libpmempool.h" #define DEFAULT_MODE 0664 /* * pmempool_create -- context and args for create command */ struct pmempool_create { int verbose; char *fname; int fexists; char *inherit_fname; int max_size; char *str_type; struct pmem_pool_params params; struct pmem_pool_params inherit_params; char *str_size; char *str_mode; char *str_bsize; uint64_t csize; int write_btt_layout; int force; char *layout; struct options *opts; int clearbadblocks; }; /* * pmempool_create_default -- default args for create command */ static const struct pmempool_create pmempool_create_default = { .verbose = 0, .fname = NULL, .fexists = 0, .inherit_fname = NULL, .max_size = 0, .str_type = NULL, .str_bsize = NULL, .csize = 0, .write_btt_layout = 0, .force = 0, .layout = NULL, .clearbadblocks = 0, .params = { .type = PMEM_POOL_TYPE_UNKNOWN, .size = 0, .mode = DEFAULT_MODE, } }; /* * help_str -- string for help message */ static const char * const help_str = "Create pmem pool of specified size, type and name\n" "\n" "Common options:\n" " -s, --size size of pool\n" " -M, --max-size use maximum available space on file system\n" " -m, --mode set permissions to (the default is 0664)\n" " -i, --inherit take required parameters from specified pool file\n" " -b, --clear-bad-blocks clear bad blocks in existing files\n" " -f, --force remove the pool first\n" " -v, --verbose increase verbosity level\n" " -h, --help display this help and exit\n" "\n" "Options for PMEMBLK:\n" " -w, --write-layout force writing the BTT layout\n" "\n" "Options for PMEMOBJ:\n" " -l, --layout layout name stored in pool's header\n" "\n" "For complete documentation see %s-create(1) manual page.\n" ; /* * long_options -- command line options */ static const struct option long_options[] = { {"size", required_argument, NULL, 's' | OPT_ALL}, {"verbose", no_argument, NULL, 'v' | OPT_ALL}, {"help", no_argument, NULL, 'h' | OPT_ALL}, {"max-size", no_argument, NULL, 'M' | OPT_ALL}, {"inherit", required_argument, NULL, 'i' | OPT_ALL}, {"mode", required_argument, NULL, 'm' | OPT_ALL}, {"write-layout", no_argument, NULL, 'w' | OPT_BLK}, {"layout", required_argument, NULL, 'l' | OPT_OBJ}, {"force", no_argument, NULL, 'f' | OPT_ALL}, {"clear-bad-blocks", no_argument, NULL, 'b' | OPT_ALL}, {NULL, 0, NULL, 0 }, }; /* * print_usage -- print application usage short description */ static void print_usage(const char *appname) { printf("Usage: %s create [] [] \n", appname); } /* * print_version -- print version string */ static void print_version(const char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * pmempool_create_help -- print help message for create command */ void pmempool_create_help(const char *appname) { print_usage(appname); print_version(appname); printf(help_str, appname); } /* * pmempool_create_obj -- create pmem obj pool */ static int pmempool_create_obj(struct pmempool_create *pcp) { PMEMobjpool *pop = pmemobj_create(pcp->fname, pcp->layout, pcp->params.size, pcp->params.mode); if (!pop) { outv_err("'%s' -- %s\n", pcp->fname, pmemobj_errormsg()); return -1; } pmemobj_close(pop); return 0; } /* * pmempool_create_blk -- create pmem blk pool */ static int pmempool_create_blk(struct pmempool_create *pcp) { ASSERTne(pcp->params.blk.bsize, 0); int ret = 0; PMEMblkpool *pbp = pmemblk_create(pcp->fname, pcp->params.blk.bsize, pcp->params.size, pcp->params.mode); if (!pbp) { outv_err("'%s' -- %s\n", pcp->fname, pmemblk_errormsg()); return -1; } if (pcp->write_btt_layout) { outv(1, "Writing BTT layout using block %d.\n", pcp->write_btt_layout); if (pmemblk_set_error(pbp, 0) || pmemblk_set_zero(pbp, 0)) { outv_err("writing BTT layout to block 0 failed\n"); ret = -1; } } pmemblk_close(pbp); return ret; } /* * pmempool_create_log -- create pmem log pool */ static int pmempool_create_log(struct pmempool_create *pcp) { PMEMlogpool *plp = pmemlog_create(pcp->fname, pcp->params.size, pcp->params.mode); if (!plp) { outv_err("'%s' -- %s\n", pcp->fname, pmemlog_errormsg()); return -1; } pmemlog_close(plp); return 0; } /* * pmempool_get_max_size -- return maximum allowed size of file */ #ifndef _WIN32 static int pmempool_get_max_size(const char *fname, uint64_t *sizep) { struct statvfs buf; int ret = 0; char *name = strdup(fname); if (name == NULL) { return -1; } char *dir = dirname(name); if (statvfs(dir, &buf)) ret = -1; else *sizep = buf.f_bsize * buf.f_bavail; free(name); return ret; } #else static int pmempool_get_max_size(const char *fname, uint64_t *sizep) { int ret = 0; ULARGE_INTEGER freespace; char *name = strdup(fname); if (name == NULL) { return -1; } char *dir = dirname(name); wchar_t *str = util_toUTF16(dir); if (str == NULL) { free(name); return -1; } if (GetDiskFreeSpaceExW(str, &freespace, NULL, NULL) == 0) ret = -1; else *sizep = freespace.QuadPart; free(str); free(name); return ret; } #endif /* * print_pool_params -- print some parameters of a pool */ static void print_pool_params(struct pmem_pool_params *params) { outv(1, "\ttype : %s\n", out_get_pool_type_str(params->type)); outv(1, "\tsize : %s\n", out_get_size_str(params->size, 2)); outv(1, "\tmode : 0%o\n", params->mode); switch (params->type) { case PMEM_POOL_TYPE_BLK: outv(1, "\tbsize : %s\n", out_get_size_str(params->blk.bsize, 0)); break; case PMEM_POOL_TYPE_OBJ: outv(1, "\tlayout: '%s'\n", params->obj.layout); break; default: break; } } /* * inherit_pool_params -- inherit pool parameters from specified file */ static int inherit_pool_params(struct pmempool_create *pcp) { outv(1, "Parsing pool: '%s'\n", pcp->inherit_fname); /* * If no type string passed, --inherit option must be passed * so parse file and get required parameters. */ if (pmem_pool_parse_params(pcp->inherit_fname, &pcp->inherit_params, 1)) { if (errno) perror(pcp->inherit_fname); else outv_err("%s: cannot determine type of pool\n", pcp->inherit_fname); return -1; } if (PMEM_POOL_TYPE_UNKNOWN == pcp->inherit_params.type) { outv_err("'%s' -- unknown pool type\n", pcp->inherit_fname); return -1; } print_pool_params(&pcp->inherit_params); return 0; } /* * pmempool_create_parse_args -- parse command line args */ static int pmempool_create_parse_args(struct pmempool_create *pcp, const char *appname, int argc, char *argv[], struct options *opts) { int opt, ret; while ((opt = util_options_getopt(argc, argv, "vhi:s:Mm:l:wfb", opts)) != -1) { switch (opt) { case 'v': pcp->verbose = 1; break; case 'h': pmempool_create_help(appname); exit(EXIT_SUCCESS); case 's': pcp->str_size = optarg; ret = util_parse_size(optarg, (size_t *)&pcp->params.size); if (ret || pcp->params.size == 0) { outv_err("invalid size value specified '%s'\n", optarg); return -1; } break; case 'M': pcp->max_size = 1; break; case 'm': pcp->str_mode = optarg; if (util_parse_mode(optarg, &pcp->params.mode)) { outv_err("invalid mode value specified '%s'\n", optarg); return -1; } break; case 'i': pcp->inherit_fname = optarg; break; case 'w': pcp->write_btt_layout = 1; break; case 'l': pcp->layout = optarg; break; case 'f': pcp->force = 1; break; case 'b': pcp->clearbadblocks = 1; break; default: print_usage(appname); return -1; } } /* check for , and strings */ if (optind + 2 < argc) { pcp->str_type = argv[optind]; pcp->str_bsize = argv[optind + 1]; pcp->fname = argv[optind + 2]; } else if (optind + 1 < argc) { pcp->str_type = argv[optind]; pcp->fname = argv[optind + 1]; } else if (optind < argc) { pcp->fname = argv[optind]; pcp->str_type = NULL; } else { print_usage(appname); return -1; } return 0; } static int allocate_max_size_available_file(const char *name_of_file, mode_t mode, os_off_t max_size) { int fd = os_open(name_of_file, O_CREAT | O_EXCL | O_RDWR, mode); if (fd == -1) { outv_err("!open '%s' failed", name_of_file); return -1; } os_off_t offset = 0; os_off_t length = max_size - (max_size % (os_off_t)Pagesize); int ret; do { ret = os_posix_fallocate(fd, offset, length); if (ret == 0) offset += length; else if (ret != ENOSPC) { os_close(fd); if (os_unlink(name_of_file) == -1) outv_err("!unlink '%s' failed", name_of_file); errno = ret; outv_err("!space allocation for '%s' failed", name_of_file); return -1; } length /= 2; length -= (length % (os_off_t)Pagesize); } while (length > (os_off_t)Pagesize); os_close(fd); return 0; } /* * pmempool_create_func -- main function for create command */ int pmempool_create_func(const char *appname, int argc, char *argv[]) { int ret = 0; struct pmempool_create pc = pmempool_create_default; pc.opts = util_options_alloc(long_options, sizeof(long_options) / sizeof(long_options[0]), NULL); /* parse command line arguments */ ret = pmempool_create_parse_args(&pc, appname, argc, argv, pc.opts); if (ret) exit(EXIT_FAILURE); /* set verbosity level */ out_set_vlevel(pc.verbose); umask(0); int exists = util_file_exists(pc.fname); if (exists < 0) return -1; pc.fexists = exists; int is_poolset = util_is_poolset_file(pc.fname) == 1; if (pc.inherit_fname) { if (inherit_pool_params(&pc)) { outv_err("parsing pool '%s' failed\n", pc.inherit_fname); return -1; } } /* * Parse pool type and other parameters if --inherit option * passed. It is possible to either pass --inherit option * or pool type string in command line arguments. This is * validated here. */ if (pc.str_type) { /* parse pool type string if passed in command line arguments */ pc.params.type = pmem_pool_type_parse_str(pc.str_type); if (PMEM_POOL_TYPE_UNKNOWN == pc.params.type) { outv_err("'%s' -- unknown pool type\n", pc.str_type); return -1; } if (PMEM_POOL_TYPE_BLK == pc.params.type) { if (pc.str_bsize == NULL) { outv_err("blk pool requires " "argument\n"); return -1; } if (util_parse_size(pc.str_bsize, (size_t *)&pc.params.blk.bsize)) { outv_err("cannot parse '%s' as block size\n", pc.str_bsize); return -1; } } if (PMEM_POOL_TYPE_OBJ == pc.params.type && pc.layout != NULL) { size_t max_layout = PMEMOBJ_MAX_LAYOUT; if (strlen(pc.layout) >= max_layout) { outv_err( "Layout name is too long, maximum number of characters (including the terminating null byte) is %zu\n", max_layout); return -1; } size_t len = sizeof(pc.params.obj.layout); strncpy(pc.params.obj.layout, pc.layout, len); pc.params.obj.layout[len - 1] = '\0'; } } else if (pc.inherit_fname) { pc.params.type = pc.inherit_params.type; } else { /* neither pool type string nor --inherit options passed */ print_usage(appname); return -1; } if (util_options_verify(pc.opts, pc.params.type)) return -1; if (pc.params.type != PMEM_POOL_TYPE_BLK && pc.str_bsize != NULL) { outv_err("invalid option specified for %s pool type" " -- block size\n", out_get_pool_type_str(pc.params.type)); return -1; } if (is_poolset) { if (pc.params.size) { outv_err("-s|--size cannot be used with " "poolset file\n"); return -1; } if (pc.max_size) { outv_err("-M|--max-size cannot be used with " "poolset file\n"); return -1; } } if (pc.params.size && pc.max_size) { outv_err("-M|--max-size option cannot be used with -s|--size" " option\n"); return -1; } if (pc.inherit_fname) { if (!pc.str_size && !pc.max_size) pc.params.size = pc.inherit_params.size; if (!pc.str_mode) pc.params.mode = pc.inherit_params.mode; switch (pc.params.type) { case PMEM_POOL_TYPE_BLK: if (!pc.str_bsize) pc.params.blk.bsize = pc.inherit_params.blk.bsize; break; case PMEM_POOL_TYPE_OBJ: if (!pc.layout) { memcpy(pc.params.obj.layout, pc.inherit_params.obj.layout, sizeof(pc.params.obj.layout)); } else { size_t len = sizeof(pc.params.obj.layout); strncpy(pc.params.obj.layout, pc.layout, len - 1); pc.params.obj.layout[len - 1] = '\0'; } break; default: break; } } /* * If neither --size nor --inherit options passed, check * for --max-size option - if not passed use minimum pool size. */ uint64_t min_size = pmem_pool_get_min_size(pc.params.type); if (pc.params.size == 0) { if (pc.max_size) { outv(1, "Maximum size option passed " "- getting available space of file system.\n"); ret = pmempool_get_max_size(pc.fname, &pc.params.size); if (ret) { outv_err("cannot get available space of fs\n"); return -1; } if (pc.params.size == 0) { outv_err("No space left on device\n"); return -1; } outv(1, "Available space is %s\n", out_get_size_str(pc.params.size, 2)); if (allocate_max_size_available_file(pc.fname, pc.params.mode, (os_off_t)pc.params.size)) return -1; /* * We are going to create pool based * on file size instead of the pc.params.size. */ pc.params.size = 0; } else { if (!pc.fexists) { outv(1, "No size option passed " "- picking minimum pool size.\n"); pc.params.size = min_size; } } } else { if (pc.params.size < min_size) { outv_err("size must be >= %lu bytes\n", min_size); return -1; } } if (pc.force) pmempool_rm(pc.fname, PMEMPOOL_RM_FORCE); outv(1, "Creating pool: %s\n", pc.fname); print_pool_params(&pc.params); if (pc.clearbadblocks) { int ret = util_pool_clear_badblocks(pc.fname, 1 /* ignore non-existing */); if (ret) { outv_err("'%s' -- clearing bad blocks failed\n", pc.fname); return -1; } } switch (pc.params.type) { case PMEM_POOL_TYPE_BLK: ret = pmempool_create_blk(&pc); break; case PMEM_POOL_TYPE_LOG: ret = pmempool_create_log(&pc); break; case PMEM_POOL_TYPE_OBJ: ret = pmempool_create_obj(&pc); break; default: ret = -1; break; } if (ret) { outv_err("creating pool file failed\n"); if (!pc.fexists) util_unlink(pc.fname); } util_options_free(pc.opts); return ret; } pmdk-1.8/src/tools/pmempool/.gitignore0000664000000000000000000000001113615011243016550 0ustar rootrootpmempool pmdk-1.8/src/tools/pmempool/transform.c0000664000000000000000000001212413615011243016747 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * transform.c -- pmempool transform command source file */ #include #include #include #include #include #include #include #include #include #include "common.h" #include "output.h" #include "transform.h" #include "libpmempool.h" /* * pmempool_transform_context -- context and arguments for transform command */ struct pmempool_transform_context { unsigned flags; /* flags which modify the command execution */ char *poolset_file_src; /* a path to a source poolset file */ char *poolset_file_dst; /* a path to a target poolset file */ }; /* * pmempool_transform_default -- default arguments for transform command */ static const struct pmempool_transform_context pmempool_transform_default = { .flags = 0, .poolset_file_src = NULL, .poolset_file_dst = NULL, }; /* * help_str -- string for help message */ static const char * const help_str = "Modify internal structure of a poolset\n" "\n" "Common options:\n" " -d, --dry-run do not apply changes, only check for viability of" " transformation\n" " -v, --verbose increase verbosity level\n" " -h, --help display this help and exit\n" "\n" "For complete documentation see %s-transform(1) manual page.\n" ; /* * long_options -- command line options */ static const struct option long_options[] = { {"dry-run", no_argument, NULL, 'd'}, {"help", no_argument, NULL, 'h'}, {"verbose", no_argument, NULL, 'v'}, {NULL, 0, NULL, 0 }, }; /* * print_usage -- print application usage short description */ static void print_usage(const char *appname) { printf("usage: %s transform [] " " \n", appname); } /* * print_version -- print version string */ static void print_version(const char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * pmempool_transform_help -- print help message for the transform command */ void pmempool_transform_help(const char *appname) { print_usage(appname); print_version(appname); printf(help_str, appname); } /* * pmempool_check_parse_args -- parse command line arguments */ static int pmempool_transform_parse_args(struct pmempool_transform_context *ctx, const char *appname, int argc, char *argv[]) { int opt; while ((opt = getopt_long(argc, argv, "dhv", long_options, NULL)) != -1) { switch (opt) { case 'd': ctx->flags = PMEMPOOL_TRANSFORM_DRY_RUN; break; case 'h': pmempool_transform_help(appname); exit(EXIT_SUCCESS); case 'v': out_set_vlevel(1); break; default: print_usage(appname); exit(EXIT_FAILURE); } } if (optind + 1 < argc) { ctx->poolset_file_src = argv[optind]; ctx->poolset_file_dst = argv[optind + 1]; } else { print_usage(appname); exit(EXIT_FAILURE); } return 0; } /* * pmempool_transform_func -- main function for the transform command */ int pmempool_transform_func(const char *appname, int argc, char *argv[]) { int ret; struct pmempool_transform_context ctx = pmempool_transform_default; /* parse command line arguments */ if ((ret = pmempool_transform_parse_args(&ctx, appname, argc, argv))) return ret; ret = pmempool_transform(ctx.poolset_file_src, ctx.poolset_file_dst, ctx.flags); if (ret) { if (errno) outv_err("%s\n", strerror(errno)); outv_err("failed to transform %s -> %s: %s\n", ctx.poolset_file_src, ctx.poolset_file_dst, pmempool_errormsg()); return -1; } else { outv(1, "%s -> %s: transformed\n", ctx.poolset_file_src, ctx.poolset_file_dst); return 0; } } pmdk-1.8/src/tools/pmempool/synchronize.c0000664000000000000000000001162613615011243017315 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * synchronize.c -- pmempool sync command source file */ #include "synchronize.h" #include #include #include #include #include #include #include #include #include #include "common.h" #include "output.h" #include "libpmempool.h" /* * pmempool_sync_context -- context and arguments for sync command */ struct pmempool_sync_context { unsigned flags; /* flags which modify the command execution */ char *poolset_file; /* a path to a poolset file */ }; /* * pmempool_sync_default -- default arguments for sync command */ static const struct pmempool_sync_context pmempool_sync_default = { .flags = 0, .poolset_file = NULL, }; /* * help_str -- string for help message */ static const char * const help_str = "Check consistency of a pool\n" "\n" "Common options:\n" " -b, --bad-blocks fix bad blocks - it requires creating or reading special recovery files\n" " -d, --dry-run do not apply changes, only check for viability of synchronization\n" " -v, --verbose increase verbosity level\n" " -h, --help display this help and exit\n" "\n" "For complete documentation see %s-sync(1) manual page.\n" ; /* * long_options -- command line options */ static const struct option long_options[] = { {"bad-blocks", no_argument, NULL, 'b'}, {"dry-run", no_argument, NULL, 'd'}, {"help", no_argument, NULL, 'h'}, {"verbose", no_argument, NULL, 'v'}, {NULL, 0, NULL, 0 }, }; /* * print_usage -- (internal) print application usage short description */ static void print_usage(const char *appname) { printf("usage: %s sync [] \n", appname); } /* * print_version -- (internal) print version string */ static void print_version(const char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * pmempool_sync_help -- print help message for the sync command */ void pmempool_sync_help(const char *appname) { print_usage(appname); print_version(appname); printf(help_str, appname); } /* * pmempool_sync_parse_args -- (internal) parse command line arguments */ static int pmempool_sync_parse_args(struct pmempool_sync_context *ctx, const char *appname, int argc, char *argv[]) { int opt; while ((opt = getopt_long(argc, argv, "bdhv", long_options, NULL)) != -1) { switch (opt) { case 'd': ctx->flags |= PMEMPOOL_SYNC_DRY_RUN; break; case 'b': ctx->flags |= PMEMPOOL_SYNC_FIX_BAD_BLOCKS; break; case 'h': pmempool_sync_help(appname); exit(EXIT_SUCCESS); case 'v': out_set_vlevel(1); break; default: print_usage(appname); exit(EXIT_FAILURE); } } if (optind < argc) { ctx->poolset_file = argv[optind]; } else { print_usage(appname); exit(EXIT_FAILURE); } return 0; } /* * pmempool_sync_func -- main function for the sync command */ int pmempool_sync_func(const char *appname, int argc, char *argv[]) { int ret = 0; struct pmempool_sync_context ctx = pmempool_sync_default; /* parse command line arguments */ if ((ret = pmempool_sync_parse_args(&ctx, appname, argc, argv))) return ret; ret = pmempool_sync(ctx.poolset_file, ctx.flags); if (ret) { outv_err("failed to synchronize: %s\n", pmempool_errormsg()); if (errno) outv_err("%s\n", strerror(errno)); return -1; } else { outv(1, "%s: synchronized\n", ctx.poolset_file); return 0; } } pmdk-1.8/src/tools/pmempool/convert.h0000664000000000000000000000342013615011243016420 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * convert.h -- pmempool convert command header file */ #include int pmempool_convert_func(const char *appname, int argc, char *argv[]); void pmempool_convert_help(const char *appname); pmdk-1.8/src/tools/pmempool/output.c0000664000000000000000000004403713615011243016304 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * output.c -- definitions of output printing related functions */ #include #include #include #include #include #include #include #include #include #include #include #include "feature.h" #include "common.h" #include "output.h" #define _STR(s) #s #define STR(s) _STR(s) #define TIME_STR_FMT "%a %b %d %Y %H:%M:%S" #define UUID_STR_MAX 37 #define HEXDUMP_ROW_WIDTH 16 /* * 2 chars + space per byte + * space after 8 bytes and terminating NULL */ #define HEXDUMP_ROW_HEX_LEN (HEXDUMP_ROW_WIDTH * 3 + 1 + 1) /* 1 printable char per byte + terminating NULL */ #define HEXDUMP_ROW_ASCII_LEN (HEXDUMP_ROW_WIDTH + 1) #define SEPARATOR_CHAR '-' #define MAX_INDENT 32 #define INDENT_CHAR ' ' static char out_indent_str[MAX_INDENT + 1]; static int out_indent_level; static int out_vlevel; static unsigned out_column_width = 20; static FILE *out_fh; static const char *out_prefix; #define STR_MAX 256 /* * outv_check -- verify verbosity level */ int outv_check(int vlevel) { return vlevel && (out_vlevel >= vlevel); } /* * out_set_col_width -- set column width * * See: outv_field() function */ void out_set_col_width(unsigned col_width) { out_column_width = col_width; } /* * out_set_vlevel -- set verbosity level */ void out_set_vlevel(int vlevel) { out_vlevel = vlevel; if (out_fh == NULL) out_fh = stdout; } /* * out_set_prefix -- set prefix to output format */ void out_set_prefix(const char *prefix) { out_prefix = prefix; } /* * out_set_stream -- set output stream */ void out_set_stream(FILE *stream) { out_fh = stream; memset(out_indent_str, INDENT_CHAR, MAX_INDENT); } /* * outv_err -- print error message */ void outv_err(const char *fmt, ...) { va_list ap; va_start(ap, fmt); outv_err_vargs(fmt, ap); va_end(ap); } /* * outv_err_vargs -- print error message */ void outv_err_vargs(const char *fmt, va_list ap) { char *_str = strdup(fmt); if (!_str) err(1, "strdup"); char *str = _str; fprintf(stderr, "error: "); int errstr = str[0] == '!'; if (errstr) str++; char *nl = strchr(str, '\n'); if (nl) *nl = '\0'; vfprintf(stderr, str, ap); if (errstr) fprintf(stderr, ": %s", strerror(errno)); fprintf(stderr, "\n"); free(_str); } /* * outv_indent -- change indentation level by factor */ void outv_indent(int vlevel, int i) { if (!outv_check(vlevel)) return; out_indent_str[out_indent_level] = INDENT_CHAR; out_indent_level += i; if (out_indent_level < 0) out_indent_level = 0; if (out_indent_level > MAX_INDENT) out_indent_level = MAX_INDENT; out_indent_str[out_indent_level] = '\0'; } /* * _out_prefix -- print prefix if defined */ static void _out_prefix(void) { if (out_prefix) fprintf(out_fh, "%s: ", out_prefix); } /* * _out_indent -- print indent */ static void _out_indent(void) { fprintf(out_fh, "%s", out_indent_str); } /* * outv -- print message taking into account verbosity level */ void outv(int vlevel, const char *fmt, ...) { va_list ap; if (!outv_check(vlevel)) return; _out_prefix(); _out_indent(); va_start(ap, fmt); vfprintf(out_fh, fmt, ap); va_end(ap); } /* * outv_nl -- print new line without indentation */ void outv_nl(int vlevel) { if (!outv_check(vlevel)) return; _out_prefix(); fprintf(out_fh, "\n"); } void outv_title(int vlevel, const char *fmt, ...) { va_list ap; if (!outv_check(vlevel)) return; fprintf(out_fh, "\n"); _out_prefix(); _out_indent(); va_start(ap, fmt); vfprintf(out_fh, fmt, ap); va_end(ap); fprintf(out_fh, ":\n"); } /* * outv_field -- print field name and value in specified format * * Field name will have fixed width which can be changed by * out_set_column_width() function. * vlevel - verbosity level * field - field name * fmt - format form value */ void outv_field(int vlevel, const char *field, const char *fmt, ...) { va_list ap; if (!outv_check(vlevel)) return; _out_prefix(); _out_indent(); va_start(ap, fmt); fprintf(out_fh, "%-*s : ", out_column_width, field); vfprintf(out_fh, fmt, ap); fprintf(out_fh, "\n"); va_end(ap); } /* * out_get_percentage -- return percentage string */ const char * out_get_percentage(double perc) { static char str_buff[STR_MAX] = {0, }; int ret = 0; if (perc > 0.0 && perc < 0.0001) { ret = snprintf(str_buff, STR_MAX, "%e %%", perc); if (ret < 0) return ""; } else { int decimal = 0; if (perc >= 100.0 || perc < DBL_EPSILON) decimal = 0; else decimal = 6; ret = snprintf(str_buff, STR_MAX, "%.*f %%", decimal, perc); if (ret < 0 || ret >= STR_MAX) return ""; } return str_buff; } /* * out_get_size_str -- return size string * * human - if 1 return size in human-readable format * if 2 return size in bytes and human-readable format * otherwise return size in bytes. */ const char * out_get_size_str(uint64_t size, int human) { static char str_buff[STR_MAX] = {0, }; char units[] = { 'K', 'M', 'G', 'T', '\0' }; const int nunits = sizeof(units) / sizeof(units[0]); int ret = 0; if (!human) { ret = snprintf(str_buff, STR_MAX, "%"PRIu64, size); } else { int i = -1; double dsize = (double)size; uint64_t csize = size; while (csize >= 1024 && i < nunits) { csize /= 1024; dsize /= 1024.0; i++; } if (i >= 0 && i < nunits) if (human == 1) ret = snprintf(str_buff, STR_MAX, "%.1f%c", dsize, units[i]); else ret = snprintf(str_buff, STR_MAX, "%.1f%c [%" PRIu64"]", dsize, units[i], size); else ret = snprintf(str_buff, STR_MAX, "%"PRIu64, size); } if (ret < 0 || ret >= STR_MAX) return ""; return str_buff; } /* * out_get_uuid_str -- returns uuid in human readable format */ const char * out_get_uuid_str(uuid_t uuid) { static char uuid_str[UUID_STR_MAX] = {0, }; int ret = util_uuid_to_string(uuid, uuid_str); if (ret != 0) { outv(2, "failed to covert uuid to string"); return NULL; } return uuid_str; } /* * out_get_time_str -- returns time in human readable format */ const char * out_get_time_str(time_t time) { static char str_buff[STR_MAX] = {0, }; struct tm *tm = util_localtime(&time); if (tm) { strftime(str_buff, STR_MAX, TIME_STR_FMT, tm); } else { int ret = snprintf(str_buff, STR_MAX, "unknown"); if (ret < 0 || ret >= STR_MAX) return ""; } return str_buff; } /* * out_get_ascii_str -- get string with printable ASCII dump buffer * * Convert non-printable ASCII characters to dot '.' * See: util_get_printable_ascii() function. */ static int out_get_ascii_str(char *str, size_t str_len, const uint8_t *datap, size_t len) { int c = 0; size_t i; char pch; if (str_len < len) return -1; for (i = 0; i < len; i++) { pch = util_get_printable_ascii((char)datap[i]); int t = snprintf(str + c, str_len - (size_t)c, "%c", pch); if (t < 0) return -1; c += t; } return c; } /* * out_get_hex_str -- get string with hexadecimal dump of buffer * * Hexadecimal bytes in format %02x, each one followed by space, * additional space after every 8th byte. */ static int out_get_hex_str(char *str, size_t str_len, const uint8_t *datap, size_t len) { int c = 0; size_t i; int t; if (str_len < (3 * len + 1)) return -1; for (i = 0; i < len; i++) { /* add space after n*8 byte */ if (i && (i % 8) == 0) { t = snprintf(str + c, str_len - (size_t)c, " "); if (t < 0) return -1; c += t; } t = snprintf(str + c, str_len - (size_t)c, "%02x ", datap[i]); if (t < 0) return -1; c += t; } return c; } /* * outv_hexdump -- print buffer in canonical hex+ASCII format * * Print offset in hexadecimal, * sixteen space-separated, two column, hexadecimal bytes, * followed by the same sixteen bytes converted to printable ASCII characters * enclosed in '|' characters. */ void outv_hexdump(int vlevel, const void *addr, size_t len, size_t offset, int sep) { if (!outv_check(vlevel) || len <= 0) return; const uint8_t *datap = (uint8_t *)addr; uint8_t row_hex_str[HEXDUMP_ROW_HEX_LEN] = {0, }; uint8_t row_ascii_str[HEXDUMP_ROW_ASCII_LEN] = {0, }; size_t curr = 0; size_t prev = 0; int repeated = 0; int n = 0; while (len) { size_t curr_len = min(len, HEXDUMP_ROW_WIDTH); /* * Check if current row is the same as the previous one * don't check it for first and last rows. */ if (len != curr_len && curr && !memcmp(datap + prev, datap + curr, curr_len)) { if (!repeated) { /* print star only for the first repeated */ fprintf(out_fh, "*\n"); repeated = 1; } } else { repeated = 0; /* row with hexadecimal bytes */ int rh = out_get_hex_str((char *)row_hex_str, HEXDUMP_ROW_HEX_LEN, datap + curr, curr_len); /* row with printable ascii chars */ int ra = out_get_ascii_str((char *)row_ascii_str, HEXDUMP_ROW_ASCII_LEN, datap + curr, curr_len); if (ra && rh) n = fprintf(out_fh, "%08zx %-*s|%-*s|\n", curr + offset, HEXDUMP_ROW_HEX_LEN, row_hex_str, HEXDUMP_ROW_WIDTH, row_ascii_str); prev = curr; } len -= curr_len; curr += curr_len; } if (sep && n) { while (--n) fprintf(out_fh, "%c", SEPARATOR_CHAR); fprintf(out_fh, "\n"); } } /* * out_get_checksum -- return checksum string with result */ const char * out_get_checksum(void *addr, size_t len, uint64_t *csump, size_t skip_off) { static char str_buff[STR_MAX] = {0, }; int ret = 0; uint64_t csum = util_checksum_compute(addr, len, csump, skip_off); if (*csump == htole64(csum)) ret = snprintf(str_buff, STR_MAX, "0x%" PRIx64" [OK]", le64toh(csum)); else ret = snprintf(str_buff, STR_MAX, "0x%" PRIx64 " [wrong! should be: 0x%" PRIx64 "]", le64toh(*csump), le64toh(csum)); if (ret < 0 || ret >= STR_MAX) return ""; return str_buff; } /* * out_get_btt_map_entry -- return BTT map entry with flags strings */ const char * out_get_btt_map_entry(uint32_t map) { static char str_buff[STR_MAX] = {0, }; int is_init = (map & ~BTT_MAP_ENTRY_LBA_MASK) == 0; int is_zero = (map & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_ZERO; int is_error = (map & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_ERROR; int is_normal = (map & ~BTT_MAP_ENTRY_LBA_MASK) == BTT_MAP_ENTRY_NORMAL; uint32_t lba = map & BTT_MAP_ENTRY_LBA_MASK; int ret = snprintf(str_buff, STR_MAX, "0x%08x state: %s", lba, is_init ? "init" : is_zero ? "zero" : is_error ? "error" : is_normal ? "normal" : "unknown"); if (ret < 0 || ret >= STR_MAX) return ""; return str_buff; } /* * out_get_pool_type_str -- get pool type string */ const char * out_get_pool_type_str(pmem_pool_type_t type) { switch (type) { case PMEM_POOL_TYPE_LOG: return "log"; case PMEM_POOL_TYPE_BLK: return "blk"; case PMEM_POOL_TYPE_OBJ: return "obj"; case PMEM_POOL_TYPE_BTT: return "btt"; default: return "unknown"; } } /* * out_get_pool_signature -- return signature of specified pool type */ const char * out_get_pool_signature(pmem_pool_type_t type) { switch (type) { case PMEM_POOL_TYPE_LOG: return LOG_HDR_SIG; case PMEM_POOL_TYPE_BLK: return BLK_HDR_SIG; case PMEM_POOL_TYPE_OBJ: return OBJ_HDR_SIG; default: return NULL; } } /* * out_get_chunk_type_str -- get chunk type string */ const char * out_get_chunk_type_str(enum chunk_type type) { switch (type) { case CHUNK_TYPE_FOOTER: return "footer"; case CHUNK_TYPE_FREE: return "free"; case CHUNK_TYPE_USED: return "used"; case CHUNK_TYPE_RUN: return "run"; case CHUNK_TYPE_UNKNOWN: default: return "unknown"; } } /* * out_get_chunk_flags -- get names of set flags for chunk header */ const char * out_get_chunk_flags(uint16_t flags) { if (flags & CHUNK_FLAG_COMPACT_HEADER) return "compact header"; else if (flags & CHUNK_FLAG_HEADER_NONE) return "header none"; return ""; } /* * out_get_zone_magic_str -- get zone magic string with additional * information about correctness of the magic value */ const char * out_get_zone_magic_str(uint32_t magic) { static char str_buff[STR_MAX] = {0, }; const char *correct = NULL; switch (magic) { case 0: correct = "uninitialized"; break; case ZONE_HEADER_MAGIC: correct = "OK"; break; default: correct = "wrong! should be " STR(ZONE_HEADER_MAGIC); break; } int ret = snprintf(str_buff, STR_MAX, "0x%08x [%s]", magic, correct); if (ret < 0 || ret >= STR_MAX) return ""; return str_buff; } /* * out_get_pmemoid_str -- get PMEMoid string */ const char * out_get_pmemoid_str(PMEMoid oid, uint64_t uuid_lo) { static char str_buff[STR_MAX] = {0, }; int free_cor = 0; int ret = 0; char *correct = "OK"; if (oid.pool_uuid_lo && oid.pool_uuid_lo != uuid_lo) { ret = snprintf(str_buff, STR_MAX, "wrong! should be 0x%016"PRIx64, uuid_lo); if (ret < 0 || ret >= STR_MAX) err(1, "snprintf: %d", ret); correct = strdup(str_buff); if (!correct) err(1, "Cannot allocate memory for PMEMoid string\n"); free_cor = 1; } ret = snprintf(str_buff, STR_MAX, "off: 0x%016"PRIx64" pool_uuid_lo: 0x%016" PRIx64" [%s]", oid.off, oid.pool_uuid_lo, correct); if (free_cor) free(correct); if (ret < 0 || ret >= STR_MAX) err(1, "snprintf: %d", ret); return str_buff; } /* * out_get_arch_machine_class_str -- get a string representation of the machine * class */ const char * out_get_arch_machine_class_str(uint8_t machine_class) { switch (machine_class) { case PMDK_MACHINE_CLASS_64: return "64"; default: return "unknown"; } } /* * out_get_arch_data_str -- get a string representation of the data endianness */ const char * out_get_arch_data_str(uint8_t data) { switch (data) { case PMDK_DATA_LE: return "2's complement, little endian"; case PMDK_DATA_BE: return "2's complement, big endian"; default: return "unknown"; } } /* * out_get_arch_machine_str -- get a string representation of the machine type */ const char * out_get_arch_machine_str(uint16_t machine) { static char str_buff[STR_MAX] = {0, }; switch (machine) { case PMDK_MACHINE_X86_64: return "AMD X86-64"; case PMDK_MACHINE_AARCH64: return "Aarch64"; case PMDK_MACHINE_PPC64: return "PPC64"; default: break; } int ret = snprintf(str_buff, STR_MAX, "unknown %u", machine); if (ret < 0 || ret >= STR_MAX) return "unknown"; return str_buff; } /* * out_get_last_shutdown_str -- get a string representation of the finish state */ const char * out_get_last_shutdown_str(uint8_t dirty) { if (dirty) return "dirty"; else return "clean"; } /* * out_get_alignment_descr_str -- get alignment descriptor string */ const char * out_get_alignment_desc_str(uint64_t ad, uint64_t valid_ad) { static char str_buff[STR_MAX] = {0, }; int ret = 0; if (ad == valid_ad) ret = snprintf(str_buff, STR_MAX, "0x%016"PRIx64"[OK]", ad); else ret = snprintf(str_buff, STR_MAX, "0x%016"PRIx64" " "[wrong! should be 0x%016"PRIx64"]", ad, valid_ad); if (ret < 0 || ret >= STR_MAX) return ""; return str_buff; } /* * out_concat -- concatenate the new element to the list of strings * * If concatenation is successful it increments current position in the output * string and number of elements in the list. Elements are separated with ", ". */ static int out_concat(char *str_buff, int *curr, int *count, const char *str) { ASSERTne(str_buff, NULL); ASSERTne(curr, NULL); ASSERTne(str, NULL); const char *separator = (count != NULL && *count > 0) ? ", " : ""; int ret = snprintf(str_buff + *curr, (size_t)(STR_MAX - *curr), "%s%s", separator, str); if (ret < 0 || *curr + ret >= STR_MAX) return -1; *curr += ret; if (count) ++(*count); return 0; } /* * out_get_incompat_features_str -- (internal) get a string with names of * incompatibility flags */ const char * out_get_incompat_features_str(uint32_t incompat) { static char str_buff[STR_MAX] = {0}; features_t features = {POOL_FEAT_ZERO, incompat, POOL_FEAT_ZERO}; int ret = 0; if (incompat == 0) { /* print the value only */ return "0x0"; } else { /* print the value and the left square bracket */ ret = snprintf(str_buff, STR_MAX, "0x%x [", incompat); if (ret < 0 || ret >= STR_MAX) { ERR("snprintf for incompat features: %d", ret); return ""; } /* print names of known options */ int count = 0; int curr = ret; features_t found; const char *feat; while (((feat = util_feature2str(features, &found))) != NULL) { util_feature_disable(&features, found); ret = out_concat(str_buff, &curr, &count, feat); if (ret < 0) return ""; } /* check if any unknown flags are set */ if (!util_feature_is_zero(features)) { if (out_concat(str_buff, &curr, &count, "?UNKNOWN_FLAG?")) return ""; } /* print the right square bracket */ if (out_concat(str_buff, &curr, NULL, "]")) return ""; } return str_buff; } pmdk-1.8/src/tools/pmempool/bash_completion/0000775000000000000000000000000013615011243017736 5ustar rootrootpmdk-1.8/src/tools/pmempool/bash_completion/pmempool0000664000000000000000000001151613615011243021515 0ustar rootroot# # Copyright 2014-2017, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # pmempool -- bash completion script for pmempool # # # _pmempool_gen -- generates result for completion # Arguments: # $1 - values # $2 - current string # $3 - prefix for results _pmempool_gen() { COMPREPLAY=() local values=$1 local cur=$2 local prefix=$3 local i=${#COMPREPLY[@]} for v in $values do [[ "$v" == "$cur"* ]] && COMPREPLY[i++]="$prefix$v" done } # # _pmempool_get_cmds -- returns available pmempool commands # _pmempool_get_cmds() { echo -n $(pmempool --help | grep -e '^\S\+\s\+-' |\ grep -o '^\S\+' | sed '/help/d') } # # _pmempool_get_opts -- returns available options for specified command # Arguments: # $1 - command # _pmempool_get_opts() { local c=$1 local opts=$(pmempool ${c} --help | grep -o -e "-., --\S\+" |\ grep -o -e "--\S\+") echo "$opts" } # # _pmempool_get_values -- returns available values for specified option # Arguments: # $1 - command # $2 - option # $3 - values delimiter # $4 - current values, will be removed from result # _pmempool_get_values() { local cmd=$1 local opt=$2 local delim=$3 local curvals=$4 local vals=$(pmempool ${cmd} --help |\ grep -o -e "${opt}\s\+\S\+${delim}\S\+" |\ sed "s/${opt}\s\+\(\S\+${delim}\S\+\)/\1/" |\ sed "s/${delim}/ /g") if [ -n "$curvals" ] then local OLD_IFS=$IFS IFS="," for v in $curvals do vals=$(echo $vals | sed "s/$v//g") done IFS=$OLD_IFS fi echo "${vals}" } # # _pmempool_get_cmd -- returns command name if exist in specified array # Arguments: # $1 - command name # $2 - list of available commands # _pmempool_get_cmd() { local cmd=$1 local cmds=$2 [[ ${cmds} =~ ${cmd} ]] && echo -n ${cmd} } # # _pmempool_get_used_values -- returns already used values # Arguments: # $1 - current string # $2 - values delimiter # _pmempool_get_used_values() { local cur=$1 local delim=$2 local used=$(echo $cur | rev | cut -d $delim -s -f1 --complement | rev) [ -n "$used" ] && used="$used$delim" echo "$used" } # # _pmempool_get_current_value -- returns current value string # Arguments: # $1 - current string # $2 - values delimiter # _pmempool_get_current_value() { local cur=$1 local delim=$2 echo $cur | rev | cut -d $delim -f1 | rev } _pmempool() { local cur prev opts cur="${COMP_WORDS[COMP_CWORD]}" prev="${COMP_WORDS[COMP_CWORD-1]}" cmds=$(_pmempool_get_cmds) cmds_all="$cmds help" opts_pool_types="blk log obj" cmd=$(_pmempool_get_cmd "${COMP_WORDS[1]}" "$cmds_all") if [[ ${cur} == -* ]] then local opts=$(_pmempool_get_opts $cmd) _pmempool_gen "$opts" "$cur" elif [[ ${prev} == --* ]] then local used=$(_pmempool_get_used_values "$cur" ",") local _cur=$(_pmempool_get_current_value "$cur" ",") local values=$(_pmempool_get_values ${cmd} ${prev} "," $used) if [ -n "${values}" ] then # values separated by ',' may contain multiple values _pmempool_gen "$values" "$_cur" "$used" else # values separated by '|' may contain only one value values=$(_pmempool_get_values $cmd $prev "|") _pmempool_gen "$values" "$cur" fi elif [[ $cmd == create ]] then case "${COMP_WORDS[@]}" in *blk*|*log*|*obj*|*--inherit*) ;; *) _pmempool_gen "$opts_pool_types" "$cur" ;; esac elif [[ ${prev} == help ]] then _pmempool_gen "$cmds" "$cur" elif [[ ${prev} == pmempool ]] then _pmempool_gen "$cmds_all" "$cur" fi } complete -o default -F _pmempool pmempool pmdk-1.8/src/tools/pmempool/rm.c0000664000000000000000000002153513615011243015360 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rm.c -- pmempool rm command main source file */ #include #include #include #include #include #include #include "os.h" #include "out.h" #include "common.h" #include "output.h" #include "file.h" #include "rm.h" #include "set.h" #ifdef USE_RPMEM #include "librpmem.h" #endif enum ask_type { ASK_SOMETIMES, /* ask before removing write-protected files */ ASK_ALWAYS, /* always ask */ ASK_NEVER, /* never ask */ }; /* verbosity level */ static int vlevel; /* force remove and ignore errors */ static int force; /* poolset files options */ #define RM_POOLSET_NONE (0) #define RM_POOLSET_LOCAL (1 << 0) #define RM_POOLSET_REMOTE (1 << 1) #define RM_POOLSET_ALL (RM_POOLSET_LOCAL | RM_POOLSET_REMOTE) static int rm_poolset_mode; /* mode of interaction */ static enum ask_type ask_mode; /* indicates whether librpmem is available */ static int rpmem_avail; /* help message */ static const char * const help_str = "Remove pool file or all files from poolset\n" "\n" "Available options:\n" " -h, --help Print this help message.\n" " -v, --verbose Be verbose.\n" " -s, --only-pools Remove only pool files (default).\n" " -a, --all Remove all poolset files - local and remote.\n" " -l, --local Remove local poolset files\n" " -r, --remote Remove remote poolset files\n" " -f, --force Ignore nonexisting files.\n" " -i, --interactive Prompt before every single removal.\n" "\n" "For complete documentation see %s-rm(1) manual page.\n"; /* short options string */ static const char *optstr = "hvsfialr"; /* long options */ static const struct option long_options[] = { {"help", no_argument, NULL, 'h'}, {"verbose", no_argument, NULL, 'v'}, {"only-pools", no_argument, NULL, 's'}, {"all", no_argument, NULL, 'a'}, {"local", no_argument, NULL, 'l'}, {"remote", no_argument, NULL, 'r'}, {"force", no_argument, NULL, 'f'}, {"interactive", no_argument, NULL, 'i'}, {NULL, 0, NULL, 0 }, }; /* * print_usage -- print usage message */ static void print_usage(const char *appname) { printf("Usage: %s rm [] \n", appname); } /* * pmempool_rm_help -- print help message */ void pmempool_rm_help(const char *appname) { print_usage(appname); printf(help_str, appname); } /* * rm_file -- remove single file */ static int rm_file(const char *file) { int write_protected = os_access(file, W_OK) != 0; char cask = 'y'; switch (ask_mode) { case ASK_ALWAYS: cask = '?'; break; case ASK_NEVER: cask = 'y'; break; case ASK_SOMETIMES: cask = write_protected ? '?' : 'y'; break; default: outv_err("unknown state"); return 1; } const char *pre_msg = write_protected ? "write-protected " : ""; char ans = ask_Yn(cask, "remove %sfile '%s' ?", pre_msg, file); if (ans == 'y') { if (util_unlink(file)) { outv_err("cannot remove file '%s'", file); return 1; } outv(1, "removed '%s'\n", file); } return 0; } /* * remove_remote -- (internal) remove remote pool */ static int remove_remote(const char *target, const char *pool_set) { #ifdef USE_RPMEM char cask = 'y'; switch (ask_mode) { case ASK_ALWAYS: cask = '?'; break; case ASK_NEVER: case ASK_SOMETIMES: cask = 'y'; break; default: outv_err("unknown state"); return 1; } char ans = ask_Yn(cask, "remove remote pool '%s' on '%s'?", pool_set, target); if (ans == INV_ANS) outv(1, "invalid answer\n"); if (ans != 'y') return 0; if (!rpmem_avail) { if (force) { outv(1, "cannot remove '%s' on '%s' -- " "librpmem not available", pool_set, target); return 0; } outv_err("!cannot remove '%s' on '%s' -- " "librpmem not available", pool_set, target); return 1; } int flags = 0; if (rm_poolset_mode & RM_POOLSET_REMOTE) flags |= RPMEM_REMOVE_POOL_SET; if (force) flags |= RPMEM_REMOVE_FORCE; int ret = Rpmem_remove(target, pool_set, flags); if (ret) { if (force) { ret = 0; outv(1, "cannot remove '%s' on '%s'", pool_set, target); } else { /* * Callback cannot return < 0 value because it * is interpretted as error in parsing poolset file. */ ret = 1; outv_err("!cannot remove '%s' on '%s'", pool_set, target); } } else { outv(1, "removed '%s' on '%s'\n", pool_set, target); } return ret; #else outv_err("remote replication not supported"); return 1; #endif } /* * rm_poolset_cb -- (internal) callback for removing replicas */ static int rm_poolset_cb(struct part_file *pf, void *arg) { int *error = (int *)arg; int ret; if (pf->is_remote) { ret = remove_remote(pf->remote->node_addr, pf->remote->pool_desc); } else { const char *part_file = pf->part->path; outv(2, "part file : %s\n", part_file); int exists = util_file_exists(part_file); if (exists < 0) ret = 1; else if (!exists) { /* * Ignore not accessible file if force * flag is set. */ if (force) return 0; ret = 1; outv_err("!cannot remove file '%s'", part_file); } else { ret = rm_file(part_file); } } if (ret) *error = ret; return 0; } /* * rm_poolset -- remove files parsed from poolset file */ static int rm_poolset(const char *file) { int error = 0; int ret = util_poolset_foreach_part(file, rm_poolset_cb, &error); if (ret == -1) { outv_err("parsing poolset failed: %s\n", out_get_errormsg()); return ret; } if (error && !force) { outv_err("!removing '%s' failed\n", file); return error; } return 0; } /* * pmempool_rm_func -- main function for rm command */ int pmempool_rm_func(const char *appname, int argc, char *argv[]) { /* by default do not remove any poolset files */ rm_poolset_mode = RM_POOLSET_NONE; int opt; while ((opt = getopt_long(argc, argv, optstr, long_options, NULL)) != -1) { switch (opt) { case 'h': pmempool_rm_help(appname); return 0; case 'v': vlevel++; break; case 's': rm_poolset_mode = RM_POOLSET_NONE; break; case 'a': rm_poolset_mode |= RM_POOLSET_ALL; break; case 'l': rm_poolset_mode |= RM_POOLSET_LOCAL; break; case 'r': rm_poolset_mode |= RM_POOLSET_REMOTE; break; case 'f': force = 1; ask_mode = ASK_NEVER; break; case 'i': ask_mode = ASK_ALWAYS; break; default: print_usage(appname); return 1; } } out_set_vlevel(vlevel); if (optind == argc) { print_usage(appname); return 1; } #ifdef USE_RPMEM /* * Try to load librpmem, if loading failed - * assume it is not available. */ util_remote_init(); rpmem_avail = !util_remote_load(); #endif int lret = 0; for (int i = optind; i < argc; i++) { char *file = argv[i]; /* check if file exists and we can read it */ int exists = os_access(file, F_OK | R_OK) == 0; if (!exists) { /* ignore not accessible file if force flag is set */ if (force) continue; outv_err("!cannot remove '%s'", file); lret = 1; continue; } int is_poolset = util_is_poolset_file(file); if (is_poolset < 0) { outv(1, "%s: cannot determine type of file", file); if (force) continue; } if (is_poolset) outv(2, "poolset file: %s\n", file); else outv(2, "pool file : %s\n", file); int ret; if (is_poolset) { ret = rm_poolset(file); if (!ret && (rm_poolset_mode & RM_POOLSET_LOCAL)) ret = rm_file(file); } else { ret = rm_file(file); } if (ret) lret = ret; } return lret; } pmdk-1.8/src/tools/pmempool/rm.h0000664000000000000000000000334413615011243015363 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rm.h -- pmempool rm command header file */ void pmempool_rm_help(const char *appname); int pmempool_rm_func(const char *appname, int argc, char *argv[]); pmdk-1.8/src/tools/pmempool/feature.h0000664000000000000000000000336313615011243016401 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * feature.h -- pmempool feature command header file */ int pmempool_feature_func(const char *appname, int argc, char *argv[]); void pmempool_feature_help(const char *appname); pmdk-1.8/src/tools/pmempool/info.h0000664000000000000000000001356713615011243015710 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * info.h -- pmempool info command header file */ #include "vec.h" /* * Verbose levels used in application: * * VERBOSE_DEFAULT: * Default value for application's verbosity level. * This is also set for data structures which should be * printed without any command line argument. * * VERBOSE_MAX: * Maximum value for application's verbosity level. * This value is used when -v command line argument passed. * * VERBOSE_SILENT: * This value is higher than VERBOSE_MAX and it is used only * for verbosity levels of data structures which should _not_ be * printed without specified command line arguments. */ #define VERBOSE_SILENT 0 #define VERBOSE_DEFAULT 1 #define VERBOSE_MAX 2 /* * print_bb_e -- printing bad blocks options */ enum print_bb_e { PRINT_BAD_BLOCKS_NOT_SET, PRINT_BAD_BLOCKS_NO, PRINT_BAD_BLOCKS_YES, PRINT_BAD_BLOCKS_MAX }; /* * pmempool_info_args -- structure for storing command line arguments */ struct pmempool_info_args { char *file; /* input file */ unsigned col_width; /* column width for printing fields */ bool human; /* sizes in human-readable formats */ bool force; /* force parsing pool */ enum print_bb_e badblocks; /* print bad blocks */ pmem_pool_type_t type; /* forced pool type */ bool use_range; /* use range for blocks */ struct ranges ranges; /* range of block/chunks to dump */ int vlevel; /* verbosity level */ int vdata; /* verbosity level for data dump */ int vhdrdump; /* verbosity level for headers hexdump */ int vstats; /* verbosity level for statistics */ struct { size_t walk; /* data chunk size */ } log; struct { int vmap; /* verbosity level for BTT Map */ int vflog; /* verbosity level for BTT FLOG */ int vbackup; /* verbosity level for BTT Info backup */ bool skip_zeros; /* skip blocks marked with zero flag */ bool skip_error; /* skip blocks marked with error flag */ bool skip_no_flag; /* skip blocks not marked with any flag */ } blk; struct { int vlanes; /* verbosity level for lanes */ int vroot; int vobjects; int valloc; int voobhdr; int vheap; int vzonehdr; int vchunkhdr; int vbitmap; bool lanes_recovery; bool ignore_empty_obj; uint64_t chunk_types; size_t replica; struct ranges lane_ranges; struct ranges type_ranges; struct ranges zone_ranges; struct ranges chunk_ranges; } obj; }; /* * pmem_blk_stats -- structure with statistics for pmemblk */ struct pmem_blk_stats { uint32_t total; /* number of processed blocks */ uint32_t zeros; /* number of blocks marked by zero flag */ uint32_t errors; /* number of blocks marked by error flag */ uint32_t noflag; /* number of blocks not marked with any flag */ }; struct pmem_obj_class_stats { uint64_t n_units; uint64_t n_used; uint64_t unit_size; uint64_t alignment; uint32_t nallocs; uint16_t flags; }; struct pmem_obj_zone_stats { uint64_t n_chunks; uint64_t n_chunks_type[MAX_CHUNK_TYPE]; uint64_t size_chunks; uint64_t size_chunks_type[MAX_CHUNK_TYPE]; VEC(, struct pmem_obj_class_stats) class_stats; }; struct pmem_obj_type_stats { PMDK_TAILQ_ENTRY(pmem_obj_type_stats) next; uint64_t type_num; uint64_t n_objects; uint64_t n_bytes; }; struct pmem_obj_stats { uint64_t n_total_objects; uint64_t n_total_bytes; uint64_t n_zones; uint64_t n_zones_used; struct pmem_obj_zone_stats *zone_stats; PMDK_TAILQ_HEAD(obj_type_stats_head, pmem_obj_type_stats) type_stats; }; /* * pmem_info -- context for pmeminfo application */ struct pmem_info { const char *file_name; /* current file name */ struct pool_set_file *pfile; struct pmempool_info_args args; /* arguments parsed from command line */ struct options *opts; struct pool_set *poolset; pmem_pool_type_t type; struct pmem_pool_params params; struct { struct pmem_blk_stats stats; } blk; struct { struct pmemobjpool *pop; struct palloc_heap *heap; struct alloc_class_collection *alloc_classes; size_t size; struct pmem_obj_stats stats; uint64_t uuid_lo; uint64_t objid; } obj; }; int pmempool_info_func(const char *appname, int argc, char *argv[]); void pmempool_info_help(const char *appname); int pmempool_info_read(struct pmem_info *pip, void *buff, size_t nbytes, uint64_t off); int pmempool_info_blk(struct pmem_info *pip); int pmempool_info_log(struct pmem_info *pip); int pmempool_info_obj(struct pmem_info *pip); int pmempool_info_btt(struct pmem_info *pip); pmdk-1.8/src/tools/pmempool/common.c0000664000000000000000000007610313615011243016233 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * common.c -- definitions of common functions */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "common.h" #include "output.h" #include "libpmem.h" #include "libpmemblk.h" #include "libpmemlog.h" #include "libpmemobj.h" #include "btt.h" #include "file.h" #include "os.h" #include "set.h" #include "out.h" #include "mmap.h" #include "util_pmem.h" #include "badblock.h" #include "util.h" #define REQ_BUFF_SIZE 2048U #define Q_BUFF_SIZE 8192 typedef const char *(*enum_to_str_fn)(int); /* * pmem_pool_type -- return pool type based on first two pages. * If pool header's content suggests that pool may be BTT device * (first page zeroed and no correct signature for pool header), * signature from second page is checked to prove that it's BTT device layout. */ pmem_pool_type_t pmem_pool_type(const void *base_pool_addr) { struct pool_hdr *hdrp = (struct pool_hdr *)base_pool_addr; if (util_is_zeroed(hdrp, DEFAULT_HDR_SIZE)) { return util_get_pool_type_second_page(base_pool_addr); } pmem_pool_type_t type = pmem_pool_type_parse_hdr(hdrp); if (type != PMEM_POOL_TYPE_UNKNOWN) return type; else return util_get_pool_type_second_page(base_pool_addr); } /* * pmem_pool_checksum -- return true if checksum is correct * based on first two pages */ int pmem_pool_checksum(const void *base_pool_addr) { /* check whether it's btt device -> first page zeroed */ if (util_is_zeroed(base_pool_addr, DEFAULT_HDR_SIZE)) { struct btt_info bttinfo; void *sec_page_addr = (char *)base_pool_addr + DEFAULT_HDR_SIZE; memcpy(&bttinfo, sec_page_addr, sizeof(bttinfo)); btt_info_convert2h(&bttinfo); return util_checksum(&bttinfo, sizeof(bttinfo), &bttinfo.checksum, 0, 0); } else { /* it's not btt device - first page contains header */ struct pool_hdr hdrp; memcpy(&hdrp, base_pool_addr, sizeof(hdrp)); return util_checksum(&hdrp, sizeof(hdrp), &hdrp.checksum, 0, POOL_HDR_CSUM_END_OFF(&hdrp)); } } /* * pmem_pool_type_parse_hdr -- return pool type based only on signature */ pmem_pool_type_t pmem_pool_type_parse_hdr(const struct pool_hdr *hdrp) { if (memcmp(hdrp->signature, LOG_HDR_SIG, POOL_HDR_SIG_LEN) == 0) return PMEM_POOL_TYPE_LOG; else if (memcmp(hdrp->signature, BLK_HDR_SIG, POOL_HDR_SIG_LEN) == 0) return PMEM_POOL_TYPE_BLK; else if (memcmp(hdrp->signature, OBJ_HDR_SIG, POOL_HDR_SIG_LEN) == 0) return PMEM_POOL_TYPE_OBJ; else return PMEM_POOL_TYPE_UNKNOWN; } /* * pmem_pool_type_parse_str -- returns pool type from command line arg */ pmem_pool_type_t pmem_pool_type_parse_str(const char *str) { if (strcmp(str, "blk") == 0) { return PMEM_POOL_TYPE_BLK; } else if (strcmp(str, "log") == 0) { return PMEM_POOL_TYPE_LOG; } else if (strcmp(str, "obj") == 0) { return PMEM_POOL_TYPE_OBJ; } else if (strcmp(str, "btt") == 0) { return PMEM_POOL_TYPE_BTT; } else { return PMEM_POOL_TYPE_UNKNOWN; } } /* * util_get_pool_type_second_page -- return type based on second page content */ pmem_pool_type_t util_get_pool_type_second_page(const void *pool_base_addr) { struct btt_info bttinfo; void *sec_page_addr = (char *)pool_base_addr + DEFAULT_HDR_SIZE; memcpy(&bttinfo, sec_page_addr, sizeof(bttinfo)); btt_info_convert2h(&bttinfo); if (util_is_zeroed(&bttinfo, sizeof(bttinfo))) return PMEM_POOL_TYPE_UNKNOWN; if (memcmp(bttinfo.sig, BTTINFO_SIG, BTTINFO_SIG_LEN) == 0) return PMEM_POOL_TYPE_BTT; return PMEM_POOL_TYPE_UNKNOWN; } /* * util_parse_mode -- parse file mode from octal string */ int util_parse_mode(const char *str, mode_t *mode) { mode_t m = 0; int digits = 0; /* skip leading zeros */ while (*str == '0') str++; /* parse at most 3 octal digits */ while (digits < 3 && *str != '\0') { if (*str < '0' || *str > '7') return -1; m = (mode_t)(m << 3) | (mode_t)(*str - '0'); digits++; str++; } /* more than 3 octal digits */ if (digits == 3 && *str != '\0') return -1; if (mode) *mode = m; return 0; } static void util_range_limit(struct range *rangep, struct range limit) { if (rangep->first < limit.first) rangep->first = limit.first; if (rangep->last > limit.last) rangep->last = limit.last; } /* * util_parse_range_from -- parse range string as interval from specified number */ static int util_parse_range_from(char *str, struct range *rangep, struct range entire) { size_t str_len = strlen(str); if (str[str_len - 1] != '-') return -1; str[str_len - 1] = '\0'; if (util_parse_size(str, (size_t *)&rangep->first)) return -1; rangep->last = entire.last; util_range_limit(rangep, entire); return 0; } /* * util_parse_range_to -- parse range string as interval to specified number */ static int util_parse_range_to(char *str, struct range *rangep, struct range entire) { if (str[0] != '-' || str[1] == '\0') return -1; if (util_parse_size(str + 1, (size_t *)&rangep->last)) return -1; rangep->first = entire.first; util_range_limit(rangep, entire); return 0; } /* * util_parse_range_number -- parse range string as a single number */ static int util_parse_range_number(char *str, struct range *rangep, struct range entire) { if (util_parse_size(str, (size_t *)&rangep->first) != 0) return -1; rangep->last = rangep->first; if (rangep->first > entire.last || rangep->last < entire.first) return -1; util_range_limit(rangep, entire); return 0; } /* * util_parse_range -- parse single range string */ static int util_parse_range(char *str, struct range *rangep, struct range entire) { char *dash = strchr(str, '-'); if (!dash) return util_parse_range_number(str, rangep, entire); /* '-' at the beginning */ if (dash == str) return util_parse_range_to(str, rangep, entire); /* '-' at the end */ if (dash[1] == '\0') return util_parse_range_from(str, rangep, entire); *dash = '\0'; dash++; if (util_parse_size(str, (size_t *)&rangep->first)) return -1; if (util_parse_size(dash, (size_t *)&rangep->last)) return -1; if (rangep->first > rangep->last) { uint64_t tmp = rangep->first; rangep->first = rangep->last; rangep->last = tmp; } util_range_limit(rangep, entire); return 0; } /* * util_ranges_overlap -- return 1 if two ranges are overlapped */ static int util_ranges_overlap(struct range *rangep1, struct range *rangep2) { if (rangep1->last + 1 < rangep2->first || rangep2->last + 1 < rangep1->first) return 0; else return 1; } /* * util_ranges_add -- create and add range */ int util_ranges_add(struct ranges *rangesp, struct range range) { struct range *rangep = malloc(sizeof(struct range)); if (!rangep) err(1, "Cannot allocate memory for range\n"); memcpy(rangep, &range, sizeof(*rangep)); struct range *curp, *next; uint64_t first = rangep->first; uint64_t last = rangep->last; curp = PMDK_LIST_FIRST(&rangesp->head); while (curp) { next = PMDK_LIST_NEXT(curp, next); if (util_ranges_overlap(curp, rangep)) { PMDK_LIST_REMOVE(curp, next); if (curp->first < first) first = curp->first; if (curp->last > last) last = curp->last; free(curp); } curp = next; } rangep->first = first; rangep->last = last; PMDK_LIST_FOREACH(curp, &rangesp->head, next) { if (curp->first < rangep->first) { PMDK_LIST_INSERT_AFTER(curp, rangep, next); return 0; } } PMDK_LIST_INSERT_HEAD(&rangesp->head, rangep, next); return 0; } /* * util_ranges_contain -- return 1 if ranges contain the number n */ int util_ranges_contain(const struct ranges *rangesp, uint64_t n) { struct range *curp = NULL; PMDK_LIST_FOREACH(curp, &rangesp->head, next) { if (curp->first <= n && n <= curp->last) return 1; } return 0; } /* * util_ranges_empty -- return 1 if ranges are empty */ int util_ranges_empty(const struct ranges *rangesp) { return PMDK_LIST_EMPTY(&rangesp->head); } /* * util_ranges_clear -- clear list of ranges */ void util_ranges_clear(struct ranges *rangesp) { while (!PMDK_LIST_EMPTY(&rangesp->head)) { struct range *rangep = PMDK_LIST_FIRST(&rangesp->head); PMDK_LIST_REMOVE(rangep, next); free(rangep); } } /* * util_parse_ranges -- parser ranges from string * * The valid formats of range are: * - 'n-m' -- from n to m * - '-m' -- from minimum passed in entirep->first to m * - 'n-' -- from n to maximum passed in entirep->last * - 'n' -- n'th byte/block * Multiple ranges may be separated by comma: * 'n1-m1,n2-,-m3,n4' */ int util_parse_ranges(const char *ptr, struct ranges *rangesp, struct range entire) { if (ptr == NULL) return util_ranges_add(rangesp, entire); char *dup = strdup(ptr); if (!dup) err(1, "Cannot allocate memory for ranges"); char *str = dup; int ret = 0; char *next = str; do { str = next; next = strchr(str, ','); if (next != NULL) { *next = '\0'; next++; } struct range range; if (util_parse_range(str, &range, entire)) { ret = -1; goto out; } else if (util_ranges_add(rangesp, range)) { ret = -1; goto out; } } while (next != NULL); out: free(dup); return ret; } /* * pmem_pool_get_min_size -- return minimum size of pool for specified type */ uint64_t pmem_pool_get_min_size(pmem_pool_type_t type) { switch (type) { case PMEM_POOL_TYPE_LOG: return PMEMLOG_MIN_POOL; case PMEM_POOL_TYPE_BLK: return PMEMBLK_MIN_POOL; case PMEM_POOL_TYPE_OBJ: return PMEMOBJ_MIN_POOL; default: break; } return 0; } /* * util_poolset_map -- map poolset */ int util_poolset_map(const char *fname, struct pool_set **poolset, int rdonly) { if (util_is_poolset_file(fname) != 1) { int ret = util_poolset_create_set(poolset, fname, 0, 0, true); if (ret < 0) { outv_err("cannot open pool set -- '%s'", fname); return -1; } unsigned flags = (rdonly ? POOL_OPEN_COW : 0) | POOL_OPEN_IGNORE_BAD_BLOCKS; return util_pool_open_nocheck(*poolset, flags); } /* open poolset file */ int fd = util_file_open(fname, NULL, 0, O_RDONLY); if (fd < 0) return -1; struct pool_set *set; /* parse poolset file */ if (util_poolset_parse(&set, fname, fd)) { outv_err("parsing poolset file failed\n"); os_close(fd); return -1; } set->ignore_sds = true; os_close(fd); /* read the pool header from first pool set file */ const char *part0_path = PART(REP(set, 0), 0)->path; struct pool_hdr hdr; if (util_file_pread(part0_path, &hdr, sizeof(hdr), 0) != sizeof(hdr)) { outv_err("cannot read pool header from poolset\n"); goto err_pool_set; } util_poolset_free(set); util_convert2h_hdr_nocheck(&hdr); /* parse pool type from first pool set file */ pmem_pool_type_t type = pmem_pool_type_parse_hdr(&hdr); if (type == PMEM_POOL_TYPE_UNKNOWN) { outv_err("cannot determine pool type from poolset\n"); return -1; } /* * Just use one thread - there is no need for multi-threaded access * to remote pool. */ unsigned nlanes = 1; /* * Open the poolset, the values passed to util_pool_open are read * from the first poolset file, these values are then compared with * the values from all headers of poolset files. */ struct pool_attr attr; util_pool_hdr2attr(&attr, &hdr); unsigned flags = (rdonly ? POOL_OPEN_COW : 0) | POOL_OPEN_IGNORE_SDS | POOL_OPEN_IGNORE_BAD_BLOCKS; if (util_pool_open(poolset, fname, 0 /* minpartsize */, &attr, &nlanes, NULL, flags)) { outv_err("opening poolset failed\n"); return -1; } return 0; err_pool_set: util_poolset_free(set); return -1; } /* * pmem_pool_parse_params -- parse pool type, file size and block size */ int pmem_pool_parse_params(const char *fname, struct pmem_pool_params *paramsp, int check) { paramsp->type = PMEM_POOL_TYPE_UNKNOWN; char pool_str_addr[POOL_HDR_DESC_SIZE]; enum file_type type = util_file_get_type(fname); if (type < 0) return -1; int is_poolset = util_is_poolset_file(fname); if (is_poolset < 0) return -1; paramsp->is_poolset = is_poolset; int fd = util_file_open(fname, NULL, 0, O_RDONLY); if (fd < 0) return -1; /* get file size and mode */ os_stat_t stat_buf; if (os_fstat(fd, &stat_buf)) { os_close(fd); return -1; } int ret = 0; assert(stat_buf.st_size >= 0); paramsp->size = (uint64_t)stat_buf.st_size; paramsp->mode = stat_buf.st_mode; void *addr = NULL; struct pool_set *set = NULL; if (paramsp->is_poolset) { /* close the file */ os_close(fd); fd = -1; if (check) { if (util_poolset_map(fname, &set, 0)) { ret = -1; goto out_close; } } else { ret = util_poolset_create_set(&set, fname, 0, 0, true); if (ret < 0) { outv_err("cannot open pool set -- '%s'", fname); ret = -1; goto out_close; } if (util_pool_open_nocheck(set, POOL_OPEN_IGNORE_BAD_BLOCKS)) { ret = -1; goto out_close; } } paramsp->size = set->poolsize; addr = set->replica[0]->part[0].addr; /* * XXX mprotect for device dax with length not aligned to its * page granularity causes SIGBUS on the next page fault. * The length argument of this call should be changed to * set->poolsize once the kernel issue is solved. */ if (mprotect(addr, set->replica[0]->repsize, PROT_READ) < 0) { outv_err("!mprotect"); goto out_close; } } else { /* read first two pages */ if (type == TYPE_DEVDAX) { addr = util_file_map_whole(fname); if (addr == NULL) { ret = -1; goto out_close; } } else { ssize_t num = read(fd, pool_str_addr, POOL_HDR_DESC_SIZE); if (num < (ssize_t)POOL_HDR_DESC_SIZE) { outv_err("!read"); ret = -1; goto out_close; } addr = pool_str_addr; } } struct pool_hdr hdr; memcpy(&hdr, addr, sizeof(hdr)); util_convert2h_hdr_nocheck(&hdr); memcpy(paramsp->signature, hdr.signature, sizeof(paramsp->signature)); /* * Check if file is a part of pool set by comparing * the UUID with the next part UUID. If it is the same * it means the pool consist of a single file. */ paramsp->is_part = !paramsp->is_poolset && (memcmp(hdr.uuid, hdr.next_part_uuid, POOL_HDR_UUID_LEN) || memcmp(hdr.uuid, hdr.prev_part_uuid, POOL_HDR_UUID_LEN) || memcmp(hdr.uuid, hdr.next_repl_uuid, POOL_HDR_UUID_LEN) || memcmp(hdr.uuid, hdr.prev_repl_uuid, POOL_HDR_UUID_LEN)); if (check) paramsp->type = pmem_pool_type(addr); else paramsp->type = pmem_pool_type_parse_hdr(addr); paramsp->is_checksum_ok = pmem_pool_checksum(addr); if (paramsp->type == PMEM_POOL_TYPE_BLK) { struct pmemblk *pbp = addr; paramsp->blk.bsize = le32toh(pbp->bsize); } else if (paramsp->type == PMEM_POOL_TYPE_OBJ) { struct pmemobjpool *pop = addr; memcpy(paramsp->obj.layout, pop->layout, PMEMOBJ_MAX_LAYOUT); } if (paramsp->is_poolset) util_poolset_close(set, DO_NOT_DELETE_PARTS); out_close: if (fd >= 0) (void) os_close(fd); return ret; } /* * util_check_memory -- check if memory contains single value */ int util_check_memory(const uint8_t *buff, size_t len, uint8_t val) { size_t i; for (i = 0; i < len; i++) { if (buff[i] != val) return -1; } return 0; } /* * pmempool_ask_yes_no -- prints the question, * takes user answer and returns validated value */ static char pmempool_ask_yes_no(char def_ans, const char *answers, const char *qbuff) { char ret = INV_ANS; printf("%s", qbuff); size_t len = strlen(answers); size_t i; char def_anslo = (char)tolower(def_ans); printf(" ["); for (i = 0; i < len; i++) { char anslo = (char)tolower(answers[i]); printf("%c", anslo == def_anslo ? toupper(anslo) : anslo); if (i != len - 1) printf("/"); } printf("] "); char *line_of_answer = util_readline(stdin); if (line_of_answer == NULL) { outv_err("input is empty"); return '?'; } char first_letter = line_of_answer[0]; line_of_answer[0] = (char)tolower(first_letter); if (strcmp(line_of_answer, "yes\n") == 0) { if (strchr(answers, 'y') != NULL) ret = 'y'; } if (strcmp(line_of_answer, "no\n") == 0) { if (strchr(answers, 'n') != NULL) ret = 'n'; } if (strlen(line_of_answer) == 2 && line_of_answer[1] == '\n') { if (strchr(answers, line_of_answer[0]) != NULL) ret = line_of_answer[0]; } if (strlen(line_of_answer) == 1 && line_of_answer[0] == '\n') { ret = def_ans; } Free(line_of_answer); return ret; } /* * ask -- keep asking for answer until it gets valid input */ char ask(char op, char *answers, char def_ans, const char *fmt, va_list ap) { char qbuff[Q_BUFF_SIZE]; char ret = INV_ANS; int is_tty = 0; if (op != '?') return op; int p = vsnprintf(qbuff, Q_BUFF_SIZE, fmt, ap); if (p < 0) { outv_err("vsnprintf"); exit(EXIT_FAILURE); } if (p >= Q_BUFF_SIZE) { outv_err("vsnprintf: output was truncated"); exit(EXIT_FAILURE); } is_tty = isatty(fileno(stdin)); while ((ret = pmempool_ask_yes_no(def_ans, answers, qbuff)) == INV_ANS) ; if (!is_tty) printf("%c\n", ret); return ret; } char ask_Yn(char op, const char *fmt, ...) { va_list ap; va_start(ap, fmt); char ret = ask(op, "yn", 'y', fmt, ap); va_end(ap); return ret; } char ask_yN(char op, const char *fmt, ...) { va_list ap; va_start(ap, fmt); char ret = ask(op, "yn", 'n', fmt, ap); va_end(ap); return ret; } /* * util_parse_enum -- parse single enum and store to bitmap */ static int util_parse_enum(const char *str, int first, int max, uint64_t *bitmap, enum_to_str_fn enum_to_str) { for (int i = first; i < max; i++) { if (strcmp(str, enum_to_str(i)) == 0) { *bitmap |= (uint64_t)1<opts = options; opts->noptions = nopts; opts->req = req; size_t bitmap_size = howmany(nopts, 8); opts->bitmap = calloc(bitmap_size, 1); if (!opts->bitmap) err(1, "Cannot allocate memory for options bitmap"); return opts; } /* * util_options_free -- free options structure */ void util_options_free(struct options *opts) { free(opts->bitmap); free(opts); } /* * util_opt_get_index -- return index of specified option in global * array of options */ static int util_opt_get_index(const struct options *opts, int opt) { const struct option *lopt = &opts->opts[0]; int ret = 0; while (lopt->name) { if ((lopt->val & ~OPT_MASK) == opt) return ret; lopt++; ret++; } return -1; } /* * util_opt_get_req -- get required option for specified option */ static struct option_requirement * util_opt_get_req(const struct options *opts, int opt, pmem_pool_type_t type) { size_t n = 0; struct option_requirement *ret = NULL; struct option_requirement *tmp = NULL; const struct option_requirement *req = &opts->req[0]; while (req->opt) { if (req->opt == opt && (req->type & type)) { n++; tmp = realloc(ret, n * sizeof(*ret)); if (!tmp) err(1, "Cannot allocate memory for" " option requirements"); ret = tmp; ret[n - 1] = *req; } req++; } if (ret) { tmp = realloc(ret, (n + 1) * sizeof(*ret)); if (!tmp) err(1, "Cannot allocate memory for" " option requirements"); ret = tmp; memset(&ret[n], 0, sizeof(*ret)); } return ret; } /* * util_opt_check_requirements -- check if requirements has been fulfilled */ static int util_opt_check_requirements(const struct options *opts, const struct option_requirement *req) { int count = 0; int set = 0; uint64_t tmp; while ((tmp = req->req) != 0) { while (tmp) { int req_idx = util_opt_get_index(opts, tmp & OPT_REQ_MASK); if (req_idx >= 0 && util_isset(opts->bitmap, req_idx)) { set++; break; } tmp >>= OPT_REQ_SHIFT; } req++; count++; } return count != set; } /* * util_opt_print_requirements -- print requirements for specified option */ static void util_opt_print_requirements(const struct options *opts, const struct option_requirement *req) { char buff[REQ_BUFF_SIZE]; unsigned n = 0; uint64_t tmp; const struct option *opt = &opts->opts[util_opt_get_index(opts, req->opt)]; int sn; sn = snprintf(&buff[n], REQ_BUFF_SIZE - n, "option [-%c|--%s] requires: ", opt->val, opt->name); assert(sn >= 0); if (sn >= 0) n += (unsigned)sn; size_t rc = 0; while ((tmp = req->req) != 0) { if (rc != 0) { sn = snprintf(&buff[n], REQ_BUFF_SIZE - n, " and "); assert(sn >= 0); if (sn >= 0) n += (unsigned)sn; } size_t c = 0; while (tmp) { if (c == 0) sn = snprintf(&buff[n], REQ_BUFF_SIZE - n, "["); else sn = snprintf(&buff[n], REQ_BUFF_SIZE - n, "|"); assert(sn >= 0); if (sn >= 0) n += (unsigned)sn; int req_opt_ind = util_opt_get_index(opts, tmp & OPT_REQ_MASK); const struct option *req_option = &opts->opts[req_opt_ind]; sn = snprintf(&buff[n], REQ_BUFF_SIZE - n, "-%c|--%s", req_option->val, req_option->name); assert(sn >= 0); if (sn >= 0) n += (unsigned)sn; tmp >>= OPT_REQ_SHIFT; c++; } sn = snprintf(&buff[n], REQ_BUFF_SIZE - n, "]"); assert(sn >= 0); if (sn >= 0) n += (unsigned)sn; req++; rc++; } outv_err("%s\n", buff); } /* * util_opt_verify_requirements -- verify specified requirements for options */ static int util_opt_verify_requirements(const struct options *opts, size_t index, pmem_pool_type_t type) { const struct option *opt = &opts->opts[index]; int val = opt->val & ~OPT_MASK; struct option_requirement *req; if ((req = util_opt_get_req(opts, val, type)) == NULL) return 0; int ret = 0; if (util_opt_check_requirements(opts, req)) { ret = -1; util_opt_print_requirements(opts, req); } free(req); return ret; } /* * util_opt_verify_type -- check if used option matches pool type */ static int util_opt_verify_type(const struct options *opts, pmem_pool_type_t type, size_t index) { const struct option *opt = &opts->opts[index]; int val = opt->val & ~OPT_MASK; int opt_type = opt->val; opt_type >>= OPT_SHIFT; if (!(opt_type & (1<name, val, out_get_pool_type_str(type)); return -1; } return 0; } /* * util_options_getopt -- wrapper for getopt_long which sets bitmap */ int util_options_getopt(int argc, char *argv[], const char *optstr, const struct options *opts) { int opt = getopt_long(argc, argv, optstr, opts->opts, NULL); if (opt == -1 || opt == '?') return opt; opt &= ~OPT_MASK; int option_index = util_opt_get_index(opts, opt); assert(option_index >= 0); util_setbit((uint8_t *)opts->bitmap, (unsigned)option_index); return opt; } /* * util_options_verify -- verify options */ int util_options_verify(const struct options *opts, pmem_pool_type_t type) { for (size_t i = 0; i < opts->noptions; i++) { if (util_isset(opts->bitmap, i)) { if (util_opt_verify_type(opts, type, i)) return -1; if (opts->req) if (util_opt_verify_requirements(opts, i, type)) return -1; } } return 0; } /* * util_heap_max_zone -- get number of zones */ unsigned util_heap_max_zone(size_t size) { unsigned max_zone = 0; size -= sizeof(struct heap_header); while (size >= ZONE_MIN_SIZE) { max_zone++; size -= size <= ZONE_MAX_SIZE ? size : ZONE_MAX_SIZE; } return max_zone; } /* * pool_set_file_open -- opens pool set file or regular file */ struct pool_set_file * pool_set_file_open(const char *fname, int rdonly, int check) { struct pool_set_file *file = calloc(1, sizeof(*file)); if (!file) return NULL; file->replica = 0; file->fname = strdup(fname); if (!file->fname) goto err; os_stat_t buf; if (os_stat(fname, &buf)) { warn("%s", fname); goto err_free_fname; } file->mtime = buf.st_mtime; file->mode = buf.st_mode; if (S_ISBLK(file->mode)) file->fileio = true; if (file->fileio) { /* Simple file open for BTT device */ int fd = util_file_open(fname, NULL, 0, O_RDONLY); if (fd < 0) { outv_err("util_file_open failed\n"); goto err_free_fname; } os_off_t seek_size = os_lseek(fd, 0, SEEK_END); if (seek_size == -1) { outv_err("lseek SEEK_END failed\n"); os_close(fd); goto err_free_fname; } file->size = (size_t)seek_size; file->fd = fd; } else { /* * The check flag indicates whether the headers from each pool * set file part should be checked for valid values. */ if (check) { if (util_poolset_map(file->fname, &file->poolset, rdonly)) goto err_free_fname; } else { int ret = util_poolset_create_set(&file->poolset, file->fname, 0, 0, true); if (ret < 0) { outv_err("cannot open pool set -- '%s'", file->fname); goto err_free_fname; } unsigned flags = (rdonly ? POOL_OPEN_COW : 0) | POOL_OPEN_IGNORE_BAD_BLOCKS; if (util_pool_open_nocheck(file->poolset, flags)) goto err_free_fname; } /* get modification time from the first part of first replica */ const char *path = file->poolset->replica[0]->part[0].path; if (os_stat(path, &buf)) { warn("%s", path); goto err_close_poolset; } file->size = file->poolset->poolsize; file->addr = file->poolset->replica[0]->part[0].addr; } return file; err_close_poolset: util_poolset_close(file->poolset, DO_NOT_DELETE_PARTS); err_free_fname: free(file->fname); err: free(file); return NULL; } /* * pool_set_file_close -- closes pool set file or regular file */ void pool_set_file_close(struct pool_set_file *file) { if (!file->fileio) { if (file->poolset) util_poolset_close(file->poolset, DO_NOT_DELETE_PARTS); else if (file->addr) { munmap(file->addr, file->size); os_close(file->fd); } } free(file->fname); free(file); } /* * pool_set_file_read -- read from pool set file or regular file * * 'buff' has to be a buffer at least 'nbytes' long * 'off' is an offset from the beginning of the file */ int pool_set_file_read(struct pool_set_file *file, void *buff, size_t nbytes, uint64_t off) { if (off + nbytes > file->size) return -1; if (file->fileio) { ssize_t num = pread(file->fd, buff, nbytes, (os_off_t)off); if (num < (ssize_t)nbytes) return -1; } else { memcpy(buff, (char *)file->addr + off, nbytes); } return 0; } /* * pool_set_file_write -- write to pool set file or regular file * * 'buff' has to be a buffer at least 'nbytes' long * 'off' is an offset from the beginning of the file */ int pool_set_file_write(struct pool_set_file *file, void *buff, size_t nbytes, uint64_t off) { enum file_type type = util_file_get_type(file->fname); if (type < 0) return -1; if (off + nbytes > file->size) return -1; if (file->fileio) { ssize_t num = pwrite(file->fd, buff, nbytes, (os_off_t)off); if (num < (ssize_t)nbytes) return -1; } else { memcpy((char *)file->addr + off, buff, nbytes); util_persist_auto(type == TYPE_DEVDAX, (char *)file->addr + off, nbytes); } return 0; } /* * pool_set_file_set_replica -- change replica for pool set file */ int pool_set_file_set_replica(struct pool_set_file *file, size_t replica) { if (!replica) return 0; if (!file->poolset) return -1; if (replica >= file->poolset->nreplicas) return -1; if (file->poolset->replica[replica]->remote) { outv_err("reading from remote replica not supported"); return -1; } file->replica = replica; file->addr = file->poolset->replica[replica]->part[0].addr; return 0; } /* * pool_set_file_nreplicas -- return number of replicas */ size_t pool_set_file_nreplicas(struct pool_set_file *file) { return file->poolset->nreplicas; } /* * pool_set_file_map -- return mapped address at given offset */ void * pool_set_file_map(struct pool_set_file *file, uint64_t offset) { if (file->addr == MAP_FAILED) return NULL; return (char *)file->addr + offset; } /* * pool_set_file_persist -- propagates and persists changes to a memory range * * 'addr' points to the beginning of data in the master replica that has to be * propagated * 'len' is the number of bytes to be propagated to other replicas */ void pool_set_file_persist(struct pool_set_file *file, const void *addr, size_t len) { uintptr_t offset = (uintptr_t)((char *)addr - (char *)file->poolset->replica[0]->part[0].addr); for (unsigned r = 1; r < file->poolset->nreplicas; ++r) { struct pool_replica *rep = file->poolset->replica[r]; void *dst = (char *)rep->part[0].addr + offset; memcpy(dst, addr, len); util_persist(rep->is_pmem, dst, len); } struct pool_replica *rep = file->poolset->replica[0]; util_persist(rep->is_pmem, (void *)addr, len); } /* * util_pool_clear_badblocks -- clear badblocks in a pool (set or a single file) */ int util_pool_clear_badblocks(const char *path, int create) { LOG(3, "path %s create %i", path, create); struct pool_set *setp; /* do not check minsize */ int ret = util_poolset_create_set(&setp, path, 0, 0, POOL_OPEN_IGNORE_SDS); if (ret < 0) { LOG(2, "cannot open pool set -- '%s'", path); return -1; } if (badblocks_clear_poolset(setp, create)) { outv_err("clearing bad blocks in the pool set failed -- '%s'", path); errno = EIO; return -1; } return 0; } pmdk-1.8/src/tools/pmempool/Makefile0000664000000000000000000000560613615011243016237 0ustar rootroot# Copyright 2014-2020, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # Makefile -- top Makefile for pmempool # SCP_TO_REMOTE_NODES = y vpath %.c ../../libpmemobj/ vpath %.c ../../librpmem/ vpath %.c ../../rpmem_common/ TARGET = pmempool OBJS = pmempool.o\ info.o info_blk.o info_log.o info_obj.o ulog.o\ create.o dump.o check.o rm.o convert.o synchronize.o transform.o\ rpmem_ssh.o rpmem_cmd.o rpmem_util.o rpmem_common.o feature.o LIBPMEM=y LIBPMEMBLK=y LIBPMEMOBJ=y LIBPMEMLOG=y LIBPMEMPOOL=y TOOLS_COMMON=y LIBPMEMOBJ_PRIV=memblock_from_offset alloc_class_by_id\ memblock_rebuild_state alloc_class_by_run\ heap_run_foreach_object alloc_class_collection_new\ alloc_class_collection_delete LIBPMEMBLK_PRIV=btt_init btt_write btt_fini btt_info_convert2h\ btt_info_convert2le btt_flog_convert2h btt_flog_convert2le INCS += -I$(TOP)/src/common INCS += -I$(TOP)/src/rpmem_common INCS += -I$(TOP)/src/librpmem INCS += -I$(TOP)/src/libpmemlog INCS += -I$(TOP)/src/libpmemblk INCS += -I$(TOP)/src/libpmemobj CFLAGS += -DUSE_RPMEM MANPAGES = $(TOP)/doc/pmempool.1\ $(TOP)/doc/pmempool-info.1\ $(TOP)/doc/pmempool-create.1\ $(TOP)/doc/pmempool-check.1\ $(TOP)/doc/pmempool-dump.1\ $(TOP)/doc/pmempool-rm.1\ $(TOP)/doc/pmempool-convert.1\ $(TOP)/doc/pmempool-sync.1\ $(TOP)/doc/pmempool-transform.1 BASH_COMP_FILES = bash_completion/pmempool include ../Makefile.inc .PHONY: test check pmdk-1.8/src/tools/pmempool/pmempool.vcxproj.filters0000664000000000000000000001135313615011243021507 0ustar rootroot {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {93995380-89BD-4b04-88EB-625FBE52EBFB} h;hh;hpp;hxx;hm;inl;inc;xsd {c91552dc-7579-447b-ad7f-7b2307c52502} Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files libs libs libs libs libs libs libs libs libs libs libs libs libs libs libs libs libs Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Source Files pmdk-1.8/src/tools/pmempool/check.h0000664000000000000000000000336013615011243016020 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check.h -- pmempool check command header file */ int pmempool_check_func(const char *appname, int argc, char *argv[]); void pmempool_check_help(const char *appname); pmdk-1.8/src/tools/pmempool/create.h0000664000000000000000000000336413615011243016212 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * create.h -- pmempool create command header file */ int pmempool_create_func(const char *appname, int argc, char *argv[]); void pmempool_create_help(const char *appname); pmdk-1.8/src/tools/pmempool/pmempool.c0000664000000000000000000001607513615011243016575 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmempool.c -- pmempool main source file */ #include #include #include #include #include #include #include #include "common.h" #include "output.h" #include "info.h" #include "create.h" #include "dump.h" #include "check.h" #include "rm.h" #include "convert.h" #include "synchronize.h" #include "transform.h" #include "feature.h" #include "set.h" #ifndef _WIN32 #include "rpmem_common.h" #include "rpmem_util.h" #endif #define APPNAME "pmempool" /* * command -- struct for pmempool commands definition */ struct command { const char *name; const char *brief; int (*func)(const char *, int, char *[]); void (*help)(const char *); }; static const struct command *get_command(const char *cmd_str); static void print_help(const char *appname); /* * long_options -- pmempool command line arguments */ static const struct option long_options[] = { {"version", no_argument, NULL, 'V'}, {"help", no_argument, NULL, 'h'}, {NULL, 0, NULL, 0 }, }; /* * help_help -- prints help message for help command */ static void help_help(const char *appname) { printf("Usage: %s help \n", appname); } /* * help_func -- prints help message for specified command */ static int help_func(const char *appname, int argc, char *argv[]) { if (argc > 1) { char *cmd_str = argv[1]; const struct command *cmdp = get_command(cmd_str); if (cmdp && cmdp->help) { cmdp->help(appname); return 0; } else { outv_err("No help text for '%s' command\n", cmd_str); return -1; } } else { print_help(appname); return -1; } } /* * commands -- definition of all pmempool commands */ static const struct command commands[] = { { .name = "info", .brief = "print information and statistics about a pool", .func = pmempool_info_func, .help = pmempool_info_help, }, { .name = "create", .brief = "create a pool", .func = pmempool_create_func, .help = pmempool_create_help, }, { .name = "dump", .brief = "dump user data from a pool", .func = pmempool_dump_func, .help = pmempool_dump_help, }, { .name = "check", .brief = "check consistency of a pool", .func = pmempool_check_func, .help = pmempool_check_help, }, { .name = "rm", .brief = "remove pool or poolset", .func = pmempool_rm_func, .help = pmempool_rm_help, }, { .name = "convert", .brief = "perform pool layout conversion", .func = pmempool_convert_func, .help = pmempool_convert_help, }, { .name = "sync", .brief = "synchronize data between replicas", .func = pmempool_sync_func, .help = pmempool_sync_help, }, { .name = "transform", .brief = "modify internal structure of a poolset", .func = pmempool_transform_func, .help = pmempool_transform_help, }, { .name = "feature", .brief = "toggle / query pool features", .func = pmempool_feature_func, .help = pmempool_feature_help, }, { .name = "help", .brief = "print help text about a command", .func = help_func, .help = help_help, }, }; /* * number of pmempool commands */ #define COMMANDS_NUMBER (sizeof(commands) / sizeof(commands[0])) /* * print_version -- prints pmempool version message */ static void print_version(const char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * print_usage -- prints pmempool usage message */ static void print_usage(const char *appname) { printf("usage: %s [--version] [--help] []\n", appname); } /* * print_help -- prints pmempool help message */ static void print_help(const char *appname) { print_usage(appname); print_version(appname); printf("\n"); printf("Options:\n"); printf(" -V, --version display version\n"); printf(" -h, --help display this help and exit\n"); printf("\n"); printf("The available commands are:\n"); unsigned i; for (i = 0; i < COMMANDS_NUMBER; i++) { const char *format = (strlen(commands[i].name) / 8) ? "%s\t- %s\n" : "%s\t\t- %s\n"; printf(format, commands[i].name, commands[i].brief); } printf("\n"); printf("For complete documentation see %s(1) manual page.\n", appname); } /* * get_command -- returns command for specified command name */ static const struct command * get_command(const char *cmd_str) { unsigned i; for (i = 0; i < COMMANDS_NUMBER; i++) { if (strcmp(cmd_str, commands[i].name) == 0) return &commands[i]; } return NULL; } int main(int argc, char *argv[]) { int opt; int option_index; int ret = 0; #ifdef _WIN32 util_suppress_errmsg(); wchar_t **wargv = CommandLineToArgvW(GetCommandLineW(), &argc); for (int i = 0; i < argc; i++) { argv[i] = util_toUTF8(wargv[i]); if (argv[i] == NULL) { for (i--; i >= 0; i--) free(argv[i]); outv_err("Error during arguments conversion\n"); return 1; } } #endif util_init(); #ifndef _WIN32 util_remote_init(); rpmem_util_cmds_init(); #endif if (argc < 2) { print_usage(APPNAME); goto end; } while ((opt = getopt_long(2, argv, "Vh", long_options, &option_index)) != -1) { switch (opt) { case 'V': print_version(APPNAME); goto end; case 'h': print_help(APPNAME); goto end; default: print_usage(APPNAME); ret = 1; goto end; } } char *cmd_str = argv[optind]; const struct command *cmdp = get_command(cmd_str); if (cmdp) { ret = cmdp->func(APPNAME, argc - 1, argv + 1); } else { outv_err("'%s' -- unknown command\n", cmd_str); ret = 1; } #ifndef _WIN32 util_remote_fini(); rpmem_util_cmds_fini(); #endif end: #ifdef _WIN32 for (int i = argc; i > 0; i--) free(argv[i - 1]); #endif if (ret) return 1; return 0; } pmdk-1.8/src/tools/pmempool/check.c0000664000000000000000000002074613615011243016022 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check.c -- pmempool check command source file */ #include #include #include "common.h" #include "check.h" #include "output.h" #include "set.h" #include "file.h" #include "libpmempool.h" typedef enum { CHECK_RESULT_CONSISTENT, CHECK_RESULT_NOT_CONSISTENT, CHECK_RESULT_REPAIRED, CHECK_RESULT_CANNOT_REPAIR, CHECK_RESULT_SYNC_REQ, CHECK_RESULT_ERROR } check_result_t; /* * pmempool_check_context -- context and arguments for check command */ struct pmempool_check_context { int verbose; /* verbosity level */ char *fname; /* file name */ struct pool_set_file *pfile; bool repair; /* do repair */ bool backup; /* do backup */ bool advanced; /* do advanced repairs */ char *backup_fname; /* backup file name */ bool exec; /* do execute */ char ans; /* default answer on all questions or '?' */ }; /* * pmempool_check_default -- default arguments for check command */ static const struct pmempool_check_context pmempool_check_default = { .verbose = 1, .fname = NULL, .repair = false, .backup = false, .backup_fname = NULL, .advanced = false, .exec = true, .ans = '?', }; /* * help_str -- string for help message */ static const char * const help_str = "Check consistency of a pool\n" "\n" "Common options:\n" " -r, --repair try to repair a pool file if possible\n" " -y, --yes answer yes to all questions\n" " -d, --dry-run don't execute, just show what would be done\n" " -b, --backup create backup of a pool file before executing\n" " -a, --advanced perform advanced repairs\n" " -q, --quiet be quiet and don't print any messages\n" " -v, --verbose increase verbosity level\n" " -h, --help display this help and exit\n" "\n" "For complete documentation see %s-check(1) manual page.\n" ; /* * long_options -- command line options */ static const struct option long_options[] = { {"repair", no_argument, NULL, 'r'}, {"yes", no_argument, NULL, 'y'}, {"dry-run", no_argument, NULL, 'd'}, {"no-exec", no_argument, NULL, 'N'}, /* deprecated */ {"backup", required_argument, NULL, 'b'}, {"advanced", no_argument, NULL, 'a'}, {"quiet", no_argument, NULL, 'q'}, {"verbose", no_argument, NULL, 'v'}, {"help", no_argument, NULL, 'h'}, {NULL, 0, NULL, 0 }, }; /* * print_usage -- print short description of application's usage */ static void print_usage(const char *appname) { printf("Usage: %s check [] \n", appname); } /* * print_version -- print version string */ static void print_version(const char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * pmempool_check_help -- print help message for check command */ void pmempool_check_help(const char *appname) { print_usage(appname); print_version(appname); printf(help_str, appname); } /* * pmempool_check_parse_args -- parse command line arguments */ static int pmempool_check_parse_args(struct pmempool_check_context *pcp, const char *appname, int argc, char *argv[]) { int opt; while ((opt = getopt_long(argc, argv, "ahvrdNb:qy", long_options, NULL)) != -1) { switch (opt) { case 'r': pcp->repair = true; break; case 'y': pcp->ans = 'y'; break; case 'd': case 'N': pcp->exec = false; break; case 'b': pcp->backup = true; pcp->backup_fname = optarg; break; case 'a': pcp->advanced = true; break; case 'q': pcp->verbose = 0; break; case 'v': pcp->verbose = 2; break; case 'h': pmempool_check_help(appname); exit(EXIT_SUCCESS); default: print_usage(appname); exit(EXIT_FAILURE); } } if (optind < argc) { pcp->fname = argv[optind]; } else { print_usage(appname); exit(EXIT_FAILURE); } if (!pcp->repair && !pcp->exec) { outv_err("'-N' option requires '-r'\n"); exit(EXIT_FAILURE); } if (!pcp->repair && pcp->backup) { outv_err("'-b' option requires '-r'\n"); exit(EXIT_FAILURE); } return 0; } static check_result_t pmempool_check_2_check_res_t[] = { [PMEMPOOL_CHECK_RESULT_CONSISTENT] = CHECK_RESULT_CONSISTENT, [PMEMPOOL_CHECK_RESULT_NOT_CONSISTENT] = CHECK_RESULT_NOT_CONSISTENT, [PMEMPOOL_CHECK_RESULT_REPAIRED] = CHECK_RESULT_REPAIRED, [PMEMPOOL_CHECK_RESULT_CANNOT_REPAIR] = CHECK_RESULT_CANNOT_REPAIR, [PMEMPOOL_CHECK_RESULT_SYNC_REQ] = CHECK_RESULT_SYNC_REQ, [PMEMPOOL_CHECK_RESULT_ERROR] = CHECK_RESULT_ERROR, }; static const char * check_ask(const char *msg) { char answer = ask_Yn('?', "%s", msg); switch (answer) { case 'y': return "yes"; case 'n': return "no"; default: return "?"; } } static check_result_t pmempool_check_perform(struct pmempool_check_context *pc) { struct pmempool_check_args args = { .path = pc->fname, .backup_path = pc->backup_fname, .pool_type = PMEMPOOL_POOL_TYPE_DETECT, .flags = PMEMPOOL_CHECK_FORMAT_STR }; if (pc->repair) args.flags |= PMEMPOOL_CHECK_REPAIR; if (!pc->exec) args.flags |= PMEMPOOL_CHECK_DRY_RUN; if (pc->advanced) args.flags |= PMEMPOOL_CHECK_ADVANCED; if (pc->ans == 'y') args.flags |= PMEMPOOL_CHECK_ALWAYS_YES; if (pc->verbose == 2) args.flags |= PMEMPOOL_CHECK_VERBOSE; PMEMpoolcheck *ppc = pmempool_check_init(&args, sizeof(args)); if (ppc == NULL) return CHECK_RESULT_ERROR; struct pmempool_check_status *status = NULL; while ((status = pmempool_check(ppc)) != NULL) { switch (status->type) { case PMEMPOOL_CHECK_MSG_TYPE_ERROR: outv(1, "%s\n", status->str.msg); break; case PMEMPOOL_CHECK_MSG_TYPE_INFO: outv(2, "%s\n", status->str.msg); break; case PMEMPOOL_CHECK_MSG_TYPE_QUESTION: status->str.answer = check_ask(status->str.msg); break; default: pmempool_check_end(ppc); exit(EXIT_FAILURE); } } enum pmempool_check_result ret = pmempool_check_end(ppc); return pmempool_check_2_check_res_t[ret]; } /* * pmempool_check_func -- main function for check command */ int pmempool_check_func(const char *appname, int argc, char *argv[]) { int ret = 0; check_result_t res = CHECK_RESULT_CONSISTENT; struct pmempool_check_context pc = pmempool_check_default; /* parse command line arguments */ ret = pmempool_check_parse_args(&pc, appname, argc, argv); if (ret) return ret; /* set verbosity level */ out_set_vlevel(pc.verbose); res = pmempool_check_perform(&pc); switch (res) { case CHECK_RESULT_CONSISTENT: outv(2, "%s: consistent\n", pc.fname); ret = 0; break; case CHECK_RESULT_NOT_CONSISTENT: outv(1, "%s: not consistent\n", pc.fname); ret = -1; break; case CHECK_RESULT_REPAIRED: outv(1, "%s: repaired\n", pc.fname); ret = 0; break; case CHECK_RESULT_CANNOT_REPAIR: outv(1, "%s: cannot repair\n", pc.fname); ret = -1; break; case CHECK_RESULT_SYNC_REQ: outv(1, "%s: sync required\n", pc.fname); ret = 0; break; case CHECK_RESULT_ERROR: if (errno) outv_err("%s\n", strerror(errno)); if (pc.repair) outv_err("repairing failed\n"); else outv_err("checking consistency failed\n"); ret = -1; break; default: outv_err("status unknown\n"); ret = -1; break; } return ret; } pmdk-1.8/src/tools/Makefile.inc0000664000000000000000000002052413615011243015153 0ustar rootroot# Copyright 2014-2020, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/tools/Makefile.inc -- Makefile include for all tools # TOP := $(dir $(lastword $(MAKEFILE_LIST)))../.. include $(TOP)/src/common.inc INSTALL_TARGET ?= y INCS += -I. INCS += -I$(TOP)/src/include INCS += $(OS_INCS) CFLAGS += -std=gnu99 CFLAGS += -Wall CFLAGS += -Werror CFLAGS += -Wmissing-prototypes CFLAGS += -Wpointer-arith CFLAGS += -Wsign-conversion CFLAGS += -Wsign-compare ifeq ($(WCONVERSION_AVAILABLE), y) CFLAGS += -Wconversion endif CFLAGS += -fno-common CFLAGS += -DSRCVERSION='"$(SRCVERSION)"' ifeq ($(OS_DIMM),ndctl) CFLAGS += -DSDS_ENABLED endif ifeq ($(IS_ICC), n) CFLAGS += -Wunused-macros CFLAGS += -Wmissing-field-initializers endif ifeq ($(WUNREACHABLE_CODE_RETURN_AVAILABLE), y) CFLAGS += -Wunreachable-code-return endif ifeq ($(WMISSING_VARIABLE_DECLARATIONS_AVAILABLE), y) CFLAGS += -Wmissing-variable-declarations endif ifeq ($(WFLOAT_EQUAL_AVAILABLE), y) CFLAGS += -Wfloat-equal endif ifeq ($(WSWITCH_DEFAULT_AVAILABLE), y) CFLAGS += -Wswitch-default endif ifeq ($(WCAST_FUNCTION_TYPE_AVAILABLE), y) CFLAGS += -Wcast-function-type endif ifeq ($(DEBUG),1) CFLAGS += -ggdb $(EXTRA_CFLAGS_DEBUG) else CFLAGS += -O2 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 $(EXTRA_CFLAGS_RELEASE) endif ifeq ($(VALGRIND),0) CFLAGS += -DVALGRIND_ENABLED=0 CXXFLAGS += -DVALGRIND_ENABLED=0 endif ifeq ($(FAULT_INJECTION),1) CFLAGS += -DFAULT_INJECTION=1 CXXFLAGS += -DFAULT_INJECTION=1 endif ifneq ($(SANITIZE),) CFLAGS += -fsanitize=$(SANITIZE) LDFLAGS += -fsanitize=$(SANITIZE) endif LDFLAGS += $(OS_LIBS) CFLAGS += $(EXTRA_CFLAGS) LDFLAGS += -Wl,-z,relro -Wl,--warn-common -Wl,--fatal-warnings $(EXTRA_LDFLAGS) ifeq ($(DEBUG),1) LDFLAGS += -L$(TOP)/src/debug else LDFLAGS += -L$(TOP)/src/nondebug endif TARGET_DIR=$(DESTDIR)$(bindir) BASH_COMP_FILES ?= BASH_COMP_DESTDIR = $(DESTDIR)$(bashcompdir) ifneq ($(DEBUG),1) TARGET_STATIC_NONDEBUG=$(TARGET).static-nondebug endif TARGET_STATIC_DEBUG=$(TARGET).static-debug LIBSDIR=$(TOP)/src LIBSDIR_DEBUG=$(LIBSDIR)/debug LIBSDIR_NONDEBUG=$(LIBSDIR)/nondebug ifneq ($(DEBUG),) LIBSDIR_PRIV=$(LIBSDIR_DEBUG) else LIBSDIR_PRIV=$(LIBSDIR_NONDEBUG) endif PMEMLOG_PRIV_OBJ=$(LIBSDIR_PRIV)/libpmemlog/libpmemlog_unscoped.o PMEMOBJ_PRIV_OBJ=$(LIBSDIR_PRIV)/libpmemobj/libpmemobj_unscoped.o PMEMBLK_PRIV_OBJ=$(LIBSDIR_PRIV)/libpmemblk/libpmemblk_unscoped.o LIBS += $(LIBUUID) ifeq ($(LIBRT_NEEDED), y) LIBS += -lrt endif ifeq ($(TOOLS_COMMON), y) LIBPMEMCOMMON=y endif ifeq ($(LIBPMEMCOMMON), y) DYNAMIC_LIBS += -lpmemcommon STATIC_DEBUG_LIBS += $(LIBSDIR_DEBUG)/libpmemcommon.a STATIC_NONDEBUG_LIBS += $(LIBSDIR_NONDEBUG)/libpmemcommon.a CFLAGS += -I$(TOP)/src/common CFLAGS += $(LIBNDCTL_CFLAGS) LIBS += $(LIBNDCTL_LIBS) endif ifeq ($(LIBPMEMPOOL), y) LIBPMEM=y DYNAMIC_LIBS += -lpmempool STATIC_DEBUG_LIBS += $(LIBSDIR_DEBUG)/libpmempool.a STATIC_NONDEBUG_LIBS += $(LIBSDIR_NONDEBUG)/libpmempool.a endif ifeq ($(LIBPMEMBLK), y) LIBPMEM=y DYNAMIC_LIBS += -lpmemblk STATIC_DEBUG_LIBS += $(LIBSDIR_DEBUG)/libpmemblk.a STATIC_NONDEBUG_LIBS += $(LIBSDIR_NONDEBUG)/libpmemblk.a endif ifeq ($(LIBPMEMLOG), y) LIBPMEM=y DYNAMIC_LIBS += -lpmemlog STATIC_DEBUG_LIBS += $(LIBSDIR_DEBUG)/libpmemlog.a STATIC_NONDEBUG_LIBS += $(LIBSDIR_NONDEBUG)/libpmemlog.a endif ifeq ($(LIBPMEMOBJ), y) LIBPMEM=y DYNAMIC_LIBS += -lpmemobj STATIC_DEBUG_LIBS += $(LIBSDIR_DEBUG)/libpmemobj.a STATIC_NONDEBUG_LIBS += $(LIBSDIR_NONDEBUG)/libpmemobj.a endif ifeq ($(LIBPMEM),y) DYNAMIC_LIBS += -lpmem STATIC_DEBUG_LIBS += $(LIBSDIR_DEBUG)/libpmem.a STATIC_NONDEBUG_LIBS += $(LIBSDIR_NONDEBUG)/libpmem.a endif # If any of these libraries is required, we need to link libpthread ifneq ($(LIBPMEMCOMMON)$(LIBPMEM)$(LIBPMEMPOOL)$(LIBPMEMBLK)$(LIBPMEMLOG)$(LIBPMEMOBJ),) LIBS += -pthread endif # If any of these libraries is required, we need to link libdl ifneq ($(LIBPMEMCOMMON)$(LIBPMEMPOOL)$(LIBPMEMOBJ),) LIBS += $(LIBDL) endif ifeq ($(TOOLS_COMMON), y) vpath %.c $(TOP)/src/tools/pmempool OBJS += common.o output.o CFLAGS += -I$(TOP)/src/common CFLAGS += -I$(TOP)/src/libpmemlog CFLAGS += -I$(TOP)/src/libpmemblk CFLAGS += -I$(TOP)/src/libpmemobj CFLAGS += -I$(TOP)/src/tools/pmempool CFLAGS += $(UNIX98_CFLAGS) endif ifneq ($(LIBPMEMLOG_PRIV),) OBJS += pmemlog_priv.o endif ifneq ($(LIBPMEMOBJ_PRIV),) OBJS += pmemobj_priv.o endif ifneq ($(LIBPMEMBLK_PRIV),) OBJS += pmemblk_priv.o endif ifneq ($(HEADERS),) ifneq ($(filter 1 2, $(CSTYLEON)),) TMP_HEADERS := $(addsuffix tmp, $(HEADERS)) endif endif ifeq ($(COVERAGE),1) CFLAGS += $(GCOV_CFLAGS) LDFLAGS += $(GCOV_LDFLAGS) LIBS += $(GCOV_LIBS) endif MAKEFILE_DEPS=$(TOP)/src/tools/Makefile.inc $(TOP)/src/common.inc ifneq ($(TARGET),) all: $(TARGET) $(TARGET_STATIC_NONDEBUG) $(TARGET_STATIC_DEBUG) else all: endif SYNC_FILE=.synced clean: $(RM) $(OBJS) $(CLEAN_FILES) $(SYNC_FILE) $(TMP_HEADERS) clobber: clean ifneq ($(TARGET),) $(RM) $(TARGET) $(RM) $(TARGET_STATIC_NONDEBUG) $(RM) $(TARGET_STATIC_DEBUG) $(RM) -r .deps endif install: all ifeq ($(INSTALL_TARGET),y) ifneq ($(TARGET),) install -d $(TARGET_DIR) install -p -m 0755 $(TARGET) $(TARGET_DIR) endif ifneq ($(BASH_COMP_FILES),) install -d $(BASH_COMP_DESTDIR) install -p -m 0644 $(BASH_COMP_FILES) $(BASH_COMP_DESTDIR) endif endif uninstall: ifeq ($(INSTALL_TARGET),y) ifneq ($(TARGET),) $(RM) $(TARGET_DIR)/$(TARGET) endif ifneq ($(BASH_COMP_FILES),) $(RM) $(BASH_COMP_DESTDIR)/$(BASH_COMP_FILES) endif endif %.gz: % gzip -nc ./$< > $@ %.txt: % man ./$< > $@ $(TARGET) $(TARGET_STATIC_DEBUG) $(TARGET_STATIC_NONDEBUG): $(TMP_HEADERS) $(OBJS) $(MAKEFILE_DEPS) $(TARGET_STATIC_DEBUG): $(STATIC_DEBUG_LIBS) $(CC) $(LDFLAGS) -o $@ $(OBJS) $(STATIC_DEBUG_LIBS) $(LIBS) $(TARGET_STATIC_NONDEBUG): $(STATIC_NONDEBUG_LIBS) $(CC) $(LDFLAGS) -o $@ $(OBJS) $(STATIC_NONDEBUG_LIBS) $(LIBS) $(TARGET): $(CC) $(LDFLAGS) -o $@ $(OBJS) $(DYNAMIC_LIBS) $(LIBS) $(PMEMLOG_PRIV_OBJ): $(MAKE) -C $(LIBSDIR) libpmemlog pmemlog_priv.o: $(PMEMLOG_PRIV_OBJ) $(OBJCOPY) --localize-hidden $(addprefix -G, $(LIBPMEMLOG_PRIV)) $< $@ $(PMEMOBJ_PRIV_OBJ): $(MAKE) -C $(LIBSDIR) libpmemobj pmemobj_priv.o: $(PMEMOBJ_PRIV_OBJ) $(OBJCOPY) --localize-hidden $(addprefix -G, $(LIBPMEMOBJ_PRIV)) $< $@ $(PMEMBLK_PRIV_OBJ): $(MAKE) -C $(LIBSDIR) libpmemblk pmemblk_priv.o: $(PMEMBLK_PRIV_OBJ) $(OBJCOPY) --localize-hidden $(addprefix -G, $(LIBPMEMBLK_PRIV)) $< $@ objdir=. %.o: %.c $(MAKEFILE_DEPS) $(call check-cstyle, $<) @mkdir -p .deps $(CC) -MD $(CFLAGS) $(INCS) -c -o $@ $(call coverage-path, $<) $(call check-os, $@, $<) $(create-deps) %.htmp: %.h $(call check-cstyle, $<, $@) test check pcheck: all TESTCONFIG=$(TOP)/src/test/testconfig.sh DIR_SYNC=$(TOP)/src/test/.sync-dir $(TESTCONFIG): sync-remotes: all $(SYNC_FILE) $(SYNC_FILE): $(TARGET) $(TESTCONFIG) ifeq ($(SCP_TO_REMOTE_NODES), y) cp $(TARGET) $(DIR_SYNC) @touch $(SYNC_FILE) endif sparse: $(if $(TARGET), $(sparse-c)) .PHONY: all clean clobber install uninstall test check pcheck -include .deps/*.P pmdk-1.8/src/tools/daxio/0000775000000000000000000000000013615011243014044 5ustar rootrootpmdk-1.8/src/tools/daxio/README0000664000000000000000000000320113615011243014720 0ustar rootrootPersistent Memory Development Kit This is src/tools/daxio/README. This file contains the high-level description of daxio utility. The main purpose of daxio is to perform I/O on Device DAX devices or zero a Device DAX device. Since the standard I/O APIs (read/write) cannot be used with Device DAX, data transfer is performed on a memory-mapped device. The daxio may be used to dump Device DAX data to a file, restore data from a backup copy, or move/copy data to another device. There must be at least one Device DAX device involved either as the input or output. If input or output is not specified, it will default to stdin or stdout respectively. No length specified will default to input file/device length or to the output file/device length, if input is a special char file or stdin. For a Device DAX device, daxio will attempt to clear badblocks within range of writes before performing the I/O. 3. Source code -------------- The source code of daxio is located in daxio directory. By default daxio is installed in $(DESTDIR)/usr/bin directory. You can change it by passing $(TOOLSDIR) variable to "make install". For example, the following command will install daxio in ~/bin directory: $ make install DESTDIR=~ TOOLSDIR=/bin See the top-level README file for detailed information about building and installation. 4. Packaging ------------ The daxio utility is provided in separate packages. Both rpm and dpkg packages are built automatically with other packages. See the top-level README file for detailed information about building packages. 5. Versioning ------------- The versioning of daxio utility is the same as all PMDK libraries. pmdk-1.8/src/tools/daxio/daxio.c0000664000000000000000000003646213615011243015327 0ustar rootroot/* * Copyright 2018-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * daxio.c -- simple app for reading and writing data from/to * Device DAX device using mmap instead of file I/O API */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "util.h" #include "os_dimm.h" #define ALIGN_UP(size, align) (((size) + (align) - 1) & ~((align) - 1)) #define ALIGN_DOWN(size, align) ((size) & ~((align) - 1)) #define ERR(fmt, ...)\ do {\ fprintf(stderr, "daxio: " fmt, ##__VA_ARGS__);\ } while (0) #define FAIL(func)\ do {\ fprintf(stderr, "daxio: %s:%d: %s: %s\n",\ __func__, __LINE__, func, strerror(errno));\ } while (0) #define USAGE_MESSAGE \ "Usage: daxio [option] ...\n"\ "Valid options:\n"\ " -i, --input=FILE - input device/file (default stdin)\n"\ " -o, --output=FILE - output device/file (default stdout)\n"\ " -k, --skip=BYTES - skip offset for input (default 0)\n"\ " -s, --seek=BYTES - seek offset for output (default 0)\n"\ " -l, --len=BYTES - total length to perform the I/O\n"\ " -b, --clear-bad-blocks= - clear bad blocks (default: yes)\n"\ " -z, --zero - zeroing the device\n"\ " -h. --help - print this help\n"\ " -V, --version - display version of daxio\n" struct daxio_device { char *path; int fd; size_t size; /* actual file/device size */ int is_devdax; /* Device DAX only */ size_t align; /* internal device alignment */ char *addr; /* mapping base address */ size_t maplen; /* mapping length */ size_t offset; /* seek or skip */ unsigned major; unsigned minor; struct ndctl_ctx *ndctl_ctx; struct ndctl_region *region; /* parent region */ }; /* * daxio_context -- context and arguments */ struct daxio_context { size_t len; /* total length of I/O */ int zero; int clear_bad_blocks; struct daxio_device src; struct daxio_device dst; }; /* * default context */ static struct daxio_context Ctx = { SIZE_MAX, /* len */ 0, /* zero */ 1, /* clear_bad_blocks */ { NULL, -1, SIZE_MAX, 0, 0, NULL, 0, 0, 0, 0, NULL, NULL }, { NULL, -1, SIZE_MAX, 0, 0, NULL, 0, 0, 0, 0, NULL, NULL }, }; /* * print_version -- print daxio version */ static void print_version(void) { printf("%s\n", SRCVERSION); } /* * print_usage -- print short description of usage */ static void print_usage(void) { fprintf(stderr, USAGE_MESSAGE); } /* * long_options -- command line options */ static const struct option long_options[] = { {"input", required_argument, NULL, 'i'}, {"output", required_argument, NULL, 'o'}, {"skip", required_argument, NULL, 'k'}, {"seek", required_argument, NULL, 's'}, {"len", required_argument, NULL, 'l'}, {"clear-bad-blocks", required_argument, NULL, 'b'}, {"zero", no_argument, NULL, 'z'}, {"help", no_argument, NULL, 'h'}, {"version", no_argument, NULL, 'V'}, {NULL, 0, NULL, 0 }, }; /* * parse_args -- (internal) parse command line arguments */ static int parse_args(struct daxio_context *ctx, int argc, char * const argv[]) { int opt; size_t offset; size_t len; while ((opt = getopt_long(argc, argv, "i:o:k:s:l:b:zhV", long_options, NULL)) != -1) { switch (opt) { case 'i': ctx->src.path = optarg; break; case 'o': ctx->dst.path = optarg; break; case 'k': if (util_parse_size(optarg, &offset)) { ERR("'%s' -- invalid input offset\n", optarg); return -1; } ctx->src.offset = offset; break; case 's': if (util_parse_size(optarg, &offset)) { ERR("'%s' -- invalid output offset\n", optarg); return -1; } ctx->dst.offset = offset; break; case 'l': if (util_parse_size(optarg, &len)) { ERR("'%s' -- invalid length\n", optarg); return -1; } ctx->len = len; break; case 'z': ctx->zero = 1; break; case 'b': if (strcmp(optarg, "no") == 0) { ctx->clear_bad_blocks = 0; } else if (strcmp(optarg, "yes") == 0) { ctx->clear_bad_blocks = 1; } else { ERR( "'%s' -- invalid argument of the '--clear-bad-blocks' option\n", optarg); return -1; } break; case 'h': print_usage(); exit(EXIT_SUCCESS); case 'V': print_version(); exit(EXIT_SUCCESS); default: print_usage(); exit(EXIT_FAILURE); } } return 0; } /* * validate_args -- (internal) validate command line arguments */ static int validate_args(struct daxio_context *ctx) { if (ctx->zero && ctx->dst.path == NULL) { ERR("zeroing flag specified but no output file provided\n"); return -1; } if (!ctx->zero && ctx->src.path == NULL && ctx->dst.path == NULL) { ERR("an input file and/or an output file must be provided\n"); return -1; } /* if no input file provided, use stdin */ if (ctx->src.path == NULL) { if (ctx->src.offset != 0) { ERR( "skip offset specified but no input file provided\n"); return -1; } ctx->src.fd = STDIN_FILENO; ctx->src.path = "STDIN"; } /* if no output file provided, use stdout */ if (ctx->dst.path == NULL) { if (ctx->dst.offset != 0) { ERR( "seek offset specified but no output file provided\n"); return -1; } ctx->dst.fd = STDOUT_FILENO; ctx->dst.path = "STDOUT"; } return 0; } /* * match_dev_dax -- (internal) find Device DAX by major/minor device number */ static int match_dev_dax(struct daxio_device *dev, struct daxctl_region *dax_region) { struct daxctl_dev *d; daxctl_dev_foreach(dax_region, d) { if (dev->major == (unsigned)daxctl_dev_get_major(d) && dev->minor == (unsigned)daxctl_dev_get_minor(d)) { dev->size = daxctl_dev_get_size(d); return 1; } } return 0; } /* * find_dev_dax -- (internal) check if device is Device DAX * * If there is matching Device DAX, find its region, size and alignment. */ static int find_dev_dax(struct ndctl_ctx *ndctl_ctx, struct daxio_device *dev) { struct ndctl_bus *bus = NULL; struct ndctl_region *region = NULL; struct ndctl_dax *dax = NULL; struct daxctl_region *dax_region = NULL; ndctl_bus_foreach(ndctl_ctx, bus) { ndctl_region_foreach(bus, region) { ndctl_dax_foreach(region, dax) { dax_region = ndctl_dax_get_daxctl_region(dax); if (match_dev_dax(dev, dax_region)) { dev->is_devdax = 1; dev->align = ndctl_dax_get_align(dax); dev->region = region; return 1; } } } } /* try with dax regions */ struct daxctl_ctx *daxctl_ctx; if (daxctl_new(&daxctl_ctx)) return 0; int ret = 0; daxctl_region_foreach(daxctl_ctx, dax_region) { if (match_dev_dax(dev, dax_region)) { dev->is_devdax = 1; dev->align = daxctl_region_get_align(dax_region); dev->region = region; ret = 1; goto end; } } end: daxctl_unref(daxctl_ctx); return ret; } /* * setup_device -- (internal) open/mmap file/device */ static int setup_device(struct ndctl_ctx *ndctl_ctx, struct daxio_device *dev, int is_dst, int clear_bad_blocks) { int ret; int flags = O_RDWR; int prot = is_dst ? PROT_WRITE : PROT_READ; if (dev->fd != -1) { dev->size = SIZE_MAX; return 0; /* stdin/stdout */ } /* try to open file/device (if exists) */ dev->fd = open(dev->path, flags, S_IRUSR|S_IWUSR); if (dev->fd == -1) { ret = errno; if (ret == ENOENT && is_dst) { /* file does not exist - create it */ flags = O_CREAT|O_WRONLY|O_TRUNC; dev->size = SIZE_MAX; dev->fd = open(dev->path, flags, S_IRUSR|S_IWUSR); if (dev->fd == -1) { FAIL("open"); return -1; } return 0; } else { ERR("failed to open '%s': %s\n", dev->path, strerror(errno)); return -1; } } struct stat stbuf; ret = fstat(dev->fd, &stbuf); if (ret == -1) { FAIL("stat"); return -1; } /* check if this is regular file or device */ if (S_ISREG(stbuf.st_mode)) { if (is_dst) dev->size = SIZE_MAX; else dev->size = (size_t)stbuf.st_size; } else if (S_ISBLK(stbuf.st_mode)) { dev->size = (size_t)stbuf.st_size; } else if (S_ISCHR(stbuf.st_mode)) { dev->size = SIZE_MAX; dev->major = major(stbuf.st_rdev); dev->minor = minor(stbuf.st_rdev); } else { return -1; } /* check if this is Device DAX */ if (S_ISCHR(stbuf.st_mode)) find_dev_dax(ndctl_ctx, dev); if (!dev->is_devdax) return 0; if (is_dst && clear_bad_blocks) { /* XXX - clear only badblocks in range bound by offset/len */ if (os_dimm_devdax_clear_badblocks_all(dev->path)) { ERR("failed to clear bad blocks on \"%s\"\n" " Probably you have not enough permissions to do that.\n" " You can choose one of three options now:\n" " 1) run 'daxio' with 'sudo' or as 'root',\n" " 2) turn off clearing bad blocks using\n" " the '-b/--clear-bad-blocks=no' option or\n" " 3) change permissions of some resource files -\n" " - for details see the description of the CHECK_BAD_BLOCKS\n" " compat feature in the pmempool-feature(1) man page.\n", dev->path); return -1; } } if (dev->align == ULONG_MAX) { ERR("cannot determine device alignment for \"%s\"\n", dev->path); return -1; } if (dev->offset > dev->size) { ERR("'%zu' -- offset beyond device size (%zu)\n", dev->offset, dev->size); return -1; } /* align len/offset to the internal device alignment */ dev->maplen = ALIGN_UP(dev->size, dev->align); size_t offset = ALIGN_DOWN(dev->offset, dev->align); dev->offset = dev->offset - offset; dev->maplen = dev->maplen - offset; dev->addr = mmap(NULL, dev->maplen, prot, MAP_SHARED, dev->fd, (off_t)offset); if (dev->addr == MAP_FAILED) { FAIL("mmap"); return -1; } return 0; } /* * setup_devices -- (internal) open/mmap input and output */ static int setup_devices(struct ndctl_ctx *ndctl_ctx, struct daxio_context *ctx) { if (!ctx->zero && setup_device(ndctl_ctx, &ctx->src, 0, ctx->clear_bad_blocks)) return -1; return setup_device(ndctl_ctx, &ctx->dst, 1, ctx->clear_bad_blocks); } /* * adjust_io_len -- (internal) calculate I/O length if not specified */ static void adjust_io_len(struct daxio_context *ctx) { size_t src_len = ctx->src.maplen - ctx->src.offset; size_t dst_len = ctx->dst.maplen - ctx->dst.offset; size_t max_len = SIZE_MAX; if (ctx->zero) assert(ctx->dst.is_devdax); else assert(ctx->src.is_devdax || ctx->dst.is_devdax); if (ctx->src.is_devdax) max_len = src_len; if (ctx->dst.is_devdax) max_len = max_len < dst_len ? max_len : dst_len; /* if length is specified and is not bigger than mmapped region */ if (ctx->len != SIZE_MAX && ctx->len <= max_len) return; /* adjust len to device size */ ctx->len = max_len; } /* * cleanup_device -- (internal) unmap/close file/device */ static void cleanup_device(struct daxio_device *dev) { if (dev->addr) (void) munmap(dev->addr, dev->maplen); if (dev->path && dev->fd != -1) (void) close(dev->fd); } /* * cleanup_devices -- (internal) unmap/close input and output */ static void cleanup_devices(struct daxio_context *ctx) { cleanup_device(&ctx->dst); if (!ctx->zero) cleanup_device(&ctx->src); } /* * do_io -- (internal) write data to device/file */ static int do_io(struct ndctl_ctx *ndctl_ctx, struct daxio_context *ctx) { ssize_t cnt = 0; assert(ctx->src.is_devdax || ctx->dst.is_devdax); if (ctx->zero) { if (ctx->dst.offset > ctx->dst.maplen) { ERR("output offset larger than device size"); return -1; } if (ctx->dst.offset + ctx->len > ctx->dst.maplen) { ERR("output offset beyond device size"); return -1; } char *dst_addr = ctx->dst.addr + ctx->dst.offset; pmem_memset_persist(dst_addr, 0, ctx->len); cnt = (ssize_t)ctx->len; } else if (ctx->src.is_devdax && ctx->dst.is_devdax) { /* memcpy between src and dst */ char *src_addr = ctx->src.addr + ctx->src.offset; char *dst_addr = ctx->dst.addr + ctx->dst.offset; pmem_memcpy_persist(dst_addr, src_addr, ctx->len); cnt = (ssize_t)ctx->len; } else if (ctx->src.is_devdax) { /* write to file directly from mmap'ed src */ char *src_addr = ctx->src.addr + ctx->src.offset; if (ctx->dst.offset) { if (lseek(ctx->dst.fd, (off_t)ctx->dst.offset, SEEK_SET) < 0) { FAIL("lseek"); goto err; } } do { ssize_t wcnt = write(ctx->dst.fd, src_addr + cnt, ctx->len - (size_t)cnt); if (wcnt == -1) { FAIL("write"); goto err; } cnt += wcnt; } while ((size_t)cnt < ctx->len); } else if (ctx->dst.is_devdax) { /* read from file directly to mmap'ed dst */ char *dst_addr = ctx->dst.addr + ctx->dst.offset; if (ctx->src.offset) { if (lseek(ctx->src.fd, (off_t)ctx->src.offset, SEEK_SET) < 0) { FAIL("lseek"); return -1; } } do { ssize_t rcnt = read(ctx->src.fd, dst_addr + cnt, ctx->len - (size_t)cnt); if (rcnt == -1) { FAIL("read"); goto err; } /* end of file */ if (rcnt == 0) break; cnt = cnt + rcnt; } while ((size_t)cnt < ctx->len); pmem_persist(dst_addr, (size_t)cnt); if ((size_t)cnt != ctx->len) ERR("requested size %zu larger than source\n", ctx->len); } ERR("copied %zd bytes to device \"%s\"\n", cnt, ctx->dst.path); return 0; err: ERR("failed to perform I/O\n"); return -1; } int main(int argc, char **argv) { struct ndctl_ctx *ndctl_ctx; int ret = EXIT_SUCCESS; if (parse_args(&Ctx, argc, argv)) return EXIT_FAILURE; if (validate_args(&Ctx)) return EXIT_FAILURE; if (ndctl_new(&ndctl_ctx)) return EXIT_FAILURE; if (setup_devices(ndctl_ctx, &Ctx)) { ret = EXIT_FAILURE; goto err; } if (!Ctx.src.is_devdax && !Ctx.dst.is_devdax) { ERR("neither input nor output is device dax\n"); ret = EXIT_FAILURE; goto err; } adjust_io_len(&Ctx); if (do_io(ndctl_ctx, &Ctx)) ret = EXIT_FAILURE; err: cleanup_devices(&Ctx); ndctl_unref(ndctl_ctx); return ret; } pmdk-1.8/src/tools/daxio/.gitignore0000664000000000000000000000000613615011243016030 0ustar rootrootdaxio pmdk-1.8/src/tools/daxio/Makefile0000664000000000000000000000373413615011243015513 0ustar rootroot# Copyright 2018-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # Makefile -- top Makefile for daxio # TOP = ../../.. include $(TOP)/src/common.inc ifeq ($(NDCTL_ENABLE),y) SCP_TO_REMOTE_NODES = y TARGET = daxio OBJS = daxio.o LIBPMEM=y LIBPMEMCOMMON=y CFLAGS += $(LIBNDCTL_CFLAGS) LIBS += $(LIBNDCTL_LIBS) MANPAGES = $(TOP)/doc/daxio.1 # XXX: to be done # BASH_COMP_FILES = daxio.sh else $(info NOTE: Skipping daxio because ndctl is not available) endif include ../Makefile.inc .PHONY: test check pmdk-1.8/src/tools/rpmemd/0000775000000000000000000000000013615011243014224 5ustar rootrootpmdk-1.8/src/tools/rpmemd/rpmemd_db.c0000664000000000000000000003570513615011243016333 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_db.c -- rpmemd database of pool set files */ #include #include #include #include #include #include #include #include #include "queue.h" #include "set.h" #include "os.h" #include "out.h" #include "file.h" #include "sys_util.h" #include "librpmem.h" #include "rpmemd_db.h" #include "rpmemd_log.h" /* * struct rpmemd_db -- pool set database structure */ struct rpmemd_db { os_mutex_t lock; char *root_dir; mode_t mode; }; /* * declaration of the 'struct list_head' type */ PMDK_LIST_HEAD(list_head, rpmemd_db_entry); /* * struct rpmemd_db_entry -- entry in the pool set list */ struct rpmemd_db_entry { PMDK_LIST_ENTRY(rpmemd_db_entry) next; char *pool_desc; struct pool_set *set; }; /* * rpmemd_db_init -- initialize the rpmem database of pool set files */ struct rpmemd_db * rpmemd_db_init(const char *root_dir, mode_t mode) { if (root_dir[0] != '/') { RPMEMD_LOG(ERR, "root directory is not an absolute path" " -- '%s'", root_dir); errno = EINVAL; return NULL; } struct rpmemd_db *db = calloc(1, sizeof(*db)); if (!db) { RPMEMD_LOG(ERR, "!allocating the rpmem database structure"); return NULL; } db->root_dir = strdup(root_dir); if (!db->root_dir) { RPMEMD_LOG(ERR, "!allocating the root dir path"); free(db); return NULL; } db->mode = mode; util_mutex_init(&db->lock); return db; } /* * rpmemd_db_concat -- (internal) concatenate two paths */ static char * rpmemd_db_concat(const char *path1, const char *path2) { size_t len1 = strlen(path1); size_t len2 = strlen(path2); size_t new_len = len1 + len2 + 2; /* +1 for '/' in snprintf() */ if (path1[0] != '/') { RPMEMD_LOG(ERR, "the first path is not an absolute one -- '%s'", path1); errno = EINVAL; return NULL; } if (path2[0] == '/') { RPMEMD_LOG(ERR, "the second path is not a relative one -- '%s'", path2); /* set to EBADF to distinguish this case from other errors */ errno = EBADF; return NULL; } char *new_str = malloc(new_len); if (new_str == NULL) { RPMEMD_LOG(ERR, "!allocating path buffer"); return NULL; } int ret = snprintf(new_str, new_len, "%s/%s", path1, path2); if (ret < 0 || (size_t)ret != new_len - 1) { RPMEMD_LOG(ERR, "snprintf error: %d", ret); free(new_str); errno = EINVAL; return NULL; } return new_str; } /* * rpmemd_db_get_path -- (internal) get the full path of the pool set file */ static char * rpmemd_db_get_path(struct rpmemd_db *db, const char *pool_desc) { return rpmemd_db_concat(db->root_dir, pool_desc); } /* * rpmemd_db_pool_madvise -- (internal) workaround device dax alignment issue */ static int rpmemd_db_pool_madvise(struct pool_set *set) { /* * This is a workaround for an issue with using device dax with * libibverbs. The problem is that we use ibv_fork_init(3) which * makes all registered memory being madvised with MADV_DONTFORK * flag. In libpmemobj the remote replication is performed without * pool header (first 4k). In such case the address passed to * madvise(2) is aligned to 4k, but device dax can require different * alignment (default is 2MB). This workaround madvises the entire * memory region before registering it by ibv_reg_mr(3). */ const struct pool_set_part *part = &set->replica[0]->part[0]; if (part->is_dev_dax) { int ret = os_madvise(part->addr, part->filesize, MADV_DONTFORK); if (ret) { ERR("!madvise"); return -1; } } return 0; } /* * rpmemd_get_attr -- (internal) get pool attributes from remote pool attributes */ static void rpmemd_get_attr(struct pool_attr *attr, const struct rpmem_pool_attr *rattr) { LOG(3, "attr %p, rattr %p", attr, rattr); memcpy(attr->signature, rattr->signature, POOL_HDR_SIG_LEN); attr->major = rattr->major; attr->features.compat = rattr->compat_features; attr->features.incompat = rattr->incompat_features; attr->features.ro_compat = rattr->ro_compat_features; memcpy(attr->poolset_uuid, rattr->poolset_uuid, POOL_HDR_UUID_LEN); memcpy(attr->first_part_uuid, rattr->uuid, POOL_HDR_UUID_LEN); memcpy(attr->prev_repl_uuid, rattr->prev_uuid, POOL_HDR_UUID_LEN); memcpy(attr->next_repl_uuid, rattr->next_uuid, POOL_HDR_UUID_LEN); memcpy(attr->arch_flags, rattr->user_flags, POOL_HDR_ARCH_LEN); } /* * rpmemd_db_pool_create -- create a new pool set */ struct rpmemd_db_pool * rpmemd_db_pool_create(struct rpmemd_db *db, const char *pool_desc, size_t pool_size, const struct rpmem_pool_attr *rattr) { RPMEMD_ASSERT(db != NULL); util_mutex_lock(&db->lock); struct rpmemd_db_pool *prp = NULL; struct pool_set *set; char *path; int ret; prp = malloc(sizeof(struct rpmemd_db_pool)); if (!prp) { RPMEMD_LOG(ERR, "!allocating pool set db entry"); goto err_unlock; } path = rpmemd_db_get_path(db, pool_desc); if (!path) { goto err_free_prp; } struct pool_attr attr; struct pool_attr *pattr = NULL; if (rattr != NULL) { rpmemd_get_attr(&attr, rattr); pattr = &attr; } ret = util_pool_create_uuids(&set, path, 0, RPMEM_MIN_POOL, RPMEM_MIN_PART, pattr, NULL, REPLICAS_DISABLED, POOL_REMOTE); if (ret) { RPMEMD_LOG(ERR, "!cannot create pool set -- '%s'", path); goto err_free_path; } ret = util_poolset_chmod(set, db->mode); if (ret) { RPMEMD_LOG(ERR, "!cannot change pool set mode bits to 0%o", db->mode); } if (rpmemd_db_pool_madvise(set)) goto err_poolset_close; /* mark as opened */ prp->pool_addr = set->replica[0]->part[0].addr; prp->pool_size = set->poolsize; prp->set = set; free(path); util_mutex_unlock(&db->lock); return prp; err_poolset_close: util_poolset_close(set, DO_NOT_DELETE_PARTS); err_free_path: free(path); err_free_prp: free(prp); err_unlock: util_mutex_unlock(&db->lock); return NULL; } /* * rpmemd_db_pool_open -- open a pool set */ struct rpmemd_db_pool * rpmemd_db_pool_open(struct rpmemd_db *db, const char *pool_desc, size_t pool_size, struct rpmem_pool_attr *rattr) { RPMEMD_ASSERT(db != NULL); RPMEMD_ASSERT(rattr != NULL); util_mutex_lock(&db->lock); struct rpmemd_db_pool *prp = NULL; struct pool_set *set; char *path; int ret; prp = malloc(sizeof(struct rpmemd_db_pool)); if (!prp) { RPMEMD_LOG(ERR, "!allocating pool set db entry"); goto err_unlock; } path = rpmemd_db_get_path(db, pool_desc); if (!path) { goto err_free_prp; } ret = util_pool_open_remote(&set, path, 0, RPMEM_MIN_PART, rattr); if (ret) { RPMEMD_LOG(ERR, "!cannot open pool set -- '%s'", path); goto err_free_path; } if (rpmemd_db_pool_madvise(set)) goto err_poolset_close; /* mark as opened */ prp->pool_addr = set->replica[0]->part[0].addr; prp->pool_size = set->poolsize; prp->set = set; free(path); util_mutex_unlock(&db->lock); return prp; err_poolset_close: util_poolset_close(set, DO_NOT_DELETE_PARTS); err_free_path: free(path); err_free_prp: free(prp); err_unlock: util_mutex_unlock(&db->lock); return NULL; } /* * rpmemd_db_pool_close -- close a pool set */ void rpmemd_db_pool_close(struct rpmemd_db *db, struct rpmemd_db_pool *prp) { RPMEMD_ASSERT(db != NULL); util_mutex_lock(&db->lock); util_poolset_close(prp->set, DO_NOT_DELETE_PARTS); free(prp); util_mutex_unlock(&db->lock); } /* * rpmemd_db_pool_set_attr -- overwrite pool attributes */ int rpmemd_db_pool_set_attr(struct rpmemd_db_pool *prp, const struct rpmem_pool_attr *rattr) { RPMEMD_ASSERT(prp != NULL); RPMEMD_ASSERT(prp->set != NULL); RPMEMD_ASSERT(prp->set->nreplicas == 1); return util_replica_set_attr(prp->set->replica[0], rattr); } struct rm_cb_args { int force; int ret; }; /* * rm_poolset_cb -- (internal) callback for removing part files */ static int rm_poolset_cb(struct part_file *pf, void *arg) { struct rm_cb_args *args = (struct rm_cb_args *)arg; if (pf->is_remote) { RPMEMD_LOG(ERR, "removing remote replica not supported"); return -1; } int ret = util_unlink_flock(pf->part->path); if (!args->force && ret) { RPMEMD_LOG(ERR, "!unlink -- '%s'", pf->part->path); args->ret = ret; } return 0; } /* * rpmemd_db_pool_remove -- remove a pool set */ int rpmemd_db_pool_remove(struct rpmemd_db *db, const char *pool_desc, int force, int pool_set) { RPMEMD_ASSERT(db != NULL); RPMEMD_ASSERT(pool_desc != NULL); util_mutex_lock(&db->lock); struct rm_cb_args args; args.force = force; args.ret = 0; char *path; path = rpmemd_db_get_path(db, pool_desc); if (!path) { args.ret = -1; goto err_unlock; } int ret = util_poolset_foreach_part(path, rm_poolset_cb, &args); if (!force && ret) { RPMEMD_LOG(ERR, "!removing '%s' failed", path); args.ret = ret; goto err_free_path; } if (pool_set) os_unlink(path); err_free_path: free(path); err_unlock: util_mutex_unlock(&db->lock); return args.ret; } /* * rpmemd_db_fini -- deinitialize the rpmem database of pool set files */ void rpmemd_db_fini(struct rpmemd_db *db) { RPMEMD_ASSERT(db != NULL); util_mutex_destroy(&db->lock); free(db->root_dir); free(db); } /* * rpmemd_db_check_dups_set -- (internal) check for duplicates in the database */ static inline int rpmemd_db_check_dups_set(struct pool_set *set, const char *path) { for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; for (unsigned p = 0; p < rep->nparts; p++) { if (strcmp(path, rep->part[p].path) == 0) return -1; } } return 0; } /* * rpmemd_db_check_dups -- (internal) check for duplicates in the database */ static int rpmemd_db_check_dups(struct list_head *head, struct rpmemd_db *db, const char *pool_desc, struct pool_set *set) { struct rpmemd_db_entry *edb; PMDK_LIST_FOREACH(edb, head, next) { for (unsigned r = 0; r < edb->set->nreplicas; r++) { struct pool_replica *rep = edb->set->replica[r]; for (unsigned p = 0; p < rep->nparts; p++) { if (rpmemd_db_check_dups_set(set, rep->part[p].path)) { RPMEMD_LOG(ERR, "part file '%s' from " "pool set '%s' duplicated in " "pool set '%s'", rep->part[p].path, pool_desc, edb->pool_desc); errno = EEXIST; return -1; } } } } return 0; } /* * rpmemd_db_add -- (internal) add an entry for a given set to the database */ static struct rpmemd_db_entry * rpmemd_db_add(struct list_head *head, struct rpmemd_db *db, const char *pool_desc, struct pool_set *set) { struct rpmemd_db_entry *edb; edb = calloc(1, sizeof(*edb)); if (!edb) { RPMEMD_LOG(ERR, "!allocating database entry"); goto err_calloc; } edb->set = set; edb->pool_desc = strdup(pool_desc); if (!edb->pool_desc) { RPMEMD_LOG(ERR, "!allocating path for database entry"); goto err_strdup; } PMDK_LIST_INSERT_HEAD(head, edb, next); return edb; err_strdup: free(edb); err_calloc: return NULL; } /* * new_paths -- (internal) create two new paths */ static int new_paths(const char *dir, const char *name, const char *old_desc, char **path, char **new_desc) { *path = rpmemd_db_concat(dir, name); if (!(*path)) return -1; if (old_desc[0] != 0) *new_desc = rpmemd_db_concat(old_desc, name); else { *new_desc = strdup(name); if (!(*new_desc)) { RPMEMD_LOG(ERR, "!allocating new descriptor"); } } if (!(*new_desc)) { free(*path); return -1; } return 0; } /* * rpmemd_db_check_dir_r -- (internal) recursively check given directory * for duplicates */ static int rpmemd_db_check_dir_r(struct list_head *head, struct rpmemd_db *db, const char *dir, char *pool_desc) { char *new_dir, *new_desc, *full_path; struct dirent *dentry; struct pool_set *set = NULL; DIR *dirp; int ret = 0; dirp = opendir(dir); if (dirp == NULL) { RPMEMD_LOG(ERR, "cannot open the directory -- %s", dir); return -1; } while ((dentry = readdir(dirp)) != NULL) { if (strcmp(dentry->d_name, ".") == 0 || strcmp(dentry->d_name, "..") == 0) continue; if (dentry->d_type == DT_DIR) { /* directory */ if (new_paths(dir, dentry->d_name, pool_desc, &new_dir, &new_desc)) goto err_closedir; /* call recursively for a new directory */ ret = rpmemd_db_check_dir_r(head, db, new_dir, new_desc); free(new_dir); free(new_desc); if (ret) goto err_closedir; continue; } if (new_paths(dir, dentry->d_name, pool_desc, &full_path, &new_desc)) { goto err_closedir; } if (util_poolset_read(&set, full_path)) { RPMEMD_LOG(ERR, "!error reading pool set file -- %s", full_path); goto err_free_paths; } if (rpmemd_db_check_dups(head, db, new_desc, set)) { RPMEMD_LOG(ERR, "!duplicate found in pool set file" " -- %s", full_path); goto err_free_set; } if (rpmemd_db_add(head, db, new_desc, set) == NULL) { goto err_free_set; } free(new_desc); free(full_path); } closedir(dirp); return 0; err_free_set: util_poolset_close(set, DO_NOT_DELETE_PARTS); err_free_paths: free(new_desc); free(full_path); err_closedir: closedir(dirp); return -1; } /* * rpmemd_db_check_dir -- check given directory for duplicates */ int rpmemd_db_check_dir(struct rpmemd_db *db) { RPMEMD_ASSERT(db != NULL); util_mutex_lock(&db->lock); struct list_head head; PMDK_LIST_INIT(&head); int ret = rpmemd_db_check_dir_r(&head, db, db->root_dir, ""); while (!PMDK_LIST_EMPTY(&head)) { struct rpmemd_db_entry *edb = PMDK_LIST_FIRST(&head); PMDK_LIST_REMOVE(edb, next); util_poolset_close(edb->set, DO_NOT_DELETE_PARTS); free(edb->pool_desc); free(edb); } util_mutex_unlock(&db->lock); return ret; } /* * rpmemd_db_pool_is_pmem -- true if pool is in PMEM */ int rpmemd_db_pool_is_pmem(struct rpmemd_db_pool *pool) { return REP(pool->set, 0)->is_pmem; } pmdk-1.8/src/tools/rpmemd/README0000664000000000000000000000031613615011243015104 0ustar rootrootThis directory contains a tool which supports remote access to persistent memory over RDMA on server side. ** DEPENDENCIES: ** The rpmemd depends on libfabric library: https://github.com/ofiwg/libfabric pmdk-1.8/src/tools/rpmemd/rpmemd_obc.c0000664000000000000000000003300013615011243016473 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_obc.c -- rpmemd out-of-band connection definitions */ #include #include #include #include #include #include #include #include #include #include "librpmem.h" #include "rpmemd_log.h" #include "rpmem_proto.h" #include "rpmem_common.h" #include "rpmemd_obc.h" struct rpmemd_obc { int fd_in; int fd_out; }; /* * rpmemd_obc_check_proto_ver -- check protocol version */ static int rpmemd_obc_check_proto_ver(unsigned major, unsigned minor) { if (major != RPMEM_PROTO_MAJOR || minor != RPMEM_PROTO_MINOR) { RPMEMD_LOG(ERR, "unsupported protocol version -- %u.%u", major, minor); return -1; } return 0; } /* * rpmemd_obc_check_msg_hdr -- check message header */ static int rpmemd_obc_check_msg_hdr(struct rpmem_msg_hdr *hdrp) { switch (hdrp->type) { case RPMEM_MSG_TYPE_OPEN: case RPMEM_MSG_TYPE_CREATE: case RPMEM_MSG_TYPE_CLOSE: case RPMEM_MSG_TYPE_SET_ATTR: /* all messages from obc to server are fine */ break; default: RPMEMD_LOG(ERR, "invalid message type -- %u", hdrp->type); return -1; } if (hdrp->size < sizeof(struct rpmem_msg_hdr)) { RPMEMD_LOG(ERR, "invalid message size -- %lu", hdrp->size); return -1; } return 0; } /* * rpmemd_obc_check_pool_desc -- check pool descriptor */ static int rpmemd_obc_check_pool_desc(struct rpmem_msg_hdr *hdrp, size_t msg_size, struct rpmem_msg_pool_desc *pool_desc) { size_t body_size = msg_size + pool_desc->size; if (hdrp->size != body_size) { RPMEMD_LOG(ERR, "message and pool descriptor size mismatch " "-- is %lu should be %lu", hdrp->size, body_size); return -1; } if (pool_desc->size < 2) { RPMEMD_LOG(ERR, "invalid pool descriptor size -- %u " "(must be >= 2)", pool_desc->size); return -1; } if (pool_desc->desc[pool_desc->size - 1] != '\0') { RPMEMD_LOG(ERR, "invalid pool descriptor " "(must be null-terminated string)"); return -1; } size_t len = strlen((char *)pool_desc->desc) + 1; if (pool_desc->size != len) { RPMEMD_LOG(ERR, "invalid pool descriptor size -- is %lu " "should be %u", len, pool_desc->size); return -1; } return 0; } /* * rpmemd_obc_check_provider -- check provider value */ static int rpmemd_obc_check_provider(uint32_t provider) { if (provider == 0 || provider >= MAX_RPMEM_PROV) { RPMEMD_LOG(ERR, "invalid provider -- %u", provider); return -1; } return 0; } /* * rpmemd_obc_ntoh_check_msg_create -- convert and check create request message */ static int rpmemd_obc_ntoh_check_msg_create(struct rpmem_msg_hdr *hdrp) { int ret; struct rpmem_msg_create *msg = (struct rpmem_msg_create *)hdrp; rpmem_ntoh_msg_create(msg); ret = rpmemd_obc_check_proto_ver(msg->c.major, msg->c.minor); if (ret) return ret; ret = rpmemd_obc_check_pool_desc(hdrp, sizeof(*msg), &msg->pool_desc); if (ret) return ret; ret = rpmemd_obc_check_provider(msg->c.provider); if (ret) return ret; return 0; } /* * rpmemd_obc_ntoh_check_msg_open -- convert and check open request message */ static int rpmemd_obc_ntoh_check_msg_open(struct rpmem_msg_hdr *hdrp) { int ret; struct rpmem_msg_open *msg = (struct rpmem_msg_open *)hdrp; rpmem_ntoh_msg_open(msg); ret = rpmemd_obc_check_proto_ver(msg->c.major, msg->c.minor); if (ret) return ret; ret = rpmemd_obc_check_pool_desc(hdrp, sizeof(*msg), &msg->pool_desc); if (ret) return ret; ret = rpmemd_obc_check_provider(msg->c.provider); if (ret) return ret; return 0; } /* * rpmemd_obc_ntoh_check_msg_close -- convert and check close request message */ static int rpmemd_obc_ntoh_check_msg_close(struct rpmem_msg_hdr *hdrp) { struct rpmem_msg_close *msg = (struct rpmem_msg_close *)hdrp; rpmem_ntoh_msg_close(msg); /* nothing to do */ return 0; } /* * rpmemd_obc_ntoh_check_msg_set_attr -- convert and check set attributes * request message */ static int rpmemd_obc_ntoh_check_msg_set_attr(struct rpmem_msg_hdr *hdrp) { struct rpmem_msg_set_attr *msg = (struct rpmem_msg_set_attr *)hdrp; rpmem_ntoh_msg_set_attr(msg); /* nothing to do */ return 0; } typedef int (*rpmemd_obc_ntoh_check_msg_fn)(struct rpmem_msg_hdr *hdrp); static rpmemd_obc_ntoh_check_msg_fn rpmemd_obc_ntoh_check_msg[] = { [RPMEM_MSG_TYPE_CREATE] = rpmemd_obc_ntoh_check_msg_create, [RPMEM_MSG_TYPE_OPEN] = rpmemd_obc_ntoh_check_msg_open, [RPMEM_MSG_TYPE_CLOSE] = rpmemd_obc_ntoh_check_msg_close, [RPMEM_MSG_TYPE_SET_ATTR] = rpmemd_obc_ntoh_check_msg_set_attr, }; /* * rpmemd_obc_process_create -- process create request */ static int rpmemd_obc_process_create(struct rpmemd_obc *obc, struct rpmemd_obc_requests *req_cb, void *arg, struct rpmem_msg_hdr *hdrp) { struct rpmem_msg_create *msg = (struct rpmem_msg_create *)hdrp; struct rpmem_req_attr req = { .pool_size = msg->c.pool_size, .nlanes = (unsigned)msg->c.nlanes, .pool_desc = (char *)msg->pool_desc.desc, .provider = (enum rpmem_provider)msg->c.provider, .buff_size = msg->c.buff_size, }; struct rpmem_pool_attr *rattr = NULL; struct rpmem_pool_attr rpmem_attr; unpack_rpmem_pool_attr(&msg->pool_attr, &rpmem_attr); if (!util_is_zeroed(&rpmem_attr, sizeof(rpmem_attr))) rattr = &rpmem_attr; return req_cb->create(obc, arg, &req, rattr); } /* * rpmemd_obc_process_open -- process open request */ static int rpmemd_obc_process_open(struct rpmemd_obc *obc, struct rpmemd_obc_requests *req_cb, void *arg, struct rpmem_msg_hdr *hdrp) { struct rpmem_msg_open *msg = (struct rpmem_msg_open *)hdrp; struct rpmem_req_attr req = { .pool_size = msg->c.pool_size, .nlanes = (unsigned)msg->c.nlanes, .pool_desc = (const char *)msg->pool_desc.desc, .provider = (enum rpmem_provider)msg->c.provider, .buff_size = msg->c.buff_size, }; return req_cb->open(obc, arg, &req); } /* * rpmemd_obc_process_close -- process close request */ static int rpmemd_obc_process_close(struct rpmemd_obc *obc, struct rpmemd_obc_requests *req_cb, void *arg, struct rpmem_msg_hdr *hdrp) { struct rpmem_msg_close *msg = (struct rpmem_msg_close *)hdrp; return req_cb->close(obc, arg, (int)msg->flags); } /* * rpmemd_obc_process_set_attr -- process set attributes request */ static int rpmemd_obc_process_set_attr(struct rpmemd_obc *obc, struct rpmemd_obc_requests *req_cb, void *arg, struct rpmem_msg_hdr *hdrp) { struct rpmem_msg_set_attr *msg = (struct rpmem_msg_set_attr *)hdrp; struct rpmem_pool_attr *rattr = NULL; struct rpmem_pool_attr rpmem_attr; unpack_rpmem_pool_attr(&msg->pool_attr, &rpmem_attr); if (!util_is_zeroed(&rpmem_attr, sizeof(rpmem_attr))) rattr = &rpmem_attr; return req_cb->set_attr(obc, arg, rattr); } typedef int (*rpmemd_obc_process_fn)(struct rpmemd_obc *obc, struct rpmemd_obc_requests *req_cb, void *arg, struct rpmem_msg_hdr *hdrp); static rpmemd_obc_process_fn rpmemd_obc_process_cb[] = { [RPMEM_MSG_TYPE_CREATE] = rpmemd_obc_process_create, [RPMEM_MSG_TYPE_OPEN] = rpmemd_obc_process_open, [RPMEM_MSG_TYPE_CLOSE] = rpmemd_obc_process_close, [RPMEM_MSG_TYPE_SET_ATTR] = rpmemd_obc_process_set_attr, }; /* * rpmemd_obc_recv -- wrapper for read and decode data function */ static inline int rpmemd_obc_recv(struct rpmemd_obc *obc, void *buff, size_t len) { return rpmem_xread(obc->fd_in, buff, len, 0); } /* * rpmemd_obc_send -- wrapper for encode and write data function */ static inline int rpmemd_obc_send(struct rpmemd_obc *obc, const void *buff, size_t len) { return rpmem_xwrite(obc->fd_out, buff, len, 0); } /* * rpmemd_obc_msg_recv -- receive and check request message * * Return values: * 0 - success * < 0 - error * 1 - obc disconnected */ static int rpmemd_obc_msg_recv(struct rpmemd_obc *obc, struct rpmem_msg_hdr **hdrpp) { struct rpmem_msg_hdr hdr; struct rpmem_msg_hdr nhdr; struct rpmem_msg_hdr *hdrp; int ret; ret = rpmemd_obc_recv(obc, &nhdr, sizeof(nhdr)); if (ret == 1) { RPMEMD_LOG(NOTICE, "out-of-band connection disconnected"); return 1; } if (ret < 0) { RPMEMD_LOG(ERR, "!receiving message header failed"); return ret; } memcpy(&hdr, &nhdr, sizeof(hdr)); rpmem_ntoh_msg_hdr(&hdr); ret = rpmemd_obc_check_msg_hdr(&hdr); if (ret) { RPMEMD_LOG(ERR, "parsing message header failed"); return ret; } hdrp = malloc(hdr.size); if (!hdrp) { RPMEMD_LOG(ERR, "!allocating message buffer failed"); return -1; } memcpy(hdrp, &nhdr, sizeof(*hdrp)); size_t body_size = hdr.size - sizeof(hdr); ret = rpmemd_obc_recv(obc, hdrp->body, body_size); if (ret) { RPMEMD_LOG(ERR, "!receiving message body failed"); goto err_recv_body; } ret = rpmemd_obc_ntoh_check_msg[hdr.type](hdrp); if (ret) { RPMEMD_LOG(ERR, "parsing message body failed"); goto err_body; } *hdrpp = hdrp; return 0; err_body: err_recv_body: free(hdrp); return -1; } /* * rpmemd_obc_init -- initialize rpmemd */ struct rpmemd_obc * rpmemd_obc_init(int fd_in, int fd_out) { struct rpmemd_obc *obc = calloc(1, sizeof(*obc)); if (!obc) { RPMEMD_LOG(ERR, "!allocating obc failed"); goto err_calloc; } obc->fd_in = fd_in; obc->fd_out = fd_out; return obc; err_calloc: return NULL; } /* * rpmemd_obc_fini -- destroy obc */ void rpmemd_obc_fini(struct rpmemd_obc *obc) { free(obc); } /* * rpmemd_obc_status -- sends initial status to the client */ int rpmemd_obc_status(struct rpmemd_obc *obc, uint32_t status) { return rpmemd_obc_send(obc, &status, sizeof(status)); } /* * rpmemd_obc_process -- wait for and process a message from client * * Return values: * 0 - success * < 0 - error * 1 - client disconnected */ int rpmemd_obc_process(struct rpmemd_obc *obc, struct rpmemd_obc_requests *req_cb, void *arg) { RPMEMD_ASSERT(req_cb != NULL); RPMEMD_ASSERT(req_cb->create != NULL); RPMEMD_ASSERT(req_cb->open != NULL); RPMEMD_ASSERT(req_cb->close != NULL); RPMEMD_ASSERT(req_cb->set_attr != NULL); struct rpmem_msg_hdr *hdrp = NULL; int ret; ret = rpmemd_obc_msg_recv(obc, &hdrp); if (ret) return ret; RPMEMD_ASSERT(hdrp != NULL); ret = rpmemd_obc_process_cb[hdrp->type](obc, req_cb, arg, hdrp); free(hdrp); return ret; } /* * rpmemd_obc_create_resp -- send create request response message */ int rpmemd_obc_create_resp(struct rpmemd_obc *obc, int status, const struct rpmem_resp_attr *res) { struct rpmem_msg_create_resp resp = { .hdr = { .type = RPMEM_MSG_TYPE_CREATE_RESP, .size = sizeof(struct rpmem_msg_create_resp), .status = (uint32_t)status, }, .ibc = { .port = res->port, .rkey = res->rkey, .raddr = res->raddr, .persist_method = res->persist_method, .nlanes = res->nlanes, }, }; rpmem_hton_msg_create_resp(&resp); return rpmemd_obc_send(obc, &resp, sizeof(resp)); } /* * rpmemd_obc_open_resp -- send open request response message */ int rpmemd_obc_open_resp(struct rpmemd_obc *obc, int status, const struct rpmem_resp_attr *res, const struct rpmem_pool_attr *pool_attr) { struct rpmem_msg_open_resp resp = { .hdr = { .type = RPMEM_MSG_TYPE_OPEN_RESP, .size = sizeof(struct rpmem_msg_open_resp), .status = (uint32_t)status, }, .ibc = { .port = res->port, .rkey = res->rkey, .raddr = res->raddr, .persist_method = res->persist_method, .nlanes = res->nlanes, }, }; pack_rpmem_pool_attr(pool_attr, &resp.pool_attr); rpmem_hton_msg_open_resp(&resp); return rpmemd_obc_send(obc, &resp, sizeof(resp)); } /* * rpmemd_obc_close_resp -- send close request response message */ int rpmemd_obc_close_resp(struct rpmemd_obc *obc, int status) { struct rpmem_msg_close_resp resp = { .hdr = { .type = RPMEM_MSG_TYPE_CLOSE_RESP, .size = sizeof(struct rpmem_msg_close_resp), .status = (uint32_t)status, }, }; rpmem_hton_msg_close_resp(&resp); return rpmemd_obc_send(obc, &resp, sizeof(resp)); } /* * rpmemd_obc_set_attr_resp -- send set attributes request response message */ int rpmemd_obc_set_attr_resp(struct rpmemd_obc *obc, int status) { struct rpmem_msg_set_attr_resp resp = { .hdr = { .type = RPMEM_MSG_TYPE_SET_ATTR_RESP, .size = sizeof(struct rpmem_msg_set_attr_resp), .status = (uint32_t)status, }, }; rpmem_hton_msg_set_attr_resp(&resp); return rpmemd_obc_send(obc, &resp, sizeof(resp)); } pmdk-1.8/src/tools/rpmemd/rpmemd_fip.c0000664000000000000000000006767413615011243016536 0ustar rootroot/* * Copyright 2016-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_fip.c -- rpmemd libfabric provider module source file */ #include #include #include #include #include #include #include #include #include #include #include #include #include "rpmemd_log.h" #include "rpmem_common.h" #include "rpmem_proto.h" #include "rpmem_fip_msg.h" #include "rpmem_fip_common.h" #include "rpmemd_fip.h" #include "os_thread.h" #include "util.h" #include "valgrind_internal.h" #define RPMEMD_FI_ERR(e, fmt, args...)\ RPMEMD_LOG(ERR, fmt ": %s", ## args, fi_strerror((e))) #define RPMEMD_FI_CLOSE(f, fmt, args...) (\ {\ int ret = fi_close(&(f)->fid);\ if (ret)\ RPMEMD_FI_ERR(ret, fmt, ## args);\ ret;\ }) /* * rpmem_fip_lane -- base lane structure */ struct rpmem_fip_lane { struct fid_ep *ep; struct fid_cq *cq; }; /* * rpmemd_fip_lane -- daemon's lane */ struct rpmemd_fip_lane { struct rpmem_fip_lane base; /* lane base structure */ struct rpmem_fip_msg recv; /* RECV message */ struct rpmem_fip_msg send; /* SEND message */ struct rpmem_msg_persist_resp resp; /* persist response msg buffer */ int send_posted; /* send buffer has been posted */ int recv_posted; /* recv buffer has been posted */ }; /* * rpmemd_fip_thread -- thread context */ struct rpmemd_fip_thread { struct rpmemd_fip *fip; /* main context */ os_thread_t thread; /* thread structure */ struct fid_cq *cq; /* per-thread completion queue */ struct rpmemd_fip_lane **lanes; /* lanes processed by this thread */ size_t nlanes; /* number of lanes processed by this thread */ }; /* * rpmemd_fip -- main context of rpmemd_fip */ struct rpmemd_fip { struct fi_info *fi; /* fabric interface information */ struct fid_fabric *fabric; /* fabric domain */ struct fid_domain *domain; /* fabric protection domain */ struct fid_eq *eq; /* event queue */ struct fid_pep *pep; /* passive endpoint - listener */ struct fid_mr *mr; /* memory region for pool */ int (*persist)(const void *addr, size_t len); /* persist function */ void *(*memcpy_persist)(void *pmemdest, const void *src, size_t len); int (*deep_persist)(const void *addr, size_t len, void *ctx); void *ctx; void *addr; /* pool's address */ size_t size; /* size of the pool */ enum rpmem_persist_method persist_method; volatile int closing; /* flag for closing background threads */ unsigned nlanes; /* number of lanes */ size_t nthreads; /* number of threads for processing */ size_t cq_size; /* size of completion queue */ size_t lanes_per_thread; /* number of lanes per thread */ size_t buff_size; /* size of buffer for inlined data */ struct rpmemd_fip_lane *lanes; struct rpmem_fip_lane rd_lane; /* lane for read operation */ void *pmsg; /* persist message buffer */ size_t pmsg_size; /* persist message buffer size including alignment */ struct fid_mr *pmsg_mr; /* persist message memory region */ void *pmsg_mr_desc; /* persist message local descriptor */ struct rpmem_msg_persist_resp *pres; /* persist response buffer */ struct fid_mr *pres_mr; /* persist response memory region */ void *pres_mr_desc; /* persist response local descriptor */ struct rpmemd_fip_thread *threads; }; /* * rpmemd_fip_get_pmsg -- return persist message buffer */ static inline struct rpmem_msg_persist * rpmemd_fip_get_pmsg(struct rpmemd_fip *fip, size_t idx) { return (struct rpmem_msg_persist *) ((uintptr_t)fip->pmsg + idx * fip->pmsg_size); } /* * rpmemd_fip_getinfo -- obtain fabric interface information */ static int rpmemd_fip_getinfo(struct rpmemd_fip *fip, const char *service, const char *node, enum rpmem_provider provider) { int ret; struct fi_info *hints = rpmem_fip_get_hints(provider); if (!hints) { RPMEMD_LOG(ERR, "getting fabric interface hints"); ret = -1; goto err_fi_get_hints; } ret = fi_getinfo(RPMEM_FIVERSION, node, service, FI_SOURCE, hints, &fip->fi); if (ret) { RPMEMD_FI_ERR(ret, "getting fabric interface information"); goto err_fi_getinfo; } rpmem_fip_print_info(fip->fi); fi_freeinfo(hints); return 0; err_fi_getinfo: fi_freeinfo(hints); err_fi_get_hints: return ret; } /* * rpmemd_fip_set_resp -- fill the response structure */ static int rpmemd_fip_set_resp(struct rpmemd_fip *fip, struct rpmem_resp_attr *resp) { int ret; if (fip->fi->addr_format == FI_SOCKADDR_IN) { struct sockaddr_in addr_in; size_t addrlen = sizeof(addr_in); ret = fi_getname(&fip->pep->fid, &addr_in, &addrlen); if (ret) { RPMEMD_FI_ERR(ret, "getting local endpoint address"); goto err_fi_getname; } if (!addr_in.sin_port) { RPMEMD_LOG(ERR, "dynamic allocation of port failed"); goto err_port; } resp->port = htons(addr_in.sin_port); } else if (fip->fi->addr_format == FI_SOCKADDR_IN6) { struct sockaddr_in6 addr_in6; size_t addrlen = sizeof(addr_in6); ret = fi_getname(&fip->pep->fid, &addr_in6, &addrlen); if (ret) { RPMEMD_FI_ERR(ret, "getting local endpoint address"); goto err_fi_getname; } if (!addr_in6.sin6_port) { RPMEMD_LOG(ERR, "dynamic allocation of port failed"); goto err_port; } resp->port = htons(addr_in6.sin6_port); } else { RPMEMD_LOG(ERR, "invalid address format"); return -1; } resp->rkey = fi_mr_key(fip->mr); resp->persist_method = fip->persist_method; resp->raddr = (uint64_t)fip->addr; resp->nlanes = fip->nlanes; return 0; err_port: err_fi_getname: return -1; } /* * rpmemd_fip_init_fabric_res -- initialize common fabric's resources */ static int rpmemd_fip_init_fabric_res(struct rpmemd_fip *fip) { int ret; ret = fi_fabric(fip->fi->fabric_attr, &fip->fabric, NULL); if (ret) { RPMEMD_FI_ERR(ret, "opening fabric domain"); goto err_fi_fabric; } ret = fi_domain(fip->fabric, fip->fi, &fip->domain, NULL); if (ret) { RPMEMD_FI_ERR(ret, "opening fabric access domain"); goto err_fi_domain; } struct fi_eq_attr eq_attr = { .size = 0, /* use default */ .flags = 0, .wait_obj = FI_WAIT_UNSPEC, .signaling_vector = 0, .wait_set = NULL, }; ret = fi_eq_open(fip->fabric, &eq_attr, &fip->eq, NULL); if (ret) { RPMEMD_FI_ERR(ret, "opening event queue"); goto err_eq_open; } ret = fi_passive_ep(fip->fabric, fip->fi, &fip->pep, NULL); if (ret) { RPMEMD_FI_ERR(ret, "allocating passive endpoint"); goto err_pep; } ret = fi_pep_bind(fip->pep, &fip->eq->fid, 0); if (ret) { RPMEMD_FI_ERR(ret, "binding event queue to passive endpoint"); goto err_pep_bind_eq; } return 0; err_pep_bind_eq: RPMEMD_FI_CLOSE(fip->pep, "closing passive endpoint"); err_pep: RPMEMD_FI_CLOSE(fip->eq, "closing event queue"); err_eq_open: RPMEMD_FI_CLOSE(fip->domain, "closing fabric access domain"); err_fi_domain: RPMEMD_FI_CLOSE(fip->fabric, "closing fabric domain"); err_fi_fabric: return ret; } /* * rpmemd_fip_fini_fabric_res -- deinitialize common fabric resources */ static void rpmemd_fip_fini_fabric_res(struct rpmemd_fip *fip) { RPMEMD_FI_CLOSE(fip->pep, "closing passive endpoint"); RPMEMD_FI_CLOSE(fip->eq, "closing event queue"); RPMEMD_FI_CLOSE(fip->domain, "closing fabric access domain"); RPMEMD_FI_CLOSE(fip->fabric, "closing fabric domain"); } /* * rpmemd_fip_init_memory -- initialize memory pool's resources */ static int rpmemd_fip_init_memory(struct rpmemd_fip *fip) { int ret; /* * Register memory region with appropriate access bits: * - FI_REMOTE_READ - remote peer can issue READ operation, * - FI_REMOTE_WRITE - remote peer can issue WRITE operation, */ ret = fi_mr_reg(fip->domain, fip->addr, fip->size, FI_REMOTE_READ | FI_REMOTE_WRITE, 0, 0, 0, &fip->mr, NULL); if (ret) { RPMEMD_FI_ERR(ret, "registering memory"); return -1; } return 0; } /* * rpmemd_fip_fini_memory -- deinitialize memory pool's resources */ static void rpmemd_fip_fini_memory(struct rpmemd_fip *fip) { RPMEMD_FI_CLOSE(fip->mr, "unregistering memory"); } /* * rpmemd_fip_init_ep -- initialize active endpoint */ static int rpmemd_fip_init_ep(struct rpmemd_fip *fip, struct fi_info *info, struct rpmem_fip_lane *lanep) { int ret; info->tx_attr->size = rpmem_fip_wq_size(fip->persist_method, RPMEM_FIP_NODE_SERVER); info->rx_attr->size = rpmem_fip_rx_size(fip->persist_method, RPMEM_FIP_NODE_SERVER); /* create an endpoint from fabric interface info */ ret = fi_endpoint(fip->domain, info, &lanep->ep, NULL); if (ret) { RPMEMD_FI_ERR(ret, "allocating endpoint"); goto err_endpoint; } /* bind event queue to the endpoint */ ret = fi_ep_bind(lanep->ep, &fip->eq->fid, 0); if (ret) { RPMEMD_FI_ERR(ret, "binding event queue to endpoint"); goto err_bind_eq; } /* * Bind completion queue to the endpoint. * Use a single completion queue for outbound and inbound work * requests. Use selective completion implies adding FI_COMPLETE * flag to each WR which needs a completion. */ ret = fi_ep_bind(lanep->ep, &lanep->cq->fid, FI_RECV | FI_TRANSMIT | FI_SELECTIVE_COMPLETION); if (ret) { RPMEMD_FI_ERR(ret, "binding completion queue to endpoint"); goto err_bind_cq; } /* enable the endpoint */ ret = fi_enable(lanep->ep); if (ret) { RPMEMD_FI_ERR(ret, "enabling endpoint"); goto err_enable; } return 0; err_enable: err_bind_cq: err_bind_eq: RPMEMD_FI_CLOSE(lanep->ep, "closing endpoint"); err_endpoint: return -1; } /* * rpmemd_fip_fini_ep -- close endpoint */ static int rpmemd_fip_fini_ep(struct rpmem_fip_lane *lanep) { return RPMEMD_FI_CLOSE(lanep->ep, "closing endpoint"); } /* * rpmemd_fip_post_msg -- post RECV buffer */ static inline int rpmemd_fip_post_msg(struct rpmemd_fip_lane *lanep) { int ret = rpmem_fip_recvmsg(lanep->base.ep, &lanep->recv); if (ret) { RPMEMD_FI_ERR(ret, "posting recv buffer"); return ret; } lanep->recv_posted = 1; return 0; } /* * rpmemd_fip_post_resp -- post SEND buffer */ static inline int rpmemd_fip_post_resp(struct rpmemd_fip_lane *lanep) { int ret = rpmem_fip_sendmsg(lanep->base.ep, &lanep->send, sizeof(struct rpmem_msg_persist_resp)); if (ret) { RPMEMD_FI_ERR(ret, "posting send buffer"); return ret; } lanep->send_posted = 1; return 0; } /* * rpmemd_fip_post_common -- post all RECV messages */ static int rpmemd_fip_post_common(struct rpmemd_fip *fip, struct rpmemd_fip_lane *lanep) { int ret = rpmem_fip_recvmsg(lanep->base.ep, &lanep->recv); if (ret) { RPMEMD_FI_ERR(ret, "posting recv buffer"); return ret; } lanep->recv_posted = 1; return 0; } /* * rpmemd_fip_lanes_init -- initialize all lanes */ static int rpmemd_fip_lanes_init(struct rpmemd_fip *fip) { fip->lanes = calloc(fip->nlanes, sizeof(*fip->lanes)); if (!fip->lanes) { RPMEMD_ERR("!allocating lanes"); goto err_alloc; } return 0; err_alloc: return -1; } /* * rpmemd_fip_fini_lanes -- deinitialize all lanes */ static void rpmemd_fip_fini_lanes(struct rpmemd_fip *fip) { free(fip->lanes); } /* * rpmemd_fip_init_common -- initialize common resources */ static int rpmemd_fip_init_common(struct rpmemd_fip *fip) { int ret; /* allocate persist message buffer */ size_t msg_size = fip->nlanes * fip->pmsg_size; fip->pmsg = malloc(msg_size); if (!fip->pmsg) { RPMEMD_LOG(ERR, "!allocating messages buffer"); goto err_msg_malloc; } /* register persist message buffer */ ret = fi_mr_reg(fip->domain, fip->pmsg, msg_size, FI_RECV, 0, 0, 0, &fip->pmsg_mr, NULL); if (ret) { RPMEMD_FI_ERR(ret, "registering messages buffer"); goto err_mr_reg_msg; } /* get persist message buffer's local descriptor */ fip->pmsg_mr_desc = fi_mr_desc(fip->pmsg_mr); /* allocate persist response message buffer */ size_t msg_resp_size = fip->nlanes * sizeof(struct rpmem_msg_persist_resp); fip->pres = malloc(msg_resp_size); if (!fip->pres) { RPMEMD_FI_ERR(ret, "allocating messages response buffer"); goto err_msg_resp_malloc; } /* register persist response message buffer */ ret = fi_mr_reg(fip->domain, fip->pres, msg_resp_size, FI_SEND, 0, 0, 0, &fip->pres_mr, NULL); if (ret) { RPMEMD_FI_ERR(ret, "registering messages " "response buffer"); goto err_mr_reg_msg_resp; } /* get persist message buffer's local descriptor */ fip->pres_mr_desc = fi_mr_desc(fip->pres_mr); /* initialize lanes */ unsigned i; for (i = 0; i < fip->nlanes; i++) { struct rpmemd_fip_lane *lanep = &fip->lanes[i]; /* initialize RECV message */ rpmem_fip_msg_init(&lanep->recv, fip->pmsg_mr_desc, 0, lanep, rpmemd_fip_get_pmsg(fip, i), fip->pmsg_size, FI_COMPLETION); /* initialize SEND message */ rpmem_fip_msg_init(&lanep->send, fip->pres_mr_desc, 0, lanep, &fip->pres[i], sizeof(fip->pres[i]), FI_COMPLETION); } return 0; err_mr_reg_msg_resp: free(fip->pres); err_msg_resp_malloc: RPMEMD_FI_CLOSE(fip->pmsg_mr, "unregistering messages buffer"); err_mr_reg_msg: free(fip->pmsg); err_msg_malloc: return -1; } /* * rpmemd_fip_fini_common -- deinitialize common resources and return last * error code */ static int rpmemd_fip_fini_common(struct rpmemd_fip *fip) { int lret = 0; int ret; ret = RPMEMD_FI_CLOSE(fip->pmsg_mr, "unregistering messages buffer"); if (ret) lret = ret; ret = RPMEMD_FI_CLOSE(fip->pres_mr, "unregistering messages response buffer"); if (ret) lret = ret; free(fip->pmsg); free(fip->pres); return lret; } /* * rpmemd_fip_check_pmsg -- verify persist message */ static inline int rpmemd_fip_check_pmsg(struct rpmemd_fip *fip, struct rpmem_msg_persist *pmsg) { if (pmsg->lane >= fip->nlanes) { RPMEMD_LOG(ERR, "invalid lane number -- %u", pmsg->lane); return -1; } uintptr_t raddr = pmsg->addr; uintptr_t laddr = (uintptr_t)fip->addr; if (raddr < laddr || raddr + pmsg->size > laddr + fip->size) { RPMEMD_LOG(ERR, "invalid address or size requested " "for persist operation (0x%lx, %lu)", raddr, pmsg->size); return -1; } return 0; } /* * rpmemd_fip_process_send -- process FI_SEND completion */ static int rpmemd_fip_process_send(struct rpmemd_fip *fip, struct rpmemd_fip_lane *lanep) { lanep->send_posted = 0; if (lanep->recv_posted) return 0; struct rpmem_msg_persist_resp *pres = rpmem_fip_msg_get_pres(&lanep->send); *pres = lanep->resp; int ret; /* post lane's RECV buffer */ ret = rpmemd_fip_post_msg(lanep); if (unlikely(ret)) goto err; /* post lane's SEND buffer */ ret = rpmemd_fip_post_resp(lanep); err: return ret; } /* * rpmemd_fip_process_recv -- process FI_RECV completion */ static int rpmemd_fip_process_recv(struct rpmemd_fip *fip, struct rpmemd_fip_lane *lanep) { int ret = 0; lanep->recv_posted = 0; /* * Get persist message and persist message response from appropriate * buffers. The persist message is in lane's RECV buffer and the * persist response message in lane's SEND buffer. */ struct rpmem_msg_persist *pmsg = rpmem_fip_msg_get_pmsg(&lanep->recv); VALGRIND_DO_MAKE_MEM_DEFINED(pmsg, sizeof(*pmsg)); /* verify persist message */ ret = rpmemd_fip_check_pmsg(fip, pmsg); if (unlikely(ret)) goto err; unsigned mode = pmsg->flags & RPMEM_FLUSH_PERSIST_MASK; if (mode == RPMEM_DEEP_PERSIST) { fip->deep_persist((void *)pmsg->addr, pmsg->size, fip->ctx); } else if (mode == RPMEM_PERSIST_SEND) { fip->memcpy_persist((void *)pmsg->addr, pmsg->data, pmsg->size); } else { fip->persist((void *)pmsg->addr, pmsg->size); } struct rpmem_msg_persist_resp *pres = lanep->send_posted ? &lanep->resp : rpmem_fip_msg_get_pres(&lanep->send); /* return back the lane id */ pres->lane = pmsg->lane; if (!lanep->send_posted) { /* post lane's RECV buffer */ ret = rpmemd_fip_post_msg(lanep); if (unlikely(ret)) goto err; /* post lane's SEND buffer */ ret = rpmemd_fip_post_resp(lanep); } err: return ret; } /* * rpmemd_fip_cq_read -- wait for specific events on completion queue */ static int rpmemd_fip_cq_read(struct rpmemd_fip *fip, struct fid_cq *cq, struct rpmemd_fip_lane **lanep, uint64_t *event, uint64_t event_mask) { struct fi_cq_err_entry err; struct fi_cq_msg_entry cq_entry; const char *str_err; ssize_t sret; int ret; while (!fip->closing) { sret = fi_cq_sread(cq, &cq_entry, 1, NULL, RPMEM_FIP_CQ_WAIT_MS); if (unlikely(fip->closing)) break; if (unlikely(sret == -FI_EAGAIN || sret == 0)) continue; if (unlikely(sret < 0)) { ret = (int)sret; goto err_cq_read; } if (!(cq_entry.flags & event_mask)) { RPMEMD_LOG(ERR, "unexpected event received %lx", cq_entry.flags); ret = -1; goto err; } if (!cq_entry.op_context) { RPMEMD_LOG(ERR, "null context received"); ret = -1; goto err; } *event = cq_entry.flags & event_mask; *lanep = cq_entry.op_context; return 0; } return 0; err_cq_read: sret = fi_cq_readerr(cq, &err, 0); if (sret < 0) { RPMEMD_FI_ERR((int)sret, "error reading from completion queue: " "cannot read error from completion queue"); goto err; } str_err = fi_cq_strerror(cq, err.prov_errno, NULL, NULL, 0); RPMEMD_LOG(ERR, "error reading from completion queue: %s", str_err); err: return ret; } /* * rpmemd_fip_thread -- thread callback which processes persist * operation */ static void * rpmemd_fip_thread(void *arg) { struct rpmemd_fip_thread *thread = arg; struct rpmemd_fip *fip = thread->fip; struct rpmemd_fip_lane *lanep = NULL; uint64_t event = 0; int ret = 0; while (!fip->closing) { ret = rpmemd_fip_cq_read(fip, thread->cq, &lanep, &event, FI_SEND|FI_RECV); if (ret) goto err; if (unlikely(fip->closing)) break; RPMEMD_ASSERT(lanep != NULL); if (event & FI_RECV) ret = rpmemd_fip_process_recv(fip, lanep); else if (event & FI_SEND) ret = rpmemd_fip_process_send(fip, lanep); if (ret) goto err; } return 0; err: return (void *)(uintptr_t)ret; } /* * rpmemd_fip_get_def_nthreads -- get default number of threads for given * persistency method */ static size_t rpmemd_fip_get_def_nthreads(struct rpmemd_fip *fip) { RPMEMD_ASSERT(fip->nlanes > 0); switch (fip->persist_method) { case RPMEM_PM_APM: case RPMEM_PM_GPSPM: return fip->nlanes; default: RPMEMD_ASSERT(0); return 0; } } /* * rpmemd_fip_set_attr -- save required attributes in rpmemd_fip handle */ static void rpmemd_fip_set_attr(struct rpmemd_fip *fip, struct rpmemd_fip_attr *attr) { fip->addr = attr->addr; fip->size = attr->size; fip->persist_method = attr->persist_method; fip->persist = attr->persist; fip->memcpy_persist = attr->memcpy_persist; fip->deep_persist = attr->deep_persist; fip->ctx = attr->ctx; fip->buff_size = attr->buff_size; fip->pmsg_size = roundup(sizeof(struct rpmem_msg_persist) + fip->buff_size, (size_t)64); size_t max_nlanes = rpmem_fip_max_nlanes(fip->fi); RPMEMD_ASSERT(max_nlanes < UINT_MAX); fip->nlanes = min((unsigned)max_nlanes, attr->nlanes); if (attr->nthreads) { fip->nthreads = attr->nthreads; } else { /* use default */ fip->nthreads = rpmemd_fip_get_def_nthreads(fip); } fip->lanes_per_thread = (fip->nlanes - 1) / fip->nthreads + 1; size_t cq_size_per_lane = rpmem_fip_cq_size(fip->persist_method, RPMEM_FIP_NODE_SERVER); fip->cq_size = fip->lanes_per_thread * cq_size_per_lane; RPMEMD_ASSERT(fip->persist_method < MAX_RPMEM_PM); } /* * rpmemd_fip_init_thread -- init worker thread */ static int rpmemd_fip_init_thread(struct rpmemd_fip *fip, struct rpmemd_fip_thread *thread) { thread->fip = fip; thread->lanes = malloc(fip->lanes_per_thread * sizeof(*thread->lanes)); if (!thread->lanes) { RPMEMD_LOG(ERR, "!allocating thread lanes"); goto err_alloc_lanes; } struct fi_cq_attr cq_attr = { .size = fip->cq_size, .flags = 0, .format = FI_CQ_FORMAT_MSG, /* need context and flags */ .wait_obj = FI_WAIT_UNSPEC, .signaling_vector = 0, .wait_cond = FI_CQ_COND_NONE, .wait_set = NULL, }; int ret = fi_cq_open(fip->domain, &cq_attr, &thread->cq, NULL); if (ret) { RPMEMD_FI_ERR(ret, "opening completion queue"); goto err_cq_open; } return 0; err_cq_open: free(thread->lanes); err_alloc_lanes: return -1; } /* * rpmemd_fip_fini_thread -- deinitialize worker thread */ static void rpmemd_fip_fini_thread(struct rpmemd_fip *fip, struct rpmemd_fip_thread *thread) { RPMEMD_FI_CLOSE(thread->cq, "closing completion queue"); free(thread->lanes); } /* * rpmemd_fip_init_threads -- initialize worker threads */ static int rpmemd_fip_init_threads(struct rpmemd_fip *fip) { RPMEMD_ASSERT(fip->lanes != NULL); RPMEMD_ASSERT(fip->nthreads > 0); fip->threads = calloc(fip->nthreads, sizeof(*fip->threads)); if (!fip->threads) { RPMEMD_LOG(ERR, "!allocating threads"); goto err_alloc_threads; } int ret; size_t i; for (i = 0; i < fip->nthreads; i++) { ret = rpmemd_fip_init_thread(fip, &fip->threads[i]); if (ret) { RPMEMD_LOG(ERR, "!initializing thread %zu", i); goto err_init_thread; } } for (size_t i = 0; i < fip->nlanes; i++) { size_t w = i % fip->nthreads; struct rpmemd_fip_thread *thread = &fip->threads[w]; fip->lanes[i].base.cq = thread->cq; thread->lanes[thread->nlanes++] = &fip->lanes[i]; } return 0; err_init_thread: for (size_t j = 0; j < i; j++) rpmemd_fip_fini_thread(fip, &fip->threads[j]); free(fip->threads); err_alloc_threads: return -1; } /* * rpmemd_fip_fini_threads -- deinitialize worker threads */ static void rpmemd_fip_fini_threads(struct rpmemd_fip *fip) { for (size_t i = 0; i < fip->nthreads; i++) rpmemd_fip_fini_thread(fip, &fip->threads[i]); free(fip->threads); } /* * rpmemd_fip_init -- initialize fabric provider */ struct rpmemd_fip * rpmemd_fip_init(const char *node, const char *service, struct rpmemd_fip_attr *attr, struct rpmem_resp_attr *resp, enum rpmem_err *err) { int ret; RPMEMD_ASSERT(resp); RPMEMD_ASSERT(err); RPMEMD_ASSERT(attr); RPMEMD_ASSERT(attr->persist); struct rpmemd_fip *fip = calloc(1, sizeof(*fip)); if (!fip) { RPMEMD_LOG(ERR, "!allocating fabric handle"); *err = RPMEM_ERR_FATAL; return NULL; } ret = rpmemd_fip_getinfo(fip, service, node, attr->provider); if (ret) { *err = RPMEM_ERR_BADPROVIDER; goto err_getinfo; } rpmemd_fip_set_attr(fip, attr); ret = rpmemd_fip_init_fabric_res(fip); if (ret) { *err = RPMEM_ERR_FATAL; goto err_init_fabric_res; } ret = rpmemd_fip_init_memory(fip); if (ret) { *err = RPMEM_ERR_FATAL; goto err_init_memory; } ret = rpmemd_fip_lanes_init(fip); if (ret) { *err = RPMEM_ERR_FATAL; goto err_init_lanes; } ret = rpmemd_fip_init_threads(fip); if (ret) { *err = RPMEM_ERR_FATAL; goto err_init_threads; } ret = rpmemd_fip_init_common(fip); if (ret) { *err = RPMEM_ERR_FATAL; goto err_init; } ret = fi_listen(fip->pep); if (ret) { *err = RPMEM_ERR_FATAL_CONN; goto err_fi_listen; } ret = rpmemd_fip_set_resp(fip, resp); if (ret) { *err = RPMEM_ERR_FATAL; goto err_set_resp; } return fip; err_set_resp: RPMEMD_FI_CLOSE(fip->pep, "closing passive endpoint"); err_fi_listen: rpmemd_fip_fini_common(fip); err_init: rpmemd_fip_fini_threads(fip); err_init_threads: rpmemd_fip_fini_lanes(fip); err_init_lanes: rpmemd_fip_fini_memory(fip); err_init_memory: rpmemd_fip_fini_fabric_res(fip); err_init_fabric_res: fi_freeinfo(fip->fi); err_getinfo: free(fip); return NULL; } /* * rpmemd_fip_fini -- deinitialize fabric provider */ void rpmemd_fip_fini(struct rpmemd_fip *fip) { rpmemd_fip_fini_common(fip); rpmemd_fip_fini_threads(fip); rpmemd_fip_fini_lanes(fip); rpmemd_fip_fini_memory(fip); rpmemd_fip_fini_fabric_res(fip); fi_freeinfo(fip->fi); free(fip); } /* * rpmemd_fip_accept_one -- accept a single connection */ static int rpmemd_fip_accept_one(struct rpmemd_fip *fip, struct fi_info *info, struct rpmemd_fip_lane *lanep) { int ret; ret = rpmemd_fip_init_ep(fip, info, &lanep->base); if (ret) goto err_init_ep; ret = rpmemd_fip_post_common(fip, lanep); if (ret) goto err_post; ret = fi_accept(lanep->base.ep, NULL, 0); if (ret) { RPMEMD_FI_ERR(ret, "accepting connection request"); goto err_accept; } fi_freeinfo(info); return 0; err_accept: err_post: rpmemd_fip_fini_ep(&lanep->base); err_init_ep: fi_freeinfo(info); return -1; } /* * rpmemd_fip_accept -- accept a single connection request */ int rpmemd_fip_accept(struct rpmemd_fip *fip, int timeout) { int ret; struct fi_eq_cm_entry entry; uint32_t event; unsigned nreq = 0; /* number of connection requests */ unsigned ncon = 0; /* number of connected endpoints */ int connecting = 1; while (connecting && (nreq < fip->nlanes || ncon < fip->nlanes)) { ret = rpmem_fip_read_eq(fip->eq, &entry, &event, timeout); if (ret) goto err_read_eq; switch (event) { case FI_CONNREQ: ret = rpmemd_fip_accept_one(fip, entry.info, &fip->lanes[nreq]); if (ret) goto err_accept_one; nreq++; break; case FI_CONNECTED: ncon++; break; case FI_SHUTDOWN: connecting = 0; break; default: RPMEMD_ERR("unexpected event received (%u)", event); goto err_read_eq; } } return 0; err_accept_one: err_read_eq: return -1; } /* * rpmemd_fip_wait_close -- wait specified time for connection closed event */ int rpmemd_fip_wait_close(struct rpmemd_fip *fip, int timeout) { struct fi_eq_cm_entry entry; int lret = 0; uint32_t event; int ret; for (unsigned i = 0; i < fip->nlanes; i++) { ret = rpmem_fip_read_eq(fip->eq, &entry, &event, timeout); if (ret) lret = ret; if (event != FI_SHUTDOWN) { RPMEMD_ERR("unexpected event received " "(is %u expected %u)", event, FI_SHUTDOWN); errno = EINVAL; lret = -1; } } return lret; } /* * rpmemd_fip_close -- close the connection */ int rpmemd_fip_close(struct rpmemd_fip *fip) { int lret = 0; int ret; for (unsigned i = 0; i < fip->nlanes; i++) { ret = rpmemd_fip_fini_ep(&fip->lanes[i].base); if (ret) lret = ret; } return lret; } /* * rpmemd_fip_process_start -- start processing */ int rpmemd_fip_process_start(struct rpmemd_fip *fip) { unsigned i; for (i = 0; i < fip->nthreads; i++) { errno = os_thread_create(&fip->threads[i].thread, NULL, rpmemd_fip_thread, &fip->threads[i]); if (errno) { RPMEMD_ERR("!running thread thread"); goto err_thread_create; } } return 0; err_thread_create: return -1; } /* * rpmemd_fip_process_stop -- stop processing */ int rpmemd_fip_process_stop(struct rpmemd_fip *fip) { /* this stops all threads */ util_fetch_and_or32(&fip->closing, 1); int ret; int lret = 0; for (size_t i = 0; i < fip->nthreads; i++) { struct rpmemd_fip_thread *thread = &fip->threads[i]; ret = fi_cq_signal(thread->cq); if (ret) { RPMEMD_FI_ERR(ret, "sending signal to CQ"); lret = ret; } void *tret; errno = os_thread_join(&thread->thread, &tret); if (errno) { RPMEMD_LOG(ERR, "!joining cq thread"); lret = -1; } else { ret = (int)(uintptr_t)tret; if (ret) { RPMEMD_LOG(ERR, "cq thread failed with code -- %d", ret); lret = ret; } } } return lret; } pmdk-1.8/src/tools/rpmemd/rpmemd.h0000664000000000000000000000321113615011243015656 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd.h -- rpmemd main header file */ #define DAEMON_NAME "rpmemd" pmdk-1.8/src/tools/rpmemd/.gitignore0000664000000000000000000000000713615011243016211 0ustar rootrootrpmemd pmdk-1.8/src/tools/rpmemd/rpmemd_util.h0000664000000000000000000000370413615011243016722 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_util.h -- rpmemd utility functions declarations */ int rpmemd_pmem_persist(const void *addr, size_t len); int rpmemd_flush_fatal(const void *addr, size_t len); int rpmemd_apply_pm_policy(enum rpmem_persist_method *persist_method, int (**persist)(const void *addr, size_t len), void *(**memcpy_persist)(void *pmemdest, const void *src, size_t len), const int is_pmem); pmdk-1.8/src/tools/rpmemd/rpmemd_fip.h0000664000000000000000000000502513615011243016521 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_fip.h -- rpmemd libfabric provider module header file */ #include struct rpmemd_fip; struct rpmemd_fip_attr { void *addr; size_t size; unsigned nlanes; size_t nthreads; size_t buff_size; enum rpmem_provider provider; enum rpmem_persist_method persist_method; int (*persist)(const void *addr, size_t len); void *(*memcpy_persist)(void *pmemdest, const void *src, size_t len); int (*deep_persist)(const void *addr, size_t len, void *ctx); void *ctx; }; struct rpmemd_fip *rpmemd_fip_init(const char *node, const char *service, struct rpmemd_fip_attr *attr, struct rpmem_resp_attr *resp, enum rpmem_err *err); void rpmemd_fip_fini(struct rpmemd_fip *fip); int rpmemd_fip_accept(struct rpmemd_fip *fip, int timeout); int rpmemd_fip_process_start(struct rpmemd_fip *fip); int rpmemd_fip_process_stop(struct rpmemd_fip *fip); int rpmemd_fip_wait_close(struct rpmemd_fip *fip, int timeout); int rpmemd_fip_close(struct rpmemd_fip *fip); pmdk-1.8/src/tools/rpmemd/rpmemd_config.h0000664000000000000000000000473713615011243017221 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_config.h -- internal definitions for rpmemd config */ #include #include #ifndef RPMEMD_DEFAULT_LOG_FILE #define RPMEMD_DEFAULT_LOG_FILE ("/var/log/" DAEMON_NAME ".log") #endif #ifndef RPMEMD_GLOBAL_CONFIG_FILE #define RPMEMD_GLOBAL_CONFIG_FILE ("/etc/" DAEMON_NAME "/" DAEMON_NAME\ ".conf") #endif #define RPMEMD_USER_CONFIG_FILE ("." DAEMON_NAME ".conf") #define RPMEM_DEFAULT_MAX_LANES 1024 #define RPMEM_DEFAULT_NTHREADS 0 #define HOME_ENV "HOME" #define HOME_STR_PLACEHOLDER ("$" HOME_ENV) struct rpmemd_config { char *log_file; char *poolset_dir; const char *rm_poolset; bool force; bool pool_set; bool persist_apm; bool persist_general; bool use_syslog; uint64_t max_lanes; enum rpmemd_log_level log_level; size_t nthreads; }; int rpmemd_config_read(struct rpmemd_config *config, int argc, char *argv[]); void rpmemd_config_free(struct rpmemd_config *config); pmdk-1.8/src/tools/rpmemd/rpmemd_obc.h0000664000000000000000000000537313615011243016514 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_obc.h -- rpmemd out-of-band connection declarations */ #include #include #include struct rpmemd_obc; struct rpmemd_obc_requests { int (*create)(struct rpmemd_obc *obc, void *arg, const struct rpmem_req_attr *req, const struct rpmem_pool_attr *pool_attr); int (*open)(struct rpmemd_obc *obc, void *arg, const struct rpmem_req_attr *req); int (*close)(struct rpmemd_obc *obc, void *arg, int flags); int (*set_attr)(struct rpmemd_obc *obc, void *arg, const struct rpmem_pool_attr *pool_attr); }; struct rpmemd_obc *rpmemd_obc_init(int fd_in, int fd_out); void rpmemd_obc_fini(struct rpmemd_obc *obc); int rpmemd_obc_status(struct rpmemd_obc *obc, uint32_t status); int rpmemd_obc_process(struct rpmemd_obc *obc, struct rpmemd_obc_requests *req_cb, void *arg); int rpmemd_obc_create_resp(struct rpmemd_obc *obc, int status, const struct rpmem_resp_attr *res); int rpmemd_obc_open_resp(struct rpmemd_obc *obc, int status, const struct rpmem_resp_attr *res, const struct rpmem_pool_attr *pool_attr); int rpmemd_obc_set_attr_resp(struct rpmemd_obc *obc, int status); int rpmemd_obc_close_resp(struct rpmemd_obc *obc, int status); pmdk-1.8/src/tools/rpmemd/rpmemd.c0000664000000000000000000004705413615011243015666 0ustar rootroot/* * Copyright 2016-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd.c -- rpmemd main source file */ #include #include #include #include #include #include #include "librpmem.h" #include "rpmemd.h" #include "rpmemd_log.h" #include "rpmemd_config.h" #include "rpmem_common.h" #include "rpmemd_fip.h" #include "rpmemd_obc.h" #include "rpmemd_db.h" #include "rpmemd_util.h" #include "pool_hdr.h" #include "os.h" #include "os_thread.h" #include "util.h" #include "uuid.h" #include "set.h" /* * rpmemd -- rpmem handle */ struct rpmemd { struct rpmemd_obc *obc; /* out-of-band connection handle */ struct rpmemd_db *db; /* pool set database handle */ struct rpmemd_db_pool *pool; /* pool handle */ char *pool_desc; /* pool descriptor */ struct rpmemd_fip *fip; /* fabric provider handle */ struct rpmemd_config config; /* configuration */ enum rpmem_persist_method persist_method; int closing; /* set when closing connection */ int created; /* pool created */ os_thread_t fip_thread; int fip_running; }; #ifdef DEBUG /* * bool2str -- convert bool to yes/no string */ static inline const char * bool2str(int v) { return v ? "yes" : "no"; } #endif /* * str_or_null -- return null string instead of NULL pointer */ static inline const char * _str(const char *str) { if (!str) return "(null)"; return str; } /* * uuid2str -- convert uuid to string */ static const char * uuid2str(const uuid_t uuid) { static char uuid_str[64] = {0, }; int ret = util_uuid_to_string(uuid, uuid_str); if (ret != 0) { return "(error)"; } return uuid_str; } /* * rpmemd_get_pm -- returns persist method based on configuration */ static enum rpmem_persist_method rpmemd_get_pm(struct rpmemd_config *config) { enum rpmem_persist_method ret = RPMEM_PM_GPSPM; if (config->persist_apm) ret = RPMEM_PM_APM; return ret; } /* * rpmemd_db_get_status -- convert error number to status for db operation */ static int rpmemd_db_get_status(int err) { switch (err) { case EEXIST: return RPMEM_ERR_EXISTS; case EACCES: return RPMEM_ERR_NOACCESS; case ENOENT: return RPMEM_ERR_NOEXIST; case EWOULDBLOCK: return RPMEM_ERR_BUSY; case EBADF: return RPMEM_ERR_BADNAME; case EINVAL: return RPMEM_ERR_POOL_CFG; default: return RPMEM_ERR_FATAL; } } /* * rpmemd_check_pool -- verify pool parameters */ static int rpmemd_check_pool(struct rpmemd *rpmemd, const struct rpmem_req_attr *req, int *status) { if (rpmemd->pool->pool_size < RPMEM_MIN_POOL) { RPMEMD_LOG(ERR, "invalid pool size -- must be >= %zu", RPMEM_MIN_POOL); *status = RPMEM_ERR_POOL_CFG; return -1; } if (rpmemd->pool->pool_size < req->pool_size) { RPMEMD_LOG(ERR, "requested size is too big"); *status = RPMEM_ERR_BADSIZE; return -1; } return 0; } /* * rpmemd_deep_persist -- perform deep persist operation */ static int rpmemd_deep_persist(const void *addr, size_t size, void *ctx) { struct rpmemd *rpmemd = (struct rpmemd *)ctx; return util_replica_deep_persist(addr, size, rpmemd->pool->set, 0); } /* * rpmemd_common_fip_init -- initialize fabric provider */ static int rpmemd_common_fip_init(struct rpmemd *rpmemd, const struct rpmem_req_attr *req, struct rpmem_resp_attr *resp, int *status) { /* register the whole pool with header in RDMA */ void *addr = (void *)((uintptr_t)rpmemd->pool->pool_addr); struct rpmemd_fip_attr fip_attr = { .addr = addr, .size = req->pool_size, .nlanes = req->nlanes, .nthreads = rpmemd->config.nthreads, .provider = req->provider, .persist_method = rpmemd->persist_method, .deep_persist = rpmemd_deep_persist, .ctx = rpmemd, .buff_size = req->buff_size, }; const int is_pmem = rpmemd_db_pool_is_pmem(rpmemd->pool); if (rpmemd_apply_pm_policy(&fip_attr.persist_method, &fip_attr.persist, &fip_attr.memcpy_persist, is_pmem)) { *status = RPMEM_ERR_FATAL; goto err_fip_init; } const char *node = rpmem_get_ssh_conn_addr(); enum rpmem_err err; rpmemd->fip = rpmemd_fip_init(node, NULL, &fip_attr, resp, &err); if (!rpmemd->fip) { *status = (int)err; goto err_fip_init; } return 0; err_fip_init: return -1; } /* * rpmemd_print_req_attr -- print request attributes */ static void rpmemd_print_req_attr(const struct rpmem_req_attr *req) { RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "pool descriptor: '%s'", _str(req->pool_desc)); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "pool size: %lu", req->pool_size); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "nlanes: %u", req->nlanes); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "provider: %s", rpmem_provider_to_str(req->provider)); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "buff_size: %lu", req->buff_size); } /* * rpmemd_print_pool_attr -- print pool attributes */ static void rpmemd_print_pool_attr(const struct rpmem_pool_attr *attr) { if (attr == NULL) { RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "NULL"); } else { RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "signature: '%s'", _str(attr->signature)); RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "major: %u", attr->major); RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "compat_features: 0x%x", attr->compat_features); RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "incompat_features: 0x%x", attr->incompat_features); RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "ro_compat_features: 0x%x", attr->ro_compat_features); RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "poolset_uuid: %s", uuid2str(attr->poolset_uuid)); RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "uuid: %s", uuid2str(attr->uuid)); RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "next_uuid: %s", uuid2str(attr->next_uuid)); RPMEMD_LOG(INFO, RPMEMD_LOG_INDENT "prev_uuid: %s", uuid2str(attr->prev_uuid)); } } /* * rpmemd_print_resp_attr -- print response attributes */ static void rpmemd_print_resp_attr(const struct rpmem_resp_attr *attr) { RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "port: %u", attr->port); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "rkey: 0x%lx", attr->rkey); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "raddr: 0x%lx", attr->raddr); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "nlanes: %u", attr->nlanes); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "persist method: %s", rpmem_persist_method_to_str(attr->persist_method)); } /* * rpmemd_fip_thread -- background thread for establishing in-band connection */ static void * rpmemd_fip_thread(void *arg) { struct rpmemd *rpmemd = (struct rpmemd *)arg; int ret; RPMEMD_LOG(INFO, "waiting for in-band connection"); ret = rpmemd_fip_accept(rpmemd->fip, RPMEM_ACCEPT_TIMEOUT); if (ret) goto err_accept; RPMEMD_LOG(NOTICE, "in-band connection established"); ret = rpmemd_fip_process_start(rpmemd->fip); if (ret) goto err_process_start; return NULL; err_process_start: rpmemd_fip_close(rpmemd->fip); err_accept: return (void *)(uintptr_t)ret; } /* * rpmemd_fip_start_thread -- start background thread for establishing * in-band connection */ static int rpmemd_fip_start_thread(struct rpmemd *rpmemd) { errno = os_thread_create(&rpmemd->fip_thread, NULL, rpmemd_fip_thread, rpmemd); if (errno) { RPMEMD_LOG(ERR, "!creating in-band thread"); goto err_os_thread_create; } rpmemd->fip_running = 1; return 0; err_os_thread_create: return -1; } /* * rpmemd_fip_stop_thread -- stop background thread for in-band connection */ static int rpmemd_fip_stop_thread(struct rpmemd *rpmemd) { RPMEMD_ASSERT(rpmemd->fip_running); void *tret; errno = os_thread_join(&rpmemd->fip_thread, &tret); if (errno) RPMEMD_LOG(ERR, "!waiting for in-band thread"); int ret = (int)(uintptr_t)tret; if (ret) RPMEMD_LOG(ERR, "in-band thread failed -- '%d'", ret); return ret; } /* * rpmemd_fip-stop -- stop in-band thread and stop processing thread */ static int rpmemd_fip_stop(struct rpmemd *rpmemd) { int ret; int fip_ret = rpmemd_fip_stop_thread(rpmemd); if (fip_ret) { RPMEMD_LOG(ERR, "!in-band thread failed"); } if (!fip_ret) { ret = rpmemd_fip_process_stop(rpmemd->fip); if (ret) { RPMEMD_LOG(ERR, "!stopping fip process failed"); } } rpmemd->fip_running = 0; return fip_ret; } /* * rpmemd_close_pool -- close pool and remove it if required */ static int rpmemd_close_pool(struct rpmemd *rpmemd, int remove) { int ret = 0; RPMEMD_LOG(NOTICE, "closing pool"); rpmemd_db_pool_close(rpmemd->db, rpmemd->pool); RPMEMD_LOG(INFO, "pool closed"); if (remove) { RPMEMD_LOG(NOTICE, "removing '%s'", rpmemd->pool_desc); ret = rpmemd_db_pool_remove(rpmemd->db, rpmemd->pool_desc, 0, 0); if (ret) { RPMEMD_LOG(ERR, "!removing pool '%s' failed", rpmemd->pool_desc); } else { RPMEMD_LOG(INFO, "removed '%s'", rpmemd->pool_desc); } } free(rpmemd->pool_desc); return ret; } /* * rpmemd_req_cleanup -- cleanup in-band connection and all resources allocated * during open/create requests */ static void rpmemd_req_cleanup(struct rpmemd *rpmemd) { if (!rpmemd->fip_running) return; int ret; ret = rpmemd_fip_stop(rpmemd); if (!ret) { rpmemd_fip_close(rpmemd->fip); rpmemd_fip_fini(rpmemd->fip); } int remove = rpmemd->created && ret; rpmemd_close_pool(rpmemd, remove); } /* * rpmemd_req_create -- handle create request */ static int rpmemd_req_create(struct rpmemd_obc *obc, void *arg, const struct rpmem_req_attr *req, const struct rpmem_pool_attr *pool_attr) { RPMEMD_ASSERT(arg != NULL); RPMEMD_LOG(NOTICE, "create request:"); rpmemd_print_req_attr(req); RPMEMD_LOG(NOTICE, "pool attributes:"); rpmemd_print_pool_attr(pool_attr); struct rpmemd *rpmemd = (struct rpmemd *)arg; int ret; int status = 0; int err_send = 1; struct rpmem_resp_attr resp; memset(&resp, 0, sizeof(resp)); if (rpmemd->pool) { RPMEMD_LOG(ERR, "pool already opened"); ret = -1; status = RPMEM_ERR_FATAL; goto err_pool_opened; } rpmemd->pool_desc = strdup(req->pool_desc); if (!rpmemd->pool_desc) { RPMEMD_LOG(ERR, "!allocating pool descriptor"); ret = -1; status = RPMEM_ERR_FATAL; goto err_strdup; } rpmemd->pool = rpmemd_db_pool_create(rpmemd->db, req->pool_desc, 0, pool_attr); if (!rpmemd->pool) { ret = -1; status = rpmemd_db_get_status(errno); goto err_pool_create; } rpmemd->created = 1; ret = rpmemd_check_pool(rpmemd, req, &status); if (ret) goto err_pool_check; ret = rpmemd_common_fip_init(rpmemd, req, &resp, &status); if (ret) goto err_fip_init; RPMEMD_LOG(NOTICE, "create request response: (status = %u)", status); if (!status) rpmemd_print_resp_attr(&resp); ret = rpmemd_obc_create_resp(obc, status, &resp); if (ret) goto err_create_resp; ret = rpmemd_fip_start_thread(rpmemd); if (ret) goto err_fip_start; return 0; err_fip_start: err_create_resp: err_send = 0; rpmemd_fip_fini(rpmemd->fip); err_fip_init: err_pool_check: rpmemd_db_pool_close(rpmemd->db, rpmemd->pool); rpmemd_db_pool_remove(rpmemd->db, req->pool_desc, 0, 0); err_pool_create: free(rpmemd->pool_desc); err_strdup: err_pool_opened: if (err_send) ret = rpmemd_obc_create_resp(obc, status, &resp); rpmemd->closing = 1; return ret; } /* * rpmemd_req_open -- handle open request */ static int rpmemd_req_open(struct rpmemd_obc *obc, void *arg, const struct rpmem_req_attr *req) { RPMEMD_ASSERT(arg != NULL); RPMEMD_LOG(NOTICE, "open request:"); rpmemd_print_req_attr(req); struct rpmemd *rpmemd = (struct rpmemd *)arg; int ret; int status = 0; int err_send = 1; struct rpmem_resp_attr resp; memset(&resp, 0, sizeof(resp)); struct rpmem_pool_attr pool_attr; memset(&pool_attr, 0, sizeof(pool_attr)); if (rpmemd->pool) { RPMEMD_LOG(ERR, "pool already opened"); ret = -1; status = RPMEM_ERR_FATAL; goto err_pool_opened; } rpmemd->pool_desc = strdup(req->pool_desc); if (!rpmemd->pool_desc) { RPMEMD_LOG(ERR, "!allocating pool descriptor"); ret = -1; status = RPMEM_ERR_FATAL; goto err_strdup; } rpmemd->pool = rpmemd_db_pool_open(rpmemd->db, req->pool_desc, 0, &pool_attr); if (!rpmemd->pool) { ret = -1; status = rpmemd_db_get_status(errno); goto err_pool_open; } RPMEMD_LOG(NOTICE, "pool attributes:"); rpmemd_print_pool_attr(&pool_attr); ret = rpmemd_check_pool(rpmemd, req, &status); if (ret) goto err_pool_check; ret = rpmemd_common_fip_init(rpmemd, req, &resp, &status); if (ret) goto err_fip_init; RPMEMD_LOG(NOTICE, "open request response: (status = %u)", status); if (!status) rpmemd_print_resp_attr(&resp); ret = rpmemd_obc_open_resp(obc, status, &resp, &pool_attr); if (ret) goto err_open_resp; ret = rpmemd_fip_start_thread(rpmemd); if (ret) goto err_fip_start; return 0; err_fip_start: err_open_resp: err_send = 0; rpmemd_fip_fini(rpmemd->fip); err_fip_init: err_pool_check: rpmemd_db_pool_close(rpmemd->db, rpmemd->pool); err_pool_open: free(rpmemd->pool_desc); err_strdup: err_pool_opened: if (err_send) ret = rpmemd_obc_open_resp(obc, status, &resp, &pool_attr); rpmemd->closing = 1; return ret; } /* * rpmemd_req_close -- handle close request */ static int rpmemd_req_close(struct rpmemd_obc *obc, void *arg, int flags) { RPMEMD_ASSERT(arg != NULL); RPMEMD_LOG(NOTICE, "close request"); struct rpmemd *rpmemd = (struct rpmemd *)arg; rpmemd->closing = 1; int ret; int status = 0; if (!rpmemd->pool) { RPMEMD_LOG(ERR, "pool not opened"); status = RPMEM_ERR_FATAL; return rpmemd_obc_close_resp(obc, status); } ret = rpmemd_fip_stop(rpmemd); if (ret) { status = RPMEM_ERR_FATAL; } else { rpmemd_fip_close(rpmemd->fip); rpmemd_fip_fini(rpmemd->fip); } int remove = rpmemd->created && (status || (flags & RPMEM_CLOSE_FLAGS_REMOVE)); if (rpmemd_close_pool(rpmemd, remove)) RPMEMD_LOG(ERR, "closing pool failed"); RPMEMD_LOG(NOTICE, "close request response (status = %u)", status); ret = rpmemd_obc_close_resp(obc, status); return ret; } /* * rpmemd_req_set_attr -- handle set attributes request */ static int rpmemd_req_set_attr(struct rpmemd_obc *obc, void *arg, const struct rpmem_pool_attr *pool_attr) { RPMEMD_ASSERT(arg != NULL); RPMEMD_LOG(NOTICE, "set attributes request"); struct rpmemd *rpmemd = (struct rpmemd *)arg; RPMEMD_ASSERT(rpmemd->pool != NULL); int ret; int status = 0; int err_send = 1; ret = rpmemd_db_pool_set_attr(rpmemd->pool, pool_attr); if (ret) { ret = -1; status = rpmemd_db_get_status(errno); goto err_set_attr; } RPMEMD_LOG(NOTICE, "new pool attributes:"); rpmemd_print_pool_attr(pool_attr); ret = rpmemd_obc_set_attr_resp(obc, status); if (ret) goto err_set_attr_resp; return ret; err_set_attr_resp: err_send = 0; err_set_attr: if (err_send) ret = rpmemd_obc_set_attr_resp(obc, status); return ret; } static struct rpmemd_obc_requests rpmemd_req = { .create = rpmemd_req_create, .open = rpmemd_req_open, .close = rpmemd_req_close, .set_attr = rpmemd_req_set_attr, }; /* * rpmemd_print_info -- print basic info and configuration */ static void rpmemd_print_info(struct rpmemd *rpmemd) { RPMEMD_LOG(NOTICE, "ssh connection: %s", _str(os_getenv("SSH_CONNECTION"))); RPMEMD_LOG(NOTICE, "user: %s", _str(os_getenv("USER"))); RPMEMD_LOG(NOTICE, "configuration"); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "pool set directory: '%s'", _str(rpmemd->config.poolset_dir)); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "persist method: %s", rpmem_persist_method_to_str(rpmemd->persist_method)); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "number of threads: %lu", rpmemd->config.nthreads); RPMEMD_DBG(RPMEMD_LOG_INDENT "persist APM: %s", bool2str(rpmemd->config.persist_apm)); RPMEMD_DBG(RPMEMD_LOG_INDENT "persist GPSPM: %s", bool2str(rpmemd->config.persist_general)); RPMEMD_DBG(RPMEMD_LOG_INDENT "use syslog: %s", bool2str(rpmemd->config.use_syslog)); RPMEMD_DBG(RPMEMD_LOG_INDENT "log file: %s", _str(rpmemd->config.log_file)); RPMEMD_DBG(RPMEMD_LOG_INDENT "log level: %s", rpmemd_log_level_to_str(rpmemd->config.log_level)); } int main(int argc, char *argv[]) { util_init(); int send_status = 1; int ret = 1; struct rpmemd *rpmemd = calloc(1, sizeof(*rpmemd)); if (!rpmemd) { RPMEMD_LOG(ERR, "!calloc"); goto err_rpmemd; } rpmemd->obc = rpmemd_obc_init(STDIN_FILENO, STDOUT_FILENO); if (!rpmemd->obc) { RPMEMD_LOG(ERR, "out-of-band connection initialization"); goto err_obc; } if (rpmemd_log_init(DAEMON_NAME, NULL, 0)) { RPMEMD_LOG(ERR, "logging subsystem initialization failed"); goto err_log_init; } if (rpmemd_config_read(&rpmemd->config, argc, argv) != 0) { RPMEMD_LOG(ERR, "reading configuration failed"); goto err_config; } rpmemd_log_close(); rpmemd_log_level = rpmemd->config.log_level; if (rpmemd_log_init(DAEMON_NAME, rpmemd->config.log_file, rpmemd->config.use_syslog)) { RPMEMD_LOG(ERR, "logging subsystem initialization" " failed (%s, %d)", rpmemd->config.log_file, rpmemd->config.use_syslog); goto err_log_init_config; } RPMEMD_LOG(INFO, "%s version %s", DAEMON_NAME, SRCVERSION); rpmemd->persist_method = rpmemd_get_pm(&rpmemd->config); rpmemd->db = rpmemd_db_init(rpmemd->config.poolset_dir, 0666); if (!rpmemd->db) { RPMEMD_LOG(ERR, "!pool set db initialization"); goto err_db_init; } if (rpmemd->config.rm_poolset) { RPMEMD_LOG(INFO, "removing '%s'", rpmemd->config.rm_poolset); if (rpmemd_db_pool_remove(rpmemd->db, rpmemd->config.rm_poolset, rpmemd->config.force, rpmemd->config.pool_set)) { RPMEMD_LOG(ERR, "removing '%s' failed", rpmemd->config.rm_poolset); ret = errno; } else { RPMEMD_LOG(NOTICE, "removed '%s'", rpmemd->config.rm_poolset); ret = 0; } send_status = 0; goto out_rm; } ret = rpmemd_obc_status(rpmemd->obc, 0); if (ret) { RPMEMD_LOG(ERR, "writing status failed"); goto err_status; } rpmemd_print_info(rpmemd); while (!ret) { ret = rpmemd_obc_process(rpmemd->obc, &rpmemd_req, rpmemd); if (ret) { RPMEMD_LOG(ERR, "out-of-band connection" " process failed"); goto err; } if (rpmemd->closing) break; } rpmemd_db_fini(rpmemd->db); rpmemd_config_free(&rpmemd->config); rpmemd_log_close(); rpmemd_obc_fini(rpmemd->obc); free(rpmemd); return 0; err: rpmemd_req_cleanup(rpmemd); err_status: out_rm: rpmemd_db_fini(rpmemd->db); err_db_init: err_log_init_config: rpmemd_config_free(&rpmemd->config); err_config: rpmemd_log_close(); err_log_init: if (send_status) { if (rpmemd_obc_status(rpmemd->obc, (uint32_t)errno)) RPMEMD_LOG(ERR, "writing status failed"); } rpmemd_obc_fini(rpmemd->obc); err_obc: free(rpmemd); err_rpmemd: return ret; } pmdk-1.8/src/tools/rpmemd/rpmemd_db.h0000664000000000000000000000512713615011243016333 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_db.h -- internal definitions for rpmemd database of pool set files */ struct rpmemd_db; struct rpmem_pool_attr; /* * struct rpmemd_db_pool -- remote pool context */ struct rpmemd_db_pool { void *pool_addr; size_t pool_size; struct pool_set *set; }; struct rpmemd_db *rpmemd_db_init(const char *root_dir, mode_t mode); struct rpmemd_db_pool *rpmemd_db_pool_create(struct rpmemd_db *db, const char *pool_desc, size_t pool_size, const struct rpmem_pool_attr *rattr); struct rpmemd_db_pool *rpmemd_db_pool_open(struct rpmemd_db *db, const char *pool_desc, size_t pool_size, struct rpmem_pool_attr *rattr); int rpmemd_db_pool_remove(struct rpmemd_db *db, const char *pool_desc, int force, int pool_set); int rpmemd_db_pool_set_attr(struct rpmemd_db_pool *prp, const struct rpmem_pool_attr *rattr); void rpmemd_db_pool_close(struct rpmemd_db *db, struct rpmemd_db_pool *prp); void rpmemd_db_fini(struct rpmemd_db *db); int rpmemd_db_check_dir(struct rpmemd_db *db); int rpmemd_db_pool_is_pmem(struct rpmemd_db_pool *pool); pmdk-1.8/src/tools/rpmemd/rpmemd_util.c0000664000000000000000000001040213615011243016706 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_util.c -- rpmemd utility functions definitions */ #include #include #include "libpmem.h" #include "rpmem_common.h" #include "rpmemd_log.h" #include "rpmemd_util.h" /* * rpmemd_pmem_persist -- pmem_persist wrapper required to unify function * pointer type with pmem_msync */ int rpmemd_pmem_persist(const void *addr, size_t len) { pmem_persist(addr, len); return 0; } /* * rpmemd_flush_fatal -- APM specific flush function which should never be * called because APM does not require flushes */ int rpmemd_flush_fatal(const void *addr, size_t len) { RPMEMD_FATAL("rpmemd_flush_fatal should never be called"); } /* * rpmemd_persist_to_str -- convert persist function pointer to string */ static const char * rpmemd_persist_to_str(int (*persist)(const void *addr, size_t len)) { if (persist == rpmemd_pmem_persist) { return "pmem_persist"; } else if (persist == pmem_msync) { return "pmem_msync"; } else if (persist == rpmemd_flush_fatal) { return "none"; } else { return NULL; } } /* * rpmem_print_pm_policy -- print persistency method policy */ static void rpmem_print_pm_policy(enum rpmem_persist_method persist_method, int (*persist)(const void *addr, size_t len)) { RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "persist method: %s", rpmem_persist_method_to_str(persist_method)); RPMEMD_LOG(NOTICE, RPMEMD_LOG_INDENT "persist flush: %s", rpmemd_persist_to_str(persist)); } /* * rpmem_memcpy_msync -- memcpy and msync */ static void * rpmem_memcpy_msync(void *pmemdest, const void *src, size_t len) { void *ret = pmem_memcpy(pmemdest, src, len, PMEM_F_MEM_NOFLUSH); pmem_msync(pmemdest, len); return ret; } /* * rpmemd_apply_pm_policy -- choose the persistency method and the flush * function according to the pool type and the persistency method read from the * config */ int rpmemd_apply_pm_policy(enum rpmem_persist_method *persist_method, int (**persist)(const void *addr, size_t len), void *(**memcpy_persist)(void *pmemdest, const void *src, size_t len), const int is_pmem) { switch (*persist_method) { case RPMEM_PM_APM: if (is_pmem) { *persist_method = RPMEM_PM_APM; *persist = rpmemd_flush_fatal; } else { *persist_method = RPMEM_PM_GPSPM; *persist = pmem_msync; } break; case RPMEM_PM_GPSPM: *persist_method = RPMEM_PM_GPSPM; *persist = is_pmem ? rpmemd_pmem_persist : pmem_msync; break; default: RPMEMD_FATAL("invalid persist method: %d", *persist_method); return -1; } /* this is for RPMEM_PERSIST_INLINE */ if (is_pmem) *memcpy_persist = pmem_memcpy_persist; else *memcpy_persist = rpmem_memcpy_msync; RPMEMD_LOG(NOTICE, "persistency policy:"); rpmem_print_pm_policy(*persist_method, *persist); return 0; } pmdk-1.8/src/tools/rpmemd/rpmemd_config.c0000664000000000000000000004021613615011243017204 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_config.c -- rpmemd config source file */ #include #include #include #include #include #include #include #include #include #include #include #include #include "rpmemd.h" #include "rpmemd_log.h" #include "rpmemd_config.h" #include "os.h" #define CONFIG_LINE_SIZE_INIT 50 #define INVALID_CHAR_POS UINT64_MAX struct rpmemd_special_chars_pos { uint64_t equal_char; uint64_t comment_char; uint64_t EOL_char; }; enum rpmemd_option { RPD_OPT_LOG_FILE, RPD_OPT_POOLSET_DIR, RPD_OPT_PERSIST_APM, RPD_OPT_PERSIST_GENERAL, RPD_OPT_USE_SYSLOG, RPD_OPT_LOG_LEVEL, RPD_OPT_RM_POOLSET, RPD_OPT_MAX_VALUE, RPD_OPT_INVALID = UINT64_MAX, }; static const char *optstr = "c:hVr:fst:"; /* * options -- cl and config file options */ static const struct option options[] = { {"config", required_argument, NULL, 'c'}, {"help", no_argument, NULL, 'h'}, {"version", no_argument, NULL, 'V'}, {"log-file", required_argument, NULL, RPD_OPT_LOG_FILE}, {"poolset-dir", required_argument, NULL, RPD_OPT_POOLSET_DIR}, {"persist-apm", no_argument, NULL, RPD_OPT_PERSIST_APM}, {"persist-general", no_argument, NULL, RPD_OPT_PERSIST_GENERAL}, {"use-syslog", no_argument, NULL, RPD_OPT_USE_SYSLOG}, {"log-level", required_argument, NULL, RPD_OPT_LOG_LEVEL}, {"remove", required_argument, NULL, 'r'}, {"force", no_argument, NULL, 'f'}, {"pool-set", no_argument, NULL, 's'}, {"nthreads", required_argument, NULL, 't'}, {NULL, 0, NULL, 0}, }; #define VALUE_INDENT " " static const char * const help_str = "\n" "Options:\n" " -c, --config configuration file location\n" " -r, --remove remove pool described by given poolset file\n" " -f, --force ignore errors when removing a pool\n" " -t, --nthreads number of processing threads\n" " -h, --help display help message and exit\n" " -V, --version display target daemon version and exit\n" " --log-file log file location\n" " --poolset-dir pool set files directory\n" " --persist-apm enable Appliance Persistency Method\n" " --persist-general enable General Server Persistency Mechanism\n" " --use-syslog use syslog(3) for logging messages\n" " --log-level set log level value\n" VALUE_INDENT "err error conditions\n" VALUE_INDENT "warn warning conditions\n" VALUE_INDENT "notice normal, but significant, condition\n" VALUE_INDENT "info informational message\n" VALUE_INDENT "debug debug-level message\n" "\n" "For complete documentation see %s(1) manual page."; /* * print_version -- (internal) prints version message */ static void print_version(void) { RPMEMD_LOG(ERR, "%s version %s", DAEMON_NAME, SRCVERSION); } /* * print_usage -- (internal) prints usage message */ static void print_usage(const char *name) { RPMEMD_LOG(ERR, "usage: %s [--version] [--help] []", name); } /* * print_help -- (internal) prints help message */ static void print_help(const char *name) { print_usage(name); print_version(); RPMEMD_LOG(ERR, help_str, DAEMON_NAME); } /* * parse_config_string -- (internal) parse string value */ static inline char * parse_config_string(const char *value) { if (strlen(value) == 0) { errno = EINVAL; return NULL; } char *output = strdup(value); if (output == NULL) RPMEMD_FATAL("!strdup"); return output; } /* * parse_config_bool -- (internal) parse yes / no flag */ static inline int parse_config_bool(bool *config_value, const char *value) { if (value == NULL) *config_value = true; else if (strcmp("yes", value) == 0) *config_value = true; else if (strcmp("no", value) == 0) *config_value = false; else { errno = EINVAL; return -1; } return 0; } /* * set_option -- (internal) set single config option */ static int set_option(enum rpmemd_option option, const char *value, struct rpmemd_config *config) { int ret = 0; switch (option) { case RPD_OPT_LOG_FILE: free(config->log_file); config->log_file = parse_config_string(value); if (config->log_file == NULL) return -1; else config->use_syslog = false; break; case RPD_OPT_POOLSET_DIR: free(config->poolset_dir); config->poolset_dir = parse_config_string(value); if (config->poolset_dir == NULL) return -1; break; case RPD_OPT_PERSIST_APM: ret = parse_config_bool(&config->persist_apm, value); break; case RPD_OPT_PERSIST_GENERAL: ret = parse_config_bool(&config->persist_general, value); break; case RPD_OPT_USE_SYSLOG: ret = parse_config_bool(&config->use_syslog, value); break; case RPD_OPT_LOG_LEVEL: config->log_level = rpmemd_log_level_from_str(value); if (config->log_level == MAX_RPD_LOG) { errno = EINVAL; return -1; } break; default: errno = EINVAL; return -1; } return ret; } /* * get_config_line -- (internal) read single line from file */ static int get_config_line(FILE *file, char **line, uint64_t *line_max, uint8_t *line_max_increased, struct rpmemd_special_chars_pos *pos) { uint8_t line_complete = 0; uint64_t line_length = 0; char *line_part = *line; do { char *ret = fgets(line_part, (int)(*line_max - line_length), file); if (ret == NULL) return 0; for (uint64_t i = 0; i < *line_max; ++i) { if (line_part[i] == '\n') line_complete = 1; else if (line_part[i] == '\0') { line_length += i; if (line_length + 1 < *line_max) line_complete = 1; break; } else if (line_part[i] == '#' && pos->comment_char == UINT64_MAX) pos->comment_char = line_length + i; else if (line_part[i] == '=' && pos->equal_char == UINT64_MAX) pos->equal_char = line_length + i; } if (line_complete == 0) { *line = realloc(*line, sizeof(char) * (*line_max) * 2); if (*line == NULL) { RPMEMD_FATAL("!realloc"); } line_part = *line + *line_max - 1; line_length = *line_max - 1; *line_max *= 2; *line_max_increased = 1; } } while (line_complete != 1); pos->EOL_char = line_length; return 0; } /* * trim_line_element -- (internal) remove white characters */ static char * trim_line_element(char *line, uint64_t start, uint64_t end) { for (; start <= end; ++start) { if (!isspace(line[start])) break; } for (; end > start; --end) { if (!isspace(line[end - 1])) break; } if (start == end) return NULL; line[end] = '\0'; return &line[start]; } /* * parse_config_key -- (internal) lookup config key */ static enum rpmemd_option parse_config_key(const char *key) { for (int i = 0; options[i].name != 0; ++i) { if (strcmp(key, options[i].name) == 0) return (enum rpmemd_option)options[i].val; } return RPD_OPT_INVALID; } /* * parse_config_line -- (internal) parse single config line * * Return newly written option flag. Store possible errors in errno. */ static int parse_config_line(char *line, struct rpmemd_special_chars_pos *pos, struct rpmemd_config *config, uint64_t disabled) { if (pos->comment_char < pos->equal_char) pos->equal_char = INVALID_CHAR_POS; uint64_t end_of_content = pos->comment_char != INVALID_CHAR_POS ? pos->comment_char : pos->EOL_char; if (pos->equal_char == INVALID_CHAR_POS) { char *leftover = trim_line_element(line, 0, end_of_content); if (leftover != NULL) { errno = EINVAL; return -1; } else { return 0; } } char *key_name = trim_line_element(line, 0, pos->equal_char); char *value = trim_line_element(line, pos->equal_char + 1, end_of_content); if (key_name == NULL || value == NULL) { errno = EINVAL; return -1; } enum rpmemd_option key = parse_config_key(key_name); if (key != RPD_OPT_INVALID) { if ((disabled & (uint64_t)(1 << key)) == 0) if (set_option(key, value, config) != 0) return -1; } else { errno = EINVAL; return -1; } return 0; } /* * parse_config_file -- (internal) parse config file */ static int parse_config_file(const char *filename, struct rpmemd_config *config, uint64_t disabled, int required) { RPMEMD_ASSERT(filename != NULL); FILE *file = os_fopen(filename, "r"); if (file == NULL) { if (required) { RPMEMD_LOG(ERR, "!%s", filename); goto error_fopen; } else { goto optional_config_missing; } } uint8_t line_max_increased = 0; uint64_t line_max = CONFIG_LINE_SIZE_INIT; uint64_t line_num = 1; char *line = (char *)malloc(sizeof(char) * line_max); if (line == NULL) { RPMEMD_LOG(ERR, "!malloc"); goto error_malloc_line; } char *line_copy = (char *)malloc(sizeof(char) * line_max); if (line_copy == NULL) { RPMEMD_LOG(ERR, "!malloc"); goto error_malloc_line_copy; } struct rpmemd_special_chars_pos pos; do { memset(&pos, 0xff, sizeof(pos)); if (get_config_line(file, &line, &line_max, &line_max_increased, &pos) != 0) goto error; if (line_max_increased) { char *line_new = (char *)realloc(line_copy, sizeof(char) * line_max); if (line_new == NULL) { RPMEMD_LOG(ERR, "!malloc"); goto error; } line_copy = line_new; line_max_increased = 0; } if (pos.EOL_char != INVALID_CHAR_POS) { strcpy(line_copy, line); int ret = parse_config_line(line_copy, &pos, config, disabled); if (ret != 0) { size_t len = strlen(line); if (len > 0 && line[len - 1] == '\n') line[len - 1] = '\0'; RPMEMD_LOG(ERR, "Invalid config file line at " "%s:%lu\n%s", filename, line_num, line); goto error; } } ++line_num; } while (pos.EOL_char != INVALID_CHAR_POS); free(line_copy); free(line); fclose(file); optional_config_missing: return 0; error: free(line_copy); error_malloc_line_copy: free(line); error_malloc_line: fclose(file); error_fopen: return -1; } /* * parse_cl_args -- (internal) parse command line arguments */ static void parse_cl_args(int argc, char *argv[], struct rpmemd_config *config, const char **config_file, uint64_t *cl_options) { RPMEMD_ASSERT(argv != NULL); RPMEMD_ASSERT(config != NULL); int opt; int option_index = 0; while ((opt = getopt_long(argc, argv, optstr, options, &option_index)) != -1) { switch (opt) { case 'c': (*config_file) = optarg; break; case 'r': config->rm_poolset = optarg; break; case 'f': config->force = true; break; case 's': config->pool_set = true; break; case 't': errno = 0; char *endptr; config->nthreads = strtoul(optarg, &endptr, 10); if (errno || *endptr != '\0') { RPMEMD_LOG(ERR, "invalid number of threads -- '%s'", optarg); exit(-1); } break; case 'h': print_help(argv[0]); exit(0); case 'V': print_version(); exit(0); break; default: if (set_option((enum rpmemd_option)opt, optarg, config) == 0) { *cl_options |= (UINT64_C(1) << opt); } else { print_usage(argv[0]); exit(-1); } } } } /* * get_home_dir -- (internal) return user home directory * * Function will lookup user home directory in order: * 1. HOME environment variable * 2. Password file entry using real user ID */ static void get_home_dir(char *str, size_t size) { char *home = os_getenv(HOME_ENV); if (home) { int r = snprintf(str, size, "%s", home); if (r < 0) RPMEMD_FATAL("snprintf: %d", r); } else { uid_t uid = getuid(); struct passwd *pw = getpwuid(uid); if (pw == NULL) RPMEMD_FATAL("!getpwuid"); int r = snprintf(str, size, "%s", pw->pw_dir); if (r < 0) RPMEMD_FATAL("snprintf: %d", r); } } /* * concat_dir_and_file_name -- (internal) concatenate directory and file name * into single string path */ static void concat_dir_and_file_name(char *path, size_t size, const char *dir, const char *file) { int r = snprintf(path, size, "%s/%s", dir, file); if (r < 0) RPMEMD_FATAL("snprintf: %d", r); } /* * str_replace_home -- (internal) replace $HOME string with user home directory * * If function does not find $HOME string it will return haystack untouched. * Otherwise it will allocate new string with $HOME replaced with provided * home_dir path. haystack will be released and newly created string returned. */ static char * str_replace_home(char *haystack, const char *home_dir) { const size_t placeholder_len = strlen(HOME_STR_PLACEHOLDER); const size_t home_len = strlen(home_dir); size_t haystack_len = strlen(haystack); char *pos = strstr(haystack, HOME_STR_PLACEHOLDER); if (!pos) return haystack; const char *after = pos + placeholder_len; if (isalnum(*after)) return haystack; haystack_len += home_len - placeholder_len + 1; char *buf = malloc(sizeof(char) * haystack_len); if (!buf) RPMEMD_FATAL("!malloc"); *pos = '\0'; int r = snprintf(buf, haystack_len, "%s%s%s", haystack, home_dir, after); if (r < 0) RPMEMD_FATAL("snprintf: %d", r); free(haystack); return buf; } /* * config_set_default -- (internal) load default config */ static void config_set_default(struct rpmemd_config *config, const char *poolset_dir) { config->log_file = strdup(RPMEMD_DEFAULT_LOG_FILE); if (!config->log_file) RPMEMD_FATAL("!strdup"); config->poolset_dir = strdup(poolset_dir); if (!config->poolset_dir) RPMEMD_FATAL("!strdup"); config->persist_apm = false; config->persist_general = true; config->use_syslog = true; config->max_lanes = RPMEM_DEFAULT_MAX_LANES; config->log_level = RPD_LOG_ERR; config->rm_poolset = NULL; config->force = false; config->nthreads = RPMEM_DEFAULT_NTHREADS; } /* * rpmemd_config_read -- read config from cl and config files * * cl param overwrites configuration from any config file. Config file are read * in order: * 1. Global config file * 2. User config file * or * cl provided config file */ int rpmemd_config_read(struct rpmemd_config *config, int argc, char *argv[]) { const char *cl_config_file = NULL; char user_config_file[PATH_MAX]; char home_dir[PATH_MAX]; uint64_t cl_options = 0; get_home_dir(home_dir, PATH_MAX); config_set_default(config, home_dir); parse_cl_args(argc, argv, config, &cl_config_file, &cl_options); if (cl_config_file) { if (parse_config_file(cl_config_file, config, cl_options, 1)) { rpmemd_config_free(config); return 1; } } else { if (parse_config_file(RPMEMD_GLOBAL_CONFIG_FILE, config, cl_options, 0)) { rpmemd_config_free(config); return 1; } concat_dir_and_file_name(user_config_file, PATH_MAX, home_dir, RPMEMD_USER_CONFIG_FILE); if (parse_config_file(user_config_file, config, cl_options, 0)) { rpmemd_config_free(config); return 1; } } config->poolset_dir = str_replace_home(config->poolset_dir, home_dir); return 0; } /* * rpmemd_config_free -- rpmemd config release */ void rpmemd_config_free(struct rpmemd_config *config) { free(config->log_file); free(config->poolset_dir); } pmdk-1.8/src/tools/rpmemd/rpmemd_log.c0000664000000000000000000001565213615011243016526 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_log.c -- rpmemd logging functions definitions */ #include #include #include #include #include #include #include "rpmemd_log.h" #include "os.h" #include "valgrind_internal.h" #define RPMEMD_SYSLOG_OPTS (LOG_NDELAY | LOG_PID) #define RPMEMD_SYSLOG_FACILITY (LOG_USER) #define RPMEMD_DEFAULT_FH stderr #define RPMEMD_MAX_MSG ((size_t)8192) #define RPMEMD_MAX_PREFIX ((size_t)256) enum rpmemd_log_level rpmemd_log_level; static char *rpmemd_ident; static int rpmemd_use_syslog; static FILE *rpmemd_log_file; static char rpmemd_prefix_buff[RPMEMD_MAX_PREFIX]; static const char *rpmemd_log_level_str[MAX_RPD_LOG] = { [RPD_LOG_ERR] = "err", [RPD_LOG_WARN] = "warn", [RPD_LOG_NOTICE] = "notice", [RPD_LOG_INFO] = "info", [_RPD_LOG_DBG] = "debug", }; static int rpmemd_level2prio[MAX_RPD_LOG] = { [RPD_LOG_ERR] = LOG_ERR, [RPD_LOG_WARN] = LOG_WARNING, [RPD_LOG_NOTICE] = LOG_NOTICE, [RPD_LOG_INFO] = LOG_INFO, [_RPD_LOG_DBG] = LOG_DEBUG, }; /* * rpmemd_log_basename -- similar to POSIX basename, but without handling for * trailing slashes. */ static const char * rpmemd_log_basename(const char *fname) { const char *s; if (fname == NULL) return "(null)"; s = strrchr(fname, '/'); if (s != NULL) return s + 1; else return fname; } /* * rpmemd_log_level_from_str -- converts string to log level value */ enum rpmemd_log_level rpmemd_log_level_from_str(const char *str) { if (!str) return MAX_RPD_LOG; for (enum rpmemd_log_level level = 0; level < MAX_RPD_LOG; level++) { if (strcmp(rpmemd_log_level_str[level], str) == 0) return level; } return MAX_RPD_LOG; } /* * rpmemd_log_level_to_str -- converts log level enum to string */ const char * rpmemd_log_level_to_str(enum rpmemd_log_level level) { if (level >= MAX_RPD_LOG) return NULL; return rpmemd_log_level_str[level]; } /* * rpmemd_log_init -- inititalize logging subsystem * * ident - string prepended to every message * use_syslog - use syslog instead of standard output */ int rpmemd_log_init(const char *ident, const char *fname, int use_syslog) { rpmemd_use_syslog = use_syslog; if (rpmemd_use_syslog) { openlog(rpmemd_ident, RPMEMD_SYSLOG_OPTS, RPMEMD_SYSLOG_FACILITY); } else { rpmemd_ident = strdup(ident); if (!rpmemd_ident) { perror("strdup"); return -1; } if (fname) { rpmemd_log_file = os_fopen(fname, "a"); if (!rpmemd_log_file) { perror(fname); free(rpmemd_ident); rpmemd_ident = NULL; return -1; } } else { rpmemd_log_file = RPMEMD_DEFAULT_FH; } } return 0; } /* * rpmemd_log_close -- deinitialize logging subsystem */ void rpmemd_log_close(void) { if (rpmemd_use_syslog) { closelog(); } else { if (rpmemd_log_file != RPMEMD_DEFAULT_FH) fclose(rpmemd_log_file); rpmemd_log_file = NULL; free(rpmemd_ident); rpmemd_ident = NULL; } } /* * rpmemd_prefix -- set prefix for every message */ int rpmemd_prefix(const char *fmt, ...) { if (!fmt) { rpmemd_prefix_buff[0] = '\0'; return 0; } va_list ap; va_start(ap, fmt); int ret = vsnprintf(rpmemd_prefix_buff, RPMEMD_MAX_PREFIX, fmt, ap); va_end(ap); if (ret < 0) return -1; return 0; } /* * rpmemd_log -- main logging function */ void rpmemd_log(enum rpmemd_log_level level, const char *fname, int lineno, const char *fmt, ...) { if (!rpmemd_use_syslog && level > rpmemd_log_level) return; char buff[RPMEMD_MAX_MSG]; size_t cnt = 0; int ret; if (fname) { ret = snprintf(&buff[cnt], RPMEMD_MAX_MSG - cnt, "[%s:%d] ", rpmemd_log_basename(fname), lineno); if (ret < 0) RPMEMD_FATAL("snprintf failed: %d", ret); if ((unsigned)ret >= RPMEMD_MAX_MSG - cnt) RPMEMD_FATAL("overflow(1): %d >= %lu", ret, RPMEMD_MAX_MSG - cnt); cnt += (size_t)ret; } if (rpmemd_prefix_buff[0]) { ret = snprintf(&buff[cnt], RPMEMD_MAX_MSG - cnt, "%s ", rpmemd_prefix_buff); if (ret < 0) RPMEMD_FATAL("snprintf failed: %d", ret); if ((unsigned)ret >= RPMEMD_MAX_MSG - cnt) RPMEMD_FATAL("overflow(2): %d >= %lu", ret, RPMEMD_MAX_MSG - cnt); cnt += (size_t)ret; } const char *errorstr = ""; const char *prefix = ""; const char *suffix = "\n"; if (fmt) { if (*fmt == '!') { fmt++; errorstr = strerror(errno); prefix = ": "; } va_list ap; va_start(ap, fmt); ret = vsnprintf(&buff[cnt], RPMEMD_MAX_MSG - cnt, fmt, ap); va_end(ap); if (ret < 0) RPMEMD_FATAL("vsnprintf failed"); if ((unsigned)ret >= RPMEMD_MAX_MSG - cnt) RPMEMD_FATAL("overflow(3): %d >= %lu", ret, RPMEMD_MAX_MSG - cnt); cnt += (size_t)ret; ret = snprintf(&buff[cnt], RPMEMD_MAX_MSG - cnt, "%s%s%s", prefix, errorstr, suffix); if (ret < 0) RPMEMD_FATAL("snprintf failed: %d", ret); if ((unsigned)ret >= RPMEMD_MAX_MSG - cnt) RPMEMD_FATAL("overflow(4): %d >= %lu", ret, RPMEMD_MAX_MSG - cnt); cnt += (size_t)ret; } buff[cnt] = 0; if (rpmemd_use_syslog) { int prio = rpmemd_level2prio[level]; syslog(prio, "%s", buff); } else { /* to suppress drd false-positive */ /* XXX: confirm real nature of this issue: pmem/issues#863 */ #ifdef SUPPRESS_FPUTS_DRD_ERROR VALGRIND_ANNOTATE_IGNORE_READS_BEGIN(); VALGRIND_ANNOTATE_IGNORE_WRITES_BEGIN(); #endif fprintf(rpmemd_log_file, "%s", buff); fflush(rpmemd_log_file); #ifdef SUPPRESS_FPUTS_DRD_ERROR VALGRIND_ANNOTATE_IGNORE_READS_END(); VALGRIND_ANNOTATE_IGNORE_WRITES_END(); #endif } } pmdk-1.8/src/tools/rpmemd/rpmemd_log.h0000664000000000000000000000666213615011243016534 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmemd_log.h -- rpmemd logging functions declarations */ #include #include "util.h" #define FORMAT_PRINTF(a, b) __attribute__((__format__(__printf__, (a), (b)))) /* * The tab character is not allowed in rpmemd log, * because it is not well handled by syslog. * Please use RPMEMD_LOG_INDENT instead. */ #define RPMEMD_LOG_INDENT " " #ifdef DEBUG #define RPMEMD_LOG(level, fmt, arg...) do {\ COMPILE_ERROR_ON(strchr(fmt, '\t') != 0);\ rpmemd_log(RPD_LOG_##level, __FILE__, __LINE__, fmt, ## arg);\ } while (0) #else #define RPMEMD_LOG(level, fmt, arg...) do {\ COMPILE_ERROR_ON(strchr(fmt, '\t') != 0);\ rpmemd_log(RPD_LOG_##level, NULL, 0, fmt, ## arg);\ } while (0) #endif #ifdef DEBUG #define RPMEMD_DBG(fmt, arg...) do {\ COMPILE_ERROR_ON(strchr(fmt, '\t') != 0);\ rpmemd_log(_RPD_LOG_DBG, __FILE__, __LINE__, fmt, ## arg);\ } while (0) #else #define RPMEMD_DBG(fmt, arg...) do {} while (0) #endif #define RPMEMD_ERR(fmt, arg...) do {\ RPMEMD_LOG(ERR, fmt, ## arg);\ } while (0) #define RPMEMD_FATAL(fmt, arg...) do {\ RPMEMD_LOG(ERR, fmt, ## arg);\ abort();\ } while (0) #define RPMEMD_ASSERT(cond) do {\ if (!(cond)) {\ rpmemd_log(RPD_LOG_ERR, __FILE__, __LINE__,\ "assertion fault: %s", #cond);\ abort();\ }\ } while (0) enum rpmemd_log_level { RPD_LOG_ERR, RPD_LOG_WARN, RPD_LOG_NOTICE, RPD_LOG_INFO, _RPD_LOG_DBG, /* disallow to use this with LOG macro */ MAX_RPD_LOG, }; enum rpmemd_log_level rpmemd_log_level_from_str(const char *str); const char *rpmemd_log_level_to_str(enum rpmemd_log_level level); extern enum rpmemd_log_level rpmemd_log_level; int rpmemd_log_init(const char *ident, const char *fname, int use_syslog); void rpmemd_log_close(void); int rpmemd_prefix(const char *fmt, ...) FORMAT_PRINTF(1, 2); void rpmemd_log(enum rpmemd_log_level level, const char *fname, int lineno, const char *fmt, ...) FORMAT_PRINTF(4, 5); pmdk-1.8/src/tools/rpmemd/Makefile0000664000000000000000000000447313615011243015674 0ustar rootroot# Copyright 2016-2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # Makefile -- top Makefile for rpmemd # vpath %.c ../../rpmem_common/ TOP = ../../.. include $(TOP)/src/common.inc ifeq ($(BUILD_RPMEM),y) SCP_TO_REMOTE_NODES = y TARGET = rpmemd OBJS = rpmemd.o\ rpmemd_log.o\ rpmemd_config.o\ rpmem_common.o\ rpmemd_obc.o\ rpmemd_db.o\ rpmemd_fip.o\ rpmem_fip_common.o\ rpmemd_util.o LIBPMEM=y TOOLS_COMMON=y LIBPMEMBLK_PRIV=btt_info_convert2h INCS += -I$(TOP)/src/rpmem_common CFLAGS += -DRPMEMC_LOG_RPMEMD ifneq ($(DEBUG),) CFLAGS += -DDEBUG endif CFLAGS += $(shell $(PKG_CONFIG) --cflags libfabric) LIBS += -pthread LIBS += $(shell $(PKG_CONFIG) --libs libfabric) INSTALL_TARGET=y else $(info NOTE: Skipping rpmemd because $(BUILD_RPMEM_INFO)) endif include ../Makefile.inc .PHONY: test check pmdk-1.8/src/tools/.gitignore0000664000000000000000000000012513615011243014726 0ustar rootrootTAGS cscope.in.out cscope.po.out cscope.out *.static-debug *.static-nondebug .synced pmdk-1.8/src/tools/pmreorder/0000775000000000000000000000000013615011243014737 5ustar rootrootpmdk-1.8/src/tools/pmreorder/consistencycheckwrap.py0000664000000000000000000001153713615011243021551 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. from sys import exit from os import path from ctypes import cdll, c_char_p, c_int import os checkers = ["prog", "lib"] class ConsistencyCheckerBase: """ Base class for consistency checker classes. Checker of each type should implement check_consistency method. """ def check_consistency(self, filename): pass class LibChecker(ConsistencyCheckerBase): """ Allows registration of a consistency checking function and verifying the consistency of a file. The function has to be in a shared library. It is then used to check consistency of an arbitrary file. The function has to take a file name as the only parameter and return an int: 0 for inconsistent, 1 for consistent. The prototype of the function:: int func_name(const char* file_name) """ def __init__(self, library_name, func_name): """ Loads the consistency checking function from the given library. :param library_name: The full name of the library. :type library_name: str :param func_name: The name of the consistency checking function within the library. :type func_name: str :return: None """ self._lib_func = getattr(cdll.LoadLibrary(library_name), func_name) self._lib_func.argtypes = [c_char_p] self._lib_func.restype = c_int def check_consistency(self, filename): """ Checks the consistency of a given file using the previously loaded function. :param filename: The full name of the file to be checked. :type filename: str :return: 1 if file is consistent, 0 otherwise. :rtype: int :raises: Generic exception, when no function has been loaded. """ if self._lib_func is None: raise RuntimeError("Consistency check function not loaded") return self._lib_func(filename) class ProgChecker(ConsistencyCheckerBase): """ Allows registration of a consistency checking program and verifying the consistency of a file. """ def __init__(self, bin_path, bin_args): self._bin_path = bin_path self._bin_cmd = bin_args def check_consistency(self, filename): """ Checks the consistency of a given file using the previously loaded function. :param filename: The full name of the file to be checked. :type filename: str :return: 1 if file is consistent, 0 otherwise. :rtype: int :raises: Generic exception, when no function has been loaded. """ if self._bin_path is None or self._bin_cmd is None: raise RuntimeError("consistency check handle not set") return os.system(self._bin_path + " " + self._bin_cmd + " " + filename) def get_checker(checker_type, checker_path_args, name): checker_path_args = checker_path_args.split(" ", 1) checker_path = checker_path_args[0] # check for params if len(checker_path_args) > 1: args = checker_path_args[1] else: args = "" if not path.exists(checker_path): print("Invalid path:" + checker_path) exit(1) checker = None if checker_type == "prog": checker = ProgChecker(checker_path, args) elif checker_type == "lib": checker = LibChecker(checker_path, name) return checker pmdk-1.8/src/tools/pmreorder/reorderengines.py0000664000000000000000000002761713615011243020341 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. from itertools import combinations from itertools import permutations from itertools import islice from itertools import chain from random import sample from functools import partial from reorderexceptions import NotSupportedOperationException import collections class FullReorderEngine: def __init__(self): self.test_on_barrier = True """ Realizes a full reordering of stores within a given list. Example: input: (a, b, c) output: () ('a',) ('b',) ('c',) ('a', 'b') ('a', 'c') ('b', 'a') ('b', 'c') ('c', 'a') ('c', 'b') ('a', 'b', 'c') ('a', 'c', 'b') ('b', 'a', 'c') ('b', 'c', 'a') ('c', 'a', 'b') ('c', 'b', 'a') """ def generate_sequence(self, store_list): """ Generates all possible combinations of all possible lengths, based on the operations in the list. :param store_list: The list of stores to be reordered. :type store_list: list of :class:`memoryoperations.Store` :return: Yields all combinations of stores. :rtype: iterable """ for length in range(0, len(store_list) + 1): for permutation in permutations(store_list, length): yield permutation class AccumulativeReorderEngine: def __init__(self): self.test_on_barrier = True """ Realizes an accumulative reorder of stores within a given list. Example: input: (a, b, c) output: () ('a') ('a', 'b') ('a', 'b', 'c') """ def generate_sequence(self, store_list): """ Generates all accumulative lists, based on the operations in the store list. :param store_list: The list of stores to be reordered. :type store_list: list of :class:`memoryoperations.Store` :return: Yields all accumulative combinations of stores. :rtype: iterable """ for i in range(0, len(store_list) + 1): out_list = [store_list[i] for i in range(0, i)] yield out_list class AccumulativeReverseReorderEngine: def __init__(self): self.test_on_barrier = True """ Realizes an accumulative reorder of stores within a given list in reverse order. Example: input: (a, b, c) output: () ('c') ('c', 'b') ('c', 'b', 'a') """ def generate_sequence(self, store_list): """ Reverse all elements order and generates all accumulative lists. :param store_list: The list of stores to be reordered. :type store_list: list of :class:`memoryoperations.Store` :return: Yields all accumulative combinations of stores. :rtype: iterable """ store_list = list(reversed(store_list)) for i in range(len(store_list) + 1): yield [store_list[j] for j in range(i)] class SlicePartialReorderEngine: """ Generates a slice of the full reordering of stores within a given list. Example: input: (a, b, c), start = 2, stop = None, step = 2 output: ('b') ('a', 'b') ('b', 'c') """ def __init__(self, start, stop, step=1): """ Initializes the generator with the provided parameters. :param start: Number of preceding elements to be skipped. :param stop: The element at which the slice is to stop. :param step: How many values are skipped between successive calls. """ self._start = start self._stop = stop self._step = step self.test_on_barrier = True def generate_sequence(self, store_list): """ This generator yields a slice of all possible combinations. The result may be a set of combinations of different lengths, depending on the slice parameters provided at object creation. :param store_list: The list of stores to be reordered. :type store_list: list of :class:`memoryoperations.Store` :return: Yields a slice of all combinations of stores. :rtype: iterable """ for sl in islice(chain(*map(lambda x: combinations(store_list, x), range(0, len(store_list) + 1))), self._start, self._stop, self._step): yield sl class FilterPartialReorderEngine: """ Generates a filtered set of the combinations without duplication of stores within a given list. Example: input: (a, b, c), filter = filter_min_elem, kwarg1 = 2 output: (a, b) (a, c) (b, c) (a, b, c) input: (a, b, c), filter = filter_max_elem, kwarg1 = 2 output: () (a) (b) (c) (a, b) (a, c) (b, c) input: (a, b, c), filter = filter_between_elem, kwarg1 = 2, kwarg2 = 2 output: (a, b) (a, c) (b, c) """ def __init__(self, func, **kwargs): """ Initializes the generator with the provided parameters. :param func: The filter function. :param **kwargs: Arguments to the filter function. """ self._filter = func self._filter_kwargs = kwargs self.test_on_barrier = True @staticmethod def filter_min_elem(store_list, **kwargs): """ Filter stores list if number of element is less than kwarg1 """ if (len(store_list) < kwargs["kwarg1"]): return False return True @staticmethod def filter_max_elem(store_list, **kwargs): """ Filter stores list if number of element is greater than kwarg1. """ if (len(store_list) > kwargs["kwarg1"]): return False return True @staticmethod def filter_between_elem(store_list, **kwargs): """ Filter stores list if number of element is greater or equal kwarg1 and less or equal kwarg2. """ store_len = len(store_list) if (store_len >= kwargs["kwarg1"] and store_len <= kwargs["kwarg2"]): return True return False def generate_sequence(self, store_list): """ This generator yields a filtered set of combinations. :param store_list: The list of stores to be reordered. :type store_list: list of :class:`memoryoperations.Store` :return: Yields a filtered set of combinations. :rtype: iterable """ filter_fun = getattr(self, self._filter, None) for elem in filter( partial(filter_fun, **self._filter_kwargs), chain( *map(lambda x: combinations(store_list, x), range( 0, len(store_list) + 1)))): yield elem class RandomPartialReorderEngine: """ Generates a random sequence of combinations of stores. Example: input: (a, b, c), max_seq = 3 output: ('b', 'c') ('b',) ('a', 'b', 'c') """ def __init__(self, max_seq=3): """ Initializes the generator with the provided parameters. :param max_seq: The number of combinations to be generated. """ self.test_on_barrier = True self._max_seq = max_seq def generate_sequence(self, store_list): """ This generator yields a random sequence of combinations. Number of combinations without replacement has to be limited to 1000 because of exponential growth of elements. Example: for 10 element from 80 -> 1646492110120 combinations for 20 element from 80 -> 3.5353161422122E+18 combinations for 40 element from 80 -> 1.0750720873334E+23 combinations :param store_list: The list of stores to be reordered. :type store_list: list of :class:`memoryoperations.Store` :return: Yields a random sequence of combinations. :rtype: iterable """ population = list(chain(*map( lambda x: islice(combinations(store_list, x), 1000), range(0, len(store_list) + 1)))) population_size = len(population) for elem in sample(population, self._max_seq if self._max_seq <= population_size else population_size): yield elem class NoReorderEngine: def __init__(self): self.test_on_barrier = True """ A NULL reorder engine. Example: input: (a, b, c) output: (a, b, c) """ def generate_sequence(self, store_list): """ This generator does not modify the provided store list. :param store_list: The list of stores to be reordered. :type store_list: The list of :class:`memoryoperations.Store` :return: The unmodified list of stores. :rtype: iterable """ return [store_list] class NoCheckerEngine: def __init__(self): self.test_on_barrier = False """ A NULL reorder engine. Example: input: (a, b, c) output: (a, b, c) """ def generate_sequence(self, store_list): """ This generator does not modify the provided store list and does not do the check. :param store_list: The list of stores to be reordered. :type store_list: The list of :class:`memoryoperations.Store` :return: The unmodified list of stores. :rtype: iterable """ return [store_list] def get_engine(engine): if engine in engines: reorder_engine = engines[engine]() else: raise NotSupportedOperationException( "Not supported reorder engine: {}" .format(engine)) return reorder_engine engines = collections.OrderedDict([ ('NoReorderNoCheck', NoCheckerEngine), ('ReorderFull', FullReorderEngine), ('NoReorderDoCheck', NoReorderEngine), ('ReorderAccumulative', AccumulativeReorderEngine), ('ReorderReverseAccumulative', AccumulativeReverseReorderEngine), ('ReorderPartial', RandomPartialReorderEngine)]) pmdk-1.8/src/tools/pmreorder/binaryoutputhandler.py0000664000000000000000000001773413615011243021430 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import utils from reorderexceptions import InconsistentFileException class BinaryOutputHandler: """ Handle :class:`BinaryFile` objects. Creates and aggregates :class:`BinaryFile` objects for ease of use. Implements methods for batch handling of aggregated files. :ivar _files: A list of registered files, most recent last. :type _files: list """ def __init__(self, checker): """ Binary handler constructor. :param checker: consistency checker object :type checker: ConsistencyCheckerBase """ self._files = [] self._checker = checker def add_file(self, file, map_base, size): """ Create and append a mapped file to :attr:`_files`. :param file: Full path of the mapped file to be added. :type file: str :param map_base: Base address of the mapped file. :type map_base: int :param size: Size of the file. :type size: int :return: None """ self._files.append(BinaryFile(file, map_base, size, self._checker)) def remove_file(self, file): """Remove file from :attr:`_files`. :param file: File to be removed. :type file: str :return: None """ for bf in self._files: if bf.file_name is file: self._files.remove(bf) def do_store(self, store_op): """ Perform a store to the given file. The file is chosen based on the address and size of the store. :param store_op: The store operation to be performed. :type store_op: Store :return: None :raises: Generic exception - to be precised later. """ store_ok = False for bf in self._files: if utils.range_cmp(store_op, bf) == 0: bf.do_store(store_op) store_ok = True if not store_ok: raise OSError( "No suitable file found for store {}" .format(store_op)) def do_revert(self, store_op): """ Reverts a store made to a file. Performing a revert on a store that has not been made previously yields undefined behavior. :param store_op: The store to be reverted. :type store_op: Store :return: None :raises: Generic exception - to be precised later. """ revert_ok = False for bf in self._files: if utils.range_cmp(store_op, bf) == 0: bf.do_revert(store_op) revert_ok = True if not revert_ok: raise OSError( "No suitable file found for store {}" .format(store_op)) def check_consistency(self): """ Checks consistency of each registered file. :return: None :raises: Generic exception - to be precised later. """ for bf in self._files: if not bf.check_consistency(): raise InconsistentFileException( "File {} inconsistent".format(bf)) class BinaryFile(utils.Rangeable): """Binary file handler. It is a handler for binary file operations. Internally it uses mmap to write to and read from the file. :ivar _file_name: Full path of the mapped file. :type _file_name: str :ivar _map_base: Base address of the mapped file. :type _map_base: int :ivar _map_max: Max address of the mapped file. :type _map_max: int :ivar _file_map: Memory mapped from the file. :type _file_map: mmap.mmap :ivar _checker: consistency checker object :type _checker: ConsistencyCheckerBase """ def __init__(self, file_name, map_base, size, checker): """ Initializes the binary file handler. :param file_name: Full path of the mapped file to be added. :type file_name: str :param map_base: Base address of the mapped file. :type map_base: int :param size: Size of the file. :type size: int :param checker: consistency checker object :type checker: ConsistencyCheckerBase :return: None """ self._file_name = file_name self._map_base = map_base self._map_max = map_base + size # TODO consider mmaping only necessary parts on demand self._file_map = utils.memory_map(file_name) self._checker = checker def __str__(self): return self._file_name def do_store(self, store_op): """ Perform the store on the file. The store records the old value for reverting. :param store_op: The store to be performed. :type store_op: Store :return: None """ base_off = store_op.get_base_address() - self._map_base max_off = store_op.get_max_address() - self._map_base # read and save old value store_op.old_value = bytes(self._file_map[base_off:max_off]) # write out the new value self._file_map[base_off:max_off] = store_op.new_value self._file_map.flush(base_off & ~4095, 4096) def do_revert(self, store_op): """ Reverts the store. Write back the old value recorded while doing the store. Reverting a store which has not been made previously has undefined behavior. :param store_op: The store to be reverted. :type store_op: Store :return: None """ base_off = store_op.get_base_address() - self._map_base max_off = store_op.get_max_address() - self._map_base # write out the old value self._file_map[base_off:max_off] = store_op.old_value self._file_map.flush(base_off & ~4095, 4096) def check_consistency(self): """ Check consistency of the file. :return: True if consistent, False otherwise. :rtype: bool """ return self._checker.check_consistency(self._file_name) == 0 def get_base_address(self): """ Returns the base address of the file. Overrides from :class:`utils.Rangeable`. :return: The base address of the mapping passed to the constructor. :rtype: int """ return self._map_base def get_max_address(self): """ Get max address of the file mapping. Overrides from :class:`utils.Rangeable`. :return: The max address of the mapping. :rtype: int """ return self._map_max pmdk-1.8/src/tools/pmreorder/memoryoperations.py0000664000000000000000000003074513615011243020736 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. from utils import Rangeable from utils import range_cmp from utils import StackTrace from sys import byteorder class BaseOperation: """ Base class for all memory operations. """ pass class Fence(BaseOperation): """ Describes a fence operation. The exact type of the memory barrier is not important, it is interpreted as an SFENCE or MFENCE. """ class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Ignored. :type values: str :return: New Fence object. :rtype: Fence """ return Fence() class Store(BaseOperation, Rangeable): """ Describes a store operation. :ivar address: The virtual address at which to store the new value. :type address: int :ivar new_value: The new value to be written. :type new_value: bytearray :ivar size: The size of the store in bytes. :type size: int :ivar old_value: The old value read from the file. :type old_value: bytearray :ivar flushed: Indicates whether the store has been flushed. :type flushed: bool """ def __init__(self, values): """ Initializes the object based on the describing string. :param values: Pre-formatted string describing the store. :type values: str :return: None """ params = values.split(";") # calculate the offset given the registered file mapping self.address = int(params[1], 16) self.size = int(params[3], 16) self.new_value = \ int(params[2], 16).to_bytes(self.size, byteorder=byteorder) if len(params) > 4: self.trace = StackTrace(params[4:]) else: self.trace = StackTrace(["No trace available", ]) self.old_value = None self.flushed = False def __str__(self): return "addr: " + hex(self.address) + " size " + \ str(self.size) + " value " + str(self.new_value) def get_base_address(self): """ Override from :class:`utils.Rangeable`. :return: Virtual address of the store. :rtype: int """ return self.address def get_max_address(self): """ Override from :class:`utils.Rangeable`. :return: Virtual address of the first byte after the store. :rtype: int """ return self.address + self.size class Factory(): """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Pre-formatted string describing the store. :type values: str :return: New Store object. :rtype: Store """ return Store(values) class FlushBase(BaseOperation, Rangeable): """ Base class for flush operations. """ def is_in_flush(self, store_op): """ Check if a given store is within the flush. :param store_op: Store operation to check. :return: True if store is in flush, false otherwise. :rtype: bool """ raise NotImplementedError class Flush(FlushBase): """ Describes a flush operation. Examples of flush instructions are CLFLUSH, CLFLUSHOPT or CLWB. :ivar _address: Virtual address of the flush. :type _address: int :ivar _size: The size of the flush in bytes (should be cache line aligned). :type _size: int """ def __init__(self, values): """ Initializes the object based on the describing string. :param values: Pre-formatted string describing the flush. :type values: str :return: None """ params = values.split(";") self._address = int(params[1], 16) self._size = int(params[2], 16) def is_in_flush(self, store_op): """ Override from :class:`FlushBase`. :param store_op: Store operation to check. :return: True if store is in flush, false otherwise. :rtype: bool """ if range_cmp(store_op, self) == 0: return True else: return False def get_base_address(self): """ Override from :class:`utils.Rangeable`. :return: Virtual address of the flush. :rtype: int """ return self._address def get_max_address(self): """ Override from :class:`utils.Rangeable`. :return: Virtual address of the first byte after the flush. :rtype: int """ return self._address + self._size class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Pre-formatted string describing the flush. :type values: str :return: New Flush object. :rtype: Flush """ return Flush(values) class ReorderBase(BaseOperation): """ Base class for all reorder type classes. """ pass class NoReorderDoCheck(ReorderBase): """ Describes the type of reordering engine to be used. This marker class triggers writing the whole sequence of stores between barriers. """ class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Ignored. :type values: str :return: New NoReorderDoCheck object. :rtype: NoReorderDoCheck """ return NoReorderDoCheck() class ReorderFull(ReorderBase): """ Describes the type of reordering engine to be used. This marker class triggers writing all possible sequences of stores between barriers. """ class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Ignored. :type values: str :return: New ReorderFull object. :rtype: ReorderFull """ return ReorderFull() class ReorderAccumulative(ReorderBase): """ Describes the type of reordering engine to be used. This marker class triggers writing all possible accumulative sequences of stores between barriers. """ class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Ignored. :type values: str :return: New ReorderAccumulative object. :rtype: ReorderAccumulative """ return ReorderAccumulative() class ReorderReverseAccumulative(ReorderBase): """ Describes the type of reordering engine to be used. This marker class triggers writing all possible reverted accumulative sequences of stores between barriers. """ class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Ignored. :type values: str :return: New ReorderReverseAccumulative object. :rtype: ReorderReverseAccumulative """ return ReorderReverseAccumulative() class NoReorderNoCheck(ReorderBase): """ Describes the type of reordering engine to be used. This marker class triggers writing the whole sequence of stores between barriers. It additionally marks that no consistency checking is to be made. """ class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Ignored. :type values: str :return: New NoReorderNoCheck object. :rtype: NoReorderNoCheck """ return NoReorderNoCheck() class ReorderDefault(ReorderBase): """ Describes the default reordering engine to be used. This marker class triggers default reordering. """ class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Ignored. :type values: str :return: ReorderDefault object. :rtype: ReorderDefault """ return ReorderDefault() class ReorderPartial(ReorderBase): """ Describes the type of reordering engine to be used. This marker class triggers writing a subset of all possible sequences of stores between barriers. The type of partial reordering is chosen at runtime. Not yet implemented. """ class Factory: """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Ignored. :type values: str :return: New ReorderPartial object. :rtype: ReorderPartial """ return ReorderPartial() class Register_file(BaseOperation): """ Describes the file to be mapped into processes address space. :ivar name: The full name of the file. :type name: str :ivar address: The base address where the file was mapped. :type address: int :ivar size: The size of the mapping. :type size: int :ivar offset: The start offset of the mapping within the file. :type offset: int """ def __init__(self, values): """ Initializes the object based on the describing string. :param values: Pre-formatted string describing the flush. :type values: str :return: None """ params = values.split(";") self.name = params[1] self.address = int(params[2], 16) self.size = int(params[3], 16) self.offset = int(params[4], 16) class Factory(): """ Internal factory class to be used in dynamic object creation. """ def create(self, values): """ Factory object creation method. :param values: Pre-formatted string describing the file registration. :type values: str :return: New Register_file object. :rtype: Register_file """ return Register_file(values) pmdk-1.8/src/tools/pmreorder/markerparser.py0000664000000000000000000000605013615011243020010 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import os import json class MarkerParser: """ Parse marker config file and command line arg provided by user via -x parameter. """ def marker_file_parser(self, macros): """ Parse markers passed by file. They should be in json format: { "MARKER_NAME"="ENGINE_TYPE" } and separated by commas. """ markers = {} try: with open(macros) as config_file: markers = json.load(config_file) except json.decoder.JSONDecodeError: print("Invalid config macros file format: ", macros, "Use: {\"MARKER_NAME1\"=\"ENGINE_TYPE1\"," "\"MARKER_NAME2\"=\"ENGINE_TYPE2\"}") return markers def marker_cli_parser(self, macros): """ Parse markers passed by cli. They should be in specific format: MARKER_NAME=ENGINE_TYPE and separated by commas. """ try: markers_array = macros.split(",") return dict(pair.split('=') for pair in markers_array) except ValueError: print("Invalid extended macros format: ", macros, "Use: MARKER_NAME1=ENGINE_TYPE1,MARKER_NAME2=ENGINE_TYPE2") def get_markers(self, markerset): """ Parse markers based on their format. """ if markerset is not None: if os.path.exists(markerset): return self.marker_file_parser(markerset) else: return self.marker_cli_parser(markerset) pmdk-1.8/src/tools/pmreorder/loggingfacility.py0000664000000000000000000000621213615011243020465 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import logging log_levels = ["debug", "info", "warning", "error", "critical"] class LoggingBase: def debug(self, text): pass def info(self, text): pass def warning(self, text): pass def error(self, text): pass def critical(self, text): pass class DefaultFileLogger(LoggingBase): def __init__(self, name="pmreorder", **kwargs): logging.basicConfig(**kwargs) self.__logger = logging.getLogger(name) def debug(self, text): self.__logger.debug(text) def info(self, text): self.__logger.info(text) def warning(self, text): self.__logger.warning(text) def error(self, text): self.__logger.error(text) def critical(self, text): self.__logger.critical(text) class DefaultPrintLogger(LoggingBase): def debug(self, text): print("DEBUG:", text) def info(self, text): print("INFO:", text) def warning(self, text): print("WARNING:", text) def error(self, text): print("ERROR:", text) def critical(self, text): print("CRITICAL:", text) def get_logger(log_output, log_level=None): logger = None # check if log_level is valid log_level = "warning" if log_level is None else log_level numeric_level = getattr(logging, log_level.upper()) if not isinstance(numeric_level, int): raise ValueError('Invalid log level: %s'.format(log_level.upper())) if log_output is None: logger = DefaultPrintLogger() else: logger = DefaultFileLogger(filename=log_output, level=numeric_level) return logger pmdk-1.8/src/tools/pmreorder/statemachine.py0000664000000000000000000003374113615011243017766 0ustar rootroot# Copyright 2018-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import memoryoperations as memops import reorderengines from reorderexceptions import InconsistentFileException from reorderexceptions import NotSupportedOperationException class State: """ The base class of all states. :ivar _context: The reordering context. :type _context: opscontext.OpsContext :ivar trans_stores: The list of unflushed stores. :type trans_stores: list of :class:`memoryoperations.Store` """ trans_stores = [] def __init__(self, context): """ Default state constructor. :param context: The context of the reordering. :type context: opscontext.OpsContext """ self._context = context def next(self, in_op): """ Go to the next state based on the input. :Note: The next state might in fact be the same state. :param in_op: The state switch trigger operation. :type in_op: subclass of :class:`memoryoperations.BaseOperation` :return: The next state. :rtype: subclass of :class:`State` """ raise NotImplementedError def run(self, in_op): """ Perform the required operation in this state. :param in_op: The operation to be performed in this state. :type in_op: subclass of :class:`memoryoperations.BaseOperation` :return: None """ raise NotImplementedError class InitState(State): """ The initial no-op state. """ def __init__(self, context): """ Saves the reordering context. :param context: The reordering context. :type context: opscontext.OpsContext """ super(InitState, self).__init__(context) def next(self, in_op): """ Switch to the next valid state. :param in_op: Ignored. :return: The next valid state. :rtype: CollectingState """ return CollectingState(self._context) def run(self, in_op): """ Does nothing. :param in_op: Ignored. :return: always True """ return True class CollectingState(State): """ Collects appropriate operations. This state mostly aggregates stores and flushes. It also validates which stores will be made persistent and passes them on to the next state. :ivar _ops_list: The list of collected stores. :type _ops_list: list of :class:`memoryoperations.Store` :ivar _inner_state: The internal state of operations. :type _inner_state: str """ def __init__(self, context): """ Saves the reordering context. :param context: The reordering context. :type context: opscontext.OpsContext """ super(CollectingState, self).__init__(context) self._ops_list = [] self._ops_list += State.trans_stores self._inner_state = "init" def next(self, in_op): """ Switch to the next valid state. :param in_op: The state switch trigger operation. :type in_op: subclass of :class:`memoryoperations.BaseOperation` :return: The next state. :rtype: subclass of :class:`State` """ if isinstance(in_op, memops.Fence) and \ self._inner_state is "flush": return ReplayingState(self._ops_list, self._context) else: return self def run(self, in_op): """ Perform operations in this state. Based on the type of operation, different handling is employed. The recognized and handled types of operations are: * :class:`memoryoperations.ReorderBase` * :class:`memoryoperations.FlushBase` * :class:`memoryoperations.Store` * :class:`memoryoperations.Register_file` :param in_op: The operation to be performed in this state. :type in_op: subclass of :class:`memoryoperations.BaseOperation` :return: always True """ self.move_inner_state(in_op) if isinstance(in_op, memops.ReorderBase): self.substitute_reorder(in_op) elif isinstance(in_op, memops.FlushBase): self.flush_stores(in_op) elif isinstance(in_op, memops.Store): self._ops_list.append(in_op) elif isinstance(in_op, memops.Register_file): self.reg_file(in_op) return True def substitute_reorder(self, order_ops): """ Changes the reordering engine based on the log marker class. :param order_ops: The reordering marker class. :type order_ops: subclass of :class:`memoryoperations.ReorderBase` :return: None """ if isinstance(order_ops, memops.ReorderFull): self._context.reorder_engine = \ reorderengines.FullReorderEngine() self._context.test_on_barrier = \ self._context.reorder_engine.test_on_barrier elif isinstance(order_ops, memops.ReorderPartial): # TODO add macro in valgrind or # parameter inside the tool to support parameters? self._context.reorder_engine = \ reorderengines.RandomPartialReorderEngine(3) self._context.test_on_barrier = \ self._context.reorder_engine.test_on_barrier elif isinstance(order_ops, memops.ReorderAccumulative): self._context.reorder_engine = \ reorderengines.AccumulativeReorderEngine() self._context.test_on_barrier = \ self._context.reorder_engine.test_on_barrier elif isinstance(order_ops, memops.ReorderReverseAccumulative): self._context.reorder_engine = \ reorderengines.AccumulativeReverseReorderEngine() self._context.test_on_barrier = \ self._context.reorder_engine.test_on_barrier elif isinstance(order_ops, memops.NoReorderDoCheck): self._context.reorder_engine = reorderengines.NoReorderEngine() self._context.test_on_barrier = \ self._context.reorder_engine.test_on_barrier elif isinstance(order_ops, memops.NoReorderNoCheck): self._context.reorder_engine = reorderengines.NoCheckerEngine() self._context.test_on_barrier = \ self._context.reorder_engine.test_on_barrier elif isinstance(order_ops, memops.ReorderDefault): self._context.reorder_engine = self._context.default_engine self._context.test_on_barrier = self._context.default_barrier else: raise NotSupportedOperationException( "Not supported reorder engine: {}" .format(order_ops)) def flush_stores(self, flush_op): """ Marks appropriate stores as flushed. Does not align the flush, the log is expected to have the flushes properly aligned. :param flush_op: The flush operation marker. :type flush_op: subclass of :class:`memoryoperations.FlushBase` :return: None """ for st in self._ops_list: if flush_op.is_in_flush(st): st.flushed = True def reg_file(self, file_op): """ Register a new file mapped into virtual memory. :param file_op: File registration operation marker. :type file_op: memoryoperations.Register_file :return: None """ self._context.file_handler.add_file(file_op.name, file_op.address, file_op.size) def move_inner_state(self, in_op): """ Tracks the internal state of the collection. The collected stores need to be processed only at specific moments - after full persistent memory barriers (flush-fence). :param in_op: The performed operation. :type in_op: subclass of :class:`memoryoperations.BaseOperation` :return: None """ if isinstance(in_op, memops.Store) and \ self._inner_state is "init": self._inner_state = "dirty" elif isinstance(in_op, memops.FlushBase) and \ self._inner_state is "dirty": self._inner_state = "flush" elif isinstance(in_op, memops.Fence) and \ self._inner_state is "flush": self._inner_state = "fence" elif isinstance(in_op, memops.Flush) and \ self._inner_state is "init": self._inner_state = "flush" class ReplayingState(State): """ Replays all collected stores according to the reordering context. :ivar _ops_list: The list of stores to be reordered and replayed. :type _ops_list: list of :class:`memoryoperations.Store` """ def __init__(self, in_ops_list, context): """ :param in_ops_list: :param context: :return: """ super(ReplayingState, self).__init__(context) self._ops_list = in_ops_list def next(self, in_op): """ Switches to the collecting state regardless of the input. :param in_op: Ignored. :type in_op: subclass of :class:`memoryoperations.BaseOperation` :return: The next state. :rtype: CollectingState """ return CollectingState(self._context) def run(self, in_op): """ Perform operations in this state. The replaying state performs reordering and if necessary checks the consistency of the registered files. The decisions and type of reordering to be used is defined by the context. :param in_op: The operation to be performed in this state. :type in_op: subclass of :class:`memoryoperations.BaseOperation` :return: State of consistency check. """ # specifies consistency state of sequence consistency = True # consider only flushed stores flushed_stores = list(filter(lambda x: x.flushed, self._ops_list)) # already flushed stores should be removed from the transitive list common_part = list(set(flushed_stores) & set(State.trans_stores)) State.trans_stores = list(set(State.trans_stores) - set(common_part)) # do not add redundant stores State.trans_stores = list(set(State.trans_stores) | set(list(filter(lambda x: x.flushed is False, self._ops_list)))) if self._context.test_on_barrier: for seq in self._context.reorder_engine.generate_sequence( flushed_stores): for op in seq: # do stores self._context.file_handler.do_store(op) # check consistency of all files try: self._context.file_handler.check_consistency() except InconsistentFileException as e: consistency = False self._context.logger.warning(e) stacktrace = "Call trace:\n" for num, op in enumerate(seq): stacktrace += "Store [{}]:\n".format(num) stacktrace += str(op.trace) self._context.logger.warning(stacktrace) for op in reversed(seq): # revert the changes self._context.file_handler.do_revert(op) # write all flushed stores for op in flushed_stores: self._context.file_handler.do_store(op) return consistency class StateMachine: """ The state machine driver. :ivar _curr_state: The current state. :type _curr_state: subclass of :class:`State` """ def __init__(self, init_state): """ Initialize the state machine with a specified state. :param init_state: The initial state to be used. :type init_state: subclass of :class:`State` """ self._curr_state = init_state def run_all(self, operations): """ Starts the state machine. :param operations: The operations to be performed by the state machine. :type operations: list of :class:`memoryoperations.BaseOperation` :return: None """ all_consistent = True for ops in operations: self._curr_state = self._curr_state.next(ops) check = self._curr_state.run(ops) if check is False: all_consistent = check return all_consistent pmdk-1.8/src/tools/pmreorder/.gitignore0000664000000000000000000000003413615011243016724 0ustar rootroot__pycache__ *.pyc pmreorder pmdk-1.8/src/tools/pmreorder/operationfactory.py0000664000000000000000000001524013615011243020703 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import memoryoperations from reorderexceptions import NotSupportedOperationException class OperationFactory: """ An abstract memory operation factory. This object factory puts special constraints on names of classes. It creates objects based on log in string format, as such the classes have to start with a capital letter and the rest of the name has to be in lowercase. For example:: STORE -> Store FULL_REORDER -> Full_reorder The object to be created has to have and internal **Factory** class with a :func:`create` method taking a string parameter. For example see :class:`memoryoperations.Store`. :cvar __factories: The registered object factories. :type __factories: dict """ __factories = {} __suffix = ['.BEGIN', '.END'] memoryoperations.BaseOperation() @staticmethod def add_factory(id_, operation_factory): """ Explicitly register an object factory. This method should be used when the factory cannot be inferred from the name of the object to be created. :param id_: The id under which this factory is to be registered in the dictionary. :type id_: str :param operation_factory: The operation factory to be registered. :return: None """ OperationFactory.__factories[id_] = operation_factory @staticmethod def create_operation(string_operation, markers, stack): def check_marker_format(marker): """ Checks if marker has proper suffix. """ for s in OperationFactory.__suffix: if marker.endswith(s): return raise NotSupportedOperationException( "Incorrect marker format {}, suffix is missing." .format(marker)) def check_pair_consistency(stack, marker): """ Checks if markers do not cross. You can pop from stack only if end marker match previous one. Example OK: MACRO1.BEGIN MACRO2.BEGIN MACRO2.END MACRO1.END Example NOT OK: MACRO1.BEGIN MACRO2.BEGIN MACRO1.END MACRO2.END """ top = stack[-1][0] if top.endswith(OperationFactory.__suffix[0]): top = top[:-len(OperationFactory.__suffix[0])] if marker.endswith(OperationFactory.__suffix[-1]): marker = marker[:-len(OperationFactory.__suffix[-1])] if top != marker: raise NotSupportedOperationException( "Cannot cross markers: {0}, {1}" .format(top, marker)) """ Creates the object based on the pre-formatted string. The string needs to be in the specific format. Each specific value in the string has to be separated with a `;`. The first field has to be the name of the operation, the rest are operation specific values. :param string_operation: The string describing the operation. :param markers: The dict describing the pair marker-engine. :param stack: The stack describing the order of engine changes. :return: The specific object instantiated based on the string. """ id_ = string_operation.split(";")[0] id_case_sensitive = id_.lower().capitalize() # checks if id_ is one of memoryoperation classes mem_ops = getattr(memoryoperations, id_case_sensitive, None) # if class is not one of memoryoperations # it means it can be user defined marker if mem_ops is None: check_marker_format(id_) # if id_ is section BEGIN if id_.endswith(OperationFactory.__suffix[0]): # BEGIN defined by user marker_name = id_.partition('.')[0] if markers is not None and marker_name in markers: engine = markers[marker_name] try: mem_ops = getattr(memoryoperations, engine) except AttributeError: raise NotSupportedOperationException( "Not supported reorder engine: {}" .format(engine)) # BEGIN but not defined by user else: mem_ops = stack[-1][1] if issubclass(mem_ops, memoryoperations.ReorderBase): stack.append((id_, mem_ops)) # END section elif id_.endswith(OperationFactory.__suffix[-1]): check_pair_consistency(stack, id_) stack.pop() mem_ops = stack[-1][1] # here we have proper memory operation to perform, # it can be Store, Fence, ReorderDefault etc. id_ = mem_ops.__name__ if id_ not in OperationFactory.__factories: OperationFactory.__factories[id_] = mem_ops.Factory() return OperationFactory.__factories[id_].create(string_operation) pmdk-1.8/src/tools/pmreorder/opscontext.py0000664000000000000000000000774013615011243017527 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. from operationfactory import OperationFactory from binaryoutputhandler import BinaryOutputHandler import reorderengines import memoryoperations from itertools import repeat class OpsContext: """ Holds the context of the performed operations. :ivar _operations: The operations to be performed, based on the log file. :type _operations: list of strings :ivar reorder_engine: The reordering engine used at the moment. :type one of the reorderengine Class :ivar default_engine: The default reordering engine. :type default_engine: One of the reorderengines Class :ivar test_on_barrier: Check consistency on barrier. :type test_on_barrier: bool :ivar default_barrier: Default consistency barrier status. :type default_barrier: bool :ivar file_handler: The file handler used. """ def __init__(self, log_file, checker, logger, arg_engine, markers): """ Splits the operations in the log file and sets the instance variables to default values. :param log_file: The full name of the log file. :type log_file: str :return: None """ # TODO reading the whole file at once is rather naive # change in the future self._operations = open(log_file).read().split("|") engine = reorderengines.get_engine(arg_engine) self.reorder_engine = engine self.test_on_barrier = engine.test_on_barrier self.default_engine = self.reorder_engine self.default_barrier = self.default_engine.test_on_barrier self.file_handler = BinaryOutputHandler(checker) self.checker = checker self.logger = logger self.markers = markers self.stack_engines = [('START', getattr(memoryoperations, arg_engine))] # TODO this should probably be made a generator def extract_operations(self): """ Creates specific operation objects based on the labels available in the split log file. :return: list of subclasses of :class:`memoryoperations.BaseOperation` """ stop_index = start_index = 0 for i, elem in enumerate(self._operations): if "START" in elem: start_index = i elif "STOP" in elem: stop_index = i return list(map(OperationFactory.create_operation, self._operations[start_index + 1:stop_index], repeat(self.markers), repeat(self.stack_engines))) pmdk-1.8/src/tools/pmreorder/pmreorder.py0000664000000000000000000001165213615011243017315 0ustar rootroot# # Copyright 2018-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # import argparse import statemachine import opscontext import consistencycheckwrap import loggingfacility import markerparser import sys import reorderengines def main(): pmreorder_version = "unknown" ''' Argv[1] should be given in order to use -v or --version flag. It is passed from the installed script. We check whether argv[1] was given and if it's not any of regular parameters we use it as a version of pmreorder and remove it from the arguments list. ''' if len(sys.argv) > 1 and sys.argv[1][0] != "-": pmreorder_version = sys.argv[1] del sys.argv[1] # TODO unicode support # TODO parameterize reorder engine type parser = argparse.ArgumentParser(description="Store reordering tool") parser.add_argument("-l", "--logfile", required=True, help="the pmemcheck log file to process") parser.add_argument("-c", "--checker", choices=consistencycheckwrap.checkers, default=consistencycheckwrap.checkers[0], help="choose consistency checker type") parser.add_argument("-p", "--path", required=True, help="path to the consistency checker and arguments", nargs='+') parser.add_argument("-n", "--name", help="consistency check function " + "for the 'lib' checker") parser.add_argument("-o", "--output", help="set the logger output file") parser.add_argument("-e", "--output-level", choices=loggingfacility.log_levels, help="set the output log level") parser.add_argument("-x", "--extended-macros", help="list of pairs MARKER=ENGINE or " + "json config file") parser.add_argument("-v", "--version", help="print version of the pmreorder", action="version", version="%(prog)s " + pmreorder_version) engines_keys = list(reorderengines.engines.keys()) parser.add_argument("-r", "--default-engine", help="set default reorder engine " + "default=NoReorderNoChecker", choices=engines_keys, default=engines_keys[0]) args = parser.parse_args() logger = loggingfacility.get_logger( args.output, args.output_level) checker = consistencycheckwrap.get_checker( args.checker, ' '.join(args.path), args.name) markers = markerparser.MarkerParser().get_markers(args.extended_macros) # create the script context context = opscontext.OpsContext( args.logfile, checker, logger, args.default_engine, markers) # init and run the state machine a = statemachine.StateMachine(statemachine.InitState(context)) if a.run_all(context.extract_operations()) is False: sys.exit(1) if __name__ == "__main__": main() pmdk-1.8/src/tools/pmreorder/reorderexceptions.py0000664000000000000000000000321313615011243021054 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. class InconsistentFileException(Exception): pass class NotSupportedOperationException(Exception): pass pmdk-1.8/src/tools/pmreorder/utils.py0000664000000000000000000000760313615011243016457 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. import os import mmap class Rangeable: """ Interface for all rangeable objects. All rangeable objects must be able to return their base and max addresses. """ def get_base_address(self): """ Getter for the base address of the object. :return: The base address of the object. :rtype: int """ raise NotImplementedError def get_max_address(self): """ Getter for the max address of the object. :return: The max address of the object. :rtype: int """ raise NotImplementedError class StackTrace: def __init__(self, trace=None): self.trace = trace def __str__(self): ret = "" if self.trace is not None: for line in self.trace: ret += " by\t{}\n".format(line) return ret def memory_map(filename, size=0, access=mmap.ACCESS_WRITE, offset=0): """ Memory map a file. :Warning: `offset` has to be a non-negative multiple of PAGESIZE or ALLOCATIONGRANULARITY :param filename: The file to be mapped. :type filename: str :param size: Number of bytes to be mapped. If is equal 0, the whole file at the moment of the call will be mapped. :type size: int :param offset: The offset within the file to be mapped. :type offset: int :param access: The type of access provided to mmap. :return: The mapped file. :rtype: mmap.mmap """ fd = os.open(filename, os.O_RDWR) m_file = mmap.mmap(fd, size, access=access, offset=offset) os.close(fd) return m_file def range_cmp(lhs, rhs): """ A range compare function. :param lhs: The left hand side of the comparison. :type lhs: Rangeable :param rhs: The right hand side of the comparison. :type rhs: Rangeable :return: -1 if lhs is before rhs, 1 when after and 0 on overlap. :rtype: int The comparison function may be explained as:: Will return -1: |___lhs___| |___rhs___| Will return +1: |___rhs___| |___lhs___| Will return 0: |___lhs___| |___rhs___| """ if lhs.get_max_address() <= rhs.get_base_address(): return -1 elif lhs.get_base_address() >= rhs.get_max_address(): return 1 else: return 0 pmdk-1.8/src/tools/pmreorder/Makefile0000664000000000000000000000335513615011243016405 0ustar rootroot# Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # Makefile -- Makefile for pmreorder # include ../Makefile.inc FLAKE8 := $(shell flake8 --version 2>/dev/null) cstyle: ifdef FLAKE8 flake8 . else @echo "Flake8 not found. Python files check skipped." endif pmdk-1.8/src/tools/Makefile0000664000000000000000000000467713615011243014416 0ustar rootroot# Copyright 2014-2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # Makefile -- top Makefile for tools # TOP = ../.. TESTCONFIG=$(TOP)/src/test/testconfig.sh TARGETS = pmempool rpmemd daxio pmreorder SCOPEDIRS=$(TARGETS) SCOPEFILES=$(foreach dir, $(SCOPEDIRS), $(shell find $(dir) -name *.[ch] )) all : TARGET = all check : TARGET = check test : TARGET = test clean : TARGET = clean clobber: TARGET = clobber cstyle : TARGET = cstyle format : TARGET = format install: TARGET = install uninstall: TARGET = uninstall sync-remotes: TARGET = sync-remotes sparse: TARGET = sparse all clean clobber cstyle install uninstall check format test sparse: $(TARGETS) $(TESTCONFIG): sync-remotes: $(TARGETS) $(TESTCONFIG) $(TARGETS): $(MAKE) -C $@ $(TARGET) clean: $(RM) TAGS cscope.in.out cscope.out cscope.po.out clobber: clean cscope: cscope -q -b $(SCOPEFILES) ctags -e $(SCOPEFILES) .PHONY: all clean clobber cstyle format install uninstall common cscope sync-remotes $(TARGETS) pmdk-1.8/src/common/0000775000000000000000000000000013615011243013070 5ustar rootrootpmdk-1.8/src/common/alloc.h0000664000000000000000000000512113615011243014332 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef COMMON_ALLOC_H #define COMMON_ALLOC_H #include #ifdef __cplusplus extern "C" { #endif typedef void *(*Malloc_func)(size_t size); typedef void *(*Realloc_func)(void *ptr, size_t size); extern Malloc_func fn_malloc; extern Realloc_func fn_realloc; #if FAULT_INJECTION void *_flt_Malloc(size_t, const char *); void *_flt_Realloc(void *, size_t, const char *); #define Malloc(size) _flt_Malloc(size, __func__) #define Realloc(ptr, size) _flt_Realloc(ptr, size, __func__) #else void *_Malloc(size_t); void *_Realloc(void *, size_t); #define Malloc(size) _Malloc(size) #define Realloc(ptr, size) _Realloc(ptr, size) #endif void set_func_malloc(void *(*malloc_func)(size_t size)); void set_func_realloc(void *(*realloc_func)(void *ptr, size_t size)); /* * overridable names for malloc & friends used by this library */ typedef void (*Free_func)(void *ptr); typedef char *(*Strdup_func)(const char *s); extern Free_func Free; extern Strdup_func Strdup; extern void *Zalloc(size_t sz); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/os_dimm_none.c0000664000000000000000000000535113615011243015706 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_dimm_none.c -- fake dimm functions */ #include "out.h" #include "os.h" #include "os_dimm.h" /* * os_dimm_uid -- returns empty uid */ int os_dimm_uid(const char *path, char *uid, size_t *len) { LOG(3, "path %s, uid %p, len %lu", path, uid, *len); if (uid == NULL) { *len = 1; } else { *uid = '\0'; } return 0; } /* * os_dimm_usc -- returns fake unsafe shutdown count */ int os_dimm_usc(const char *path, uint64_t *usc) { LOG(3, "path %s, usc %p", path, usc); *usc = 0; return 0; } /* * os_dimm_files_namespace_badblocks -- fake os_dimm_files_namespace_badblocks() */ int os_dimm_files_namespace_badblocks(const char *path, struct badblocks *bbs) { LOG(3, "path %s", path); os_stat_t st; if (os_stat(path, &st)) { ERR("!stat %s", path); return -1; } return 0; } /* * os_dimm_devdax_clear_badblocks -- fake bad block clearing routine */ int os_dimm_devdax_clear_badblocks(const char *path, struct badblocks *bbs) { LOG(3, "path %s badblocks %p", path, bbs); return 0; } /* * os_dimm_devdax_clear_badblocks_all -- fake bad block clearing routine */ int os_dimm_devdax_clear_badblocks_all(const char *path) { LOG(3, "path %s", path); return 0; } pmdk-1.8/src/common/os_thread_posix.c0000664000000000000000000002472113615011243016434 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_thread_posix.c -- Posix thread abstraction layer */ #define _GNU_SOURCE #include #ifdef __FreeBSD__ #include #endif #include #include "os_thread.h" #include "util.h" typedef struct { pthread_t thread; } internal_os_thread_t; /* * os_once -- pthread_once abstraction layer */ int os_once(os_once_t *o, void (*func)(void)) { COMPILE_ERROR_ON(sizeof(os_once_t) < sizeof(pthread_once_t)); return pthread_once((pthread_once_t *)o, func); } /* * os_tls_key_create -- pthread_key_create abstraction layer */ int os_tls_key_create(os_tls_key_t *key, void (*destructor)(void *)) { COMPILE_ERROR_ON(sizeof(os_tls_key_t) < sizeof(pthread_key_t)); return pthread_key_create((pthread_key_t *)key, destructor); } /* * os_tls_key_delete -- pthread_key_delete abstraction layer */ int os_tls_key_delete(os_tls_key_t key) { return pthread_key_delete((pthread_key_t)key); } /* * os_tls_setspecific -- pthread_key_setspecific abstraction layer */ int os_tls_set(os_tls_key_t key, const void *value) { return pthread_setspecific((pthread_key_t)key, value); } /* * os_tls_get -- pthread_key_getspecific abstraction layer */ void * os_tls_get(os_tls_key_t key) { return pthread_getspecific((pthread_key_t)key); } /* * os_mutex_init -- pthread_mutex_init abstraction layer */ int os_mutex_init(os_mutex_t *__restrict mutex) { COMPILE_ERROR_ON(sizeof(os_mutex_t) < sizeof(pthread_mutex_t)); return pthread_mutex_init((pthread_mutex_t *)mutex, NULL); } /* * os_mutex_destroy -- pthread_mutex_destroy abstraction layer */ int os_mutex_destroy(os_mutex_t *__restrict mutex) { return pthread_mutex_destroy((pthread_mutex_t *)mutex); } /* * os_mutex_lock -- pthread_mutex_lock abstraction layer */ int os_mutex_lock(os_mutex_t *__restrict mutex) { return pthread_mutex_lock((pthread_mutex_t *)mutex); } /* * os_mutex_trylock -- pthread_mutex_trylock abstraction layer */ int os_mutex_trylock(os_mutex_t *__restrict mutex) { return pthread_mutex_trylock((pthread_mutex_t *)mutex); } /* * os_mutex_unlock -- pthread_mutex_unlock abstraction layer */ int os_mutex_unlock(os_mutex_t *__restrict mutex) { return pthread_mutex_unlock((pthread_mutex_t *)mutex); } /* * os_mutex_timedlock -- pthread_mutex_timedlock abstraction layer */ int os_mutex_timedlock(os_mutex_t *__restrict mutex, const struct timespec *abstime) { return pthread_mutex_timedlock((pthread_mutex_t *)mutex, abstime); } /* * os_rwlock_init -- pthread_rwlock_init abstraction layer */ int os_rwlock_init(os_rwlock_t *__restrict rwlock) { COMPILE_ERROR_ON(sizeof(os_rwlock_t) < sizeof(pthread_rwlock_t)); return pthread_rwlock_init((pthread_rwlock_t *)rwlock, NULL); } /* * os_rwlock_destroy -- pthread_rwlock_destroy abstraction layer */ int os_rwlock_destroy(os_rwlock_t *__restrict rwlock) { return pthread_rwlock_destroy((pthread_rwlock_t *)rwlock); } /* * os_rwlock_rdlock - pthread_rwlock_rdlock abstraction layer */ int os_rwlock_rdlock(os_rwlock_t *__restrict rwlock) { return pthread_rwlock_rdlock((pthread_rwlock_t *)rwlock); } /* * os_rwlock_wrlock -- pthread_rwlock_wrlock abstraction layer */ int os_rwlock_wrlock(os_rwlock_t *__restrict rwlock) { return pthread_rwlock_wrlock((pthread_rwlock_t *)rwlock); } /* * os_rwlock_unlock -- pthread_rwlock_unlock abstraction layer */ int os_rwlock_unlock(os_rwlock_t *__restrict rwlock) { return pthread_rwlock_unlock((pthread_rwlock_t *)rwlock); } /* * os_rwlock_tryrdlock -- pthread_rwlock_tryrdlock abstraction layer */ int os_rwlock_tryrdlock(os_rwlock_t *__restrict rwlock) { return pthread_rwlock_tryrdlock((pthread_rwlock_t *)rwlock); } /* * os_rwlock_tryrwlock -- pthread_rwlock_trywrlock abstraction layer */ int os_rwlock_trywrlock(os_rwlock_t *__restrict rwlock) { return pthread_rwlock_trywrlock((pthread_rwlock_t *)rwlock); } /* * os_rwlock_timedrdlock -- pthread_rwlock_timedrdlock abstraction layer */ int os_rwlock_timedrdlock(os_rwlock_t *__restrict rwlock, const struct timespec *abstime) { return pthread_rwlock_timedrdlock((pthread_rwlock_t *)rwlock, abstime); } /* * os_rwlock_timedwrlock -- pthread_rwlock_timedwrlock abstraction layer */ int os_rwlock_timedwrlock(os_rwlock_t *__restrict rwlock, const struct timespec *abstime) { return pthread_rwlock_timedwrlock((pthread_rwlock_t *)rwlock, abstime); } /* * os_spin_init -- pthread_spin_init abstraction layer */ int os_spin_init(os_spinlock_t *lock, int pshared) { COMPILE_ERROR_ON(sizeof(os_spinlock_t) < sizeof(pthread_spinlock_t)); return pthread_spin_init((pthread_spinlock_t *)lock, pshared); } /* * os_spin_destroy -- pthread_spin_destroy abstraction layer */ int os_spin_destroy(os_spinlock_t *lock) { return pthread_spin_destroy((pthread_spinlock_t *)lock); } /* * os_spin_lock -- pthread_spin_lock abstraction layer */ int os_spin_lock(os_spinlock_t *lock) { return pthread_spin_lock((pthread_spinlock_t *)lock); } /* * os_spin_unlock -- pthread_spin_unlock abstraction layer */ int os_spin_unlock(os_spinlock_t *lock) { return pthread_spin_unlock((pthread_spinlock_t *)lock); } /* * os_spin_trylock -- pthread_spin_trylock abstraction layer */ int os_spin_trylock(os_spinlock_t *lock) { return pthread_spin_trylock((pthread_spinlock_t *)lock); } /* * os_cond_init -- pthread_cond_init abstraction layer */ int os_cond_init(os_cond_t *__restrict cond) { COMPILE_ERROR_ON(sizeof(os_cond_t) < sizeof(pthread_cond_t)); return pthread_cond_init((pthread_cond_t *)cond, NULL); } /* * os_cond_destroy -- pthread_cond_destroy abstraction layer */ int os_cond_destroy(os_cond_t *__restrict cond) { return pthread_cond_destroy((pthread_cond_t *)cond); } /* * os_cond_broadcast -- pthread_cond_broadcast abstraction layer */ int os_cond_broadcast(os_cond_t *__restrict cond) { return pthread_cond_broadcast((pthread_cond_t *)cond); } /* * os_cond_signal -- pthread_cond_signal abstraction layer */ int os_cond_signal(os_cond_t *__restrict cond) { return pthread_cond_signal((pthread_cond_t *)cond); } /* * os_cond_timedwait -- pthread_cond_timedwait abstraction layer */ int os_cond_timedwait(os_cond_t *__restrict cond, os_mutex_t *__restrict mutex, const struct timespec *abstime) { return pthread_cond_timedwait((pthread_cond_t *)cond, (pthread_mutex_t *)mutex, abstime); } /* * os_cond_wait -- pthread_cond_wait abstraction layer */ int os_cond_wait(os_cond_t *__restrict cond, os_mutex_t *__restrict mutex) { return pthread_cond_wait((pthread_cond_t *)cond, (pthread_mutex_t *)mutex); } /* * os_thread_create -- pthread_create abstraction layer */ int os_thread_create(os_thread_t *thread, const os_thread_attr_t *attr, void *(*start_routine)(void *), void *arg) { COMPILE_ERROR_ON(sizeof(os_thread_t) < sizeof(internal_os_thread_t)); internal_os_thread_t *thread_info = (internal_os_thread_t *)thread; return pthread_create(&thread_info->thread, (pthread_attr_t *)attr, start_routine, arg); } /* * os_thread_join -- pthread_join abstraction layer */ int os_thread_join(os_thread_t *thread, void **result) { internal_os_thread_t *thread_info = (internal_os_thread_t *)thread; return pthread_join(thread_info->thread, result); } /* * os_thread_self -- pthread_self abstraction layer */ void os_thread_self(os_thread_t *thread) { internal_os_thread_t *thread_info = (internal_os_thread_t *)thread; thread_info->thread = pthread_self(); } /* * os_thread_atfork -- pthread_atfork abstraction layer */ int os_thread_atfork(void (*prepare)(void), void (*parent)(void), void (*child)(void)) { return pthread_atfork(prepare, parent, child); } /* * os_thread_setaffinity_np -- pthread_atfork abstraction layer */ int os_thread_setaffinity_np(os_thread_t *thread, size_t set_size, const os_cpu_set_t *set) { COMPILE_ERROR_ON(sizeof(os_cpu_set_t) < sizeof(cpu_set_t)); internal_os_thread_t *thread_info = (internal_os_thread_t *)thread; return pthread_setaffinity_np(thread_info->thread, set_size, (cpu_set_t *)set); } /* * os_cpu_zero -- CP_ZERO abstraction layer */ void os_cpu_zero(os_cpu_set_t *set) { CPU_ZERO((cpu_set_t *)set); } /* * os_cpu_set -- CP_SET abstraction layer */ void os_cpu_set(size_t cpu, os_cpu_set_t *set) { CPU_SET(cpu, (cpu_set_t *)set); } /* * os_semaphore_init -- initializes semaphore instance */ int os_semaphore_init(os_semaphore_t *sem, unsigned value) { COMPILE_ERROR_ON(sizeof(os_semaphore_t) < sizeof(sem_t)); return sem_init((sem_t *)sem, 0, value); } /* * os_semaphore_destroy -- destroys a semaphore instance */ int os_semaphore_destroy(os_semaphore_t *sem) { return sem_destroy((sem_t *)sem); } /* * os_semaphore_wait -- decreases the value of the semaphore */ int os_semaphore_wait(os_semaphore_t *sem) { return sem_wait((sem_t *)sem); } /* * os_semaphore_trywait -- tries to decrease the value of the semaphore */ int os_semaphore_trywait(os_semaphore_t *sem) { return sem_trywait((sem_t *)sem); } /* * os_semaphore_post -- increases the value of the semaphore */ int os_semaphore_post(os_semaphore_t *sem) { return sem_post((sem_t *)sem); } pmdk-1.8/src/common/uuid.c0000664000000000000000000000640513615011243014207 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * uuid.c -- uuid utilities */ #include #include #include #include "uuid.h" #include "out.h" /* * util_uuid_to_string -- generate a string form of the uuid */ int util_uuid_to_string(const uuid_t u, char *buf) { int len; /* size that is returned from sprintf call */ if (buf == NULL) { LOG(2, "invalid buffer for uuid string"); return -1; } if (u == NULL) { LOG(2, "invalid uuid structure"); return -1; } struct uuid *uuid = (struct uuid *)u; len = snprintf(buf, POOL_HDR_UUID_STR_LEN, "%08x-%04x-%04x-%02x%02x-%02x%02x%02x%02x%02x%02x", uuid->time_low, uuid->time_mid, uuid->time_hi_and_ver, uuid->clock_seq_hi, uuid->clock_seq_low, uuid->node[0], uuid->node[1], uuid->node[2], uuid->node[3], uuid->node[4], uuid->node[5]); if (len != POOL_HDR_UUID_STR_LEN - 1) { LOG(2, "snprintf(uuid): %d", len); return -1; } return 0; } /* * util_uuid_from_string -- generate a binary form of the uuid * * uuid string read from /proc/sys/kernel/random/uuid. UUID string * format example: * f81d4fae-7dec-11d0-a765-00a0c91e6bf6 */ int util_uuid_from_string(const char *uuid, struct uuid *ud) { if (strlen(uuid) != 36) { LOG(2, "invalid uuid string"); return -1; } if (uuid[8] != '-' || uuid[13] != '-' || uuid[18] != '-' || uuid[23] != '-') { LOG(2, "invalid uuid string"); return -1; } int n = sscanf(uuid, "%08x-%04hx-%04hx-%02hhx%02hhx-" "%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx", &ud->time_low, &ud->time_mid, &ud->time_hi_and_ver, &ud->clock_seq_hi, &ud->clock_seq_low, &ud->node[0], &ud->node[1], &ud->node[2], &ud->node[3], &ud->node[4], &ud->node[5]); if (n != 11) { LOG(2, "sscanf(uuid)"); return -1; } return 0; } pmdk-1.8/src/common/util_posix.c0000664000000000000000000000762413615011243015444 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * util_posix.c -- Abstraction layer for misc utilities (Posix implementation) */ #include #include #include #include #include #include "os.h" #include "out.h" #include "util.h" /* pass through for Posix */ void util_strerror(int errnum, char *buff, size_t bufflen) { strerror_r(errnum, buff, bufflen); } /* * util_strwinerror -- should never be called on posix OS - abort() */ void util_strwinerror(char *buff, size_t bufflen) { abort(); } /* * util_part_realpath -- get canonicalized absolute pathname * * As paths used in a poolset file have to be absolute (checked when parsing * a poolset file), here we only have to resolve symlinks. */ char * util_part_realpath(const char *path) { return realpath(path, NULL); } /* * util_compare_file_inodes -- compare device and inodes of two files; * this resolves hard links */ int util_compare_file_inodes(const char *path1, const char *path2) { struct stat sb1, sb2; if (os_stat(path1, &sb1)) { if (errno != ENOENT) { ERR("!stat failed for %s", path1); return -1; } LOG(1, "stat failed for %s", path1); errno = 0; return strcmp(path1, path2) != 0; } if (os_stat(path2, &sb2)) { if (errno != ENOENT) { ERR("!stat failed for %s", path2); return -1; } LOG(1, "stat failed for %s", path2); errno = 0; return strcmp(path1, path2) != 0; } return sb1.st_dev != sb2.st_dev || sb1.st_ino != sb2.st_ino; } /* * util_aligned_malloc -- allocate aligned memory */ void * util_aligned_malloc(size_t alignment, size_t size) { void *retval = NULL; errno = posix_memalign(&retval, alignment, size); return retval; } /* * util_aligned_free -- free allocated memory in util_aligned_malloc */ void util_aligned_free(void *ptr) { free(ptr); } /* * util_getexecname -- return name of current executable */ char * util_getexecname(char *path, size_t pathlen) { ASSERT(pathlen != 0); ssize_t cc; #ifdef __FreeBSD__ #include #include int mib[4] = {CTL_KERN, KERN_PROC, KERN_PROC_PATHNAME, -1}; cc = (sysctl(mib, 4, path, &pathlen, NULL, 0) == -1) ? -1 : (ssize_t)pathlen; #else cc = readlink("/proc/self/exe", path, pathlen); #endif if (cc == -1) { strncpy(path, "unknown", pathlen); path[pathlen - 1] = '\0'; } else { path[cc] = '\0'; } return path; } pmdk-1.8/src/common/vec.h0000664000000000000000000001131713615011243014021 0ustar rootroot/* * Copyright 2017-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * vec.h -- vector interface */ #ifndef PMDK_VEC_H #define PMDK_VEC_H 1 #include #include "valgrind_internal.h" #include "util.h" #include "out.h" #include "alloc.h" #ifdef __cplusplus extern "C" { #endif #define VEC_INIT_SIZE (64) #define VEC(name, type)\ struct name {\ type *buffer;\ size_t size;\ size_t capacity;\ } #define VEC_INITIALIZER {NULL, 0, 0} #define VEC_INIT(vec) do {\ (vec)->buffer = NULL;\ (vec)->size = 0;\ (vec)->capacity = 0;\ } while (0) #define VEC_MOVE(vecl, vecr) do {\ Free((vecl)->buffer);\ (vecl)->buffer = (vecr)->buffer;\ (vecl)->size = (vecr)->size;\ (vecl)->capacity = (vecr)->capacity;\ (vecr)->buffer = NULL;\ (vecr)->size = 0;\ (vecr)->capacity = 0;\ } while (0) #define VEC_REINIT(vec) do {\ VALGRIND_ANNOTATE_NEW_MEMORY((vec), sizeof(*vec));\ VALGRIND_ANNOTATE_NEW_MEMORY((vec)->buffer,\ (sizeof(*(vec)->buffer) * ((vec)->capacity)));\ (vec)->size = 0;\ } while (0) static inline int vec_reserve(void *vec, size_t ncapacity, size_t s) { size_t ncap = ncapacity == 0 ? VEC_INIT_SIZE : ncapacity; VEC(vvec, void) *vecp = (struct vvec *)vec; void *tbuf = Realloc(vecp->buffer, s * ncap); if (tbuf == NULL) { ERR("!Realloc"); return -1; } vecp->buffer = tbuf; vecp->capacity = ncap; return 0; } #define VEC_RESERVE(vec, ncapacity)\ (((vec)->size == 0 || (ncapacity) > (vec)->size) ?\ vec_reserve((void *)vec, ncapacity, sizeof(*(vec)->buffer)) :\ 0) #define VEC_POP_BACK(vec) do {\ (vec)->size -= 1;\ } while (0) #define VEC_FRONT(vec)\ (vec)->buffer[0] #define VEC_BACK(vec)\ (vec)->buffer[(vec)->size - 1] #define VEC_ERASE_BY_POS(vec, pos) do {\ if ((pos) != ((vec)->size - 1))\ (vec)->buffer[(pos)] = VEC_BACK(vec);\ VEC_POP_BACK(vec);\ } while (0) #define VEC_ERASE_BY_PTR(vec, element) do {\ if ((element) != &VEC_BACK(vec))\ *(element) = VEC_BACK(vec);\ VEC_POP_BACK(vec);\ } while (0) #define VEC_INSERT(vec, element)\ ((vec)->buffer[(vec)->size - 1] = (element), 0) #define VEC_INC_SIZE(vec)\ (((vec)->size++), 0) #define VEC_INC_BACK(vec)\ ((vec)->capacity == (vec)->size ?\ (VEC_RESERVE((vec), ((vec)->capacity * 2)) == 0 ?\ VEC_INC_SIZE(vec) : -1) :\ VEC_INC_SIZE(vec)) #define VEC_PUSH_BACK(vec, element)\ (VEC_INC_BACK(vec) == 0? VEC_INSERT(vec, element) : -1) #define VEC_FOREACH(el, vec)\ for (size_t _vec_i = 0;\ _vec_i < (vec)->size && (((el) = (vec)->buffer[_vec_i]), 1);\ ++_vec_i) #define VEC_FOREACH_REVERSE(el, vec)\ for (size_t _vec_i = ((vec)->size);\ _vec_i != 0 && (((el) = (vec)->buffer[_vec_i - 1]), 1);\ --_vec_i) #define VEC_FOREACH_BY_POS(elpos, vec)\ for ((elpos) = 0; (elpos) < (vec)->size; ++(elpos)) #define VEC_FOREACH_BY_PTR(el, vec)\ for (size_t _vec_i = 0;\ _vec_i < (vec)->size && (((el) = &(vec)->buffer[_vec_i]), 1);\ ++_vec_i) #define VEC_SIZE(vec)\ ((vec)->size) #define VEC_CAPACITY(vec)\ ((vec)->capacity) #define VEC_ARR(vec)\ ((vec)->buffer) #define VEC_GET(vec, id)\ (&(vec)->buffer[id]) #define VEC_CLEAR(vec) do {\ (vec)->size = 0;\ } while (0) #define VEC_DELETE(vec) do {\ Free((vec)->buffer);\ (vec)->buffer = NULL;\ (vec)->size = 0;\ (vec)->capacity = 0;\ } while (0) #ifdef __cplusplus } #endif #endif /* PMDK_VEC_H */ pmdk-1.8/src/common/util_windows.c0000664000000000000000000001555113615011243015772 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * util_windows.c -- misc utilities with OS-specific implementation */ #include #include #include #include #include "alloc.h" #include "util.h" #include "out.h" #include "file.h" /* Windows CRT doesn't support all errors, add unmapped here */ #define ENOTSUP_STR "Operation not supported" #define ECANCELED_STR "Operation canceled" #define ENOERROR 0 #define ENOERROR_STR "Success" #define UNMAPPED_STR "Unmapped error" /* * util_strerror -- return string describing error number * * XXX: There are many other POSIX error codes that are not recognized by * strerror_s(), so eventually we may want to implement this in a similar * fashion as strsignal(). */ void util_strerror(int errnum, char *buff, size_t bufflen) { switch (errnum) { case ENOERROR: strcpy_s(buff, bufflen, ENOERROR_STR); break; case ENOTSUP: strcpy_s(buff, bufflen, ENOTSUP_STR); break; case ECANCELED: strcpy_s(buff, bufflen, ECANCELED_STR); break; default: if (strerror_s(buff, bufflen, errnum)) strcpy_s(buff, bufflen, UNMAPPED_STR); } } /* * util_strwinerror -- return string describing windows error codes */ void util_strwinerror(char *buff, size_t bufflen) { wchar_t *error_str; unsigned long err = GetLastError(); if (FormatMessageW(FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM | FORMAT_MESSAGE_IGNORE_INSERTS, NULL, err, 0, (LPWSTR)&error_str, 0, NULL) == 0) { sprintf_s(buff, bufflen, "GetLastError() == %lu", err); return; } if (util_toUTF8_buff(error_str, buff, bufflen)) { LocalFree(error_str); sprintf_s(buff, bufflen, "GetLastError() == %lu", err); return; } LocalFree(error_str); } /* * util_part_realpath -- get canonicalized absolute pathname for a part file * * On Windows, paths cannot be symlinks and paths used in a poolset have to * be absolute (checked when parsing a poolset file), so we just return * the path. */ char * util_part_realpath(const char *path) { return strdup(path); } /* * util_compare_file_inodes -- compare device and inodes of two files */ int util_compare_file_inodes(const char *path1, const char *path2) { return strcmp(path1, path2) != 0; } /* * util_aligned_malloc -- allocate aligned memory */ void * util_aligned_malloc(size_t alignment, size_t size) { return _aligned_malloc(size, alignment); } /* * util_aligned_free -- free allocated memory in util_aligned_malloc */ void util_aligned_free(void *ptr) { _aligned_free(ptr); } /* * util_toUTF8 -- allocating conversion from wide char string to UTF8 */ char * util_toUTF8(const wchar_t *wstr) { int size = WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, wstr, -1, NULL, 0, NULL, NULL); if (size == 0) goto err; char *str = Malloc(size * sizeof(char)); if (str == NULL) goto out; if (WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, wstr, -1, str, size, NULL, NULL) == 0) { Free(str); goto err; } out: return str; err: errno = EINVAL; return NULL; } /* * util_free_UTF8 -- free UTF8 string */ void util_free_UTF8(char *str) { Free(str); } /* * util_toUTF16 -- allocating conversion from UTF8 to wide char string */ wchar_t * util_toUTF16(const char *str) { int size = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, str, -1, NULL, 0); if (size == 0) goto err; wchar_t *wstr = Malloc(size * sizeof(wchar_t)); if (wstr == NULL) goto out; if (MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, str, -1, wstr, size) == 0) { Free(wstr); goto err; } out: return wstr; err: errno = EINVAL; return NULL; } /* * util_free_UTF16 -- free wide char string */ void util_free_UTF16(wchar_t *wstr) { Free(wstr); } /* * util_toUTF16_buff -- non-allocating conversion from UTF8 to wide char string * * The user responsible for supplying a large enough out buffer. */ int util_toUTF16_buff(const char *in, wchar_t *out, size_t out_size) { ASSERT(out != NULL); int size = MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, in, -1, NULL, 0); if (size == 0 || out_size < size) goto err; if (MultiByteToWideChar(CP_UTF8, MB_ERR_INVALID_CHARS, in, -1, out, size) == 0) goto err; return 0; err: errno = EINVAL; return -1; } /* * util_toUTF8_buff -- non-allocating conversion from wide char string to UTF8 * * The user responsible for supplying a large enough out buffer. */ int util_toUTF8_buff(const wchar_t *in, char *out, size_t out_size) { ASSERT(out != NULL); int size = WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, in, -1, NULL, 0, NULL, NULL); if (size == 0 || out_size < size) goto err; if (WideCharToMultiByte(CP_UTF8, WC_ERR_INVALID_CHARS, in, -1, out, size, NULL, NULL) == 0) goto err; return 0; err: errno = EINVAL; return -1; } /* * util_getexecname -- return name of current executable */ char * util_getexecname(char *path, size_t pathlen) { ssize_t cc; if ((cc = GetModuleFileNameA(NULL, path, (DWORD)pathlen)) == 0) strcpy(path, "unknown"); else path[cc] = '\0'; return path; } /* * util_suppress_errmsg -- suppresses "abort" window on Windows if env variable * is set, useful for automatic tests */ void util_suppress_errmsg(void) { if (os_getenv("PMDK_NO_ABORT_MSG") != NULL) { DWORD err = GetErrorMode(); SetErrorMode(err | SEM_NOGPFAULTERRORBOX | SEM_FAILCRITICALERRORS); _set_abort_behavior(0, _WRITE_ABORT_MSG | _CALL_REPORTFAULT); } } pmdk-1.8/src/common/uuid_windows.c0000664000000000000000000000360113615011243015754 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * uuid_windows.c -- pool set utilities with OS-specific implementation */ #include "uuid.h" #include "out.h" /* * util_uuid_generate -- generate a uuid */ int util_uuid_generate(uuid_t uuid) { HRESULT res = CoCreateGuid((GUID *)(uuid)); if (res != S_OK) { ERR("CoCreateGuid"); return -1; } return 0; } pmdk-1.8/src/common/util.c0000664000000000000000000002304513615011243014215 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * util.c -- very basic utilities */ #include #include #include #include #include #include #include #include "util.h" #include "valgrind_internal.h" #include "alloc.h" /* library-wide page size */ unsigned long long Pagesize; /* allocation/mmap granularity */ unsigned long long Mmap_align; #if ANY_VG_TOOL_ENABLED /* initialized to true if the process is running inside Valgrind */ unsigned _On_valgrind; #endif #if VG_PMEMCHECK_ENABLED #define LIB_LOG_LEN 20 #define FUNC_LOG_LEN 50 #define SUFFIX_LEN 7 /* true if pmreorder instrumentization has to be enabled */ int _Pmreorder_emit; /* * util_emit_log -- emits lib and func name with appropriate suffix * to pmemcheck store log file */ void util_emit_log(const char *lib, const char *func, int order) { char lib_name[LIB_LOG_LEN]; char func_name[FUNC_LOG_LEN]; char suffix[SUFFIX_LEN]; size_t lib_len = strlen(lib); size_t func_len = strlen(func); if (order == 0) strcpy(suffix, ".BEGIN"); else strcpy(suffix, ".END"); size_t suffix_len = strlen(suffix); if (lib_len + suffix_len + 1 > LIB_LOG_LEN) { VALGRIND_EMIT_LOG("Library name is too long"); return; } if (func_len + suffix_len + 1 > FUNC_LOG_LEN) { VALGRIND_EMIT_LOG("Function name is too long"); return; } strcpy(lib_name, lib); strcat(lib_name, suffix); strcpy(func_name, func); strcat(func_name, suffix); if (order == 0) { VALGRIND_EMIT_LOG(func_name); VALGRIND_EMIT_LOG(lib_name); } else { VALGRIND_EMIT_LOG(lib_name); VALGRIND_EMIT_LOG(func_name); } } #endif /* * util_is_zeroed -- check if given memory range is all zero */ int util_is_zeroed(const void *addr, size_t len) { const char *a = addr; if (len == 0) return 1; if (a[0] == 0 && memcmp(a, a + 1, len - 1) == 0) return 1; return 0; } /* * util_checksum_compute -- compute Fletcher64 checksum * * csump points to where the checksum lives, so that location * is treated as zeros while calculating the checksum. The * checksummed data is assumed to be in little endian order. */ uint64_t util_checksum_compute(void *addr, size_t len, uint64_t *csump, size_t skip_off) { if (len % 4 != 0) abort(); uint32_t *p32 = addr; uint32_t *p32end = (uint32_t *)((char *)addr + len); uint32_t *skip; uint32_t lo32 = 0; uint32_t hi32 = 0; if (skip_off) skip = (uint32_t *)((char *)addr + skip_off); else skip = (uint32_t *)((char *)addr + len); while (p32 < p32end) if (p32 == (uint32_t *)csump || p32 >= skip) { /* lo32 += 0; treat first 32-bits as zero */ p32++; hi32 += lo32; /* lo32 += 0; treat second 32-bits as zero */ p32++; hi32 += lo32; } else { lo32 += le32toh(*p32); ++p32; hi32 += lo32; } return (uint64_t)hi32 << 32 | lo32; } /* * util_checksum -- compute Fletcher64 checksum * * csump points to where the checksum lives, so that location * is treated as zeros while calculating the checksum. * If insert is true, the calculated checksum is inserted into * the range at *csump. Otherwise the calculated checksum is * checked against *csump and the result returned (true means * the range checksummed correctly). */ int util_checksum(void *addr, size_t len, uint64_t *csump, int insert, size_t skip_off) { uint64_t csum = util_checksum_compute(addr, len, csump, skip_off); if (insert) { *csump = htole64(csum); return 1; } return *csump == htole64(csum); } /* * util_checksum_seq -- compute sequential Fletcher64 checksum * * Merges checksum from the old buffer with checksum for current buffer. */ uint64_t util_checksum_seq(const void *addr, size_t len, uint64_t csum) { if (len % 4 != 0) abort(); const uint32_t *p32 = addr; const uint32_t *p32end = (const uint32_t *)((const char *)addr + len); uint32_t lo32 = (uint32_t)csum; uint32_t hi32 = (uint32_t)(csum >> 32); while (p32 < p32end) { lo32 += le32toh(*p32); ++p32; hi32 += lo32; } return (uint64_t)hi32 << 32 | lo32; } /* * util_fgets -- fgets wrapper with conversion CRLF to LF */ char * util_fgets(char *buffer, int max, FILE *stream) { char *str = fgets(buffer, max, stream); if (str == NULL) goto end; int len = (int)strlen(str); if (len < 2) goto end; if (str[len - 2] == '\r' && str[len - 1] == '\n') { str[len - 2] = '\n'; str[len - 1] = '\0'; } end: return str; } struct suff { const char *suff; uint64_t mag; }; /* * util_parse_size -- parse size from string */ int util_parse_size(const char *str, size_t *sizep) { const struct suff suffixes[] = { { "B", 1ULL }, { "K", 1ULL << 10 }, /* JEDEC */ { "M", 1ULL << 20 }, { "G", 1ULL << 30 }, { "T", 1ULL << 40 }, { "P", 1ULL << 50 }, { "KiB", 1ULL << 10 }, /* IEC */ { "MiB", 1ULL << 20 }, { "GiB", 1ULL << 30 }, { "TiB", 1ULL << 40 }, { "PiB", 1ULL << 50 }, { "kB", 1000ULL }, /* SI */ { "MB", 1000ULL * 1000 }, { "GB", 1000ULL * 1000 * 1000 }, { "TB", 1000ULL * 1000 * 1000 * 1000 }, { "PB", 1000ULL * 1000 * 1000 * 1000 * 1000 } }; int res = -1; unsigned i; size_t size = 0; char unit[9] = {0}; int ret = sscanf(str, "%zu%8s", &size, unit); if (ret == 1) { res = 0; } else if (ret == 2) { for (i = 0; i < ARRAY_SIZE(suffixes); ++i) { if (strcmp(suffixes[i].suff, unit) == 0) { size = size * suffixes[i].mag; res = 0; break; } } } else { return -1; } if (sizep && res == 0) *sizep = size; return res; } /* * util_init -- initialize the utils * * This is called from the library initialization code. */ void util_init(void) { /* XXX - replace sysconf() with util_get_sys_xxx() */ if (Pagesize == 0) Pagesize = (unsigned long) sysconf(_SC_PAGESIZE); #ifndef _WIN32 Mmap_align = Pagesize; #else if (Mmap_align == 0) { SYSTEM_INFO si; GetSystemInfo(&si); Mmap_align = si.dwAllocationGranularity; } #endif #if ANY_VG_TOOL_ENABLED _On_valgrind = RUNNING_ON_VALGRIND; #endif #if VG_PMEMCHECK_ENABLED if (On_valgrind) { char *pmreorder_env = getenv("PMREORDER_EMIT_LOG"); if (pmreorder_env) _Pmreorder_emit = atoi(pmreorder_env); } else { _Pmreorder_emit = 0; } #endif } /* * util_concat_str -- concatenate two strings */ char * util_concat_str(const char *s1, const char *s2) { char *result = malloc(strlen(s1) + strlen(s2) + 1); if (!result) return NULL; strcpy(result, s1); strcat(result, s2); return result; } /* * util_localtime -- a wrapper for localtime function * * localtime can set nonzero errno even if it succeeds (e.g. when there is no * /etc/localtime file under Linux) and we do not want the errno to be polluted * in such cases. */ struct tm * util_localtime(const time_t *timep) { int oerrno = errno; struct tm *tm = localtime(timep); if (tm != NULL) errno = oerrno; return tm; } /* * util_safe_strcpy -- copies string from src to dst, returns -1 * when length of source string (including null-terminator) * is greater than max_length, 0 otherwise * * For gcc (found in version 8.1.1) calling this function with * max_length equal to dst size produces -Wstringop-truncation warning * * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85902 */ #ifdef STRINGOP_TRUNCATION_SUPPORTED #pragma GCC diagnostic push #pragma GCC diagnostic ignored "-Wstringop-truncation" #endif int util_safe_strcpy(char *dst, const char *src, size_t max_length) { if (max_length == 0) return -1; strncpy(dst, src, max_length); return dst[max_length - 1] == '\0' ? 0 : -1; } #ifdef STRINGOP_TRUNCATION_SUPPORTED #pragma GCC diagnostic pop #endif #define PARSER_MAX_LINE (PATH_MAX + 1024) /* * util_readline -- read line from stream */ char * util_readline(FILE *fh) { size_t bufsize = PARSER_MAX_LINE; size_t position = 0; char *buffer = NULL; do { char *tmp = buffer; buffer = Realloc(buffer, bufsize); if (buffer == NULL) { Free(tmp); return NULL; } /* ensure if we can cast bufsize to int */ char *s = util_fgets(buffer + position, (int)bufsize / 2, fh); if (s == NULL) { Free(buffer); return NULL; } position = strlen(buffer); bufsize *= 2; } while (!feof(fh) && buffer[position - 1] != '\n'); return buffer; } pmdk-1.8/src/common/out.h0000664000000000000000000001663513615011243014063 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * out.h -- definitions for "out" module */ #ifndef PMDK_OUT_H #define PMDK_OUT_H 1 #include #include #include #include "util.h" #ifdef __cplusplus extern "C" { #endif /* * Suppress errors which are after appropriate ASSERT* macro for nondebug * builds. */ #if !defined(DEBUG) && (defined(__clang_analyzer__) || defined(__COVERITY__) ||\ defined(__KLOCWORK__)) #define OUT_FATAL_DISCARD_NORETURN __attribute__((noreturn)) #else #define OUT_FATAL_DISCARD_NORETURN #endif #ifndef EVALUATE_DBG_EXPRESSIONS #if defined(DEBUG) || defined(__clang_analyzer__) || defined(__COVERITY__) ||\ defined(__KLOCWORK__) #define EVALUATE_DBG_EXPRESSIONS 1 #else #define EVALUATE_DBG_EXPRESSIONS 0 #endif #endif #ifdef DEBUG #define OUT_LOG out_log #define OUT_NONL out_nonl #define OUT_FATAL out_fatal #define OUT_FATAL_ABORT out_fatal #else static __attribute__((always_inline)) inline void out_log_discard(const char *file, int line, const char *func, int level, const char *fmt, ...) { (void) file; (void) line; (void) func; (void) level; (void) fmt; } static __attribute__((always_inline)) inline void out_nonl_discard(int level, const char *fmt, ...) { (void) level; (void) fmt; } static __attribute__((always_inline)) OUT_FATAL_DISCARD_NORETURN inline void out_fatal_discard(const char *file, int line, const char *func, const char *fmt, ...) { (void) file; (void) line; (void) func; (void) fmt; } static __attribute__((always_inline)) NORETURN inline void out_fatal_abort(const char *file, int line, const char *func, const char *fmt, ...) { (void) file; (void) line; (void) func; (void) fmt; abort(); } #define OUT_LOG out_log_discard #define OUT_NONL out_nonl_discard #define OUT_FATAL out_fatal_discard #define OUT_FATAL_ABORT out_fatal_abort #endif #if defined(__KLOCWORK__) #define TEST_ALWAYS_TRUE_EXPR(cnd) #define TEST_ALWAYS_EQ_EXPR(cnd) #define TEST_ALWAYS_NE_EXPR(cnd) #else #define TEST_ALWAYS_TRUE_EXPR(cnd)\ if (__builtin_constant_p(cnd))\ ASSERT_COMPILE_ERROR_ON(cnd); #define TEST_ALWAYS_EQ_EXPR(lhs, rhs)\ if (__builtin_constant_p(lhs) && __builtin_constant_p(rhs))\ ASSERT_COMPILE_ERROR_ON((lhs) == (rhs)); #define TEST_ALWAYS_NE_EXPR(lhs, rhs)\ if (__builtin_constant_p(lhs) && __builtin_constant_p(rhs))\ ASSERT_COMPILE_ERROR_ON((lhs) != (rhs)); #endif /* produce debug/trace output */ #define LOG(level, ...) do { \ if (!EVALUATE_DBG_EXPRESSIONS) break;\ OUT_LOG(__FILE__, __LINE__, __func__, level, __VA_ARGS__);\ } while (0) /* produce debug/trace output without prefix and new line */ #define LOG_NONL(level, ...) do { \ if (!EVALUATE_DBG_EXPRESSIONS) break; \ OUT_NONL(level, __VA_ARGS__); \ } while (0) /* produce output and exit */ #define FATAL(...)\ OUT_FATAL_ABORT(__FILE__, __LINE__, __func__, __VA_ARGS__) /* assert a condition is true at runtime */ #define ASSERT_rt(cnd) do { \ if (!EVALUATE_DBG_EXPRESSIONS || (cnd)) break; \ OUT_FATAL(__FILE__, __LINE__, __func__, "assertion failure: %s", #cnd);\ } while (0) /* assertion with extra info printed if assertion fails at runtime */ #define ASSERTinfo_rt(cnd, info) do { \ if (!EVALUATE_DBG_EXPRESSIONS || (cnd)) break; \ OUT_FATAL(__FILE__, __LINE__, __func__, \ "assertion failure: %s (%s = %s)", #cnd, #info, info);\ } while (0) /* assert two integer values are equal at runtime */ #define ASSERTeq_rt(lhs, rhs) do { \ if (!EVALUATE_DBG_EXPRESSIONS || ((lhs) == (rhs))) break; \ OUT_FATAL(__FILE__, __LINE__, __func__,\ "assertion failure: %s (0x%llx) == %s (0x%llx)", #lhs,\ (unsigned long long)(lhs), #rhs, (unsigned long long)(rhs)); \ } while (0) /* assert two integer values are not equal at runtime */ #define ASSERTne_rt(lhs, rhs) do { \ if (!EVALUATE_DBG_EXPRESSIONS || ((lhs) != (rhs))) break; \ OUT_FATAL(__FILE__, __LINE__, __func__,\ "assertion failure: %s (0x%llx) != %s (0x%llx)", #lhs,\ (unsigned long long)(lhs), #rhs, (unsigned long long)(rhs)); \ } while (0) /* assert a condition is true */ #define ASSERT(cnd)\ do {\ /*\ * Detect useless asserts on always true expression. Please use\ * COMPILE_ERROR_ON(!cnd) or ASSERT_rt(cnd) in such cases.\ */\ TEST_ALWAYS_TRUE_EXPR(cnd);\ ASSERT_rt(cnd);\ } while (0) /* assertion with extra info printed if assertion fails */ #define ASSERTinfo(cnd, info)\ do {\ /* See comment in ASSERT. */\ TEST_ALWAYS_TRUE_EXPR(cnd);\ ASSERTinfo_rt(cnd, info);\ } while (0) /* assert two integer values are equal */ #define ASSERTeq(lhs, rhs)\ do {\ /* See comment in ASSERT. */\ TEST_ALWAYS_EQ_EXPR(lhs, rhs);\ ASSERTeq_rt(lhs, rhs);\ } while (0) /* assert two integer values are not equal */ #define ASSERTne(lhs, rhs)\ do {\ /* See comment in ASSERT. */\ TEST_ALWAYS_NE_EXPR(lhs, rhs);\ ASSERTne_rt(lhs, rhs);\ } while (0) #define ERR(...)\ out_err(__FILE__, __LINE__, __func__, __VA_ARGS__) void out_init(const char *log_prefix, const char *log_level_var, const char *log_file_var, int major_version, int minor_version); void out_fini(void); void out(const char *fmt, ...) FORMAT_PRINTF(1, 2); void out_nonl(int level, const char *fmt, ...) FORMAT_PRINTF(2, 3); void out_log(const char *file, int line, const char *func, int level, const char *fmt, ...) FORMAT_PRINTF(5, 6); void out_err(const char *file, int line, const char *func, const char *fmt, ...) FORMAT_PRINTF(4, 5); void NORETURN out_fatal(const char *file, int line, const char *func, const char *fmt, ...) FORMAT_PRINTF(4, 5); void out_set_print_func(void (*print_func)(const char *s)); void out_set_vsnprintf_func(int (*vsnprintf_func)(char *str, size_t size, const char *format, va_list ap)); #ifdef _WIN32 #ifndef PMDK_UTF8_API #define out_get_errormsg out_get_errormsgW #else #define out_get_errormsg out_get_errormsgU #endif #endif #ifndef _WIN32 const char *out_get_errormsg(void); #else const char *out_get_errormsgU(void); const wchar_t *out_get_errormsgW(void); #endif #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/libpmemcommon.vcxproj.filters0000664000000000000000000001171313615011243021015 0ustar rootroot {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {93995380-89BD-4b04-88EB-625FBE52EBFB} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files pmdk-1.8/src/common/file.h0000664000000000000000000001063313615011243014163 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * file.h -- internal definitions for file module */ #ifndef PMDK_FILE_H #define PMDK_FILE_H 1 #include #include #include #include #include #include "os.h" #ifdef __cplusplus extern "C" { #endif #ifdef _WIN32 #define NAME_MAX _MAX_FNAME #endif struct file_info { char filename[NAME_MAX + 1]; int is_dir; }; struct dir_handle { const char *path; #ifdef _WIN32 HANDLE handle; char *_file; #else DIR *dirp; #endif }; enum file_type { OTHER_ERROR = -2, NOT_EXISTS = -1, TYPE_NORMAL = 1, TYPE_DEVDAX = 2 }; int util_file_dir_open(struct dir_handle *a, const char *path); int util_file_dir_next(struct dir_handle *a, struct file_info *info); int util_file_dir_close(struct dir_handle *a); int util_file_dir_remove(const char *path); int util_file_exists(const char *path); enum file_type util_stat_get_type(const os_stat_t *st); enum file_type util_fd_get_type(int fd); enum file_type util_file_get_type(const char *path); int util_ddax_region_find(const char *path); ssize_t util_file_get_size(const char *path); ssize_t util_fd_get_size(int fd); size_t util_file_device_dax_alignment(const char *path); void *util_file_map_whole(const char *path); int util_file_zero(const char *path, os_off_t off, size_t len); ssize_t util_file_pread(const char *path, void *buffer, size_t size, os_off_t offset); ssize_t util_file_pwrite(const char *path, const void *buffer, size_t size, os_off_t offset); int util_tmpfile(const char *dir, const char *templ, int flags); int util_is_absolute_path(const char *path); int util_file_create(const char *path, size_t size, size_t minsize); int util_file_open(const char *path, size_t *size, size_t minsize, int flags); int util_unlink(const char *path); int util_unlink_flock(const char *path); int util_file_mkdir(const char *path, mode_t mode); int util_write_all(int fd, const char *buf, size_t count); #ifndef _WIN32 #define util_read read #define util_write write #else static inline ssize_t util_read(int fd, void *buf, size_t count) { /* * Simulate short read, because Windows' _read uses "unsigned" as * a type of the last argument and "int" as a return type. * We have to limit "count" to what _read can return as a success, * not what it can accept. */ if (count > INT_MAX) count = INT_MAX; return _read(fd, buf, (unsigned)count); } static inline ssize_t util_write(int fd, const void *buf, size_t count) { /* * Simulate short write, because Windows' _write uses "unsigned" as * a type of the last argument and "int" as a return type. * We have to limit "count" to what _write can return as a success, * not what it can accept. */ if (count > INT_MAX) count = INT_MAX; return _write(fd, buf, (unsigned)count); } #define S_ISCHR(m) (((m) & S_IFMT) == S_IFCHR) #define S_ISDIR(m) (((m) & S_IFMT) == S_IFDIR) #endif #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/os.h0000664000000000000000000000747713615011243013701 0ustar rootroot/* * Copyright 2017-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os.h -- os abstraction layer */ #ifndef PMDK_OS_H #define PMDK_OS_H 1 #include #include #include #include "errno_freebsd.h" #ifdef __cplusplus extern "C" { #endif #ifndef _WIN32 #define OS_DIR_SEPARATOR '/' #define OS_DIR_SEP_STR "/" #else #define OS_DIR_SEPARATOR '\\' #define OS_DIR_SEP_STR "\\" #endif #ifndef _WIN32 /* madvise() */ #ifdef __FreeBSD__ #define os_madvise minherit #define MADV_DONTFORK INHERIT_NONE #else #define os_madvise madvise #endif /* dlopen() */ #ifdef __FreeBSD__ #define RTLD_DEEPBIND 0 /* XXX */ #endif /* major(), minor() */ #ifdef __FreeBSD__ #define os_major (unsigned)major #define os_minor (unsigned)minor #else #define os_major major #define os_minor minor #endif #endif /* #ifndef _WIN32 */ struct iovec; /* os_flock */ #define OS_LOCK_SH 1 #define OS_LOCK_EX 2 #define OS_LOCK_NB 4 #define OS_LOCK_UN 8 #ifndef _WIN32 typedef struct stat os_stat_t; #define os_fstat fstat #define os_lseek lseek #else typedef struct _stat64 os_stat_t; #define os_fstat _fstat64 #define os_lseek _lseeki64 #endif #define os_close close #define os_fclose fclose #ifndef _WIN32 typedef off_t os_off_t; #else /* XXX: os_off_t defined in platform.h */ #endif int os_open(const char *pathname, int flags, ...); int os_fsync(int fd); int os_fsync_dir(const char *dir_name); int os_stat(const char *pathname, os_stat_t *buf); int os_unlink(const char *pathname); int os_access(const char *pathname, int mode); FILE *os_fopen(const char *pathname, const char *mode); FILE *os_fdopen(int fd, const char *mode); int os_chmod(const char *pathname, mode_t mode); int os_mkstemp(char *temp); int os_posix_fallocate(int fd, os_off_t offset, os_off_t len); int os_ftruncate(int fd, os_off_t length); int os_flock(int fd, int operation); ssize_t os_writev(int fd, const struct iovec *iov, int iovcnt); int os_clock_gettime(int id, struct timespec *ts); unsigned os_rand_r(unsigned *seedp); int os_unsetenv(const char *name); int os_setenv(const char *name, const char *value, int overwrite); char *os_getenv(const char *name); const char *os_strsignal(int sig); int os_execv(const char *path, char *const argv[]); /* * XXX: missing APis (used in ut_file.c) * * rename * read * write */ #ifdef __cplusplus } #endif #endif /* os.h */ pmdk-1.8/src/common/badblock.h0000664000000000000000000000423313615011243015004 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * badblock.h - common part of bad blocks API */ #ifndef PMDK_BADBLOCK_POOLSET_H #define PMDK_BADBLOCK_POOLSET_H 1 #include "set.h" #ifdef __cplusplus extern "C" { #endif struct badblocks *badblocks_new(void); void badblocks_delete(struct badblocks *bbs); int badblocks_check_poolset(struct pool_set *set, int create); int badblocks_clear_poolset(struct pool_set *set, int create); char *badblocks_recovery_file_alloc(const char *file, unsigned rep, unsigned part); int badblocks_recovery_file_exists(struct pool_set *set); #ifdef __cplusplus } #endif #endif /* PMDK_BADBLOCK_POOLSET_H */ pmdk-1.8/src/common/os_dimm_windows.c0000664000000000000000000001345313615011243016443 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_dimm_windows.c -- implementation of DIMMs API based on winapi */ #include "out.h" #include "os.h" #include "os_dimm.h" #include "util.h" #define GUID_SIZE sizeof("XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX") /* * os_dimm_volume -- returns volume handle */ static HANDLE os_dimm_volume_handle(const char *path) { wchar_t mount[MAX_PATH]; wchar_t volume[MAX_PATH]; wchar_t *wpath = util_toUTF16(path); if (wpath == NULL) return INVALID_HANDLE_VALUE; if (!GetVolumePathNameW(wpath, mount, MAX_PATH)) { ERR("!GetVolumePathNameW"); util_free_UTF16(wpath); return INVALID_HANDLE_VALUE; } util_free_UTF16(wpath); /* get volume name - "\\?\Volume{VOLUME_GUID}\" */ if (!GetVolumeNameForVolumeMountPointW(mount, volume, MAX_PATH) || wcslen(volume) == 0 || volume[wcslen(volume) - 1] != L'\\') { ERR("!GetVolumeNameForVolumeMountPointW"); return INVALID_HANDLE_VALUE; } /* * Remove trailing \\ as "CreateFile processes a volume GUID path with * an appended backslash as the root directory of the volume." */ volume[wcslen(volume) - 1] = L'\0'; HANDLE h = CreateFileW(volume, /* path to the file */ /* request access to send ioctl to the file */ FILE_READ_ATTRIBUTES, /* do not block access to the file */ FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE, NULL, /* security attributes */ OPEN_EXISTING, /* open only if it exists */ FILE_ATTRIBUTE_NORMAL, /* no attributes */ NULL); /* used only for new files */ if (h == INVALID_HANDLE_VALUE) ERR("!CreateFileW"); return h; } /* * os_dimm_uid -- returns a file uid based on dimm guid */ int os_dimm_uid(const char *path, char *uid, size_t *len) { LOG(3, "path %s, uid %p, len %lu", path, uid, *len); if (uid == NULL) { *len = GUID_SIZE; } else { ASSERT(*len >= GUID_SIZE); HANDLE vHandle = os_dimm_volume_handle(path); if (vHandle == INVALID_HANDLE_VALUE) return -1; STORAGE_DEVICE_NUMBER_EX sdn; sdn.DeviceNumber = -1; DWORD dwBytesReturned = 0; if (!DeviceIoControl(vHandle, IOCTL_STORAGE_GET_DEVICE_NUMBER_EX, NULL, 0, &sdn, sizeof(sdn), &dwBytesReturned, NULL)) { /* * IOCTL_STORAGE_GET_DEVICE_NUMBER_EX is not supported * on this server */ memset(uid, 0, *len); CloseHandle(vHandle); return 0; } GUID guid = sdn.DeviceGuid; snprintf(uid, GUID_SIZE, "%08lX-%04hX-%04hX-%02hhX%02hhX-%02hhX%02hhX%02hhX%02hhX%02hhX%02hhX", guid.Data1, guid.Data2, guid.Data3, guid.Data4[0], guid.Data4[1], guid.Data4[2], guid.Data4[3], guid.Data4[4], guid.Data4[5], guid.Data4[6], guid.Data4[7]); CloseHandle(vHandle); } return 0; } /* * os_dimm_usc -- returns unsafe shutdown count */ int os_dimm_usc(const char *path, uint64_t *usc) { LOG(3, "path %s, usc %p", path, usc); *usc = 0; HANDLE vHandle = os_dimm_volume_handle(path); if (vHandle == INVALID_HANDLE_VALUE) return -1; STORAGE_PROPERTY_QUERY prop; DWORD dwSize; prop.PropertyId = StorageDeviceUnsafeShutdownCount; prop.QueryType = PropertyExistsQuery; prop.AdditionalParameters[0] = 0; STORAGE_DEVICE_UNSAFE_SHUTDOWN_COUNT ret; BOOL bResult = DeviceIoControl(vHandle, IOCTL_STORAGE_QUERY_PROPERTY, &prop, sizeof(prop), &ret, sizeof(ret), (LPDWORD)&dwSize, (LPOVERLAPPED)NULL); if (!bResult) { CloseHandle(vHandle); return 0; /* STORAGE_PROPERTY_QUERY not supported */ } prop.QueryType = PropertyStandardQuery; bResult = DeviceIoControl(vHandle, IOCTL_STORAGE_QUERY_PROPERTY, &prop, sizeof(prop), &ret, sizeof(ret), (LPDWORD)&dwSize, (LPOVERLAPPED)NULL); CloseHandle(vHandle); if (!bResult) return -1; *usc = ret.UnsafeShutdownCount; return 0; } /* * os_dimm_files_namespace_badblocks -- fake os_dimm_files_namespace_badblocks() */ int os_dimm_files_namespace_badblocks(const char *path, struct badblocks *bbs) { LOG(3, "path %s", path); os_stat_t st; if (os_stat(path, &st)) { ERR("!stat %s", path); return -1; } return 0; } /* * os_dimm_devdax_clear_badblocks -- fake bad block clearing routine */ int os_dimm_devdax_clear_badblocks(const char *path, struct badblocks *bbs) { LOG(3, "path %s badblocks %p", path, bbs); return 0; } /* * os_dimm_devdax_clear_badblocks_all -- fake bad block clearing routine */ int os_dimm_devdax_clear_badblocks_all(const char *path) { LOG(3, "path %s", path); return 0; } pmdk-1.8/src/common/mmap_windows.c0000664000000000000000000001147113615011243015744 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * Copyright (c) 2015-2017, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * mmap_windows.c -- memory-mapped files for Windows */ #include #include "mmap.h" #include "out.h" /* * util_map_hint_unused -- use VirtualQuery to determine hint address * * This is a helper function for util_map_hint(). * It iterates through memory regions and looks for the first unused address * in the process address space that is: * - greater or equal 'minaddr' argument, * - large enough to hold range of given length, * - aligned to the specified unit. */ char * util_map_hint_unused(void *minaddr, size_t len, size_t align) { LOG(3, "minaddr %p len %zu align %zu", minaddr, len, align); ASSERT(align > 0); MEMORY_BASIC_INFORMATION mi; char *lo = NULL; /* beginning of current range in maps file */ char *hi = NULL; /* end of current range in maps file */ char *raddr = minaddr; /* ignore regions below 'minaddr' */ if (raddr == NULL) raddr += Pagesize; raddr = (char *)roundup((uintptr_t)raddr, align); while ((uintptr_t)raddr < UINTPTR_MAX - len) { size_t ret = VirtualQuery(raddr, &mi, sizeof(mi)); if (ret == 0) { ERR("VirtualQuery %p", raddr); return MAP_FAILED; } LOG(4, "addr %p len %zu state %d", mi.BaseAddress, mi.RegionSize, mi.State); if ((mi.State != MEM_FREE) || (mi.RegionSize < len)) { raddr = (char *)mi.BaseAddress + mi.RegionSize; raddr = (char *)roundup((uintptr_t)raddr, align); LOG(4, "nearest aligned addr %p", raddr); } else { LOG(4, "unused region of size %zu found at %p", mi.RegionSize, mi.BaseAddress); return mi.BaseAddress; } } LOG(4, "end of address space reached"); return MAP_FAILED; } /* * util_map_hint -- determine hint address for mmap() * * XXX - Windows doesn't support large DAX pages yet, so there is * no point in aligning for the same. */ char * util_map_hint(size_t len, size_t req_align) { LOG(3, "len %zu req_align %zu", len, req_align); char *hint_addr = MAP_FAILED; /* choose the desired alignment based on the requested length */ size_t align = util_map_hint_align(len, req_align); if (Mmap_no_random) { LOG(4, "user-defined hint %p", Mmap_hint); hint_addr = util_map_hint_unused(Mmap_hint, len, align); } else { /* * Create dummy mapping to find an unused region of given size. * Request for increased size for later address alignment. * * Use MAP_NORESERVE flag to only reserve the range of pages * rather than commit. We don't want the pages to be actually * backed by the operating system paging file, as the swap * file is usually too small to handle terabyte pools. */ char *addr = mmap(NULL, len + align, PROT_READ, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0); if (addr != MAP_FAILED) { LOG(4, "system choice %p", addr); hint_addr = (char *)roundup((uintptr_t)addr, align); munmap(addr, len + align); } } LOG(4, "hint %p", hint_addr); return hint_addr; } /* * util_map_sync -- memory map given file into memory */ void * util_map_sync(void *addr, size_t len, int proto, int flags, int fd, os_off_t offset, int *map_sync) { LOG(15, "addr %p len %zu proto %x flags %x fd %d offset %ld", addr, len, proto, flags, fd, offset); if (map_sync) *map_sync = 0; return mmap(addr, len, proto, flags, fd, offset); } pmdk-1.8/src/common/valgrind/0000775000000000000000000000000013615011243014676 5ustar rootrootpmdk-1.8/src/common/valgrind/drd.h0000664000000000000000000005470613615011243015634 0ustar rootroot/* ---------------------------------------------------------------- Notice that the following BSD-style license applies to this one file (drd.h) only. The rest of Valgrind is licensed under the terms of the GNU General Public License, version 2, unless otherwise indicated. See the COPYING file in the source distribution for details. ---------------------------------------------------------------- This file is part of DRD, a Valgrind tool for verification of multithreaded programs. Copyright (C) 2006-2017 Bart Van Assche . All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 3. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 4. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ---------------------------------------------------------------- Notice that the above BSD-style license applies to this one file (drd.h) only. The entire rest of Valgrind is licensed under the terms of the GNU General Public License, version 2. See the COPYING file in the source distribution for details. ---------------------------------------------------------------- */ #ifndef __VALGRIND_DRD_H #define __VALGRIND_DRD_H #include "valgrind.h" /** Obtain the thread ID assigned by Valgrind's core. */ #define DRD_GET_VALGRIND_THREADID \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__DRD_GET_VALGRIND_THREAD_ID, \ 0, 0, 0, 0, 0) /** Obtain the thread ID assigned by DRD. */ #define DRD_GET_DRD_THREADID \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__DRD_GET_DRD_THREAD_ID, \ 0, 0, 0, 0, 0) /** Tell DRD not to complain about data races for the specified variable. */ #define DRD_IGNORE_VAR(x) ANNOTATE_BENIGN_RACE_SIZED(&(x), sizeof(x), "") /** Tell DRD to no longer ignore data races for the specified variable. */ #define DRD_STOP_IGNORING_VAR(x) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_FINISH_SUPPRESSION, \ &(x), sizeof(x), 0, 0, 0) /** * Tell DRD to trace all memory accesses for the specified variable * until the memory that was allocated for the variable is freed. */ #define DRD_TRACE_VAR(x) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_START_TRACE_ADDR, \ &(x), sizeof(x), 0, 0, 0) /** * Tell DRD to stop tracing memory accesses for the specified variable. */ #define DRD_STOP_TRACING_VAR(x) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_STOP_TRACE_ADDR, \ &(x), sizeof(x), 0, 0, 0) /** * @defgroup RaceDetectionAnnotations Data race detection annotations. * * @see See also the source file producer-consumer. */ #define ANNOTATE_PCQ_CREATE(pcq) do { } while(0) /** Tell DRD that a FIFO queue has been destroyed. */ #define ANNOTATE_PCQ_DESTROY(pcq) do { } while(0) /** * Tell DRD that an element has been added to the FIFO queue at address pcq. */ #define ANNOTATE_PCQ_PUT(pcq) do { } while(0) /** * Tell DRD that an element has been removed from the FIFO queue at address pcq, * and that DRD should insert a happens-before relationship between the memory * accesses that occurred before the corresponding ANNOTATE_PCQ_PUT(pcq) * annotation and the memory accesses after this annotation. Correspondence * between PUT and GET annotations happens in FIFO order. Since locking * of the queue is needed anyway to add elements to or to remove elements from * the queue, for DRD all four FIFO annotations are defined as no-ops. */ #define ANNOTATE_PCQ_GET(pcq) do { } while(0) /** * Tell DRD that data races at the specified address are expected and must not * be reported. */ #define ANNOTATE_BENIGN_RACE(addr, descr) \ ANNOTATE_BENIGN_RACE_SIZED(addr, sizeof(*addr), descr) /* Same as ANNOTATE_BENIGN_RACE(addr, descr), but applies to the memory range [addr, addr + size). */ #define ANNOTATE_BENIGN_RACE_SIZED(addr, size, descr) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_START_SUPPRESSION, \ addr, size, 0, 0, 0) /** Tell DRD to ignore all reads performed by the current thread. */ #define ANNOTATE_IGNORE_READS_BEGIN() \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_RECORD_LOADS, \ 0, 0, 0, 0, 0); /** Tell DRD to no longer ignore the reads performed by the current thread. */ #define ANNOTATE_IGNORE_READS_END() \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_RECORD_LOADS, \ 1, 0, 0, 0, 0); /** Tell DRD to ignore all writes performed by the current thread. */ #define ANNOTATE_IGNORE_WRITES_BEGIN() \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_RECORD_STORES, \ 0, 0, 0, 0, 0) /** Tell DRD to no longer ignore the writes performed by the current thread. */ #define ANNOTATE_IGNORE_WRITES_END() \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_RECORD_STORES, \ 1, 0, 0, 0, 0) /** Tell DRD to ignore all memory accesses performed by the current thread. */ #define ANNOTATE_IGNORE_READS_AND_WRITES_BEGIN() \ do { ANNOTATE_IGNORE_READS_BEGIN(); ANNOTATE_IGNORE_WRITES_BEGIN(); } while(0) /** * Tell DRD to no longer ignore the memory accesses performed by the current * thread. */ #define ANNOTATE_IGNORE_READS_AND_WRITES_END() \ do { ANNOTATE_IGNORE_READS_END(); ANNOTATE_IGNORE_WRITES_END(); } while(0) /** * Tell DRD that size bytes starting at addr has been allocated by a custom * memory allocator. */ #define ANNOTATE_NEW_MEMORY(addr, size) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_CLEAN_MEMORY, \ addr, size, 0, 0, 0) /** Ask DRD to report every access to the specified address. */ #define ANNOTATE_TRACE_MEMORY(addr) DRD_TRACE_VAR(*(char*)(addr)) /** * Tell DRD to assign the specified name to the current thread. This name will * be used in error messages printed by DRD. */ #define ANNOTATE_THREAD_NAME(name) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DRD_SET_THREAD_NAME, \ name, 0, 0, 0, 0) /*@}*/ /* !! ABIWARNING !! ABIWARNING !! ABIWARNING !! ABIWARNING !! This enum comprises an ABI exported by Valgrind to programs which use client requests. DO NOT CHANGE THE ORDER OF THESE ENTRIES, NOR DELETE ANY -- add new ones at the end. */ enum { /* Ask the DRD tool to discard all information about memory accesses */ /* and client objects for the specified range. This client request is */ /* binary compatible with the similarly named Helgrind client request. */ VG_USERREQ__DRD_CLEAN_MEMORY = VG_USERREQ_TOOL_BASE('H','G'), /* args: Addr, SizeT. */ /* Ask the DRD tool the thread ID assigned by Valgrind. */ VG_USERREQ__DRD_GET_VALGRIND_THREAD_ID = VG_USERREQ_TOOL_BASE('D','R'), /* args: none. */ /* Ask the DRD tool the thread ID assigned by DRD. */ VG_USERREQ__DRD_GET_DRD_THREAD_ID, /* args: none. */ /* To tell the DRD tool to suppress data race detection on the */ /* specified address range. */ VG_USERREQ__DRD_START_SUPPRESSION, /* args: start address, size in bytes */ /* To tell the DRD tool no longer to suppress data race detection on */ /* the specified address range. */ VG_USERREQ__DRD_FINISH_SUPPRESSION, /* args: start address, size in bytes */ /* To ask the DRD tool to trace all accesses to the specified range. */ VG_USERREQ__DRD_START_TRACE_ADDR, /* args: Addr, SizeT. */ /* To ask the DRD tool to stop tracing accesses to the specified range. */ VG_USERREQ__DRD_STOP_TRACE_ADDR, /* args: Addr, SizeT. */ /* Tell DRD whether or not to record memory loads in the calling thread. */ VG_USERREQ__DRD_RECORD_LOADS, /* args: Bool. */ /* Tell DRD whether or not to record memory stores in the calling thread. */ VG_USERREQ__DRD_RECORD_STORES, /* args: Bool. */ /* Set the name of the thread that performs this client request. */ VG_USERREQ__DRD_SET_THREAD_NAME, /* args: null-terminated character string. */ /* Tell DRD that a DRD annotation has not yet been implemented. */ VG_USERREQ__DRD_ANNOTATION_UNIMP, /* args: char*. */ /* Tell DRD that a user-defined semaphore synchronization object * is about to be created. */ VG_USERREQ__DRD_ANNOTATE_SEM_INIT_PRE, /* args: Addr, UInt value. */ /* Tell DRD that a user-defined semaphore synchronization object * has been destroyed. */ VG_USERREQ__DRD_ANNOTATE_SEM_DESTROY_POST, /* args: Addr. */ /* Tell DRD that a user-defined semaphore synchronization * object is going to be acquired (semaphore wait). */ VG_USERREQ__DRD_ANNOTATE_SEM_WAIT_PRE, /* args: Addr. */ /* Tell DRD that a user-defined semaphore synchronization * object has been acquired (semaphore wait). */ VG_USERREQ__DRD_ANNOTATE_SEM_WAIT_POST, /* args: Addr. */ /* Tell DRD that a user-defined semaphore synchronization * object is about to be released (semaphore post). */ VG_USERREQ__DRD_ANNOTATE_SEM_POST_PRE, /* args: Addr. */ /* Tell DRD to ignore the inter-thread ordering introduced by a mutex. */ VG_USERREQ__DRD_IGNORE_MUTEX_ORDERING, /* args: Addr. */ /* Tell DRD that a user-defined reader-writer synchronization object * has been created. */ VG_USERREQ__DRD_ANNOTATE_RWLOCK_CREATE = VG_USERREQ_TOOL_BASE('H','G') + 256 + 14, /* args: Addr. */ /* Tell DRD that a user-defined reader-writer synchronization object * is about to be destroyed. */ VG_USERREQ__DRD_ANNOTATE_RWLOCK_DESTROY = VG_USERREQ_TOOL_BASE('H','G') + 256 + 15, /* args: Addr. */ /* Tell DRD that a lock on a user-defined reader-writer synchronization * object has been acquired. */ VG_USERREQ__DRD_ANNOTATE_RWLOCK_ACQUIRED = VG_USERREQ_TOOL_BASE('H','G') + 256 + 17, /* args: Addr, Int is_rw. */ /* Tell DRD that a lock on a user-defined reader-writer synchronization * object is about to be released. */ VG_USERREQ__DRD_ANNOTATE_RWLOCK_RELEASED = VG_USERREQ_TOOL_BASE('H','G') + 256 + 18, /* args: Addr, Int is_rw. */ /* Tell DRD that a Helgrind annotation has not yet been implemented. */ VG_USERREQ__HELGRIND_ANNOTATION_UNIMP = VG_USERREQ_TOOL_BASE('H','G') + 256 + 32, /* args: char*. */ /* Tell DRD to insert a happens-before annotation. */ VG_USERREQ__DRD_ANNOTATE_HAPPENS_BEFORE = VG_USERREQ_TOOL_BASE('H','G') + 256 + 33, /* args: Addr. */ /* Tell DRD to insert a happens-after annotation. */ VG_USERREQ__DRD_ANNOTATE_HAPPENS_AFTER = VG_USERREQ_TOOL_BASE('H','G') + 256 + 34, /* args: Addr. */ }; /** * @addtogroup RaceDetectionAnnotations */ /*@{*/ #ifdef __cplusplus /* ANNOTATE_UNPROTECTED_READ is the preferred way to annotate racy reads. Instead of doing ANNOTATE_IGNORE_READS_BEGIN(); ... = x; ANNOTATE_IGNORE_READS_END(); one can use ... = ANNOTATE_UNPROTECTED_READ(x); */ template inline T ANNOTATE_UNPROTECTED_READ(const volatile T& x) { ANNOTATE_IGNORE_READS_BEGIN(); const T result = x; ANNOTATE_IGNORE_READS_END(); return result; } /* Apply ANNOTATE_BENIGN_RACE_SIZED to a static variable. */ #define ANNOTATE_BENIGN_RACE_STATIC(static_var, description) \ namespace { \ static class static_var##_annotator \ { \ public: \ static_var##_annotator() \ { \ ANNOTATE_BENIGN_RACE_SIZED(&static_var, sizeof(static_var), \ #static_var ": " description); \ } \ } the_##static_var##_annotator; \ } #endif /*@}*/ #endif /* __VALGRIND_DRD_H */ pmdk-1.8/src/common/valgrind/helgrind.h0000664000000000000000000011373313615011243016653 0ustar rootroot/* ---------------------------------------------------------------- Notice that the above BSD-style license applies to this one file (helgrind.h) only. The entire rest of Valgrind is licensed under the terms of the GNU General Public License, version 2. See the COPYING file in the source distribution for details. ---------------------------------------------------------------- This file is part of Helgrind, a Valgrind tool for detecting errors in threaded programs. Copyright (C) 2007-2017 OpenWorks LLP info@open-works.co.uk Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 3. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 4. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ---------------------------------------------------------------- Notice that the above BSD-style license applies to this one file (helgrind.h) only. The entire rest of Valgrind is licensed under the terms of the GNU General Public License, version 2. See the COPYING file in the source distribution for details. ---------------------------------------------------------------- */ #ifndef __HELGRIND_H #define __HELGRIND_H #include "valgrind.h" /* !! ABIWARNING !! ABIWARNING !! ABIWARNING !! ABIWARNING !! This enum comprises an ABI exported by Valgrind to programs which use client requests. DO NOT CHANGE THE ORDER OF THESE ENTRIES, NOR DELETE ANY -- add new ones at the end. */ typedef enum { VG_USERREQ__HG_CLEAN_MEMORY = VG_USERREQ_TOOL_BASE('H','G'), /* The rest are for Helgrind's internal use. Not for end-user use. Do not use them unless you are a Valgrind developer. */ /* Notify the tool what this thread's pthread_t is. */ _VG_USERREQ__HG_SET_MY_PTHREAD_T = VG_USERREQ_TOOL_BASE('H','G') + 256, _VG_USERREQ__HG_PTH_API_ERROR, /* char*, int */ _VG_USERREQ__HG_PTHREAD_JOIN_POST, /* pthread_t of quitter */ _VG_USERREQ__HG_PTHREAD_MUTEX_INIT_POST, /* pth_mx_t*, long mbRec */ _VG_USERREQ__HG_PTHREAD_MUTEX_DESTROY_PRE, /* pth_mx_t*, long isInit */ _VG_USERREQ__HG_PTHREAD_MUTEX_UNLOCK_PRE, /* pth_mx_t* */ _VG_USERREQ__HG_PTHREAD_MUTEX_UNLOCK_POST, /* pth_mx_t* */ _VG_USERREQ__HG_PTHREAD_MUTEX_ACQUIRE_PRE, /* void*, long isTryLock */ _VG_USERREQ__HG_PTHREAD_MUTEX_ACQUIRE_POST, /* void* */ _VG_USERREQ__HG_PTHREAD_COND_SIGNAL_PRE, /* pth_cond_t* */ _VG_USERREQ__HG_PTHREAD_COND_BROADCAST_PRE, /* pth_cond_t* */ _VG_USERREQ__HG_PTHREAD_COND_WAIT_PRE, /* pth_cond_t*, pth_mx_t* */ _VG_USERREQ__HG_PTHREAD_COND_WAIT_POST, /* pth_cond_t*, pth_mx_t* */ _VG_USERREQ__HG_PTHREAD_COND_DESTROY_PRE, /* pth_cond_t*, long isInit */ _VG_USERREQ__HG_PTHREAD_RWLOCK_INIT_POST, /* pth_rwlk_t* */ _VG_USERREQ__HG_PTHREAD_RWLOCK_DESTROY_PRE, /* pth_rwlk_t* */ _VG_USERREQ__HG_PTHREAD_RWLOCK_LOCK_PRE, /* pth_rwlk_t*, long isW */ _VG_USERREQ__HG_PTHREAD_RWLOCK_ACQUIRED, /* void*, long isW */ _VG_USERREQ__HG_PTHREAD_RWLOCK_RELEASED, /* void* */ _VG_USERREQ__HG_PTHREAD_RWLOCK_UNLOCK_POST, /* pth_rwlk_t* */ _VG_USERREQ__HG_POSIX_SEM_INIT_POST, /* sem_t*, ulong value */ _VG_USERREQ__HG_POSIX_SEM_DESTROY_PRE, /* sem_t* */ _VG_USERREQ__HG_POSIX_SEM_RELEASED, /* void* */ _VG_USERREQ__HG_POSIX_SEM_ACQUIRED, /* void* */ _VG_USERREQ__HG_PTHREAD_BARRIER_INIT_PRE, /* pth_bar_t*, ulong, ulong */ _VG_USERREQ__HG_PTHREAD_BARRIER_WAIT_PRE, /* pth_bar_t* */ _VG_USERREQ__HG_PTHREAD_BARRIER_DESTROY_PRE, /* pth_bar_t* */ _VG_USERREQ__HG_PTHREAD_SPIN_INIT_OR_UNLOCK_PRE, /* pth_slk_t* */ _VG_USERREQ__HG_PTHREAD_SPIN_INIT_OR_UNLOCK_POST, /* pth_slk_t* */ _VG_USERREQ__HG_PTHREAD_SPIN_LOCK_PRE, /* pth_slk_t* */ _VG_USERREQ__HG_PTHREAD_SPIN_LOCK_POST, /* pth_slk_t* */ _VG_USERREQ__HG_PTHREAD_SPIN_DESTROY_PRE, /* pth_slk_t* */ _VG_USERREQ__HG_CLIENTREQ_UNIMP, /* char* */ _VG_USERREQ__HG_USERSO_SEND_PRE, /* arbitrary UWord SO-tag */ _VG_USERREQ__HG_USERSO_RECV_POST, /* arbitrary UWord SO-tag */ _VG_USERREQ__HG_USERSO_FORGET_ALL, /* arbitrary UWord SO-tag */ _VG_USERREQ__HG_RESERVED2, /* Do not use */ _VG_USERREQ__HG_RESERVED3, /* Do not use */ _VG_USERREQ__HG_RESERVED4, /* Do not use */ _VG_USERREQ__HG_ARANGE_MAKE_UNTRACKED, /* Addr a, ulong len */ _VG_USERREQ__HG_ARANGE_MAKE_TRACKED, /* Addr a, ulong len */ _VG_USERREQ__HG_PTHREAD_BARRIER_RESIZE_PRE, /* pth_bar_t*, ulong */ _VG_USERREQ__HG_CLEAN_MEMORY_HEAPBLOCK, /* Addr start_of_block */ _VG_USERREQ__HG_PTHREAD_COND_INIT_POST, /* pth_cond_t*, pth_cond_attr_t*/ _VG_USERREQ__HG_GNAT_MASTER_HOOK, /* void*d,void*m,Word ml */ _VG_USERREQ__HG_GNAT_MASTER_COMPLETED_HOOK, /* void*s,Word ml */ _VG_USERREQ__HG_GET_ABITS, /* Addr a,Addr abits, ulong len */ _VG_USERREQ__HG_PTHREAD_CREATE_BEGIN, _VG_USERREQ__HG_PTHREAD_CREATE_END, _VG_USERREQ__HG_PTHREAD_MUTEX_LOCK_PRE, /* pth_mx_t*,long isTryLock */ _VG_USERREQ__HG_PTHREAD_MUTEX_LOCK_POST, /* pth_mx_t *,long tookLock */ _VG_USERREQ__HG_PTHREAD_RWLOCK_LOCK_POST, /* pth_rwlk_t*,long isW,long */ _VG_USERREQ__HG_PTHREAD_RWLOCK_UNLOCK_PRE, /* pth_rwlk_t* */ _VG_USERREQ__HG_POSIX_SEM_POST_PRE, /* sem_t* */ _VG_USERREQ__HG_POSIX_SEM_POST_POST, /* sem_t* */ _VG_USERREQ__HG_POSIX_SEM_WAIT_PRE, /* sem_t* */ _VG_USERREQ__HG_POSIX_SEM_WAIT_POST, /* sem_t*, long tookLock */ _VG_USERREQ__HG_PTHREAD_COND_SIGNAL_POST, /* pth_cond_t* */ _VG_USERREQ__HG_PTHREAD_COND_BROADCAST_POST,/* pth_cond_t* */ _VG_USERREQ__HG_RTLD_BIND_GUARD, /* int flags */ _VG_USERREQ__HG_RTLD_BIND_CLEAR, /* int flags */ _VG_USERREQ__HG_GNAT_DEPENDENT_MASTER_JOIN /* void*d, void*m */ } Vg_TCheckClientRequest; /*----------------------------------------------------------------*/ /*--- ---*/ /*--- Implementation-only facilities. Not for end-user use. ---*/ /*--- For end-user facilities see below (the next section in ---*/ /*--- this file.) ---*/ /*--- ---*/ /*----------------------------------------------------------------*/ /* Do a client request. These are macros rather than a functions so as to avoid having an extra frame in stack traces. NB: these duplicate definitions in hg_intercepts.c. But here, we have to make do with weaker typing (no definition of Word etc) and no assertions, whereas in helgrind.h we can use those facilities. Obviously it's important the two sets of definitions are kept in sync. The commented-out asserts should actually hold, but unfortunately they can't be allowed to be visible here, because that would require the end-user code to #include . */ #define DO_CREQ_v_W(_creqF, _ty1F,_arg1F) \ do { \ long int _arg1; \ /* assert(sizeof(_ty1F) == sizeof(long int)); */ \ _arg1 = (long int)(_arg1F); \ VALGRIND_DO_CLIENT_REQUEST_STMT( \ (_creqF), \ _arg1, 0,0,0,0); \ } while (0) #define DO_CREQ_W_W(_resF, _dfltF, _creqF, _ty1F,_arg1F) \ do { \ long int _arg1; \ /* assert(sizeof(_ty1F) == sizeof(long int)); */ \ _arg1 = (long int)(_arg1F); \ _qzz_res = VALGRIND_DO_CLIENT_REQUEST_EXPR( \ (_dfltF), \ (_creqF), \ _arg1, 0,0,0,0); \ _resF = _qzz_res; \ } while (0) #define DO_CREQ_v_WW(_creqF, _ty1F,_arg1F, _ty2F,_arg2F) \ do { \ long int _arg1, _arg2; \ /* assert(sizeof(_ty1F) == sizeof(long int)); */ \ /* assert(sizeof(_ty2F) == sizeof(long int)); */ \ _arg1 = (long int)(_arg1F); \ _arg2 = (long int)(_arg2F); \ VALGRIND_DO_CLIENT_REQUEST_STMT( \ (_creqF), \ _arg1,_arg2,0,0,0); \ } while (0) #define DO_CREQ_v_WWW(_creqF, _ty1F,_arg1F, \ _ty2F,_arg2F, _ty3F, _arg3F) \ do { \ long int _arg1, _arg2, _arg3; \ /* assert(sizeof(_ty1F) == sizeof(long int)); */ \ /* assert(sizeof(_ty2F) == sizeof(long int)); */ \ /* assert(sizeof(_ty3F) == sizeof(long int)); */ \ _arg1 = (long int)(_arg1F); \ _arg2 = (long int)(_arg2F); \ _arg3 = (long int)(_arg3F); \ VALGRIND_DO_CLIENT_REQUEST_STMT( \ (_creqF), \ _arg1,_arg2,_arg3,0,0); \ } while (0) #define DO_CREQ_W_WWW(_resF, _dfltF, _creqF, _ty1F,_arg1F, \ _ty2F,_arg2F, _ty3F, _arg3F) \ do { \ long int _qzz_res; \ long int _arg1, _arg2, _arg3; \ /* assert(sizeof(_ty1F) == sizeof(long int)); */ \ _arg1 = (long int)(_arg1F); \ _arg2 = (long int)(_arg2F); \ _arg3 = (long int)(_arg3F); \ _qzz_res = VALGRIND_DO_CLIENT_REQUEST_EXPR( \ (_dfltF), \ (_creqF), \ _arg1,_arg2,_arg3,0,0); \ _resF = _qzz_res; \ } while (0) #define _HG_CLIENTREQ_UNIMP(_qzz_str) \ DO_CREQ_v_W(_VG_USERREQ__HG_CLIENTREQ_UNIMP, \ (char*),(_qzz_str)) /*----------------------------------------------------------------*/ /*--- ---*/ /*--- Helgrind-native requests. These allow access to ---*/ /*--- the same set of annotation primitives that are used ---*/ /*--- to build the POSIX pthread wrappers. ---*/ /*--- ---*/ /*----------------------------------------------------------------*/ /* ---------------------------------------------------------- For describing ordinary mutexes (non-rwlocks). For rwlock descriptions see ANNOTATE_RWLOCK_* below. ---------------------------------------------------------- */ /* Notify here immediately after mutex creation. _mbRec == 0 for a non-recursive mutex, 1 for a recursive mutex. */ #define VALGRIND_HG_MUTEX_INIT_POST(_mutex, _mbRec) \ DO_CREQ_v_WW(_VG_USERREQ__HG_PTHREAD_MUTEX_INIT_POST, \ void*,(_mutex), long,(_mbRec)) /* Notify here immediately before mutex acquisition. _isTryLock == 0 for a normal acquisition, 1 for a "try" style acquisition. */ #define VALGRIND_HG_MUTEX_LOCK_PRE(_mutex, _isTryLock) \ DO_CREQ_v_WW(_VG_USERREQ__HG_PTHREAD_MUTEX_ACQUIRE_PRE, \ void*,(_mutex), long,(_isTryLock)) /* Notify here immediately after a successful mutex acquisition. */ #define VALGRIND_HG_MUTEX_LOCK_POST(_mutex) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_MUTEX_ACQUIRE_POST, \ void*,(_mutex)) /* Notify here immediately before a mutex release. */ #define VALGRIND_HG_MUTEX_UNLOCK_PRE(_mutex) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_MUTEX_UNLOCK_PRE, \ void*,(_mutex)) /* Notify here immediately after a mutex release. */ #define VALGRIND_HG_MUTEX_UNLOCK_POST(_mutex) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_MUTEX_UNLOCK_POST, \ void*,(_mutex)) /* Notify here immediately before mutex destruction. */ #define VALGRIND_HG_MUTEX_DESTROY_PRE(_mutex) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_MUTEX_DESTROY_PRE, \ void*,(_mutex)) /* ---------------------------------------------------------- For describing semaphores. ---------------------------------------------------------- */ /* Notify here immediately after semaphore creation. */ #define VALGRIND_HG_SEM_INIT_POST(_sem, _value) \ DO_CREQ_v_WW(_VG_USERREQ__HG_POSIX_SEM_INIT_POST, \ void*, (_sem), unsigned long, (_value)) /* Notify here immediately after a semaphore wait (an acquire-style operation) */ #define VALGRIND_HG_SEM_WAIT_POST(_sem) \ DO_CREQ_v_W(_VG_USERREQ__HG_POSIX_SEM_ACQUIRED, \ void*,(_sem)) /* Notify here immediately before semaphore post (a release-style operation) */ #define VALGRIND_HG_SEM_POST_PRE(_sem) \ DO_CREQ_v_W(_VG_USERREQ__HG_POSIX_SEM_RELEASED, \ void*,(_sem)) /* Notify here immediately before semaphore destruction. */ #define VALGRIND_HG_SEM_DESTROY_PRE(_sem) \ DO_CREQ_v_W(_VG_USERREQ__HG_POSIX_SEM_DESTROY_PRE, \ void*, (_sem)) /* ---------------------------------------------------------- For describing barriers. ---------------------------------------------------------- */ /* Notify here immediately before barrier creation. _count is the capacity. _resizable == 0 means the barrier may not be resized, 1 means it may be. */ #define VALGRIND_HG_BARRIER_INIT_PRE(_bar, _count, _resizable) \ DO_CREQ_v_WWW(_VG_USERREQ__HG_PTHREAD_BARRIER_INIT_PRE, \ void*,(_bar), \ unsigned long,(_count), \ unsigned long,(_resizable)) /* Notify here immediately before arrival at a barrier. */ #define VALGRIND_HG_BARRIER_WAIT_PRE(_bar) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_BARRIER_WAIT_PRE, \ void*,(_bar)) /* Notify here immediately before a resize (change of barrier capacity). If _newcount >= the existing capacity, then there is no change in the state of any threads waiting at the barrier. If _newcount < the existing capacity, and >= _newcount threads are currently waiting at the barrier, then this notification is considered to also have the effect of telling the checker that all waiting threads have now moved past the barrier. (I can't think of any other sane semantics.) */ #define VALGRIND_HG_BARRIER_RESIZE_PRE(_bar, _newcount) \ DO_CREQ_v_WW(_VG_USERREQ__HG_PTHREAD_BARRIER_RESIZE_PRE, \ void*,(_bar), \ unsigned long,(_newcount)) /* Notify here immediately before barrier destruction. */ #define VALGRIND_HG_BARRIER_DESTROY_PRE(_bar) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_BARRIER_DESTROY_PRE, \ void*,(_bar)) /* ---------------------------------------------------------- For describing memory ownership changes. ---------------------------------------------------------- */ /* Clean memory state. This makes Helgrind forget everything it knew about the specified memory range. Effectively this announces that the specified memory range now "belongs" to the calling thread, so that: (1) the calling thread can access it safely without synchronisation, and (2) all other threads must sync with this one to access it safely. This is particularly useful for memory allocators that wish to recycle memory. */ #define VALGRIND_HG_CLEAN_MEMORY(_qzz_start, _qzz_len) \ DO_CREQ_v_WW(VG_USERREQ__HG_CLEAN_MEMORY, \ void*,(_qzz_start), \ unsigned long,(_qzz_len)) /* The same, but for the heap block starting at _qzz_blockstart. This allows painting when we only know the address of an object, but not its size, which is sometimes the case in C++ code involving inheritance, and in which RTTI is not, for whatever reason, available. Returns the number of bytes painted, which can be zero for a zero-sized block. Hence, return values >= 0 indicate success (the block was found), and the value -1 indicates block not found, and -2 is returned when not running on Helgrind. */ #define VALGRIND_HG_CLEAN_MEMORY_HEAPBLOCK(_qzz_blockstart) \ (__extension__ \ ({long int _npainted; \ DO_CREQ_W_W(_npainted, (-2)/*default*/, \ _VG_USERREQ__HG_CLEAN_MEMORY_HEAPBLOCK, \ void*,(_qzz_blockstart)); \ _npainted; \ })) /* ---------------------------------------------------------- For error control. ---------------------------------------------------------- */ /* Tell H that an address range is not to be "tracked" until further notice. This puts it in the NOACCESS state, in which case we ignore all reads and writes to it. Useful for ignoring ranges of memory where there might be races we don't want to see. If the memory is subsequently reallocated via malloc/new/stack allocation, then it is put back in the trackable state. Hence it is safe in the situation where checking is disabled, the containing area is deallocated and later reallocated for some other purpose. */ #define VALGRIND_HG_DISABLE_CHECKING(_qzz_start, _qzz_len) \ DO_CREQ_v_WW(_VG_USERREQ__HG_ARANGE_MAKE_UNTRACKED, \ void*,(_qzz_start), \ unsigned long,(_qzz_len)) /* And put it back into the normal "tracked" state, that is, make it once again subject to the normal race-checking machinery. This puts it in the same state as new memory allocated by this thread -- that is, basically owned exclusively by this thread. */ #define VALGRIND_HG_ENABLE_CHECKING(_qzz_start, _qzz_len) \ DO_CREQ_v_WW(_VG_USERREQ__HG_ARANGE_MAKE_TRACKED, \ void*,(_qzz_start), \ unsigned long,(_qzz_len)) /* Checks the accessibility bits for addresses [zza..zza+zznbytes-1]. If zzabits array is provided, copy the accessibility bits in zzabits. Return values: -2 if not running on helgrind -1 if any parts of zzabits is not addressable >= 0 : success. When success, it returns the nr of addressable bytes found. So, to check that a whole range is addressable, check VALGRIND_HG_GET_ABITS(addr,NULL,len) == len In addition, if you want to examine the addressability of each byte of the range, you need to provide a non NULL ptr as second argument, pointing to an array of unsigned char of length len. Addressable bytes are indicated with 0xff. Non-addressable bytes are indicated with 0x00. */ #define VALGRIND_HG_GET_ABITS(zza,zzabits,zznbytes) \ (__extension__ \ ({long int _res; \ DO_CREQ_W_WWW(_res, (-2)/*default*/, \ _VG_USERREQ__HG_GET_ABITS, \ void*,(zza), void*,(zzabits), \ unsigned long,(zznbytes)); \ _res; \ })) /* End-user request for Ada applications compiled with GNAT. Helgrind understands the Ada concept of Ada task dependencies and terminations. See Ada Reference Manual section 9.3 "Task Dependence - Termination of Tasks". However, in some cases, the master of (terminated) tasks completes only when the application exits. An example of this is dynamically allocated tasks with an access type defined at Library Level. By default, the state of such tasks in Helgrind will be 'exited but join not done yet'. Many tasks in such a state are however causing Helgrind CPU and memory to increase significantly. VALGRIND_HG_GNAT_DEPENDENT_MASTER_JOIN can be used to indicate to Helgrind that a not yet completed master has however already 'seen' the termination of a dependent : this is conceptually the same as a pthread_join and causes the cleanup of the dependent as done by Helgrind when a master completes. This allows to avoid the overhead in helgrind caused by such tasks. A typical usage for a master to indicate it has done conceptually a join with a dependent task before the master completes is: while not Dep_Task'Terminated loop ... do whatever to wait for Dep_Task termination. end loop; VALGRIND_HG_GNAT_DEPENDENT_MASTER_JOIN (Dep_Task'Identity, Ada.Task_Identification.Current_Task); Note that VALGRIND_HG_GNAT_DEPENDENT_MASTER_JOIN should be a binding to a C function built with the below macro. */ #define VALGRIND_HG_GNAT_DEPENDENT_MASTER_JOIN(_qzz_dep, _qzz_master) \ DO_CREQ_v_WW(_VG_USERREQ__HG_GNAT_DEPENDENT_MASTER_JOIN, \ void*,(_qzz_dep), \ void*,(_qzz_master)) /*----------------------------------------------------------------*/ /*--- ---*/ /*--- ThreadSanitizer-compatible requests ---*/ /*--- (mostly unimplemented) ---*/ /*--- ---*/ /*----------------------------------------------------------------*/ /* A quite-broad set of annotations, as used in the ThreadSanitizer project. This implementation aims to be a (source-level) compatible implementation of the macros defined in: http://code.google.com/p/data-race-test/source /browse/trunk/dynamic_annotations/dynamic_annotations.h (some of the comments below are taken from the above file) The implementation here is very incomplete, and intended as a starting point. Many of the macros are unimplemented. Rather than allowing unimplemented macros to silently do nothing, they cause an assertion. Intention is to implement them on demand. The major use of these macros is to make visible to race detectors, the behaviour (effects) of user-implemented synchronisation primitives, that the detectors could not otherwise deduce from the normal observation of pthread etc calls. Some of the macros are no-ops in Helgrind. That's because Helgrind is a pure happens-before detector, whereas ThreadSanitizer uses a hybrid lockset and happens-before scheme, which requires more accurate annotations for correct operation. The macros are listed in the same order as in dynamic_annotations.h (URL just above). I should point out that I am less than clear about the intended semantics of quite a number of them. Comments and clarifications welcomed! */ /* ---------------------------------------------------------------- These four allow description of user-level condition variables, apparently in the style of POSIX's pthread_cond_t. Currently unimplemented and will assert. ---------------------------------------------------------------- */ /* Report that wait on the condition variable at address CV has succeeded and the lock at address LOCK is now held. CV and LOCK are completely arbitrary memory addresses which presumably mean something to the application, but are meaningless to Helgrind. */ #define ANNOTATE_CONDVAR_LOCK_WAIT(cv, lock) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_CONDVAR_LOCK_WAIT") /* Report that wait on the condition variable at CV has succeeded. Variant w/o lock. */ #define ANNOTATE_CONDVAR_WAIT(cv) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_CONDVAR_WAIT") /* Report that we are about to signal on the condition variable at address CV. */ #define ANNOTATE_CONDVAR_SIGNAL(cv) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_CONDVAR_SIGNAL") /* Report that we are about to signal_all on the condition variable at CV. */ #define ANNOTATE_CONDVAR_SIGNAL_ALL(cv) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_CONDVAR_SIGNAL_ALL") /* ---------------------------------------------------------------- Create completely arbitrary happens-before edges between threads. If threads T1 .. Tn all do ANNOTATE_HAPPENS_BEFORE(obj) and later (w.r.t. some notional global clock for the computation) thread Tm does ANNOTATE_HAPPENS_AFTER(obj), then Helgrind will regard all memory accesses done by T1 .. Tn before the ..BEFORE.. call as happening-before all memory accesses done by Tm after the ..AFTER.. call. Hence Helgrind won't complain about races if Tm's accesses afterwards are to the same locations as accesses before by any of T1 .. Tn. OBJ is a machine word (unsigned long, or void*), is completely arbitrary, and denotes the identity of some synchronisation object you're modelling. You must do the _BEFORE call just before the real sync event on the signaller's side, and _AFTER just after the real sync event on the waiter's side. If none of the rest of these macros make sense to you, at least take the time to understand these two. They form the very essence of describing arbitrary inter-thread synchronisation events to Helgrind. You can get a long way just with them alone. See also, extensive discussion on semantics of this in https://bugs.kde.org/show_bug.cgi?id=243935 ANNOTATE_HAPPENS_BEFORE_FORGET_ALL(obj) is interim until such time as bug 243935 is fully resolved. It instructs Helgrind to forget about any ANNOTATE_HAPPENS_BEFORE calls on the specified object, in effect putting it back in its original state. Once in that state, a use of ANNOTATE_HAPPENS_AFTER on it has no effect on the calling thread. An implementation may optionally release resources it has associated with 'obj' when ANNOTATE_HAPPENS_BEFORE_FORGET_ALL(obj) happens. Users are recommended to use ANNOTATE_HAPPENS_BEFORE_FORGET_ALL to indicate when a synchronisation object is no longer needed, so as to avoid potential indefinite resource leaks. ---------------------------------------------------------------- */ #define ANNOTATE_HAPPENS_BEFORE(obj) \ DO_CREQ_v_W(_VG_USERREQ__HG_USERSO_SEND_PRE, void*,(obj)) #define ANNOTATE_HAPPENS_AFTER(obj) \ DO_CREQ_v_W(_VG_USERREQ__HG_USERSO_RECV_POST, void*,(obj)) #define ANNOTATE_HAPPENS_BEFORE_FORGET_ALL(obj) \ DO_CREQ_v_W(_VG_USERREQ__HG_USERSO_FORGET_ALL, void*,(obj)) /* ---------------------------------------------------------------- Memory publishing. The TSan sources say: Report that the bytes in the range [pointer, pointer+size) are about to be published safely. The race checker will create a happens-before arc from the call ANNOTATE_PUBLISH_MEMORY_RANGE(pointer, size) to subsequent accesses to this memory. I'm not sure I understand what this means exactly, nor whether it is relevant for a pure h-b detector. Leaving unimplemented for now. ---------------------------------------------------------------- */ #define ANNOTATE_PUBLISH_MEMORY_RANGE(pointer, size) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_PUBLISH_MEMORY_RANGE") /* DEPRECATED. Don't use it. */ /* #define ANNOTATE_UNPUBLISH_MEMORY_RANGE(pointer, size) */ /* DEPRECATED. Don't use it. */ /* #define ANNOTATE_SWAP_MEMORY_RANGE(pointer, size) */ /* ---------------------------------------------------------------- TSan sources say: Instruct the tool to create a happens-before arc between MU->Unlock() and MU->Lock(). This annotation may slow down the race detector; normally it is used only when it would be difficult to annotate each of the mutex's critical sections individually using the annotations above. If MU is a posix pthread_mutex_t then Helgrind will do this anyway. In any case, leave as unimp for now. I'm unsure about the intended behaviour. ---------------------------------------------------------------- */ #define ANNOTATE_PURE_HAPPENS_BEFORE_MUTEX(mu) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_PURE_HAPPENS_BEFORE_MUTEX") /* Deprecated. Use ANNOTATE_PURE_HAPPENS_BEFORE_MUTEX. */ /* #define ANNOTATE_MUTEX_IS_USED_AS_CONDVAR(mu) */ /* ---------------------------------------------------------------- TSan sources say: Annotations useful when defining memory allocators, or when memory that was protected in one way starts to be protected in another. Report that a new memory at "address" of size "size" has been allocated. This might be used when the memory has been retrieved from a free list and is about to be reused, or when a the locking discipline for a variable changes. AFAICS this is the same as VALGRIND_HG_CLEAN_MEMORY. ---------------------------------------------------------------- */ #define ANNOTATE_NEW_MEMORY(address, size) \ VALGRIND_HG_CLEAN_MEMORY((address), (size)) /* ---------------------------------------------------------------- TSan sources say: Annotations useful when defining FIFO queues that transfer data between threads. All unimplemented. Am not claiming to understand this (yet). ---------------------------------------------------------------- */ /* Report that the producer-consumer queue object at address PCQ has been created. The ANNOTATE_PCQ_* annotations should be used only for FIFO queues. For non-FIFO queues use ANNOTATE_HAPPENS_BEFORE (for put) and ANNOTATE_HAPPENS_AFTER (for get). */ #define ANNOTATE_PCQ_CREATE(pcq) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_PCQ_CREATE") /* Report that the queue at address PCQ is about to be destroyed. */ #define ANNOTATE_PCQ_DESTROY(pcq) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_PCQ_DESTROY") /* Report that we are about to put an element into a FIFO queue at address PCQ. */ #define ANNOTATE_PCQ_PUT(pcq) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_PCQ_PUT") /* Report that we've just got an element from a FIFO queue at address PCQ. */ #define ANNOTATE_PCQ_GET(pcq) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_PCQ_GET") /* ---------------------------------------------------------------- Annotations that suppress errors. It is usually better to express the program's synchronization using the other annotations, but these can be used when all else fails. Currently these are all unimplemented. I can't think of a simple way to implement them without at least some performance overhead. ---------------------------------------------------------------- */ /* Report that we may have a benign race at "pointer", with size "sizeof(*(pointer))". "pointer" must be a non-void* pointer. Insert at the point where "pointer" has been allocated, preferably close to the point where the race happens. See also ANNOTATE_BENIGN_RACE_STATIC. XXX: what's this actually supposed to do? And what's the type of DESCRIPTION? When does the annotation stop having an effect? */ #define ANNOTATE_BENIGN_RACE(pointer, description) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_BENIGN_RACE") /* Same as ANNOTATE_BENIGN_RACE(address, description), but applies to the memory range [address, address+size). */ #define ANNOTATE_BENIGN_RACE_SIZED(address, size, description) \ VALGRIND_HG_DISABLE_CHECKING(address, size) /* Request the analysis tool to ignore all reads in the current thread until ANNOTATE_IGNORE_READS_END is called. Useful to ignore intentional racey reads, while still checking other reads and all writes. */ #define ANNOTATE_IGNORE_READS_BEGIN() \ _HG_CLIENTREQ_UNIMP("ANNOTATE_IGNORE_READS_BEGIN") /* Stop ignoring reads. */ #define ANNOTATE_IGNORE_READS_END() \ _HG_CLIENTREQ_UNIMP("ANNOTATE_IGNORE_READS_END") /* Similar to ANNOTATE_IGNORE_READS_BEGIN, but ignore writes. */ #define ANNOTATE_IGNORE_WRITES_BEGIN() \ _HG_CLIENTREQ_UNIMP("ANNOTATE_IGNORE_WRITES_BEGIN") /* Stop ignoring writes. */ #define ANNOTATE_IGNORE_WRITES_END() \ _HG_CLIENTREQ_UNIMP("ANNOTATE_IGNORE_WRITES_END") /* Start ignoring all memory accesses (reads and writes). */ #define ANNOTATE_IGNORE_READS_AND_WRITES_BEGIN() \ do { \ ANNOTATE_IGNORE_READS_BEGIN(); \ ANNOTATE_IGNORE_WRITES_BEGIN(); \ } while (0) /* Stop ignoring all memory accesses. */ #define ANNOTATE_IGNORE_READS_AND_WRITES_END() \ do { \ ANNOTATE_IGNORE_WRITES_END(); \ ANNOTATE_IGNORE_READS_END(); \ } while (0) /* ---------------------------------------------------------------- Annotations useful for debugging. Again, so for unimplemented, partly for performance reasons. ---------------------------------------------------------------- */ /* Request to trace every access to ADDRESS. */ #define ANNOTATE_TRACE_MEMORY(address) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_TRACE_MEMORY") /* Report the current thread name to a race detector. */ #define ANNOTATE_THREAD_NAME(name) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_THREAD_NAME") /* ---------------------------------------------------------------- Annotations for describing behaviour of user-implemented lock primitives. In all cases, the LOCK argument is a completely arbitrary machine word (unsigned long, or void*) and can be any value which gives a unique identity to the lock objects being modelled. We just pretend they're ordinary posix rwlocks. That'll probably give some rather confusing wording in error messages, claiming that the arbitrary LOCK values are pthread_rwlock_t*'s, when in fact they are not. Ah well. ---------------------------------------------------------------- */ /* Report that a lock has just been created at address LOCK. */ #define ANNOTATE_RWLOCK_CREATE(lock) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_RWLOCK_INIT_POST, \ void*,(lock)) /* Report that the lock at address LOCK is about to be destroyed. */ #define ANNOTATE_RWLOCK_DESTROY(lock) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_RWLOCK_DESTROY_PRE, \ void*,(lock)) /* Report that the lock at address LOCK has just been acquired. is_w=1 for writer lock, is_w=0 for reader lock. */ #define ANNOTATE_RWLOCK_ACQUIRED(lock, is_w) \ DO_CREQ_v_WW(_VG_USERREQ__HG_PTHREAD_RWLOCK_ACQUIRED, \ void*,(lock), unsigned long,(is_w)) /* Report that the lock at address LOCK is about to be released. */ #define ANNOTATE_RWLOCK_RELEASED(lock, is_w) \ DO_CREQ_v_W(_VG_USERREQ__HG_PTHREAD_RWLOCK_RELEASED, \ void*,(lock)) /* is_w is ignored */ /* ------------------------------------------------------------- Annotations useful when implementing barriers. They are not normally needed by modules that merely use barriers. The "barrier" argument is a pointer to the barrier object. ---------------------------------------------------------------- */ /* Report that the "barrier" has been initialized with initial "count". If 'reinitialization_allowed' is true, initialization is allowed to happen multiple times w/o calling barrier_destroy() */ #define ANNOTATE_BARRIER_INIT(barrier, count, reinitialization_allowed) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_BARRIER_INIT") /* Report that we are about to enter barrier_wait("barrier"). */ #define ANNOTATE_BARRIER_WAIT_BEFORE(barrier) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_BARRIER_DESTROY") /* Report that we just exited barrier_wait("barrier"). */ #define ANNOTATE_BARRIER_WAIT_AFTER(barrier) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_BARRIER_DESTROY") /* Report that the "barrier" has been destroyed. */ #define ANNOTATE_BARRIER_DESTROY(barrier) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_BARRIER_DESTROY") /* ---------------------------------------------------------------- Annotations useful for testing race detectors. ---------------------------------------------------------------- */ /* Report that we expect a race on the variable at ADDRESS. Use only in unit tests for a race detector. */ #define ANNOTATE_EXPECT_RACE(address, description) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_EXPECT_RACE") /* A no-op. Insert where you like to test the interceptors. */ #define ANNOTATE_NO_OP(arg) \ _HG_CLIENTREQ_UNIMP("ANNOTATE_NO_OP") /* Force the race detector to flush its state. The actual effect depends on * the implementation of the detector. */ #define ANNOTATE_FLUSH_STATE() \ _HG_CLIENTREQ_UNIMP("ANNOTATE_FLUSH_STATE") #endif /* __HELGRIND_H */ pmdk-1.8/src/common/valgrind/README0000664000000000000000000000013313615011243015553 0ustar rootrootFiles in this directory were imported from Valgrind 3.14: https://github.com/pmem/valgrind pmdk-1.8/src/common/valgrind/valgrind.h0000664000000000000000000137522113615011243016670 0ustar rootroot/* -*- c -*- ---------------------------------------------------------------- Notice that the following BSD-style license applies to this one file (valgrind.h) only. The rest of Valgrind is licensed under the terms of the GNU General Public License, version 2, unless otherwise indicated. See the COPYING file in the source distribution for details. ---------------------------------------------------------------- This file is part of Valgrind, a dynamic binary instrumentation framework. Copyright (C) 2000-2017 Julian Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 3. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 4. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ---------------------------------------------------------------- Notice that the above BSD-style license applies to this one file (valgrind.h) only. The entire rest of Valgrind is licensed under the terms of the GNU General Public License, version 2. See the COPYING file in the source distribution for details. ---------------------------------------------------------------- */ /* This file is for inclusion into client (your!) code. You can use these macros to manipulate and query Valgrind's execution inside your own programs. The resulting executables will still run without Valgrind, just a little bit more slowly than they otherwise would, but otherwise unchanged. When not running on valgrind, each client request consumes very few (eg. 7) instructions, so the resulting performance loss is negligible unless you plan to execute client requests millions of times per second. Nevertheless, if that is still a problem, you can compile with the NVALGRIND symbol defined (gcc -DNVALGRIND) so that client requests are not even compiled in. */ #ifndef __VALGRIND_H #define __VALGRIND_H /* ------------------------------------------------------------------ */ /* VERSION NUMBER OF VALGRIND */ /* ------------------------------------------------------------------ */ /* Specify Valgrind's version number, so that user code can conditionally compile based on our version number. Note that these were introduced at version 3.6 and so do not exist in version 3.5 or earlier. The recommended way to use them to check for "version X.Y or later" is (eg) #if defined(__VALGRIND_MAJOR__) && defined(__VALGRIND_MINOR__) \ && (__VALGRIND_MAJOR__ > 3 \ || (__VALGRIND_MAJOR__ == 3 && __VALGRIND_MINOR__ >= 6)) */ #define __VALGRIND_MAJOR__ 3 #define __VALGRIND_MINOR__ 14 #include /* Nb: this file might be included in a file compiled with -ansi. So we can't use C++ style "//" comments nor the "asm" keyword (instead use "__asm__"). */ /* Derive some tags indicating what the target platform is. Note that in this file we're using the compiler's CPP symbols for identifying architectures, which are different to the ones we use within the rest of Valgrind. Note, __powerpc__ is active for both 32 and 64-bit PPC, whereas __powerpc64__ is only active for the latter (on Linux, that is). Misc note: how to find out what's predefined in gcc by default: gcc -Wp,-dM somefile.c */ #undef PLAT_x86_darwin #undef PLAT_amd64_darwin #undef PLAT_x86_win32 #undef PLAT_amd64_win64 #undef PLAT_x86_linux #undef PLAT_amd64_linux #undef PLAT_ppc32_linux #undef PLAT_ppc64be_linux #undef PLAT_ppc64le_linux #undef PLAT_arm_linux #undef PLAT_arm64_linux #undef PLAT_s390x_linux #undef PLAT_mips32_linux #undef PLAT_mips64_linux #undef PLAT_x86_solaris #undef PLAT_amd64_solaris #if defined(__APPLE__) && defined(__i386__) # define PLAT_x86_darwin 1 #elif defined(__APPLE__) && defined(__x86_64__) # define PLAT_amd64_darwin 1 #elif (defined(__MINGW32__) && !defined(__MINGW64__)) \ || defined(__CYGWIN32__) \ || (defined(_WIN32) && defined(_M_IX86)) # define PLAT_x86_win32 1 #elif defined(__MINGW64__) \ || (defined(_WIN64) && defined(_M_X64)) # define PLAT_amd64_win64 1 #elif defined(__linux__) && defined(__i386__) # define PLAT_x86_linux 1 #elif defined(__linux__) && defined(__x86_64__) && !defined(__ILP32__) # define PLAT_amd64_linux 1 #elif defined(__linux__) && defined(__powerpc__) && !defined(__powerpc64__) # define PLAT_ppc32_linux 1 #elif defined(__linux__) && defined(__powerpc__) && defined(__powerpc64__) && _CALL_ELF != 2 /* Big Endian uses ELF version 1 */ # define PLAT_ppc64be_linux 1 #elif defined(__linux__) && defined(__powerpc__) && defined(__powerpc64__) && _CALL_ELF == 2 /* Little Endian uses ELF version 2 */ # define PLAT_ppc64le_linux 1 #elif defined(__linux__) && defined(__arm__) && !defined(__aarch64__) # define PLAT_arm_linux 1 #elif defined(__linux__) && defined(__aarch64__) && !defined(__arm__) # define PLAT_arm64_linux 1 #elif defined(__linux__) && defined(__s390__) && defined(__s390x__) # define PLAT_s390x_linux 1 #elif defined(__linux__) && defined(__mips__) && (__mips==64) # define PLAT_mips64_linux 1 #elif defined(__linux__) && defined(__mips__) && (__mips!=64) # define PLAT_mips32_linux 1 #elif defined(__sun) && defined(__i386__) # define PLAT_x86_solaris 1 #elif defined(__sun) && defined(__x86_64__) # define PLAT_amd64_solaris 1 #else /* If we're not compiling for our target platform, don't generate any inline asms. */ # if !defined(NVALGRIND) # define NVALGRIND 1 # endif #endif /* ------------------------------------------------------------------ */ /* ARCHITECTURE SPECIFICS for SPECIAL INSTRUCTIONS. There is nothing */ /* in here of use to end-users -- skip to the next section. */ /* ------------------------------------------------------------------ */ /* * VALGRIND_DO_CLIENT_REQUEST(): a statement that invokes a Valgrind client * request. Accepts both pointers and integers as arguments. * * VALGRIND_DO_CLIENT_REQUEST_STMT(): a statement that invokes a Valgrind * client request that does not return a value. * VALGRIND_DO_CLIENT_REQUEST_EXPR(): a C expression that invokes a Valgrind * client request and whose value equals the client request result. Accepts * both pointers and integers as arguments. Note that such calls are not * necessarily pure functions -- they may have side effects. */ #define VALGRIND_DO_CLIENT_REQUEST(_zzq_rlval, _zzq_default, \ _zzq_request, _zzq_arg1, _zzq_arg2, \ _zzq_arg3, _zzq_arg4, _zzq_arg5) \ do { (_zzq_rlval) = VALGRIND_DO_CLIENT_REQUEST_EXPR((_zzq_default), \ (_zzq_request), (_zzq_arg1), (_zzq_arg2), \ (_zzq_arg3), (_zzq_arg4), (_zzq_arg5)); } while (0) #define VALGRIND_DO_CLIENT_REQUEST_STMT(_zzq_request, _zzq_arg1, \ _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ do { (void) VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ (_zzq_request), (_zzq_arg1), (_zzq_arg2), \ (_zzq_arg3), (_zzq_arg4), (_zzq_arg5)); } while (0) #if defined(NVALGRIND) /* Define NVALGRIND to completely remove the Valgrind magic sequence from the compiled code (analogous to NDEBUG's effects on assert()) */ #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ (_zzq_default) #else /* ! NVALGRIND */ /* The following defines the magic code sequences which the JITter spots and handles magically. Don't look too closely at them as they will rot your brain. The assembly code sequences for all architectures is in this one file. This is because this file must be stand-alone, and we don't want to have multiple files. For VALGRIND_DO_CLIENT_REQUEST, we must ensure that the default value gets put in the return slot, so that everything works when this is executed not under Valgrind. Args are passed in a memory block, and so there's no intrinsic limit to the number that could be passed, but it's currently five. The macro args are: _zzq_rlval result lvalue _zzq_default default value (result returned when running on real CPU) _zzq_request request code _zzq_arg1..5 request params The other two macros are used to support function wrapping, and are a lot simpler. VALGRIND_GET_NR_CONTEXT returns the value of the guest's NRADDR pseudo-register and whatever other information is needed to safely run the call original from the wrapper: on ppc64-linux, the R2 value at the divert point is also needed. This information is abstracted into a user-visible type, OrigFn. VALGRIND_CALL_NOREDIR_* behaves the same as the following on the guest, but guarantees that the branch instruction will not be redirected: x86: call *%eax, amd64: call *%rax, ppc32/ppc64: branch-and-link-to-r11. VALGRIND_CALL_NOREDIR is just text, not a complete inline asm, since it needs to be combined with more magic inline asm stuff to be useful. */ /* ----------------- x86-{linux,darwin,solaris} ---------------- */ #if defined(PLAT_x86_linux) || defined(PLAT_x86_darwin) \ || (defined(PLAT_x86_win32) && defined(__GNUC__)) \ || defined(PLAT_x86_solaris) typedef struct { unsigned int nraddr; /* where's the code? */ } OrigFn; #define __SPECIAL_INSTRUCTION_PREAMBLE \ "roll $3, %%edi ; roll $13, %%edi\n\t" \ "roll $29, %%edi ; roll $19, %%edi\n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ __extension__ \ ({volatile unsigned int _zzq_args[6]; \ volatile unsigned int _zzq_result; \ _zzq_args[0] = (unsigned int)(_zzq_request); \ _zzq_args[1] = (unsigned int)(_zzq_arg1); \ _zzq_args[2] = (unsigned int)(_zzq_arg2); \ _zzq_args[3] = (unsigned int)(_zzq_arg3); \ _zzq_args[4] = (unsigned int)(_zzq_arg4); \ _zzq_args[5] = (unsigned int)(_zzq_arg5); \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %EDX = client_request ( %EAX ) */ \ "xchgl %%ebx,%%ebx" \ : "=d" (_zzq_result) \ : "a" (&_zzq_args[0]), "0" (_zzq_default) \ : "cc", "memory" \ ); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ volatile unsigned int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %EAX = guest_NRADDR */ \ "xchgl %%ecx,%%ecx" \ : "=a" (__addr) \ : \ : "cc", "memory" \ ); \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_CALL_NOREDIR_EAX \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* call-noredir *%EAX */ \ "xchgl %%edx,%%edx\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "xchgl %%edi,%%edi\n\t" \ : : : "cc", "memory" \ ); \ } while (0) #endif /* PLAT_x86_linux || PLAT_x86_darwin || (PLAT_x86_win32 && __GNUC__) || PLAT_x86_solaris */ /* ------------------------- x86-Win32 ------------------------- */ #if defined(PLAT_x86_win32) && !defined(__GNUC__) typedef struct { unsigned int nraddr; /* where's the code? */ } OrigFn; #if defined(_MSC_VER) #define __SPECIAL_INSTRUCTION_PREAMBLE \ __asm rol edi, 3 __asm rol edi, 13 \ __asm rol edi, 29 __asm rol edi, 19 #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ valgrind_do_client_request_expr((uintptr_t)(_zzq_default), \ (uintptr_t)(_zzq_request), (uintptr_t)(_zzq_arg1), \ (uintptr_t)(_zzq_arg2), (uintptr_t)(_zzq_arg3), \ (uintptr_t)(_zzq_arg4), (uintptr_t)(_zzq_arg5)) static __inline uintptr_t valgrind_do_client_request_expr(uintptr_t _zzq_default, uintptr_t _zzq_request, uintptr_t _zzq_arg1, uintptr_t _zzq_arg2, uintptr_t _zzq_arg3, uintptr_t _zzq_arg4, uintptr_t _zzq_arg5) { volatile uintptr_t _zzq_args[6]; volatile unsigned int _zzq_result; _zzq_args[0] = (uintptr_t)(_zzq_request); _zzq_args[1] = (uintptr_t)(_zzq_arg1); _zzq_args[2] = (uintptr_t)(_zzq_arg2); _zzq_args[3] = (uintptr_t)(_zzq_arg3); _zzq_args[4] = (uintptr_t)(_zzq_arg4); _zzq_args[5] = (uintptr_t)(_zzq_arg5); __asm { __asm lea eax, _zzq_args __asm mov edx, _zzq_default __SPECIAL_INSTRUCTION_PREAMBLE /* %EDX = client_request ( %EAX ) */ __asm xchg ebx,ebx __asm mov _zzq_result, edx } return _zzq_result; } #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ volatile unsigned int __addr; \ __asm { __SPECIAL_INSTRUCTION_PREAMBLE \ /* %EAX = guest_NRADDR */ \ __asm xchg ecx,ecx \ __asm mov __addr, eax \ } \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_CALL_NOREDIR_EAX ERROR #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm { __SPECIAL_INSTRUCTION_PREAMBLE \ __asm xchg edi,edi \ } \ } while (0) #else #error Unsupported compiler. #endif #endif /* PLAT_x86_win32 */ /* ----------------- amd64-{linux,darwin,solaris} --------------- */ #if defined(PLAT_amd64_linux) || defined(PLAT_amd64_darwin) \ || defined(PLAT_amd64_solaris) \ || (defined(PLAT_amd64_win64) && defined(__GNUC__)) typedef struct { unsigned long int nraddr; /* where's the code? */ } OrigFn; #define __SPECIAL_INSTRUCTION_PREAMBLE \ "rolq $3, %%rdi ; rolq $13, %%rdi\n\t" \ "rolq $61, %%rdi ; rolq $51, %%rdi\n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ __extension__ \ ({ volatile unsigned long int _zzq_args[6]; \ volatile unsigned long int _zzq_result; \ _zzq_args[0] = (unsigned long int)(_zzq_request); \ _zzq_args[1] = (unsigned long int)(_zzq_arg1); \ _zzq_args[2] = (unsigned long int)(_zzq_arg2); \ _zzq_args[3] = (unsigned long int)(_zzq_arg3); \ _zzq_args[4] = (unsigned long int)(_zzq_arg4); \ _zzq_args[5] = (unsigned long int)(_zzq_arg5); \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %RDX = client_request ( %RAX ) */ \ "xchgq %%rbx,%%rbx" \ : "=d" (_zzq_result) \ : "a" (&_zzq_args[0]), "0" (_zzq_default) \ : "cc", "memory" \ ); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ volatile unsigned long int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %RAX = guest_NRADDR */ \ "xchgq %%rcx,%%rcx" \ : "=a" (__addr) \ : \ : "cc", "memory" \ ); \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_CALL_NOREDIR_RAX \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* call-noredir *%RAX */ \ "xchgq %%rdx,%%rdx\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "xchgq %%rdi,%%rdi\n\t" \ : : : "cc", "memory" \ ); \ } while (0) #endif /* PLAT_amd64_linux || PLAT_amd64_darwin || PLAT_amd64_solaris */ /* ------------------------- amd64-Win64 ------------------------- */ #if defined(PLAT_amd64_win64) && !defined(__GNUC__) #error Unsupported compiler. #endif /* PLAT_amd64_win64 */ /* ------------------------ ppc32-linux ------------------------ */ #if defined(PLAT_ppc32_linux) typedef struct { unsigned int nraddr; /* where's the code? */ } OrigFn; #define __SPECIAL_INSTRUCTION_PREAMBLE \ "rlwinm 0,0,3,0,31 ; rlwinm 0,0,13,0,31\n\t" \ "rlwinm 0,0,29,0,31 ; rlwinm 0,0,19,0,31\n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ \ __extension__ \ ({ unsigned int _zzq_args[6]; \ unsigned int _zzq_result; \ unsigned int* _zzq_ptr; \ _zzq_args[0] = (unsigned int)(_zzq_request); \ _zzq_args[1] = (unsigned int)(_zzq_arg1); \ _zzq_args[2] = (unsigned int)(_zzq_arg2); \ _zzq_args[3] = (unsigned int)(_zzq_arg3); \ _zzq_args[4] = (unsigned int)(_zzq_arg4); \ _zzq_args[5] = (unsigned int)(_zzq_arg5); \ _zzq_ptr = _zzq_args; \ __asm__ volatile("mr 3,%1\n\t" /*default*/ \ "mr 4,%2\n\t" /*ptr*/ \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* %R3 = client_request ( %R4 ) */ \ "or 1,1,1\n\t" \ "mr %0,3" /*result*/ \ : "=b" (_zzq_result) \ : "b" (_zzq_default), "b" (_zzq_ptr) \ : "cc", "memory", "r3", "r4"); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ unsigned int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %R3 = guest_NRADDR */ \ "or 2,2,2\n\t" \ "mr %0,3" \ : "=b" (__addr) \ : \ : "cc", "memory", "r3" \ ); \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* branch-and-link-to-noredir *%R11 */ \ "or 3,3,3\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "or 5,5,5\n\t" \ ); \ } while (0) #endif /* PLAT_ppc32_linux */ /* ------------------------ ppc64-linux ------------------------ */ #if defined(PLAT_ppc64be_linux) typedef struct { unsigned long int nraddr; /* where's the code? */ unsigned long int r2; /* what tocptr do we need? */ } OrigFn; #define __SPECIAL_INSTRUCTION_PREAMBLE \ "rotldi 0,0,3 ; rotldi 0,0,13\n\t" \ "rotldi 0,0,61 ; rotldi 0,0,51\n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ \ __extension__ \ ({ unsigned long int _zzq_args[6]; \ unsigned long int _zzq_result; \ unsigned long int* _zzq_ptr; \ _zzq_args[0] = (unsigned long int)(_zzq_request); \ _zzq_args[1] = (unsigned long int)(_zzq_arg1); \ _zzq_args[2] = (unsigned long int)(_zzq_arg2); \ _zzq_args[3] = (unsigned long int)(_zzq_arg3); \ _zzq_args[4] = (unsigned long int)(_zzq_arg4); \ _zzq_args[5] = (unsigned long int)(_zzq_arg5); \ _zzq_ptr = _zzq_args; \ __asm__ volatile("mr 3,%1\n\t" /*default*/ \ "mr 4,%2\n\t" /*ptr*/ \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* %R3 = client_request ( %R4 ) */ \ "or 1,1,1\n\t" \ "mr %0,3" /*result*/ \ : "=b" (_zzq_result) \ : "b" (_zzq_default), "b" (_zzq_ptr) \ : "cc", "memory", "r3", "r4"); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ unsigned long int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %R3 = guest_NRADDR */ \ "or 2,2,2\n\t" \ "mr %0,3" \ : "=b" (__addr) \ : \ : "cc", "memory", "r3" \ ); \ _zzq_orig->nraddr = __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %R3 = guest_NRADDR_GPR2 */ \ "or 4,4,4\n\t" \ "mr %0,3" \ : "=b" (__addr) \ : \ : "cc", "memory", "r3" \ ); \ _zzq_orig->r2 = __addr; \ } #define VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* branch-and-link-to-noredir *%R11 */ \ "or 3,3,3\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "or 5,5,5\n\t" \ ); \ } while (0) #endif /* PLAT_ppc64be_linux */ #if defined(PLAT_ppc64le_linux) typedef struct { unsigned long int nraddr; /* where's the code? */ unsigned long int r2; /* what tocptr do we need? */ } OrigFn; #define __SPECIAL_INSTRUCTION_PREAMBLE \ "rotldi 0,0,3 ; rotldi 0,0,13\n\t" \ "rotldi 0,0,61 ; rotldi 0,0,51\n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ \ __extension__ \ ({ unsigned long int _zzq_args[6]; \ unsigned long int _zzq_result; \ unsigned long int* _zzq_ptr; \ _zzq_args[0] = (unsigned long int)(_zzq_request); \ _zzq_args[1] = (unsigned long int)(_zzq_arg1); \ _zzq_args[2] = (unsigned long int)(_zzq_arg2); \ _zzq_args[3] = (unsigned long int)(_zzq_arg3); \ _zzq_args[4] = (unsigned long int)(_zzq_arg4); \ _zzq_args[5] = (unsigned long int)(_zzq_arg5); \ _zzq_ptr = _zzq_args; \ __asm__ volatile("mr 3,%1\n\t" /*default*/ \ "mr 4,%2\n\t" /*ptr*/ \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* %R3 = client_request ( %R4 ) */ \ "or 1,1,1\n\t" \ "mr %0,3" /*result*/ \ : "=b" (_zzq_result) \ : "b" (_zzq_default), "b" (_zzq_ptr) \ : "cc", "memory", "r3", "r4"); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ unsigned long int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %R3 = guest_NRADDR */ \ "or 2,2,2\n\t" \ "mr %0,3" \ : "=b" (__addr) \ : \ : "cc", "memory", "r3" \ ); \ _zzq_orig->nraddr = __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %R3 = guest_NRADDR_GPR2 */ \ "or 4,4,4\n\t" \ "mr %0,3" \ : "=b" (__addr) \ : \ : "cc", "memory", "r3" \ ); \ _zzq_orig->r2 = __addr; \ } #define VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* branch-and-link-to-noredir *%R12 */ \ "or 3,3,3\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "or 5,5,5\n\t" \ ); \ } while (0) #endif /* PLAT_ppc64le_linux */ /* ------------------------- arm-linux ------------------------- */ #if defined(PLAT_arm_linux) typedef struct { unsigned int nraddr; /* where's the code? */ } OrigFn; #define __SPECIAL_INSTRUCTION_PREAMBLE \ "mov r12, r12, ror #3 ; mov r12, r12, ror #13 \n\t" \ "mov r12, r12, ror #29 ; mov r12, r12, ror #19 \n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ \ __extension__ \ ({volatile unsigned int _zzq_args[6]; \ volatile unsigned int _zzq_result; \ _zzq_args[0] = (unsigned int)(_zzq_request); \ _zzq_args[1] = (unsigned int)(_zzq_arg1); \ _zzq_args[2] = (unsigned int)(_zzq_arg2); \ _zzq_args[3] = (unsigned int)(_zzq_arg3); \ _zzq_args[4] = (unsigned int)(_zzq_arg4); \ _zzq_args[5] = (unsigned int)(_zzq_arg5); \ __asm__ volatile("mov r3, %1\n\t" /*default*/ \ "mov r4, %2\n\t" /*ptr*/ \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* R3 = client_request ( R4 ) */ \ "orr r10, r10, r10\n\t" \ "mov %0, r3" /*result*/ \ : "=r" (_zzq_result) \ : "r" (_zzq_default), "r" (&_zzq_args[0]) \ : "cc","memory", "r3", "r4"); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ unsigned int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* R3 = guest_NRADDR */ \ "orr r11, r11, r11\n\t" \ "mov %0, r3" \ : "=r" (__addr) \ : \ : "cc", "memory", "r3" \ ); \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* branch-and-link-to-noredir *%R4 */ \ "orr r12, r12, r12\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "orr r9, r9, r9\n\t" \ : : : "cc", "memory" \ ); \ } while (0) #endif /* PLAT_arm_linux */ /* ------------------------ arm64-linux ------------------------- */ #if defined(PLAT_arm64_linux) typedef struct { unsigned long int nraddr; /* where's the code? */ } OrigFn; #define __SPECIAL_INSTRUCTION_PREAMBLE \ "ror x12, x12, #3 ; ror x12, x12, #13 \n\t" \ "ror x12, x12, #51 ; ror x12, x12, #61 \n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ \ __extension__ \ ({volatile unsigned long int _zzq_args[6]; \ volatile unsigned long int _zzq_result; \ _zzq_args[0] = (unsigned long int)(_zzq_request); \ _zzq_args[1] = (unsigned long int)(_zzq_arg1); \ _zzq_args[2] = (unsigned long int)(_zzq_arg2); \ _zzq_args[3] = (unsigned long int)(_zzq_arg3); \ _zzq_args[4] = (unsigned long int)(_zzq_arg4); \ _zzq_args[5] = (unsigned long int)(_zzq_arg5); \ __asm__ volatile("mov x3, %1\n\t" /*default*/ \ "mov x4, %2\n\t" /*ptr*/ \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* X3 = client_request ( X4 ) */ \ "orr x10, x10, x10\n\t" \ "mov %0, x3" /*result*/ \ : "=r" (_zzq_result) \ : "r" ((unsigned long int)(_zzq_default)), \ "r" (&_zzq_args[0]) \ : "cc","memory", "x3", "x4"); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ unsigned long int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* X3 = guest_NRADDR */ \ "orr x11, x11, x11\n\t" \ "mov %0, x3" \ : "=r" (__addr) \ : \ : "cc", "memory", "x3" \ ); \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* branch-and-link-to-noredir X8 */ \ "orr x12, x12, x12\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "orr x9, x9, x9\n\t" \ : : : "cc", "memory" \ ); \ } while (0) #endif /* PLAT_arm64_linux */ /* ------------------------ s390x-linux ------------------------ */ #if defined(PLAT_s390x_linux) typedef struct { unsigned long int nraddr; /* where's the code? */ } OrigFn; /* __SPECIAL_INSTRUCTION_PREAMBLE will be used to identify Valgrind specific * code. This detection is implemented in platform specific toIR.c * (e.g. VEX/priv/guest_s390_decoder.c). */ #define __SPECIAL_INSTRUCTION_PREAMBLE \ "lr 15,15\n\t" \ "lr 1,1\n\t" \ "lr 2,2\n\t" \ "lr 3,3\n\t" #define __CLIENT_REQUEST_CODE "lr 2,2\n\t" #define __GET_NR_CONTEXT_CODE "lr 3,3\n\t" #define __CALL_NO_REDIR_CODE "lr 4,4\n\t" #define __VEX_INJECT_IR_CODE "lr 5,5\n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ __extension__ \ ({volatile unsigned long int _zzq_args[6]; \ volatile unsigned long int _zzq_result; \ _zzq_args[0] = (unsigned long int)(_zzq_request); \ _zzq_args[1] = (unsigned long int)(_zzq_arg1); \ _zzq_args[2] = (unsigned long int)(_zzq_arg2); \ _zzq_args[3] = (unsigned long int)(_zzq_arg3); \ _zzq_args[4] = (unsigned long int)(_zzq_arg4); \ _zzq_args[5] = (unsigned long int)(_zzq_arg5); \ __asm__ volatile(/* r2 = args */ \ "lgr 2,%1\n\t" \ /* r3 = default */ \ "lgr 3,%2\n\t" \ __SPECIAL_INSTRUCTION_PREAMBLE \ __CLIENT_REQUEST_CODE \ /* results = r3 */ \ "lgr %0, 3\n\t" \ : "=d" (_zzq_result) \ : "a" (&_zzq_args[0]), "0" (_zzq_default) \ : "cc", "2", "3", "memory" \ ); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ volatile unsigned long int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ __GET_NR_CONTEXT_CODE \ "lgr %0, 3\n\t" \ : "=a" (__addr) \ : \ : "cc", "3", "memory" \ ); \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_CALL_NOREDIR_R1 \ __SPECIAL_INSTRUCTION_PREAMBLE \ __CALL_NO_REDIR_CODE #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ __VEX_INJECT_IR_CODE); \ } while (0) #endif /* PLAT_s390x_linux */ /* ------------------------- mips32-linux ---------------- */ #if defined(PLAT_mips32_linux) typedef struct { unsigned int nraddr; /* where's the code? */ } OrigFn; /* .word 0x342 * .word 0x742 * .word 0xC2 * .word 0x4C2*/ #define __SPECIAL_INSTRUCTION_PREAMBLE \ "srl $0, $0, 13\n\t" \ "srl $0, $0, 29\n\t" \ "srl $0, $0, 3\n\t" \ "srl $0, $0, 19\n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ __extension__ \ ({ volatile unsigned int _zzq_args[6]; \ volatile unsigned int _zzq_result; \ _zzq_args[0] = (unsigned int)(_zzq_request); \ _zzq_args[1] = (unsigned int)(_zzq_arg1); \ _zzq_args[2] = (unsigned int)(_zzq_arg2); \ _zzq_args[3] = (unsigned int)(_zzq_arg3); \ _zzq_args[4] = (unsigned int)(_zzq_arg4); \ _zzq_args[5] = (unsigned int)(_zzq_arg5); \ __asm__ volatile("move $11, %1\n\t" /*default*/ \ "move $12, %2\n\t" /*ptr*/ \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* T3 = client_request ( T4 ) */ \ "or $13, $13, $13\n\t" \ "move %0, $11\n\t" /*result*/ \ : "=r" (_zzq_result) \ : "r" (_zzq_default), "r" (&_zzq_args[0]) \ : "$11", "$12", "memory"); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ volatile unsigned int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* %t9 = guest_NRADDR */ \ "or $14, $14, $14\n\t" \ "move %0, $11" /*result*/ \ : "=r" (__addr) \ : \ : "$11" \ ); \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_CALL_NOREDIR_T9 \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* call-noredir *%t9 */ \ "or $15, $15, $15\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "or $11, $11, $11\n\t" \ ); \ } while (0) #endif /* PLAT_mips32_linux */ /* ------------------------- mips64-linux ---------------- */ #if defined(PLAT_mips64_linux) typedef struct { unsigned long nraddr; /* where's the code? */ } OrigFn; /* dsll $0,$0, 3 * dsll $0,$0, 13 * dsll $0,$0, 29 * dsll $0,$0, 19*/ #define __SPECIAL_INSTRUCTION_PREAMBLE \ "dsll $0,$0, 3 ; dsll $0,$0,13\n\t" \ "dsll $0,$0,29 ; dsll $0,$0,19\n\t" #define VALGRIND_DO_CLIENT_REQUEST_EXPR( \ _zzq_default, _zzq_request, \ _zzq_arg1, _zzq_arg2, _zzq_arg3, _zzq_arg4, _zzq_arg5) \ __extension__ \ ({ volatile unsigned long int _zzq_args[6]; \ volatile unsigned long int _zzq_result; \ _zzq_args[0] = (unsigned long int)(_zzq_request); \ _zzq_args[1] = (unsigned long int)(_zzq_arg1); \ _zzq_args[2] = (unsigned long int)(_zzq_arg2); \ _zzq_args[3] = (unsigned long int)(_zzq_arg3); \ _zzq_args[4] = (unsigned long int)(_zzq_arg4); \ _zzq_args[5] = (unsigned long int)(_zzq_arg5); \ __asm__ volatile("move $11, %1\n\t" /*default*/ \ "move $12, %2\n\t" /*ptr*/ \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* $11 = client_request ( $12 ) */ \ "or $13, $13, $13\n\t" \ "move %0, $11\n\t" /*result*/ \ : "=r" (_zzq_result) \ : "r" (_zzq_default), "r" (&_zzq_args[0]) \ : "$11", "$12", "memory"); \ _zzq_result; \ }) #define VALGRIND_GET_NR_CONTEXT(_zzq_rlval) \ { volatile OrigFn* _zzq_orig = &(_zzq_rlval); \ volatile unsigned long int __addr; \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ /* $11 = guest_NRADDR */ \ "or $14, $14, $14\n\t" \ "move %0, $11" /*result*/ \ : "=r" (__addr) \ : \ : "$11"); \ _zzq_orig->nraddr = __addr; \ } #define VALGRIND_CALL_NOREDIR_T9 \ __SPECIAL_INSTRUCTION_PREAMBLE \ /* call-noredir $25 */ \ "or $15, $15, $15\n\t" #define VALGRIND_VEX_INJECT_IR() \ do { \ __asm__ volatile(__SPECIAL_INSTRUCTION_PREAMBLE \ "or $11, $11, $11\n\t" \ ); \ } while (0) #endif /* PLAT_mips64_linux */ /* Insert assembly code for other platforms here... */ #endif /* NVALGRIND */ /* ------------------------------------------------------------------ */ /* PLATFORM SPECIFICS for FUNCTION WRAPPING. This is all very */ /* ugly. It's the least-worst tradeoff I can think of. */ /* ------------------------------------------------------------------ */ /* This section defines magic (a.k.a appalling-hack) macros for doing guaranteed-no-redirection macros, so as to get from function wrappers to the functions they are wrapping. The whole point is to construct standard call sequences, but to do the call itself with a special no-redirect call pseudo-instruction that the JIT understands and handles specially. This section is long and repetitious, and I can't see a way to make it shorter. The naming scheme is as follows: CALL_FN_{W,v}_{v,W,WW,WWW,WWWW,5W,6W,7W,etc} 'W' stands for "word" and 'v' for "void". Hence there are different macros for calling arity 0, 1, 2, 3, 4, etc, functions, and for each, the possibility of returning a word-typed result, or no result. */ /* Use these to write the name of your wrapper. NOTE: duplicates VG_WRAP_FUNCTION_Z{U,Z} in pub_tool_redir.h. NOTE also: inserts the default behaviour equivalance class tag "0000" into the name. See pub_tool_redir.h for details -- normally you don't need to think about this, though. */ /* Use an extra level of macroisation so as to ensure the soname/fnname args are fully macro-expanded before pasting them together. */ #define VG_CONCAT4(_aa,_bb,_cc,_dd) _aa##_bb##_cc##_dd #define I_WRAP_SONAME_FNNAME_ZU(soname,fnname) \ VG_CONCAT4(_vgw00000ZU_,soname,_,fnname) #define I_WRAP_SONAME_FNNAME_ZZ(soname,fnname) \ VG_CONCAT4(_vgw00000ZZ_,soname,_,fnname) /* Use this macro from within a wrapper function to collect the context (address and possibly other info) of the original function. Once you have that you can then use it in one of the CALL_FN_ macros. The type of the argument _lval is OrigFn. */ #define VALGRIND_GET_ORIG_FN(_lval) VALGRIND_GET_NR_CONTEXT(_lval) /* Also provide end-user facilities for function replacement, rather than wrapping. A replacement function differs from a wrapper in that it has no way to get hold of the original function being called, and hence no way to call onwards to it. In a replacement function, VALGRIND_GET_ORIG_FN always returns zero. */ #define I_REPLACE_SONAME_FNNAME_ZU(soname,fnname) \ VG_CONCAT4(_vgr00000ZU_,soname,_,fnname) #define I_REPLACE_SONAME_FNNAME_ZZ(soname,fnname) \ VG_CONCAT4(_vgr00000ZZ_,soname,_,fnname) /* Derivatives of the main macros below, for calling functions returning void. */ #define CALL_FN_v_v(fnptr) \ do { volatile unsigned long _junk; \ CALL_FN_W_v(_junk,fnptr); } while (0) #define CALL_FN_v_W(fnptr, arg1) \ do { volatile unsigned long _junk; \ CALL_FN_W_W(_junk,fnptr,arg1); } while (0) #define CALL_FN_v_WW(fnptr, arg1,arg2) \ do { volatile unsigned long _junk; \ CALL_FN_W_WW(_junk,fnptr,arg1,arg2); } while (0) #define CALL_FN_v_WWW(fnptr, arg1,arg2,arg3) \ do { volatile unsigned long _junk; \ CALL_FN_W_WWW(_junk,fnptr,arg1,arg2,arg3); } while (0) #define CALL_FN_v_WWWW(fnptr, arg1,arg2,arg3,arg4) \ do { volatile unsigned long _junk; \ CALL_FN_W_WWWW(_junk,fnptr,arg1,arg2,arg3,arg4); } while (0) #define CALL_FN_v_5W(fnptr, arg1,arg2,arg3,arg4,arg5) \ do { volatile unsigned long _junk; \ CALL_FN_W_5W(_junk,fnptr,arg1,arg2,arg3,arg4,arg5); } while (0) #define CALL_FN_v_6W(fnptr, arg1,arg2,arg3,arg4,arg5,arg6) \ do { volatile unsigned long _junk; \ CALL_FN_W_6W(_junk,fnptr,arg1,arg2,arg3,arg4,arg5,arg6); } while (0) #define CALL_FN_v_7W(fnptr, arg1,arg2,arg3,arg4,arg5,arg6,arg7) \ do { volatile unsigned long _junk; \ CALL_FN_W_7W(_junk,fnptr,arg1,arg2,arg3,arg4,arg5,arg6,arg7); } while (0) /* ----------------- x86-{linux,darwin,solaris} ---------------- */ #if defined(PLAT_x86_linux) || defined(PLAT_x86_darwin) \ || defined(PLAT_x86_solaris) /* These regs are trashed by the hidden call. No need to mention eax as gcc can already see that, plus causes gcc to bomb. */ #define __CALLER_SAVED_REGS /*"eax"*/ "ecx", "edx" /* Macros to save and align the stack before making a function call and restore it afterwards as gcc may not keep the stack pointer aligned if it doesn't realise calls are being made to other functions. */ #define VALGRIND_ALIGN_STACK \ "movl %%esp,%%edi\n\t" \ "andl $0xfffffff0,%%esp\n\t" #define VALGRIND_RESTORE_STACK \ "movl %%edi,%%esp\n\t" /* These CALL_FN_ macros assume that on x86-linux, sizeof(unsigned long) == 4. */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[1]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[2]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $12, %%esp\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $8, %%esp\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[4]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $4, %%esp\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[5]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[6]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $12, %%esp\n\t" \ "pushl 20(%%eax)\n\t" \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[7]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $8, %%esp\n\t" \ "pushl 24(%%eax)\n\t" \ "pushl 20(%%eax)\n\t" \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[8]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $4, %%esp\n\t" \ "pushl 28(%%eax)\n\t" \ "pushl 24(%%eax)\n\t" \ "pushl 20(%%eax)\n\t" \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[9]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "pushl 32(%%eax)\n\t" \ "pushl 28(%%eax)\n\t" \ "pushl 24(%%eax)\n\t" \ "pushl 20(%%eax)\n\t" \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[10]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $12, %%esp\n\t" \ "pushl 36(%%eax)\n\t" \ "pushl 32(%%eax)\n\t" \ "pushl 28(%%eax)\n\t" \ "pushl 24(%%eax)\n\t" \ "pushl 20(%%eax)\n\t" \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[11]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $8, %%esp\n\t" \ "pushl 40(%%eax)\n\t" \ "pushl 36(%%eax)\n\t" \ "pushl 32(%%eax)\n\t" \ "pushl 28(%%eax)\n\t" \ "pushl 24(%%eax)\n\t" \ "pushl 20(%%eax)\n\t" \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5, \ arg6,arg7,arg8,arg9,arg10, \ arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[12]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "subl $4, %%esp\n\t" \ "pushl 44(%%eax)\n\t" \ "pushl 40(%%eax)\n\t" \ "pushl 36(%%eax)\n\t" \ "pushl 32(%%eax)\n\t" \ "pushl 28(%%eax)\n\t" \ "pushl 24(%%eax)\n\t" \ "pushl 20(%%eax)\n\t" \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5, \ arg6,arg7,arg8,arg9,arg10, \ arg11,arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[13]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ _argvec[12] = (unsigned long)(arg12); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "pushl 48(%%eax)\n\t" \ "pushl 44(%%eax)\n\t" \ "pushl 40(%%eax)\n\t" \ "pushl 36(%%eax)\n\t" \ "pushl 32(%%eax)\n\t" \ "pushl 28(%%eax)\n\t" \ "pushl 24(%%eax)\n\t" \ "pushl 20(%%eax)\n\t" \ "pushl 16(%%eax)\n\t" \ "pushl 12(%%eax)\n\t" \ "pushl 8(%%eax)\n\t" \ "pushl 4(%%eax)\n\t" \ "movl (%%eax), %%eax\n\t" /* target->%eax */ \ VALGRIND_CALL_NOREDIR_EAX \ VALGRIND_RESTORE_STACK \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "edi" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_x86_linux || PLAT_x86_darwin || PLAT_x86_solaris */ /* ---------------- amd64-{linux,darwin,solaris} --------------- */ #if defined(PLAT_amd64_linux) || defined(PLAT_amd64_darwin) \ || defined(PLAT_amd64_solaris) /* ARGREGS: rdi rsi rdx rcx r8 r9 (the rest on stack in R-to-L order) */ /* These regs are trashed by the hidden call. */ #define __CALLER_SAVED_REGS /*"rax",*/ "rcx", "rdx", "rsi", \ "rdi", "r8", "r9", "r10", "r11" /* This is all pretty complex. It's so as to make stack unwinding work reliably. See bug 243270. The basic problem is the sub and add of 128 of %rsp in all of the following macros. If gcc believes the CFA is in %rsp, then unwinding may fail, because what's at the CFA is not what gcc "expected" when it constructs the CFIs for the places where the macros are instantiated. But we can't just add a CFI annotation to increase the CFA offset by 128, to match the sub of 128 from %rsp, because we don't know whether gcc has chosen %rsp as the CFA at that point, or whether it has chosen some other register (eg, %rbp). In the latter case, adding a CFI annotation to change the CFA offset is simply wrong. So the solution is to get hold of the CFA using __builtin_dwarf_cfa(), put it in a known register, and add a CFI annotation to say what the register is. We choose %rbp for this (perhaps perversely), because: (1) %rbp is already subject to unwinding. If a new register was chosen then the unwinder would have to unwind it in all stack traces, which is expensive, and (2) %rbp is already subject to precise exception updates in the JIT. If a new register was chosen, we'd have to have precise exceptions for it too, which reduces performance of the generated code. However .. one extra complication. We can't just whack the result of __builtin_dwarf_cfa() into %rbp and then add %rbp to the list of trashed registers at the end of the inline assembly fragments; gcc won't allow %rbp to appear in that list. Hence instead we need to stash %rbp in %r15 for the duration of the asm, and say that %r15 is trashed instead. gcc seems happy to go with that. Oh .. and this all needs to be conditionalised so that it is unchanged from before this commit, when compiled with older gccs that don't support __builtin_dwarf_cfa. Furthermore, since this header file is freestanding, it has to be independent of config.h, and so the following conditionalisation cannot depend on configure time checks. Although it's not clear from 'defined(__GNUC__) && defined(__GCC_HAVE_DWARF2_CFI_ASM)', this expression excludes Darwin. .cfi directives in Darwin assembly appear to be completely different and I haven't investigated how they work. For even more entertainment value, note we have to use the completely undocumented __builtin_dwarf_cfa(), which appears to really compute the CFA, whereas __builtin_frame_address(0) claims to but actually doesn't. See https://bugs.kde.org/show_bug.cgi?id=243270#c47 */ #if defined(__GNUC__) && defined(__GCC_HAVE_DWARF2_CFI_ASM) # define __FRAME_POINTER \ ,"r"(__builtin_dwarf_cfa()) # define VALGRIND_CFI_PROLOGUE \ "movq %%rbp, %%r15\n\t" \ "movq %2, %%rbp\n\t" \ ".cfi_remember_state\n\t" \ ".cfi_def_cfa rbp, 0\n\t" # define VALGRIND_CFI_EPILOGUE \ "movq %%r15, %%rbp\n\t" \ ".cfi_restore_state\n\t" #else # define __FRAME_POINTER # define VALGRIND_CFI_PROLOGUE # define VALGRIND_CFI_EPILOGUE #endif /* Macros to save and align the stack before making a function call and restore it afterwards as gcc may not keep the stack pointer aligned if it doesn't realise calls are being made to other functions. */ #define VALGRIND_ALIGN_STACK \ "movq %%rsp,%%r14\n\t" \ "andq $0xfffffffffffffff0,%%rsp\n\t" #define VALGRIND_RESTORE_STACK \ "movq %%r14,%%rsp\n\t" /* These CALL_FN_ macros assume that on amd64-linux, sizeof(unsigned long) == 8. */ /* NB 9 Sept 07. There is a nasty kludge here in all these CALL_FN_ macros. In order not to trash the stack redzone, we need to drop %rsp by 128 before the hidden call, and restore afterwards. The nastiness is that it is only by luck that the stack still appears to be unwindable during the hidden call - since then the behaviour of any routine using this macro does not match what the CFI data says. Sigh. Why is this important? Imagine that a wrapper has a stack allocated local, and passes to the hidden call, a pointer to it. Because gcc does not know about the hidden call, it may allocate that local in the redzone. Unfortunately the hidden call may then trash it before it comes to use it. So we must step clear of the redzone, for the duration of the hidden call, to make it safe. Probably the same problem afflicts the other redzone-style ABIs too (ppc64-linux); but for those, the stack is self describing (none of this CFI nonsense) so at least messing with the stack pointer doesn't give a danger of non-unwindable stack. */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[1]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[2]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[4]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[5]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[6]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "movq 40(%%rax), %%r8\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[7]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "movq 48(%%rax), %%r9\n\t" \ "movq 40(%%rax), %%r8\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[8]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $136,%%rsp\n\t" \ "pushq 56(%%rax)\n\t" \ "movq 48(%%rax), %%r9\n\t" \ "movq 40(%%rax), %%r8\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[9]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "pushq 64(%%rax)\n\t" \ "pushq 56(%%rax)\n\t" \ "movq 48(%%rax), %%r9\n\t" \ "movq 40(%%rax), %%r8\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[10]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $136,%%rsp\n\t" \ "pushq 72(%%rax)\n\t" \ "pushq 64(%%rax)\n\t" \ "pushq 56(%%rax)\n\t" \ "movq 48(%%rax), %%r9\n\t" \ "movq 40(%%rax), %%r8\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[11]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "pushq 80(%%rax)\n\t" \ "pushq 72(%%rax)\n\t" \ "pushq 64(%%rax)\n\t" \ "pushq 56(%%rax)\n\t" \ "movq 48(%%rax), %%r9\n\t" \ "movq 40(%%rax), %%r8\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[12]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $136,%%rsp\n\t" \ "pushq 88(%%rax)\n\t" \ "pushq 80(%%rax)\n\t" \ "pushq 72(%%rax)\n\t" \ "pushq 64(%%rax)\n\t" \ "pushq 56(%%rax)\n\t" \ "movq 48(%%rax), %%r9\n\t" \ "movq 40(%%rax), %%r8\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11,arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[13]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ _argvec[12] = (unsigned long)(arg12); \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ VALGRIND_ALIGN_STACK \ "subq $128,%%rsp\n\t" \ "pushq 96(%%rax)\n\t" \ "pushq 88(%%rax)\n\t" \ "pushq 80(%%rax)\n\t" \ "pushq 72(%%rax)\n\t" \ "pushq 64(%%rax)\n\t" \ "pushq 56(%%rax)\n\t" \ "movq 48(%%rax), %%r9\n\t" \ "movq 40(%%rax), %%r8\n\t" \ "movq 32(%%rax), %%rcx\n\t" \ "movq 24(%%rax), %%rdx\n\t" \ "movq 16(%%rax), %%rsi\n\t" \ "movq 8(%%rax), %%rdi\n\t" \ "movq (%%rax), %%rax\n\t" /* target->%rax */ \ VALGRIND_CALL_NOREDIR_RAX \ VALGRIND_RESTORE_STACK \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=a" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r14", "r15" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_amd64_linux || PLAT_amd64_darwin || PLAT_amd64_solaris */ /* ------------------------ ppc32-linux ------------------------ */ #if defined(PLAT_ppc32_linux) /* This is useful for finding out about the on-stack stuff: extern int f9 ( int,int,int,int,int,int,int,int,int ); extern int f10 ( int,int,int,int,int,int,int,int,int,int ); extern int f11 ( int,int,int,int,int,int,int,int,int,int,int ); extern int f12 ( int,int,int,int,int,int,int,int,int,int,int,int ); int g9 ( void ) { return f9(11,22,33,44,55,66,77,88,99); } int g10 ( void ) { return f10(11,22,33,44,55,66,77,88,99,110); } int g11 ( void ) { return f11(11,22,33,44,55,66,77,88,99,110,121); } int g12 ( void ) { return f12(11,22,33,44,55,66,77,88,99,110,121,132); } */ /* ARGREGS: r3 r4 r5 r6 r7 r8 r9 r10 (the rest on stack somewhere) */ /* These regs are trashed by the hidden call. */ #define __CALLER_SAVED_REGS \ "lr", "ctr", "xer", \ "cr0", "cr1", "cr2", "cr3", "cr4", "cr5", "cr6", "cr7", \ "r0", "r2", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10", \ "r11", "r12", "r13" /* Macros to save and align the stack before making a function call and restore it afterwards as gcc may not keep the stack pointer aligned if it doesn't realise calls are being made to other functions. */ #define VALGRIND_ALIGN_STACK \ "mr 28,1\n\t" \ "rlwinm 1,1,0,0,27\n\t" #define VALGRIND_RESTORE_STACK \ "mr 1,28\n\t" /* These CALL_FN_ macros assume that on ppc32-linux, sizeof(unsigned long) == 4. */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[1]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[2]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[4]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[5]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[6]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 7,20(11)\n\t" \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[7]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 7,20(11)\n\t" \ "lwz 8,24(11)\n\t" \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[8]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 7,20(11)\n\t" \ "lwz 8,24(11)\n\t" \ "lwz 9,28(11)\n\t" \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[9]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 7,20(11)\n\t" \ "lwz 8,24(11)\n\t" \ "lwz 9,28(11)\n\t" \ "lwz 10,32(11)\n\t" /* arg8->r10 */ \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[10]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ _argvec[9] = (unsigned long)arg9; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "addi 1,1,-16\n\t" \ /* arg9 */ \ "lwz 3,36(11)\n\t" \ "stw 3,8(1)\n\t" \ /* args1-8 */ \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 7,20(11)\n\t" \ "lwz 8,24(11)\n\t" \ "lwz 9,28(11)\n\t" \ "lwz 10,32(11)\n\t" /* arg8->r10 */ \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[11]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ _argvec[9] = (unsigned long)arg9; \ _argvec[10] = (unsigned long)arg10; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "addi 1,1,-16\n\t" \ /* arg10 */ \ "lwz 3,40(11)\n\t" \ "stw 3,12(1)\n\t" \ /* arg9 */ \ "lwz 3,36(11)\n\t" \ "stw 3,8(1)\n\t" \ /* args1-8 */ \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 7,20(11)\n\t" \ "lwz 8,24(11)\n\t" \ "lwz 9,28(11)\n\t" \ "lwz 10,32(11)\n\t" /* arg8->r10 */ \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[12]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ _argvec[9] = (unsigned long)arg9; \ _argvec[10] = (unsigned long)arg10; \ _argvec[11] = (unsigned long)arg11; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "addi 1,1,-32\n\t" \ /* arg11 */ \ "lwz 3,44(11)\n\t" \ "stw 3,16(1)\n\t" \ /* arg10 */ \ "lwz 3,40(11)\n\t" \ "stw 3,12(1)\n\t" \ /* arg9 */ \ "lwz 3,36(11)\n\t" \ "stw 3,8(1)\n\t" \ /* args1-8 */ \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 7,20(11)\n\t" \ "lwz 8,24(11)\n\t" \ "lwz 9,28(11)\n\t" \ "lwz 10,32(11)\n\t" /* arg8->r10 */ \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11,arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[13]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ _argvec[9] = (unsigned long)arg9; \ _argvec[10] = (unsigned long)arg10; \ _argvec[11] = (unsigned long)arg11; \ _argvec[12] = (unsigned long)arg12; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "addi 1,1,-32\n\t" \ /* arg12 */ \ "lwz 3,48(11)\n\t" \ "stw 3,20(1)\n\t" \ /* arg11 */ \ "lwz 3,44(11)\n\t" \ "stw 3,16(1)\n\t" \ /* arg10 */ \ "lwz 3,40(11)\n\t" \ "stw 3,12(1)\n\t" \ /* arg9 */ \ "lwz 3,36(11)\n\t" \ "stw 3,8(1)\n\t" \ /* args1-8 */ \ "lwz 3,4(11)\n\t" /* arg1->r3 */ \ "lwz 4,8(11)\n\t" \ "lwz 5,12(11)\n\t" \ "lwz 6,16(11)\n\t" /* arg4->r6 */ \ "lwz 7,20(11)\n\t" \ "lwz 8,24(11)\n\t" \ "lwz 9,28(11)\n\t" \ "lwz 10,32(11)\n\t" /* arg8->r10 */ \ "lwz 11,0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ VALGRIND_RESTORE_STACK \ "mr %0,3" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_ppc32_linux */ /* ------------------------ ppc64-linux ------------------------ */ #if defined(PLAT_ppc64be_linux) /* ARGREGS: r3 r4 r5 r6 r7 r8 r9 r10 (the rest on stack somewhere) */ /* These regs are trashed by the hidden call. */ #define __CALLER_SAVED_REGS \ "lr", "ctr", "xer", \ "cr0", "cr1", "cr2", "cr3", "cr4", "cr5", "cr6", "cr7", \ "r0", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10", \ "r11", "r12", "r13" /* Macros to save and align the stack before making a function call and restore it afterwards as gcc may not keep the stack pointer aligned if it doesn't realise calls are being made to other functions. */ #define VALGRIND_ALIGN_STACK \ "mr 28,1\n\t" \ "rldicr 1,1,0,59\n\t" #define VALGRIND_RESTORE_STACK \ "mr 1,28\n\t" /* These CALL_FN_ macros assume that on ppc64-linux, sizeof(unsigned long) == 8. */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+0]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+1]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+2]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+3]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+4]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+5]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 7, 40(11)\n\t" /* arg5->r7 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+6]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 7, 40(11)\n\t" /* arg5->r7 */ \ "ld 8, 48(11)\n\t" /* arg6->r8 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+7]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 7, 40(11)\n\t" /* arg5->r7 */ \ "ld 8, 48(11)\n\t" /* arg6->r8 */ \ "ld 9, 56(11)\n\t" /* arg7->r9 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+8]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 7, 40(11)\n\t" /* arg5->r7 */ \ "ld 8, 48(11)\n\t" /* arg6->r8 */ \ "ld 9, 56(11)\n\t" /* arg7->r9 */ \ "ld 10, 64(11)\n\t" /* arg8->r10 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+9]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ _argvec[2+9] = (unsigned long)arg9; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "addi 1,1,-128\n\t" /* expand stack frame */ \ /* arg9 */ \ "ld 3,72(11)\n\t" \ "std 3,112(1)\n\t" \ /* args1-8 */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 7, 40(11)\n\t" /* arg5->r7 */ \ "ld 8, 48(11)\n\t" /* arg6->r8 */ \ "ld 9, 56(11)\n\t" /* arg7->r9 */ \ "ld 10, 64(11)\n\t" /* arg8->r10 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+10]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ _argvec[2+9] = (unsigned long)arg9; \ _argvec[2+10] = (unsigned long)arg10; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "addi 1,1,-128\n\t" /* expand stack frame */ \ /* arg10 */ \ "ld 3,80(11)\n\t" \ "std 3,120(1)\n\t" \ /* arg9 */ \ "ld 3,72(11)\n\t" \ "std 3,112(1)\n\t" \ /* args1-8 */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 7, 40(11)\n\t" /* arg5->r7 */ \ "ld 8, 48(11)\n\t" /* arg6->r8 */ \ "ld 9, 56(11)\n\t" /* arg7->r9 */ \ "ld 10, 64(11)\n\t" /* arg8->r10 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+11]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ _argvec[2+9] = (unsigned long)arg9; \ _argvec[2+10] = (unsigned long)arg10; \ _argvec[2+11] = (unsigned long)arg11; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "addi 1,1,-144\n\t" /* expand stack frame */ \ /* arg11 */ \ "ld 3,88(11)\n\t" \ "std 3,128(1)\n\t" \ /* arg10 */ \ "ld 3,80(11)\n\t" \ "std 3,120(1)\n\t" \ /* arg9 */ \ "ld 3,72(11)\n\t" \ "std 3,112(1)\n\t" \ /* args1-8 */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 7, 40(11)\n\t" /* arg5->r7 */ \ "ld 8, 48(11)\n\t" /* arg6->r8 */ \ "ld 9, 56(11)\n\t" /* arg7->r9 */ \ "ld 10, 64(11)\n\t" /* arg8->r10 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11,arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+12]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ _argvec[2+9] = (unsigned long)arg9; \ _argvec[2+10] = (unsigned long)arg10; \ _argvec[2+11] = (unsigned long)arg11; \ _argvec[2+12] = (unsigned long)arg12; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 11,%1\n\t" \ "std 2,-16(11)\n\t" /* save tocptr */ \ "ld 2,-8(11)\n\t" /* use nraddr's tocptr */ \ "addi 1,1,-144\n\t" /* expand stack frame */ \ /* arg12 */ \ "ld 3,96(11)\n\t" \ "std 3,136(1)\n\t" \ /* arg11 */ \ "ld 3,88(11)\n\t" \ "std 3,128(1)\n\t" \ /* arg10 */ \ "ld 3,80(11)\n\t" \ "std 3,120(1)\n\t" \ /* arg9 */ \ "ld 3,72(11)\n\t" \ "std 3,112(1)\n\t" \ /* args1-8 */ \ "ld 3, 8(11)\n\t" /* arg1->r3 */ \ "ld 4, 16(11)\n\t" /* arg2->r4 */ \ "ld 5, 24(11)\n\t" /* arg3->r5 */ \ "ld 6, 32(11)\n\t" /* arg4->r6 */ \ "ld 7, 40(11)\n\t" /* arg5->r7 */ \ "ld 8, 48(11)\n\t" /* arg6->r8 */ \ "ld 9, 56(11)\n\t" /* arg7->r9 */ \ "ld 10, 64(11)\n\t" /* arg8->r10 */ \ "ld 11, 0(11)\n\t" /* target->r11 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R11 \ "mr 11,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(11)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_ppc64be_linux */ /* ------------------------- ppc64le-linux ----------------------- */ #if defined(PLAT_ppc64le_linux) /* ARGREGS: r3 r4 r5 r6 r7 r8 r9 r10 (the rest on stack somewhere) */ /* These regs are trashed by the hidden call. */ #define __CALLER_SAVED_REGS \ "lr", "ctr", "xer", \ "cr0", "cr1", "cr2", "cr3", "cr4", "cr5", "cr6", "cr7", \ "r0", "r3", "r4", "r5", "r6", "r7", "r8", "r9", "r10", \ "r11", "r12", "r13" /* Macros to save and align the stack before making a function call and restore it afterwards as gcc may not keep the stack pointer aligned if it doesn't realise calls are being made to other functions. */ #define VALGRIND_ALIGN_STACK \ "mr 28,1\n\t" \ "rldicr 1,1,0,59\n\t" #define VALGRIND_RESTORE_STACK \ "mr 1,28\n\t" /* These CALL_FN_ macros assume that on ppc64-linux, sizeof(unsigned long) == 8. */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+0]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+1]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+2]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+3]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+4]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+5]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 7, 40(12)\n\t" /* arg5->r7 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+6]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 7, 40(12)\n\t" /* arg5->r7 */ \ "ld 8, 48(12)\n\t" /* arg6->r8 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+7]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 7, 40(12)\n\t" /* arg5->r7 */ \ "ld 8, 48(12)\n\t" /* arg6->r8 */ \ "ld 9, 56(12)\n\t" /* arg7->r9 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+8]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 7, 40(12)\n\t" /* arg5->r7 */ \ "ld 8, 48(12)\n\t" /* arg6->r8 */ \ "ld 9, 56(12)\n\t" /* arg7->r9 */ \ "ld 10, 64(12)\n\t" /* arg8->r10 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+9]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ _argvec[2+9] = (unsigned long)arg9; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "addi 1,1,-128\n\t" /* expand stack frame */ \ /* arg9 */ \ "ld 3,72(12)\n\t" \ "std 3,96(1)\n\t" \ /* args1-8 */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 7, 40(12)\n\t" /* arg5->r7 */ \ "ld 8, 48(12)\n\t" /* arg6->r8 */ \ "ld 9, 56(12)\n\t" /* arg7->r9 */ \ "ld 10, 64(12)\n\t" /* arg8->r10 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+10]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ _argvec[2+9] = (unsigned long)arg9; \ _argvec[2+10] = (unsigned long)arg10; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "addi 1,1,-128\n\t" /* expand stack frame */ \ /* arg10 */ \ "ld 3,80(12)\n\t" \ "std 3,104(1)\n\t" \ /* arg9 */ \ "ld 3,72(12)\n\t" \ "std 3,96(1)\n\t" \ /* args1-8 */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 7, 40(12)\n\t" /* arg5->r7 */ \ "ld 8, 48(12)\n\t" /* arg6->r8 */ \ "ld 9, 56(12)\n\t" /* arg7->r9 */ \ "ld 10, 64(12)\n\t" /* arg8->r10 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+11]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ _argvec[2+9] = (unsigned long)arg9; \ _argvec[2+10] = (unsigned long)arg10; \ _argvec[2+11] = (unsigned long)arg11; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "addi 1,1,-144\n\t" /* expand stack frame */ \ /* arg11 */ \ "ld 3,88(12)\n\t" \ "std 3,112(1)\n\t" \ /* arg10 */ \ "ld 3,80(12)\n\t" \ "std 3,104(1)\n\t" \ /* arg9 */ \ "ld 3,72(12)\n\t" \ "std 3,96(1)\n\t" \ /* args1-8 */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 7, 40(12)\n\t" /* arg5->r7 */ \ "ld 8, 48(12)\n\t" /* arg6->r8 */ \ "ld 9, 56(12)\n\t" /* arg7->r9 */ \ "ld 10, 64(12)\n\t" /* arg8->r10 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11,arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3+12]; \ volatile unsigned long _res; \ /* _argvec[0] holds current r2 across the call */ \ _argvec[1] = (unsigned long)_orig.r2; \ _argvec[2] = (unsigned long)_orig.nraddr; \ _argvec[2+1] = (unsigned long)arg1; \ _argvec[2+2] = (unsigned long)arg2; \ _argvec[2+3] = (unsigned long)arg3; \ _argvec[2+4] = (unsigned long)arg4; \ _argvec[2+5] = (unsigned long)arg5; \ _argvec[2+6] = (unsigned long)arg6; \ _argvec[2+7] = (unsigned long)arg7; \ _argvec[2+8] = (unsigned long)arg8; \ _argvec[2+9] = (unsigned long)arg9; \ _argvec[2+10] = (unsigned long)arg10; \ _argvec[2+11] = (unsigned long)arg11; \ _argvec[2+12] = (unsigned long)arg12; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "mr 12,%1\n\t" \ "std 2,-16(12)\n\t" /* save tocptr */ \ "ld 2,-8(12)\n\t" /* use nraddr's tocptr */ \ "addi 1,1,-144\n\t" /* expand stack frame */ \ /* arg12 */ \ "ld 3,96(12)\n\t" \ "std 3,120(1)\n\t" \ /* arg11 */ \ "ld 3,88(12)\n\t" \ "std 3,112(1)\n\t" \ /* arg10 */ \ "ld 3,80(12)\n\t" \ "std 3,104(1)\n\t" \ /* arg9 */ \ "ld 3,72(12)\n\t" \ "std 3,96(1)\n\t" \ /* args1-8 */ \ "ld 3, 8(12)\n\t" /* arg1->r3 */ \ "ld 4, 16(12)\n\t" /* arg2->r4 */ \ "ld 5, 24(12)\n\t" /* arg3->r5 */ \ "ld 6, 32(12)\n\t" /* arg4->r6 */ \ "ld 7, 40(12)\n\t" /* arg5->r7 */ \ "ld 8, 48(12)\n\t" /* arg6->r8 */ \ "ld 9, 56(12)\n\t" /* arg7->r9 */ \ "ld 10, 64(12)\n\t" /* arg8->r10 */ \ "ld 12, 0(12)\n\t" /* target->r12 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R12 \ "mr 12,%1\n\t" \ "mr %0,3\n\t" \ "ld 2,-16(12)\n\t" /* restore tocptr */ \ VALGRIND_RESTORE_STACK \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[2]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r28" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_ppc64le_linux */ /* ------------------------- arm-linux ------------------------- */ #if defined(PLAT_arm_linux) /* These regs are trashed by the hidden call. */ #define __CALLER_SAVED_REGS "r0", "r1", "r2", "r3","r4", "r12", "r14" /* Macros to save and align the stack before making a function call and restore it afterwards as gcc may not keep the stack pointer aligned if it doesn't realise calls are being made to other functions. */ /* This is a bit tricky. We store the original stack pointer in r10 as it is callee-saves. gcc doesn't allow the use of r11 for some reason. Also, we can't directly "bic" the stack pointer in thumb mode since r13 isn't an allowed register number in that context. So use r4 as a temporary, since that is about to get trashed anyway, just after each use of this macro. Side effect is we need to be very careful about any future changes, since VALGRIND_ALIGN_STACK simply assumes r4 is usable. */ #define VALGRIND_ALIGN_STACK \ "mov r10, sp\n\t" \ "mov r4, sp\n\t" \ "bic r4, r4, #7\n\t" \ "mov sp, r4\n\t" #define VALGRIND_RESTORE_STACK \ "mov sp, r10\n\t" /* These CALL_FN_ macros assume that on arm-linux, sizeof(unsigned long) == 4. */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[1]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[2]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r0, [%1, #4] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[4]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[5]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[6]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "sub sp, sp, #4 \n\t" \ "ldr r0, [%1, #20] \n\t" \ "push {r0} \n\t" \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[7]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r0, [%1, #20] \n\t" \ "ldr r1, [%1, #24] \n\t" \ "push {r0, r1} \n\t" \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[8]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "sub sp, sp, #4 \n\t" \ "ldr r0, [%1, #20] \n\t" \ "ldr r1, [%1, #24] \n\t" \ "ldr r2, [%1, #28] \n\t" \ "push {r0, r1, r2} \n\t" \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[9]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r0, [%1, #20] \n\t" \ "ldr r1, [%1, #24] \n\t" \ "ldr r2, [%1, #28] \n\t" \ "ldr r3, [%1, #32] \n\t" \ "push {r0, r1, r2, r3} \n\t" \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[10]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "sub sp, sp, #4 \n\t" \ "ldr r0, [%1, #20] \n\t" \ "ldr r1, [%1, #24] \n\t" \ "ldr r2, [%1, #28] \n\t" \ "ldr r3, [%1, #32] \n\t" \ "ldr r4, [%1, #36] \n\t" \ "push {r0, r1, r2, r3, r4} \n\t" \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[11]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r0, [%1, #40] \n\t" \ "push {r0} \n\t" \ "ldr r0, [%1, #20] \n\t" \ "ldr r1, [%1, #24] \n\t" \ "ldr r2, [%1, #28] \n\t" \ "ldr r3, [%1, #32] \n\t" \ "ldr r4, [%1, #36] \n\t" \ "push {r0, r1, r2, r3, r4} \n\t" \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5, \ arg6,arg7,arg8,arg9,arg10, \ arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[12]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "sub sp, sp, #4 \n\t" \ "ldr r0, [%1, #40] \n\t" \ "ldr r1, [%1, #44] \n\t" \ "push {r0, r1} \n\t" \ "ldr r0, [%1, #20] \n\t" \ "ldr r1, [%1, #24] \n\t" \ "ldr r2, [%1, #28] \n\t" \ "ldr r3, [%1, #32] \n\t" \ "ldr r4, [%1, #36] \n\t" \ "push {r0, r1, r2, r3, r4} \n\t" \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5, \ arg6,arg7,arg8,arg9,arg10, \ arg11,arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[13]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ _argvec[12] = (unsigned long)(arg12); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr r0, [%1, #40] \n\t" \ "ldr r1, [%1, #44] \n\t" \ "ldr r2, [%1, #48] \n\t" \ "push {r0, r1, r2} \n\t" \ "ldr r0, [%1, #20] \n\t" \ "ldr r1, [%1, #24] \n\t" \ "ldr r2, [%1, #28] \n\t" \ "ldr r3, [%1, #32] \n\t" \ "ldr r4, [%1, #36] \n\t" \ "push {r0, r1, r2, r3, r4} \n\t" \ "ldr r0, [%1, #4] \n\t" \ "ldr r1, [%1, #8] \n\t" \ "ldr r2, [%1, #12] \n\t" \ "ldr r3, [%1, #16] \n\t" \ "ldr r4, [%1] \n\t" /* target->r4 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_R4 \ VALGRIND_RESTORE_STACK \ "mov %0, r0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "r10" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_arm_linux */ /* ------------------------ arm64-linux ------------------------ */ #if defined(PLAT_arm64_linux) /* These regs are trashed by the hidden call. */ #define __CALLER_SAVED_REGS \ "x0", "x1", "x2", "x3","x4", "x5", "x6", "x7", "x8", "x9", \ "x10", "x11", "x12", "x13", "x14", "x15", "x16", "x17", \ "x18", "x19", "x20", "x30", \ "v0", "v1", "v2", "v3", "v4", "v5", "v6", "v7", "v8", "v9", \ "v10", "v11", "v12", "v13", "v14", "v15", "v16", "v17", \ "v18", "v19", "v20", "v21", "v22", "v23", "v24", "v25", \ "v26", "v27", "v28", "v29", "v30", "v31" /* x21 is callee-saved, so we can use it to save and restore SP around the hidden call. */ #define VALGRIND_ALIGN_STACK \ "mov x21, sp\n\t" \ "bic sp, x21, #15\n\t" #define VALGRIND_RESTORE_STACK \ "mov sp, x21\n\t" /* These CALL_FN_ macros assume that on arm64-linux, sizeof(unsigned long) == 8. */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[1]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[2]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x0, [%1, #8] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[4]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[5]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[6]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x4, [%1, #40] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[7]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x4, [%1, #40] \n\t" \ "ldr x5, [%1, #48] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[8]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x4, [%1, #40] \n\t" \ "ldr x5, [%1, #48] \n\t" \ "ldr x6, [%1, #56] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[9]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x4, [%1, #40] \n\t" \ "ldr x5, [%1, #48] \n\t" \ "ldr x6, [%1, #56] \n\t" \ "ldr x7, [%1, #64] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[10]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "sub sp, sp, #0x20 \n\t" \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x4, [%1, #40] \n\t" \ "ldr x5, [%1, #48] \n\t" \ "ldr x6, [%1, #56] \n\t" \ "ldr x7, [%1, #64] \n\t" \ "ldr x8, [%1, #72] \n\t" \ "str x8, [sp, #0] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[11]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "sub sp, sp, #0x20 \n\t" \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x4, [%1, #40] \n\t" \ "ldr x5, [%1, #48] \n\t" \ "ldr x6, [%1, #56] \n\t" \ "ldr x7, [%1, #64] \n\t" \ "ldr x8, [%1, #72] \n\t" \ "str x8, [sp, #0] \n\t" \ "ldr x8, [%1, #80] \n\t" \ "str x8, [sp, #8] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[12]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "sub sp, sp, #0x30 \n\t" \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x4, [%1, #40] \n\t" \ "ldr x5, [%1, #48] \n\t" \ "ldr x6, [%1, #56] \n\t" \ "ldr x7, [%1, #64] \n\t" \ "ldr x8, [%1, #72] \n\t" \ "str x8, [sp, #0] \n\t" \ "ldr x8, [%1, #80] \n\t" \ "str x8, [sp, #8] \n\t" \ "ldr x8, [%1, #88] \n\t" \ "str x8, [sp, #16] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10,arg11, \ arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[13]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ _argvec[12] = (unsigned long)(arg12); \ __asm__ volatile( \ VALGRIND_ALIGN_STACK \ "sub sp, sp, #0x30 \n\t" \ "ldr x0, [%1, #8] \n\t" \ "ldr x1, [%1, #16] \n\t" \ "ldr x2, [%1, #24] \n\t" \ "ldr x3, [%1, #32] \n\t" \ "ldr x4, [%1, #40] \n\t" \ "ldr x5, [%1, #48] \n\t" \ "ldr x6, [%1, #56] \n\t" \ "ldr x7, [%1, #64] \n\t" \ "ldr x8, [%1, #72] \n\t" \ "str x8, [sp, #0] \n\t" \ "ldr x8, [%1, #80] \n\t" \ "str x8, [sp, #8] \n\t" \ "ldr x8, [%1, #88] \n\t" \ "str x8, [sp, #16] \n\t" \ "ldr x8, [%1, #96] \n\t" \ "str x8, [sp, #24] \n\t" \ "ldr x8, [%1] \n\t" /* target->x8 */ \ VALGRIND_BRANCH_AND_LINK_TO_NOREDIR_X8 \ VALGRIND_RESTORE_STACK \ "mov %0, x0" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS, "x21" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_arm64_linux */ /* ------------------------- s390x-linux ------------------------- */ #if defined(PLAT_s390x_linux) /* Similar workaround as amd64 (see above), but we use r11 as frame pointer and save the old r11 in r7. r11 might be used for argvec, therefore we copy argvec in r1 since r1 is clobbered after the call anyway. */ #if defined(__GNUC__) && defined(__GCC_HAVE_DWARF2_CFI_ASM) # define __FRAME_POINTER \ ,"d"(__builtin_dwarf_cfa()) # define VALGRIND_CFI_PROLOGUE \ ".cfi_remember_state\n\t" \ "lgr 1,%1\n\t" /* copy the argvec pointer in r1 */ \ "lgr 7,11\n\t" \ "lgr 11,%2\n\t" \ ".cfi_def_cfa r11, 0\n\t" # define VALGRIND_CFI_EPILOGUE \ "lgr 11, 7\n\t" \ ".cfi_restore_state\n\t" #else # define __FRAME_POINTER # define VALGRIND_CFI_PROLOGUE \ "lgr 1,%1\n\t" # define VALGRIND_CFI_EPILOGUE #endif /* Nb: On s390 the stack pointer is properly aligned *at all times* according to the s390 GCC maintainer. (The ABI specification is not precise in this regard.) Therefore, VALGRIND_ALIGN_STACK and VALGRIND_RESTORE_STACK are not defined here. */ /* These regs are trashed by the hidden call. Note that we overwrite r14 in s390_irgen_noredir (VEX/priv/guest_s390_irgen.c) to give the function a proper return address. All others are ABI defined call clobbers. */ #define __CALLER_SAVED_REGS "0","1","2","3","4","5","14", \ "f0","f1","f2","f3","f4","f5","f6","f7" /* Nb: Although r11 is modified in the asm snippets below (inside VALGRIND_CFI_PROLOGUE) it is not listed in the clobber section, for two reasons: (1) r11 is restored in VALGRIND_CFI_EPILOGUE, so effectively it is not modified (2) GCC will complain that r11 cannot appear inside a clobber section, when compiled with -O -fno-omit-frame-pointer */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[1]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-160\n\t" \ "lg 1, 0(1)\n\t" /* target->r1 */ \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,160\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "d" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) /* The call abi has the arguments in r2-r6 and stack */ #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[2]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-160\n\t" \ "lg 2, 8(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,160\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1, arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-160\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,160\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1, arg2, arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[4]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-160\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,160\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1, arg2, arg3, arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[5]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-160\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,160\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1, arg2, arg3, arg4, arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[6]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-160\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 6,40(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,160\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"6","7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1, arg2, arg3, arg4, arg5, \ arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[7]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-168\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 6,40(1)\n\t" \ "mvc 160(8,15), 48(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,168\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"6","7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1, arg2, arg3, arg4, arg5, \ arg6, arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[8]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-176\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 6,40(1)\n\t" \ "mvc 160(8,15), 48(1)\n\t" \ "mvc 168(8,15), 56(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,176\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"6","7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1, arg2, arg3, arg4, arg5, \ arg6, arg7 ,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[9]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-184\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 6,40(1)\n\t" \ "mvc 160(8,15), 48(1)\n\t" \ "mvc 168(8,15), 56(1)\n\t" \ "mvc 176(8,15), 64(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,184\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"6","7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1, arg2, arg3, arg4, arg5, \ arg6, arg7 ,arg8, arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[10]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ _argvec[9] = (unsigned long)arg9; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-192\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 6,40(1)\n\t" \ "mvc 160(8,15), 48(1)\n\t" \ "mvc 168(8,15), 56(1)\n\t" \ "mvc 176(8,15), 64(1)\n\t" \ "mvc 184(8,15), 72(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,192\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"6","7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1, arg2, arg3, arg4, arg5, \ arg6, arg7 ,arg8, arg9, arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[11]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ _argvec[9] = (unsigned long)arg9; \ _argvec[10] = (unsigned long)arg10; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-200\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 6,40(1)\n\t" \ "mvc 160(8,15), 48(1)\n\t" \ "mvc 168(8,15), 56(1)\n\t" \ "mvc 176(8,15), 64(1)\n\t" \ "mvc 184(8,15), 72(1)\n\t" \ "mvc 192(8,15), 80(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,200\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"6","7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1, arg2, arg3, arg4, arg5, \ arg6, arg7 ,arg8, arg9, arg10, arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[12]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ _argvec[9] = (unsigned long)arg9; \ _argvec[10] = (unsigned long)arg10; \ _argvec[11] = (unsigned long)arg11; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-208\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 6,40(1)\n\t" \ "mvc 160(8,15), 48(1)\n\t" \ "mvc 168(8,15), 56(1)\n\t" \ "mvc 176(8,15), 64(1)\n\t" \ "mvc 184(8,15), 72(1)\n\t" \ "mvc 192(8,15), 80(1)\n\t" \ "mvc 200(8,15), 88(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,208\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"6","7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1, arg2, arg3, arg4, arg5, \ arg6, arg7 ,arg8, arg9, arg10, arg11, arg12)\ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[13]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)arg1; \ _argvec[2] = (unsigned long)arg2; \ _argvec[3] = (unsigned long)arg3; \ _argvec[4] = (unsigned long)arg4; \ _argvec[5] = (unsigned long)arg5; \ _argvec[6] = (unsigned long)arg6; \ _argvec[7] = (unsigned long)arg7; \ _argvec[8] = (unsigned long)arg8; \ _argvec[9] = (unsigned long)arg9; \ _argvec[10] = (unsigned long)arg10; \ _argvec[11] = (unsigned long)arg11; \ _argvec[12] = (unsigned long)arg12; \ __asm__ volatile( \ VALGRIND_CFI_PROLOGUE \ "aghi 15,-216\n\t" \ "lg 2, 8(1)\n\t" \ "lg 3,16(1)\n\t" \ "lg 4,24(1)\n\t" \ "lg 5,32(1)\n\t" \ "lg 6,40(1)\n\t" \ "mvc 160(8,15), 48(1)\n\t" \ "mvc 168(8,15), 56(1)\n\t" \ "mvc 176(8,15), 64(1)\n\t" \ "mvc 184(8,15), 72(1)\n\t" \ "mvc 192(8,15), 80(1)\n\t" \ "mvc 200(8,15), 88(1)\n\t" \ "mvc 208(8,15), 96(1)\n\t" \ "lg 1, 0(1)\n\t" \ VALGRIND_CALL_NOREDIR_R1 \ "lgr %0, 2\n\t" \ "aghi 15,216\n\t" \ VALGRIND_CFI_EPILOGUE \ : /*out*/ "=d" (_res) \ : /*in*/ "a" (&_argvec[0]) __FRAME_POINTER \ : /*trash*/ "cc", "memory", __CALLER_SAVED_REGS,"6","7" \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_s390x_linux */ /* ------------------------- mips32-linux ----------------------- */ #if defined(PLAT_mips32_linux) /* These regs are trashed by the hidden call. */ #define __CALLER_SAVED_REGS "$2", "$3", "$4", "$5", "$6", \ "$7", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24", \ "$25", "$31" /* These CALL_FN_ macros assume that on mips-linux, sizeof(unsigned long) == 4. */ #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[1]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "subu $29, $29, 16 \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 16\n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[2]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "subu $29, $29, 16 \n\t" \ "lw $4, 4(%1) \n\t" /* arg1*/ \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 16 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[3]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "subu $29, $29, 16 \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 16 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[4]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "subu $29, $29, 16 \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 16 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[5]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "subu $29, $29, 16 \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 16 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[6]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "lw $4, 20(%1) \n\t" \ "subu $29, $29, 24\n\t" \ "sw $4, 16($29) \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 24 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[7]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "lw $4, 20(%1) \n\t" \ "subu $29, $29, 32\n\t" \ "sw $4, 16($29) \n\t" \ "lw $4, 24(%1) \n\t" \ "nop\n\t" \ "sw $4, 20($29) \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 32 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[8]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "lw $4, 20(%1) \n\t" \ "subu $29, $29, 32\n\t" \ "sw $4, 16($29) \n\t" \ "lw $4, 24(%1) \n\t" \ "sw $4, 20($29) \n\t" \ "lw $4, 28(%1) \n\t" \ "sw $4, 24($29) \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 32 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[9]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "lw $4, 20(%1) \n\t" \ "subu $29, $29, 40\n\t" \ "sw $4, 16($29) \n\t" \ "lw $4, 24(%1) \n\t" \ "sw $4, 20($29) \n\t" \ "lw $4, 28(%1) \n\t" \ "sw $4, 24($29) \n\t" \ "lw $4, 32(%1) \n\t" \ "sw $4, 28($29) \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 40 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[10]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "lw $4, 20(%1) \n\t" \ "subu $29, $29, 40\n\t" \ "sw $4, 16($29) \n\t" \ "lw $4, 24(%1) \n\t" \ "sw $4, 20($29) \n\t" \ "lw $4, 28(%1) \n\t" \ "sw $4, 24($29) \n\t" \ "lw $4, 32(%1) \n\t" \ "sw $4, 28($29) \n\t" \ "lw $4, 36(%1) \n\t" \ "sw $4, 32($29) \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 40 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[11]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "lw $4, 20(%1) \n\t" \ "subu $29, $29, 48\n\t" \ "sw $4, 16($29) \n\t" \ "lw $4, 24(%1) \n\t" \ "sw $4, 20($29) \n\t" \ "lw $4, 28(%1) \n\t" \ "sw $4, 24($29) \n\t" \ "lw $4, 32(%1) \n\t" \ "sw $4, 28($29) \n\t" \ "lw $4, 36(%1) \n\t" \ "sw $4, 32($29) \n\t" \ "lw $4, 40(%1) \n\t" \ "sw $4, 36($29) \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 48 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5, \ arg6,arg7,arg8,arg9,arg10, \ arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[12]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "lw $4, 20(%1) \n\t" \ "subu $29, $29, 48\n\t" \ "sw $4, 16($29) \n\t" \ "lw $4, 24(%1) \n\t" \ "sw $4, 20($29) \n\t" \ "lw $4, 28(%1) \n\t" \ "sw $4, 24($29) \n\t" \ "lw $4, 32(%1) \n\t" \ "sw $4, 28($29) \n\t" \ "lw $4, 36(%1) \n\t" \ "sw $4, 32($29) \n\t" \ "lw $4, 40(%1) \n\t" \ "sw $4, 36($29) \n\t" \ "lw $4, 44(%1) \n\t" \ "sw $4, 40($29) \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 48 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5, \ arg6,arg7,arg8,arg9,arg10, \ arg11,arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long _argvec[13]; \ volatile unsigned long _res; \ _argvec[0] = (unsigned long)_orig.nraddr; \ _argvec[1] = (unsigned long)(arg1); \ _argvec[2] = (unsigned long)(arg2); \ _argvec[3] = (unsigned long)(arg3); \ _argvec[4] = (unsigned long)(arg4); \ _argvec[5] = (unsigned long)(arg5); \ _argvec[6] = (unsigned long)(arg6); \ _argvec[7] = (unsigned long)(arg7); \ _argvec[8] = (unsigned long)(arg8); \ _argvec[9] = (unsigned long)(arg9); \ _argvec[10] = (unsigned long)(arg10); \ _argvec[11] = (unsigned long)(arg11); \ _argvec[12] = (unsigned long)(arg12); \ __asm__ volatile( \ "subu $29, $29, 8 \n\t" \ "sw $28, 0($29) \n\t" \ "sw $31, 4($29) \n\t" \ "lw $4, 20(%1) \n\t" \ "subu $29, $29, 56\n\t" \ "sw $4, 16($29) \n\t" \ "lw $4, 24(%1) \n\t" \ "sw $4, 20($29) \n\t" \ "lw $4, 28(%1) \n\t" \ "sw $4, 24($29) \n\t" \ "lw $4, 32(%1) \n\t" \ "sw $4, 28($29) \n\t" \ "lw $4, 36(%1) \n\t" \ "sw $4, 32($29) \n\t" \ "lw $4, 40(%1) \n\t" \ "sw $4, 36($29) \n\t" \ "lw $4, 44(%1) \n\t" \ "sw $4, 40($29) \n\t" \ "lw $4, 48(%1) \n\t" \ "sw $4, 44($29) \n\t" \ "lw $4, 4(%1) \n\t" \ "lw $5, 8(%1) \n\t" \ "lw $6, 12(%1) \n\t" \ "lw $7, 16(%1) \n\t" \ "lw $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "addu $29, $29, 56 \n\t" \ "lw $28, 0($29) \n\t" \ "lw $31, 4($29) \n\t" \ "addu $29, $29, 8 \n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) _res; \ } while (0) #endif /* PLAT_mips32_linux */ /* ------------------------- mips64-linux ------------------------- */ #if defined(PLAT_mips64_linux) /* These regs are trashed by the hidden call. */ #define __CALLER_SAVED_REGS "$2", "$3", "$4", "$5", "$6", \ "$7", "$8", "$9", "$10", "$11", "$12", "$13", "$14", "$15", "$24", \ "$25", "$31" /* These CALL_FN_ macros assume that on mips64-linux, sizeof(long long) == 8. */ #define MIPS64_LONG2REG_CAST(x) ((long long)(long)x) #define CALL_FN_W_v(lval, orig) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[1]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ __asm__ volatile( \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "0" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_W(lval, orig, arg1) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[2]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ __asm__ volatile( \ "ld $4, 8(%1)\n\t" /* arg1*/ \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_WW(lval, orig, arg1,arg2) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[3]; \ volatile unsigned long long _res; \ _argvec[0] = _orig.nraddr; \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ __asm__ volatile( \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_WWW(lval, orig, arg1,arg2,arg3) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[4]; \ volatile unsigned long long _res; \ _argvec[0] = _orig.nraddr; \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ __asm__ volatile( \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_WWWW(lval, orig, arg1,arg2,arg3,arg4) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[5]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ __asm__ volatile( \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_5W(lval, orig, arg1,arg2,arg3,arg4,arg5) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[6]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ _argvec[5] = MIPS64_LONG2REG_CAST(arg5); \ __asm__ volatile( \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $8, 40(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_6W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[7]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ _argvec[5] = MIPS64_LONG2REG_CAST(arg5); \ _argvec[6] = MIPS64_LONG2REG_CAST(arg6); \ __asm__ volatile( \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $8, 40(%1)\n\t" \ "ld $9, 48(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_7W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[8]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ _argvec[5] = MIPS64_LONG2REG_CAST(arg5); \ _argvec[6] = MIPS64_LONG2REG_CAST(arg6); \ _argvec[7] = MIPS64_LONG2REG_CAST(arg7); \ __asm__ volatile( \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $8, 40(%1)\n\t" \ "ld $9, 48(%1)\n\t" \ "ld $10, 56(%1)\n\t" \ "ld $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_8W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[9]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ _argvec[5] = MIPS64_LONG2REG_CAST(arg5); \ _argvec[6] = MIPS64_LONG2REG_CAST(arg6); \ _argvec[7] = MIPS64_LONG2REG_CAST(arg7); \ _argvec[8] = MIPS64_LONG2REG_CAST(arg8); \ __asm__ volatile( \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $8, 40(%1)\n\t" \ "ld $9, 48(%1)\n\t" \ "ld $10, 56(%1)\n\t" \ "ld $11, 64(%1)\n\t" \ "ld $25, 0(%1) \n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_9W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[10]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ _argvec[5] = MIPS64_LONG2REG_CAST(arg5); \ _argvec[6] = MIPS64_LONG2REG_CAST(arg6); \ _argvec[7] = MIPS64_LONG2REG_CAST(arg7); \ _argvec[8] = MIPS64_LONG2REG_CAST(arg8); \ _argvec[9] = MIPS64_LONG2REG_CAST(arg9); \ __asm__ volatile( \ "dsubu $29, $29, 8\n\t" \ "ld $4, 72(%1)\n\t" \ "sd $4, 0($29)\n\t" \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $8, 40(%1)\n\t" \ "ld $9, 48(%1)\n\t" \ "ld $10, 56(%1)\n\t" \ "ld $11, 64(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "daddu $29, $29, 8\n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_10W(lval, orig, arg1,arg2,arg3,arg4,arg5,arg6, \ arg7,arg8,arg9,arg10) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[11]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ _argvec[5] = MIPS64_LONG2REG_CAST(arg5); \ _argvec[6] = MIPS64_LONG2REG_CAST(arg6); \ _argvec[7] = MIPS64_LONG2REG_CAST(arg7); \ _argvec[8] = MIPS64_LONG2REG_CAST(arg8); \ _argvec[9] = MIPS64_LONG2REG_CAST(arg9); \ _argvec[10] = MIPS64_LONG2REG_CAST(arg10); \ __asm__ volatile( \ "dsubu $29, $29, 16\n\t" \ "ld $4, 72(%1)\n\t" \ "sd $4, 0($29)\n\t" \ "ld $4, 80(%1)\n\t" \ "sd $4, 8($29)\n\t" \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $8, 40(%1)\n\t" \ "ld $9, 48(%1)\n\t" \ "ld $10, 56(%1)\n\t" \ "ld $11, 64(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "daddu $29, $29, 16\n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_11W(lval, orig, arg1,arg2,arg3,arg4,arg5, \ arg6,arg7,arg8,arg9,arg10, \ arg11) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[12]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ _argvec[5] = MIPS64_LONG2REG_CAST(arg5); \ _argvec[6] = MIPS64_LONG2REG_CAST(arg6); \ _argvec[7] = MIPS64_LONG2REG_CAST(arg7); \ _argvec[8] = MIPS64_LONG2REG_CAST(arg8); \ _argvec[9] = MIPS64_LONG2REG_CAST(arg9); \ _argvec[10] = MIPS64_LONG2REG_CAST(arg10); \ _argvec[11] = MIPS64_LONG2REG_CAST(arg11); \ __asm__ volatile( \ "dsubu $29, $29, 24\n\t" \ "ld $4, 72(%1)\n\t" \ "sd $4, 0($29)\n\t" \ "ld $4, 80(%1)\n\t" \ "sd $4, 8($29)\n\t" \ "ld $4, 88(%1)\n\t" \ "sd $4, 16($29)\n\t" \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $8, 40(%1)\n\t" \ "ld $9, 48(%1)\n\t" \ "ld $10, 56(%1)\n\t" \ "ld $11, 64(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "daddu $29, $29, 24\n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #define CALL_FN_W_12W(lval, orig, arg1,arg2,arg3,arg4,arg5, \ arg6,arg7,arg8,arg9,arg10, \ arg11,arg12) \ do { \ volatile OrigFn _orig = (orig); \ volatile unsigned long long _argvec[13]; \ volatile unsigned long long _res; \ _argvec[0] = MIPS64_LONG2REG_CAST(_orig.nraddr); \ _argvec[1] = MIPS64_LONG2REG_CAST(arg1); \ _argvec[2] = MIPS64_LONG2REG_CAST(arg2); \ _argvec[3] = MIPS64_LONG2REG_CAST(arg3); \ _argvec[4] = MIPS64_LONG2REG_CAST(arg4); \ _argvec[5] = MIPS64_LONG2REG_CAST(arg5); \ _argvec[6] = MIPS64_LONG2REG_CAST(arg6); \ _argvec[7] = MIPS64_LONG2REG_CAST(arg7); \ _argvec[8] = MIPS64_LONG2REG_CAST(arg8); \ _argvec[9] = MIPS64_LONG2REG_CAST(arg9); \ _argvec[10] = MIPS64_LONG2REG_CAST(arg10); \ _argvec[11] = MIPS64_LONG2REG_CAST(arg11); \ _argvec[12] = MIPS64_LONG2REG_CAST(arg12); \ __asm__ volatile( \ "dsubu $29, $29, 32\n\t" \ "ld $4, 72(%1)\n\t" \ "sd $4, 0($29)\n\t" \ "ld $4, 80(%1)\n\t" \ "sd $4, 8($29)\n\t" \ "ld $4, 88(%1)\n\t" \ "sd $4, 16($29)\n\t" \ "ld $4, 96(%1)\n\t" \ "sd $4, 24($29)\n\t" \ "ld $4, 8(%1)\n\t" \ "ld $5, 16(%1)\n\t" \ "ld $6, 24(%1)\n\t" \ "ld $7, 32(%1)\n\t" \ "ld $8, 40(%1)\n\t" \ "ld $9, 48(%1)\n\t" \ "ld $10, 56(%1)\n\t" \ "ld $11, 64(%1)\n\t" \ "ld $25, 0(%1)\n\t" /* target->t9 */ \ VALGRIND_CALL_NOREDIR_T9 \ "daddu $29, $29, 32\n\t" \ "move %0, $2\n" \ : /*out*/ "=r" (_res) \ : /*in*/ "r" (&_argvec[0]) \ : /*trash*/ "memory", __CALLER_SAVED_REGS \ ); \ lval = (__typeof__(lval)) (long)_res; \ } while (0) #endif /* PLAT_mips64_linux */ /* ------------------------------------------------------------------ */ /* ARCHITECTURE INDEPENDENT MACROS for CLIENT REQUESTS. */ /* */ /* ------------------------------------------------------------------ */ /* Some request codes. There are many more of these, but most are not exposed to end-user view. These are the public ones, all of the form 0x1000 + small_number. Core ones are in the range 0x00000000--0x0000ffff. The non-public ones start at 0x2000. */ /* These macros are used by tools -- they must be public, but don't embed them into other programs. */ #define VG_USERREQ_TOOL_BASE(a,b) \ ((unsigned int)(((a)&0xff) << 24 | ((b)&0xff) << 16)) #define VG_IS_TOOL_USERREQ(a, b, v) \ (VG_USERREQ_TOOL_BASE(a,b) == ((v) & 0xffff0000)) /* !! ABIWARNING !! ABIWARNING !! ABIWARNING !! ABIWARNING !! This enum comprises an ABI exported by Valgrind to programs which use client requests. DO NOT CHANGE THE NUMERIC VALUES OF THESE ENTRIES, NOR DELETE ANY -- add new ones at the end of the most relevant group. */ typedef enum { VG_USERREQ__RUNNING_ON_VALGRIND = 0x1001, VG_USERREQ__DISCARD_TRANSLATIONS = 0x1002, /* These allow any function to be called from the simulated CPU but run on the real CPU. Nb: the first arg passed to the function is always the ThreadId of the running thread! So CLIENT_CALL0 actually requires a 1 arg function, etc. */ VG_USERREQ__CLIENT_CALL0 = 0x1101, VG_USERREQ__CLIENT_CALL1 = 0x1102, VG_USERREQ__CLIENT_CALL2 = 0x1103, VG_USERREQ__CLIENT_CALL3 = 0x1104, /* Can be useful in regression testing suites -- eg. can send Valgrind's output to /dev/null and still count errors. */ VG_USERREQ__COUNT_ERRORS = 0x1201, /* Allows the client program and/or gdbserver to execute a monitor command. */ VG_USERREQ__GDB_MONITOR_COMMAND = 0x1202, /* These are useful and can be interpreted by any tool that tracks malloc() et al, by using vg_replace_malloc.c. */ VG_USERREQ__MALLOCLIKE_BLOCK = 0x1301, VG_USERREQ__RESIZEINPLACE_BLOCK = 0x130b, VG_USERREQ__FREELIKE_BLOCK = 0x1302, /* Memory pool support. */ VG_USERREQ__CREATE_MEMPOOL = 0x1303, VG_USERREQ__DESTROY_MEMPOOL = 0x1304, VG_USERREQ__MEMPOOL_ALLOC = 0x1305, VG_USERREQ__MEMPOOL_FREE = 0x1306, VG_USERREQ__MEMPOOL_TRIM = 0x1307, VG_USERREQ__MOVE_MEMPOOL = 0x1308, VG_USERREQ__MEMPOOL_CHANGE = 0x1309, VG_USERREQ__MEMPOOL_EXISTS = 0x130a, /* Allow printfs to valgrind log. */ /* The first two pass the va_list argument by value, which assumes it is the same size as or smaller than a UWord, which generally isn't the case. Hence are deprecated. The second two pass the vargs by reference and so are immune to this problem. */ /* both :: char* fmt, va_list vargs (DEPRECATED) */ VG_USERREQ__PRINTF = 0x1401, VG_USERREQ__PRINTF_BACKTRACE = 0x1402, /* both :: char* fmt, va_list* vargs */ VG_USERREQ__PRINTF_VALIST_BY_REF = 0x1403, VG_USERREQ__PRINTF_BACKTRACE_VALIST_BY_REF = 0x1404, /* Stack support. */ VG_USERREQ__STACK_REGISTER = 0x1501, VG_USERREQ__STACK_DEREGISTER = 0x1502, VG_USERREQ__STACK_CHANGE = 0x1503, /* Wine support */ VG_USERREQ__LOAD_PDB_DEBUGINFO = 0x1601, /* Querying of debug info. */ VG_USERREQ__MAP_IP_TO_SRCLOC = 0x1701, /* Disable/enable error reporting level. Takes a single Word arg which is the delta to this thread's error disablement indicator. Hence 1 disables or further disables errors, and -1 moves back towards enablement. Other values are not allowed. */ VG_USERREQ__CHANGE_ERR_DISABLEMENT = 0x1801, /* Some requests used for Valgrind internal, such as self-test or self-hosting. */ /* Initialise IR injection */ VG_USERREQ__VEX_INIT_FOR_IRI = 0x1901, /* Used by Inner Valgrind to inform Outer Valgrind where to find the list of inner guest threads */ VG_USERREQ__INNER_THREADS = 0x1902 } Vg_ClientRequest; #if !defined(__GNUC__) # define __extension__ /* */ #endif /* Returns the number of Valgrinds this code is running under. That is, 0 if running natively, 1 if running under Valgrind, 2 if running under Valgrind which is running under another Valgrind, etc. */ #define RUNNING_ON_VALGRIND \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* if not */, \ VG_USERREQ__RUNNING_ON_VALGRIND, \ 0, 0, 0, 0, 0) \ /* Discard translation of code in the range [_qzz_addr .. _qzz_addr + _qzz_len - 1]. Useful if you are debugging a JITter or some such, since it provides a way to make sure valgrind will retranslate the invalidated area. Returns no value. */ #define VALGRIND_DISCARD_TRANSLATIONS(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DISCARD_TRANSLATIONS, \ _qzz_addr, _qzz_len, 0, 0, 0) #define VALGRIND_INNER_THREADS(_qzz_addr) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__INNER_THREADS, \ _qzz_addr, 0, 0, 0, 0) /* These requests are for getting Valgrind itself to print something. Possibly with a backtrace. This is a really ugly hack. The return value is the number of characters printed, excluding the "**** " part at the start and the backtrace (if present). */ #if defined(__GNUC__) || defined(__INTEL_COMPILER) && !defined(_MSC_VER) /* Modern GCC will optimize the static routine out if unused, and unused attribute will shut down warnings about it. */ static int VALGRIND_PRINTF(const char *format, ...) __attribute__((format(__printf__, 1, 2), __unused__)); #endif static int #if defined(_MSC_VER) __inline #endif VALGRIND_PRINTF(const char *format, ...) { #if defined(NVALGRIND) (void)format; return 0; #else /* NVALGRIND */ #if defined(_MSC_VER) || defined(__MINGW64__) uintptr_t _qzz_res; #else unsigned long _qzz_res; #endif va_list vargs; va_start(vargs, format); #if defined(_MSC_VER) || defined(__MINGW64__) _qzz_res = VALGRIND_DO_CLIENT_REQUEST_EXPR(0, VG_USERREQ__PRINTF_VALIST_BY_REF, (uintptr_t)format, (uintptr_t)&vargs, 0, 0, 0); #else _qzz_res = VALGRIND_DO_CLIENT_REQUEST_EXPR(0, VG_USERREQ__PRINTF_VALIST_BY_REF, (unsigned long)format, (unsigned long)&vargs, 0, 0, 0); #endif va_end(vargs); return (int)_qzz_res; #endif /* NVALGRIND */ } #if defined(__GNUC__) || defined(__INTEL_COMPILER) && !defined(_MSC_VER) static int VALGRIND_PRINTF_BACKTRACE(const char *format, ...) __attribute__((format(__printf__, 1, 2), __unused__)); #endif static int #if defined(_MSC_VER) __inline #endif VALGRIND_PRINTF_BACKTRACE(const char *format, ...) { #if defined(NVALGRIND) (void)format; return 0; #else /* NVALGRIND */ #if defined(_MSC_VER) || defined(__MINGW64__) uintptr_t _qzz_res; #else unsigned long _qzz_res; #endif va_list vargs; va_start(vargs, format); #if defined(_MSC_VER) || defined(__MINGW64__) _qzz_res = VALGRIND_DO_CLIENT_REQUEST_EXPR(0, VG_USERREQ__PRINTF_BACKTRACE_VALIST_BY_REF, (uintptr_t)format, (uintptr_t)&vargs, 0, 0, 0); #else _qzz_res = VALGRIND_DO_CLIENT_REQUEST_EXPR(0, VG_USERREQ__PRINTF_BACKTRACE_VALIST_BY_REF, (unsigned long)format, (unsigned long)&vargs, 0, 0, 0); #endif va_end(vargs); return (int)_qzz_res; #endif /* NVALGRIND */ } /* These requests allow control to move from the simulated CPU to the real CPU, calling an arbitrary function. Note that the current ThreadId is inserted as the first argument. So this call: VALGRIND_NON_SIMD_CALL2(f, arg1, arg2) requires f to have this signature: Word f(Word tid, Word arg1, Word arg2) where "Word" is a word-sized type. Note that these client requests are not entirely reliable. For example, if you call a function with them that subsequently calls printf(), there's a high chance Valgrind will crash. Generally, your prospects of these working are made higher if the called function does not refer to any global variables, and does not refer to any libc or other functions (printf et al). Any kind of entanglement with libc or dynamic linking is likely to have a bad outcome, for tricky reasons which we've grappled with a lot in the past. */ #define VALGRIND_NON_SIMD_CALL0(_qyy_fn) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__CLIENT_CALL0, \ _qyy_fn, \ 0, 0, 0, 0) #define VALGRIND_NON_SIMD_CALL1(_qyy_fn, _qyy_arg1) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__CLIENT_CALL1, \ _qyy_fn, \ _qyy_arg1, 0, 0, 0) #define VALGRIND_NON_SIMD_CALL2(_qyy_fn, _qyy_arg1, _qyy_arg2) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__CLIENT_CALL2, \ _qyy_fn, \ _qyy_arg1, _qyy_arg2, 0, 0) #define VALGRIND_NON_SIMD_CALL3(_qyy_fn, _qyy_arg1, _qyy_arg2, _qyy_arg3) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__CLIENT_CALL3, \ _qyy_fn, \ _qyy_arg1, _qyy_arg2, \ _qyy_arg3, 0) /* Counts the number of errors that have been recorded by a tool. Nb: the tool must record the errors with VG_(maybe_record_error)() or VG_(unique_error)() for them to be counted. */ #define VALGRIND_COUNT_ERRORS \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR( \ 0 /* default return */, \ VG_USERREQ__COUNT_ERRORS, \ 0, 0, 0, 0, 0) /* Several Valgrind tools (Memcheck, Massif, Helgrind, DRD) rely on knowing when heap blocks are allocated in order to give accurate results. This happens automatically for the standard allocator functions such as malloc(), calloc(), realloc(), memalign(), new, new[], free(), delete, delete[], etc. But if your program uses a custom allocator, this doesn't automatically happen, and Valgrind will not do as well. For example, if you allocate superblocks with mmap() and then allocates chunks of the superblocks, all Valgrind's observations will be at the mmap() level and it won't know that the chunks should be considered separate entities. In Memcheck's case, that means you probably won't get heap block overrun detection (because there won't be redzones marked as unaddressable) and you definitely won't get any leak detection. The following client requests allow a custom allocator to be annotated so that it can be handled accurately by Valgrind. VALGRIND_MALLOCLIKE_BLOCK marks a region of memory as having been allocated by a malloc()-like function. For Memcheck (an illustrative case), this does two things: - It records that the block has been allocated. This means any addresses within the block mentioned in error messages will be identified as belonging to the block. It also means that if the block isn't freed it will be detected by the leak checker. - It marks the block as being addressable and undefined (if 'is_zeroed' is not set), or addressable and defined (if 'is_zeroed' is set). This controls how accesses to the block by the program are handled. 'addr' is the start of the usable block (ie. after any redzone), 'sizeB' is its size. 'rzB' is the redzone size if the allocator can apply redzones -- these are blocks of padding at the start and end of each block. Adding redzones is recommended as it makes it much more likely Valgrind will spot block overruns. `is_zeroed' indicates if the memory is zeroed (or filled with another predictable value), as is the case for calloc(). VALGRIND_MALLOCLIKE_BLOCK should be put immediately after the point where a heap block -- that will be used by the client program -- is allocated. It's best to put it at the outermost level of the allocator if possible; for example, if you have a function my_alloc() which calls internal_alloc(), and the client request is put inside internal_alloc(), stack traces relating to the heap block will contain entries for both my_alloc() and internal_alloc(), which is probably not what you want. For Memcheck users: if you use VALGRIND_MALLOCLIKE_BLOCK to carve out custom blocks from within a heap block, B, that has been allocated with malloc/calloc/new/etc, then block B will be *ignored* during leak-checking -- the custom blocks will take precedence. VALGRIND_FREELIKE_BLOCK is the partner to VALGRIND_MALLOCLIKE_BLOCK. For Memcheck, it does two things: - It records that the block has been deallocated. This assumes that the block was annotated as having been allocated via VALGRIND_MALLOCLIKE_BLOCK. Otherwise, an error will be issued. - It marks the block as being unaddressable. VALGRIND_FREELIKE_BLOCK should be put immediately after the point where a heap block is deallocated. VALGRIND_RESIZEINPLACE_BLOCK informs a tool about reallocation. For Memcheck, it does four things: - It records that the size of a block has been changed. This assumes that the block was annotated as having been allocated via VALGRIND_MALLOCLIKE_BLOCK. Otherwise, an error will be issued. - If the block shrunk, it marks the freed memory as being unaddressable. - If the block grew, it marks the new area as undefined and defines a red zone past the end of the new block. - The V-bits of the overlap between the old and the new block are preserved. VALGRIND_RESIZEINPLACE_BLOCK should be put after allocation of the new block and before deallocation of the old block. In many cases, these three client requests will not be enough to get your allocator working well with Memcheck. More specifically, if your allocator writes to freed blocks in any way then a VALGRIND_MAKE_MEM_UNDEFINED call will be necessary to mark the memory as addressable just before the zeroing occurs, otherwise you'll get a lot of invalid write errors. For example, you'll need to do this if your allocator recycles freed blocks, but it zeroes them before handing them back out (via VALGRIND_MALLOCLIKE_BLOCK). Alternatively, if your allocator reuses freed blocks for allocator-internal data structures, VALGRIND_MAKE_MEM_UNDEFINED calls will also be necessary. Really, what's happening is a blurring of the lines between the client program and the allocator... after VALGRIND_FREELIKE_BLOCK is called, the memory should be considered unaddressable to the client program, but the allocator knows more than the rest of the client program and so may be able to safely access it. Extra client requests are necessary for Valgrind to understand the distinction between the allocator and the rest of the program. Ignored if addr == 0. */ #define VALGRIND_MALLOCLIKE_BLOCK(addr, sizeB, rzB, is_zeroed) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__MALLOCLIKE_BLOCK, \ addr, sizeB, rzB, is_zeroed, 0) /* See the comment for VALGRIND_MALLOCLIKE_BLOCK for details. Ignored if addr == 0. */ #define VALGRIND_RESIZEINPLACE_BLOCK(addr, oldSizeB, newSizeB, rzB) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__RESIZEINPLACE_BLOCK, \ addr, oldSizeB, newSizeB, rzB, 0) /* See the comment for VALGRIND_MALLOCLIKE_BLOCK for details. Ignored if addr == 0. */ #define VALGRIND_FREELIKE_BLOCK(addr, rzB) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__FREELIKE_BLOCK, \ addr, rzB, 0, 0, 0) /* Create a memory pool. */ #define VALGRIND_CREATE_MEMPOOL(pool, rzB, is_zeroed) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__CREATE_MEMPOOL, \ pool, rzB, is_zeroed, 0, 0) /* Create a memory pool with some flags specifying extended behaviour. When flags is zero, the behaviour is identical to VALGRIND_CREATE_MEMPOOL. The flag VALGRIND_MEMPOOL_METAPOOL specifies that the pieces of memory associated with the pool using VALGRIND_MEMPOOL_ALLOC will be used by the application as superblocks to dole out MALLOC_LIKE blocks using VALGRIND_MALLOCLIKE_BLOCK. In other words, a meta pool is a "2 levels" pool : first level is the blocks described by VALGRIND_MEMPOOL_ALLOC. The second level blocks are described using VALGRIND_MALLOCLIKE_BLOCK. Note that the association between the pool and the second level blocks is implicit : second level blocks will be located inside first level blocks. It is necessary to use the VALGRIND_MEMPOOL_METAPOOL flag for such 2 levels pools, as otherwise valgrind will detect overlapping memory blocks, and will abort execution (e.g. during leak search). Such a meta pool can also be marked as an 'auto free' pool using the flag VALGRIND_MEMPOOL_AUTO_FREE, which must be OR-ed together with the VALGRIND_MEMPOOL_METAPOOL. For an 'auto free' pool, VALGRIND_MEMPOOL_FREE will automatically free the second level blocks that are contained inside the first level block freed with VALGRIND_MEMPOOL_FREE. In other words, calling VALGRIND_MEMPOOL_FREE will cause implicit calls to VALGRIND_FREELIKE_BLOCK for all the second level blocks included in the first level block. Note: it is an error to use the VALGRIND_MEMPOOL_AUTO_FREE flag without the VALGRIND_MEMPOOL_METAPOOL flag. */ #define VALGRIND_MEMPOOL_AUTO_FREE 1 #define VALGRIND_MEMPOOL_METAPOOL 2 #define VALGRIND_CREATE_MEMPOOL_EXT(pool, rzB, is_zeroed, flags) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__CREATE_MEMPOOL, \ pool, rzB, is_zeroed, flags, 0) /* Destroy a memory pool. */ #define VALGRIND_DESTROY_MEMPOOL(pool) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DESTROY_MEMPOOL, \ pool, 0, 0, 0, 0) /* Associate a piece of memory with a memory pool. */ #define VALGRIND_MEMPOOL_ALLOC(pool, addr, size) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__MEMPOOL_ALLOC, \ pool, addr, size, 0, 0) /* Disassociate a piece of memory from a memory pool. */ #define VALGRIND_MEMPOOL_FREE(pool, addr) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__MEMPOOL_FREE, \ pool, addr, 0, 0, 0) /* Disassociate any pieces outside a particular range. */ #define VALGRIND_MEMPOOL_TRIM(pool, addr, size) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__MEMPOOL_TRIM, \ pool, addr, size, 0, 0) /* Resize and/or move a piece associated with a memory pool. */ #define VALGRIND_MOVE_MEMPOOL(poolA, poolB) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__MOVE_MEMPOOL, \ poolA, poolB, 0, 0, 0) /* Resize and/or move a piece associated with a memory pool. */ #define VALGRIND_MEMPOOL_CHANGE(pool, addrA, addrB, size) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__MEMPOOL_CHANGE, \ pool, addrA, addrB, size, 0) /* Return 1 if a mempool exists, else 0. */ #define VALGRIND_MEMPOOL_EXISTS(pool) \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__MEMPOOL_EXISTS, \ pool, 0, 0, 0, 0) /* Mark a piece of memory as being a stack. Returns a stack id. start is the lowest addressable stack byte, end is the highest addressable stack byte. */ #define VALGRIND_STACK_REGISTER(start, end) \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__STACK_REGISTER, \ start, end, 0, 0, 0) /* Unmark the piece of memory associated with a stack id as being a stack. */ #define VALGRIND_STACK_DEREGISTER(id) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__STACK_DEREGISTER, \ id, 0, 0, 0, 0) /* Change the start and end address of the stack id. start is the new lowest addressable stack byte, end is the new highest addressable stack byte. */ #define VALGRIND_STACK_CHANGE(id, start, end) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__STACK_CHANGE, \ id, start, end, 0, 0) /* Load PDB debug info for Wine PE image_map. */ #define VALGRIND_LOAD_PDB_DEBUGINFO(fd, ptr, total_size, delta) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__LOAD_PDB_DEBUGINFO, \ fd, ptr, total_size, delta, 0) /* Map a code address to a source file name and line number. buf64 must point to a 64-byte buffer in the caller's address space. The result will be dumped in there and is guaranteed to be zero terminated. If no info is found, the first byte is set to zero. */ #define VALGRIND_MAP_IP_TO_SRCLOC(addr, buf64) \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__MAP_IP_TO_SRCLOC, \ addr, buf64, 0, 0, 0) /* Disable error reporting for this thread. Behaves in a stack like way, so you can safely call this multiple times provided that VALGRIND_ENABLE_ERROR_REPORTING is called the same number of times to re-enable reporting. The first call of this macro disables reporting. Subsequent calls have no effect except to increase the number of VALGRIND_ENABLE_ERROR_REPORTING calls needed to re-enable reporting. Child threads do not inherit this setting from their parents -- they are always created with reporting enabled. */ #define VALGRIND_DISABLE_ERROR_REPORTING \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__CHANGE_ERR_DISABLEMENT, \ 1, 0, 0, 0, 0) /* Re-enable error reporting, as per comments on VALGRIND_DISABLE_ERROR_REPORTING. */ #define VALGRIND_ENABLE_ERROR_REPORTING \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__CHANGE_ERR_DISABLEMENT, \ -1, 0, 0, 0, 0) /* Execute a monitor command from the client program. If a connection is opened with GDB, the output will be sent according to the output mode set for vgdb. If no connection is opened, output will go to the log output. Returns 1 if command not recognised, 0 otherwise. */ #define VALGRIND_MONITOR_COMMAND(command) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0, VG_USERREQ__GDB_MONITOR_COMMAND, \ command, 0, 0, 0, 0) #undef PLAT_x86_darwin #undef PLAT_amd64_darwin #undef PLAT_x86_win32 #undef PLAT_amd64_win64 #undef PLAT_x86_linux #undef PLAT_amd64_linux #undef PLAT_ppc32_linux #undef PLAT_ppc64be_linux #undef PLAT_ppc64le_linux #undef PLAT_arm_linux #undef PLAT_s390x_linux #undef PLAT_mips32_linux #undef PLAT_mips64_linux #undef PLAT_x86_solaris #undef PLAT_amd64_solaris #endif /* __VALGRIND_H */ pmdk-1.8/src/common/valgrind/.cstyleignore0000664000000000000000000000006313615011243017405 0ustar rootrootdrd.h helgrind.h memcheck.h pmemcheck.h valgrind.h pmdk-1.8/src/common/valgrind/pmemcheck.h0000664000000000000000000002455113615011243017012 0ustar rootroot/* * Copyright (c) 2014-2015, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of Intel Corporation nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef __PMEMCHECK_H #define __PMEMCHECK_H /* This file is for inclusion into client (your!) code. You can use these macros to manipulate and query memory permissions inside your own programs. See comment near the top of valgrind.h on how to use them. */ #include "valgrind.h" /* !! ABIWARNING !! ABIWARNING !! ABIWARNING !! ABIWARNING !! This enum comprises an ABI exported by Valgrind to programs which use client requests. DO NOT CHANGE THE ORDER OF THESE ENTRIES, NOR DELETE ANY -- add new ones at the end. */ typedef enum { VG_USERREQ__PMC_REGISTER_PMEM_MAPPING = VG_USERREQ_TOOL_BASE('P','C'), VG_USERREQ__PMC_REGISTER_PMEM_FILE, VG_USERREQ__PMC_REMOVE_PMEM_MAPPING, VG_USERREQ__PMC_CHECK_IS_PMEM_MAPPING, VG_USERREQ__PMC_PRINT_PMEM_MAPPINGS, VG_USERREQ__PMC_DO_FLUSH, VG_USERREQ__PMC_DO_FENCE, VG_USERREQ__PMC_RESERVED1, /* Do not use. */ VG_USERREQ__PMC_WRITE_STATS, VG_USERREQ__PMC_RESERVED2, /* Do not use. */ VG_USERREQ__PMC_RESERVED3, /* Do not use. */ VG_USERREQ__PMC_RESERVED4, /* Do not use. */ VG_USERREQ__PMC_RESERVED5, /* Do not use. */ VG_USERREQ__PMC_RESERVED7, /* Do not use. */ VG_USERREQ__PMC_RESERVED8, /* Do not use. */ VG_USERREQ__PMC_RESERVED9, /* Do not use. */ VG_USERREQ__PMC_RESERVED10, /* Do not use. */ VG_USERREQ__PMC_SET_CLEAN, /* transaction support */ VG_USERREQ__PMC_START_TX, VG_USERREQ__PMC_START_TX_N, VG_USERREQ__PMC_END_TX, VG_USERREQ__PMC_END_TX_N, VG_USERREQ__PMC_ADD_TO_TX, VG_USERREQ__PMC_ADD_TO_TX_N, VG_USERREQ__PMC_REMOVE_FROM_TX, VG_USERREQ__PMC_REMOVE_FROM_TX_N, VG_USERREQ__PMC_ADD_THREAD_TO_TX_N, VG_USERREQ__PMC_REMOVE_THREAD_FROM_TX_N, VG_USERREQ__PMC_ADD_TO_GLOBAL_TX_IGNORE, VG_USERREQ__PMC_RESERVED6, /* Do not use. */ VG_USERREQ__PMC_EMIT_LOG, } Vg_PMemCheckClientRequest; /* Client-code macros to manipulate pmem mappings */ /** Register a persistent memory mapping region */ #define VALGRIND_PMC_REGISTER_PMEM_MAPPING(_qzz_addr, _qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_REGISTER_PMEM_MAPPING, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /** Register a persistent memory file */ #define VALGRIND_PMC_REGISTER_PMEM_FILE(_qzz_desc, _qzz_addr_base, \ _qzz_size, _qzz_offset) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_REGISTER_PMEM_FILE, \ (_qzz_desc), (_qzz_addr_base), (_qzz_size), \ (_qzz_offset), 0) /** Remove a persistent memory mapping region */ #define VALGRIND_PMC_REMOVE_PMEM_MAPPING(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_REMOVE_PMEM_MAPPING, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /** Check if the given range is a registered persistent memory mapping */ #define VALGRIND_PMC_CHECK_IS_PMEM_MAPPING(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_CHECK_IS_PMEM_MAPPING, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /** Register an SFENCE */ #define VALGRIND_PMC_PRINT_PMEM_MAPPINGS \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__PMC_PRINT_PMEM_MAPPINGS, \ 0, 0, 0, 0, 0) /** Register a CLFLUSH-like operation */ #define VALGRIND_PMC_DO_FLUSH(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_DO_FLUSH, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /** Register an SFENCE */ #define VALGRIND_PMC_DO_FENCE \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__PMC_DO_FENCE, \ 0, 0, 0, 0, 0) /** Write tool stats */ #define VALGRIND_PMC_WRITE_STATS \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__PMC_WRITE_STATS, \ 0, 0, 0, 0, 0) /** Emit user log */ #define VALGRIND_PMC_EMIT_LOG(_qzz_emit_log) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_EMIT_LOG, \ (_qzz_emit_log), 0, 0, 0, 0) /** Set a region of persistent memory as clean */ #define VALGRIND_PMC_SET_CLEAN(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_SET_CLEAN, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /** Support for transactions */ /** Start an implicit persistent memory transaction */ #define VALGRIND_PMC_START_TX \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__PMC_START_TX, \ 0, 0, 0, 0, 0) /** Start an explicit persistent memory transaction */ #define VALGRIND_PMC_START_TX_N(_qzz_txn) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_START_TX_N, \ (_qzz_txn), 0, 0, 0, 0) /** End an implicit persistent memory transaction */ #define VALGRIND_PMC_END_TX \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__PMC_END_TX, \ 0, 0, 0, 0, 0) /** End an explicit persistent memory transaction */ #define VALGRIND_PMC_END_TX_N(_qzz_txn) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_END_TX_N, \ (_qzz_txn), 0, 0, 0, 0) /** Add a persistent memory region to the implicit transaction */ #define VALGRIND_PMC_ADD_TO_TX(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_ADD_TO_TX, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /** Add a persistent memory region to an explicit transaction */ #define VALGRIND_PMC_ADD_TO_TX_N(_qzz_txn,_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_ADD_TO_TX_N, \ (_qzz_txn), (_qzz_addr), (_qzz_len), 0, 0) /** Remove a persistent memory region from the implicit transaction */ #define VALGRIND_PMC_REMOVE_FROM_TX(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_REMOVE_FROM_TX, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /** Remove a persistent memory region from an explicit transaction */ #define VALGRIND_PMC_REMOVE_FROM_TX_N(_qzz_txn,_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_REMOVE_FROM_TX_N, \ (_qzz_txn), (_qzz_addr), (_qzz_len), 0, 0) /** End an explicit persistent memory transaction */ #define VALGRIND_PMC_ADD_THREAD_TX_N(_qzz_txn) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_ADD_THREAD_TO_TX_N, \ (_qzz_txn), 0, 0, 0, 0) /** End an explicit persistent memory transaction */ #define VALGRIND_PMC_REMOVE_THREAD_FROM_TX_N(_qzz_txn) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__PMC_REMOVE_THREAD_FROM_TX_N, \ (_qzz_txn), 0, 0, 0, 0) /** Remove a persistent memory region from the implicit transaction */ #define VALGRIND_PMC_ADD_TO_GLOBAL_TX_IGNORE(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__PMC_ADD_TO_GLOBAL_TX_IGNORE,\ (_qzz_addr), (_qzz_len), 0, 0, 0) #endif pmdk-1.8/src/common/valgrind/memcheck.h0000664000000000000000000003640513615011243016633 0ustar rootroot /* ---------------------------------------------------------------- Notice that the following BSD-style license applies to this one file (memcheck.h) only. The rest of Valgrind is licensed under the terms of the GNU General Public License, version 2, unless otherwise indicated. See the COPYING file in the source distribution for details. ---------------------------------------------------------------- This file is part of MemCheck, a heavyweight Valgrind tool for detecting memory errors. Copyright (C) 2000-2017 Julian Seward. All rights reserved. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 3. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 4. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ---------------------------------------------------------------- Notice that the above BSD-style license applies to this one file (memcheck.h) only. The entire rest of Valgrind is licensed under the terms of the GNU General Public License, version 2. See the COPYING file in the source distribution for details. ---------------------------------------------------------------- */ #ifndef __MEMCHECK_H #define __MEMCHECK_H /* This file is for inclusion into client (your!) code. You can use these macros to manipulate and query memory permissions inside your own programs. See comment near the top of valgrind.h on how to use them. */ #include "valgrind.h" /* !! ABIWARNING !! ABIWARNING !! ABIWARNING !! ABIWARNING !! This enum comprises an ABI exported by Valgrind to programs which use client requests. DO NOT CHANGE THE ORDER OF THESE ENTRIES, NOR DELETE ANY -- add new ones at the end. */ typedef enum { VG_USERREQ__MAKE_MEM_NOACCESS = VG_USERREQ_TOOL_BASE('M','C'), VG_USERREQ__MAKE_MEM_UNDEFINED, VG_USERREQ__MAKE_MEM_DEFINED, VG_USERREQ__DISCARD, VG_USERREQ__CHECK_MEM_IS_ADDRESSABLE, VG_USERREQ__CHECK_MEM_IS_DEFINED, VG_USERREQ__DO_LEAK_CHECK, VG_USERREQ__COUNT_LEAKS, VG_USERREQ__GET_VBITS, VG_USERREQ__SET_VBITS, VG_USERREQ__CREATE_BLOCK, VG_USERREQ__MAKE_MEM_DEFINED_IF_ADDRESSABLE, /* Not next to VG_USERREQ__COUNT_LEAKS because it was added later. */ VG_USERREQ__COUNT_LEAK_BLOCKS, VG_USERREQ__ENABLE_ADDR_ERROR_REPORTING_IN_RANGE, VG_USERREQ__DISABLE_ADDR_ERROR_REPORTING_IN_RANGE, VG_USERREQ__CHECK_MEM_IS_UNADDRESSABLE, VG_USERREQ__CHECK_MEM_IS_UNDEFINED, /* This is just for memcheck's internal use - don't use it */ _VG_USERREQ__MEMCHECK_RECORD_OVERLAP_ERROR = VG_USERREQ_TOOL_BASE('M','C') + 256 } Vg_MemCheckClientRequest; /* Client-code macros to manipulate the state of memory. */ /* Mark memory at _qzz_addr as unaddressable for _qzz_len bytes. */ #define VALGRIND_MAKE_MEM_NOACCESS(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__MAKE_MEM_NOACCESS, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /* Similarly, mark memory at _qzz_addr as addressable but undefined for _qzz_len bytes. */ #define VALGRIND_MAKE_MEM_UNDEFINED(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__MAKE_MEM_UNDEFINED, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /* Similarly, mark memory at _qzz_addr as addressable and defined for _qzz_len bytes. */ #define VALGRIND_MAKE_MEM_DEFINED(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__MAKE_MEM_DEFINED, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /* Similar to VALGRIND_MAKE_MEM_DEFINED except that addressability is not altered: bytes which are addressable are marked as defined, but those which are not addressable are left unchanged. */ #define VALGRIND_MAKE_MEM_DEFINED_IF_ADDRESSABLE(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__MAKE_MEM_DEFINED_IF_ADDRESSABLE, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /* Create a block-description handle. The description is an ascii string which is included in any messages pertaining to addresses within the specified memory range. Has no other effect on the properties of the memory range. */ #define VALGRIND_CREATE_BLOCK(_qzz_addr,_qzz_len, _qzz_desc) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__CREATE_BLOCK, \ (_qzz_addr), (_qzz_len), (_qzz_desc), \ 0, 0) /* Discard a block-description-handle. Returns 1 for an invalid handle, 0 for a valid handle. */ #define VALGRIND_DISCARD(_qzz_blkindex) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__DISCARD, \ 0, (_qzz_blkindex), 0, 0, 0) /* Client-code macros to check the state of memory. */ /* Check that memory at _qzz_addr is addressable for _qzz_len bytes. If suitable addressability is not established, Valgrind prints an error message and returns the address of the first offending byte. Otherwise it returns zero. */ #define VALGRIND_CHECK_MEM_IS_ADDRESSABLE(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__CHECK_MEM_IS_ADDRESSABLE, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /* Check that memory at _qzz_addr is addressable and defined for _qzz_len bytes. If suitable addressability and definedness are not established, Valgrind prints an error message and returns the address of the first offending byte. Otherwise it returns zero. */ #define VALGRIND_CHECK_MEM_IS_DEFINED(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__CHECK_MEM_IS_DEFINED, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /* Use this macro to force the definedness and addressability of an lvalue to be checked. If suitable addressability and definedness are not established, Valgrind prints an error message and returns the address of the first offending byte. Otherwise it returns zero. */ #define VALGRIND_CHECK_VALUE_IS_DEFINED(__lvalue) \ VALGRIND_CHECK_MEM_IS_DEFINED( \ (volatile unsigned char *)&(__lvalue), \ (unsigned long)(sizeof (__lvalue))) /* Check that memory at _qzz_addr is unaddressable for _qzz_len bytes. If any byte in this range is addressable, Valgrind returns the address of the first offending byte. Otherwise it returns zero. */ #define VALGRIND_CHECK_MEM_IS_UNADDRESSABLE(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__CHECK_MEM_IS_UNADDRESSABLE,\ (_qzz_addr), (_qzz_len), 0, 0, 0) /* Check that memory at _qzz_addr is undefined for _qzz_len bytes. If any byte in this range is defined or unaddressable, Valgrind returns the address of the first offending byte. Otherwise it returns zero. */ #define VALGRIND_CHECK_MEM_IS_UNDEFINED(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__CHECK_MEM_IS_UNDEFINED, \ (_qzz_addr), (_qzz_len), 0, 0, 0) /* Do a full memory leak check (like --leak-check=full) mid-execution. */ #define VALGRIND_DO_LEAK_CHECK \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DO_LEAK_CHECK, \ 0, 0, 0, 0, 0) /* Same as VALGRIND_DO_LEAK_CHECK but only showing the entries for which there was an increase in leaked bytes or leaked nr of blocks since the previous leak search. */ #define VALGRIND_DO_ADDED_LEAK_CHECK \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DO_LEAK_CHECK, \ 0, 1, 0, 0, 0) /* Same as VALGRIND_DO_ADDED_LEAK_CHECK but showing entries with increased or decreased leaked bytes/blocks since previous leak search. */ #define VALGRIND_DO_CHANGED_LEAK_CHECK \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DO_LEAK_CHECK, \ 0, 2, 0, 0, 0) /* Do a summary memory leak check (like --leak-check=summary) mid-execution. */ #define VALGRIND_DO_QUICK_LEAK_CHECK \ VALGRIND_DO_CLIENT_REQUEST_STMT(VG_USERREQ__DO_LEAK_CHECK, \ 1, 0, 0, 0, 0) /* Return number of leaked, dubious, reachable and suppressed bytes found by all previous leak checks. They must be lvalues. */ #define VALGRIND_COUNT_LEAKS(leaked, dubious, reachable, suppressed) \ /* For safety on 64-bit platforms we assign the results to private unsigned long variables, then assign these to the lvalues the user specified, which works no matter what type 'leaked', 'dubious', etc are. We also initialise '_qzz_leaked', etc because VG_USERREQ__COUNT_LEAKS doesn't mark the values returned as defined. */ \ { \ unsigned long _qzz_leaked = 0, _qzz_dubious = 0; \ unsigned long _qzz_reachable = 0, _qzz_suppressed = 0; \ VALGRIND_DO_CLIENT_REQUEST_STMT( \ VG_USERREQ__COUNT_LEAKS, \ &_qzz_leaked, &_qzz_dubious, \ &_qzz_reachable, &_qzz_suppressed, 0); \ leaked = _qzz_leaked; \ dubious = _qzz_dubious; \ reachable = _qzz_reachable; \ suppressed = _qzz_suppressed; \ } /* Return number of leaked, dubious, reachable and suppressed bytes found by all previous leak checks. They must be lvalues. */ #define VALGRIND_COUNT_LEAK_BLOCKS(leaked, dubious, reachable, suppressed) \ /* For safety on 64-bit platforms we assign the results to private unsigned long variables, then assign these to the lvalues the user specified, which works no matter what type 'leaked', 'dubious', etc are. We also initialise '_qzz_leaked', etc because VG_USERREQ__COUNT_LEAKS doesn't mark the values returned as defined. */ \ { \ unsigned long _qzz_leaked = 0, _qzz_dubious = 0; \ unsigned long _qzz_reachable = 0, _qzz_suppressed = 0; \ VALGRIND_DO_CLIENT_REQUEST_STMT( \ VG_USERREQ__COUNT_LEAK_BLOCKS, \ &_qzz_leaked, &_qzz_dubious, \ &_qzz_reachable, &_qzz_suppressed, 0); \ leaked = _qzz_leaked; \ dubious = _qzz_dubious; \ reachable = _qzz_reachable; \ suppressed = _qzz_suppressed; \ } /* Get the validity data for addresses [zza..zza+zznbytes-1] and copy it into the provided zzvbits array. Return values: 0 if not running on valgrind 1 success 2 [previously indicated unaligned arrays; these are now allowed] 3 if any parts of zzsrc/zzvbits are not addressable. The metadata is not copied in cases 0, 2 or 3 so it should be impossible to segfault your system by using this call. */ #define VALGRIND_GET_VBITS(zza,zzvbits,zznbytes) \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__GET_VBITS, \ (const char*)(zza), \ (char*)(zzvbits), \ (zznbytes), 0, 0) /* Set the validity data for addresses [zza..zza+zznbytes-1], copying it from the provided zzvbits array. Return values: 0 if not running on valgrind 1 success 2 [previously indicated unaligned arrays; these are now allowed] 3 if any parts of zza/zzvbits are not addressable. The metadata is not copied in cases 0, 2 or 3 so it should be impossible to segfault your system by using this call. */ #define VALGRIND_SET_VBITS(zza,zzvbits,zznbytes) \ (unsigned)VALGRIND_DO_CLIENT_REQUEST_EXPR(0, \ VG_USERREQ__SET_VBITS, \ (const char*)(zza), \ (const char*)(zzvbits), \ (zznbytes), 0, 0 ) /* Disable and re-enable reporting of addressing errors in the specified address range. */ #define VALGRIND_DISABLE_ADDR_ERROR_REPORTING_IN_RANGE(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__DISABLE_ADDR_ERROR_REPORTING_IN_RANGE, \ (_qzz_addr), (_qzz_len), 0, 0, 0) #define VALGRIND_ENABLE_ADDR_ERROR_REPORTING_IN_RANGE(_qzz_addr,_qzz_len) \ VALGRIND_DO_CLIENT_REQUEST_EXPR(0 /* default return */, \ VG_USERREQ__ENABLE_ADDR_ERROR_REPORTING_IN_RANGE, \ (_qzz_addr), (_qzz_len), 0, 0, 0) #endif pmdk-1.8/src/common/queue.h0000664000000000000000000005322513615011243014374 0ustar rootroot/* * Source: glibc 2.24 (git://sourceware.org/glibc.git /misc/sys/queue.h) * * Copyright (c) 1991, 1993 * The Regents of the University of California. All rights reserved. * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * 1. Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * 2. Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in the * documentation and/or other materials provided with the distribution. * 3. Neither the name of the University nor the names of its contributors * may be used to endorse or promote products derived from this software * without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. * * @(#)queue.h 8.5 (Berkeley) 8/20/94 */ #ifndef _PMDK_QUEUE_H_ #define _PMDK_QUEUE_H_ /* * This file defines five types of data structures: singly-linked lists, * lists, simple queues, tail queues, and circular queues. * * A singly-linked list is headed by a single forward pointer. The * elements are singly linked for minimum space and pointer manipulation * overhead at the expense of O(n) removal for arbitrary elements. New * elements can be added to the list after an existing element or at the * head of the list. Elements being removed from the head of the list * should use the explicit macro for this purpose for optimum * efficiency. A singly-linked list may only be traversed in the forward * direction. Singly-linked lists are ideal for applications with large * datasets and few or no removals or for implementing a LIFO queue. * * A list is headed by a single forward pointer (or an array of forward * pointers for a hash table header). The elements are doubly linked * so that an arbitrary element can be removed without a need to * traverse the list. New elements can be added to the list before * or after an existing element or at the head of the list. A list * may only be traversed in the forward direction. * * A simple queue is headed by a pair of pointers, one the head of the * list and the other to the tail of the list. The elements are singly * linked to save space, so elements can only be removed from the * head of the list. New elements can be added to the list after * an existing element, at the head of the list, or at the end of the * list. A simple queue may only be traversed in the forward direction. * * A tail queue is headed by a pair of pointers, one to the head of the * list and the other to the tail of the list. The elements are doubly * linked so that an arbitrary element can be removed without a need to * traverse the list. New elements can be added to the list before or * after an existing element, at the head of the list, or at the end of * the list. A tail queue may be traversed in either direction. * * A circle queue is headed by a pair of pointers, one to the head of the * list and the other to the tail of the list. The elements are doubly * linked so that an arbitrary element can be removed without a need to * traverse the list. New elements can be added to the list before or after * an existing element, at the head of the list, or at the end of the list. * A circle queue may be traversed in either direction, but has a more * complex end of list detection. * * For details on the use of these macros, see the queue(3) manual page. */ /* * XXX This is a workaround for a bug in the llvm's static analyzer. For more * info see https://github.com/pmem/issues/issues/309. */ #ifdef __clang_analyzer__ static void custom_assert(void) { abort(); } #define ANALYZER_ASSERT(x) (__builtin_expect(!(x), 0) ? (void)0 : custom_assert()) #else #define ANALYZER_ASSERT(x) do {} while (0) #endif /* * List definitions. */ #define PMDK_LIST_HEAD(name, type) \ struct name { \ struct type *lh_first; /* first element */ \ } #define PMDK_LIST_HEAD_INITIALIZER(head) \ { NULL } #ifdef __cplusplus #define PMDK__CAST_AND_ASSIGN(x, y) x = (__typeof__(x))y; #else #define PMDK__CAST_AND_ASSIGN(x, y) x = (void *)(y); #endif #define PMDK_LIST_ENTRY(type) \ struct { \ struct type *le_next; /* next element */ \ struct type **le_prev; /* address of previous next element */ \ } /* * List functions. */ #define PMDK_LIST_INIT(head) do { \ (head)->lh_first = NULL; \ } while (/*CONSTCOND*/0) #define PMDK_LIST_INSERT_AFTER(listelm, elm, field) do { \ if (((elm)->field.le_next = (listelm)->field.le_next) != NULL) \ (listelm)->field.le_next->field.le_prev = \ &(elm)->field.le_next; \ (listelm)->field.le_next = (elm); \ (elm)->field.le_prev = &(listelm)->field.le_next; \ } while (/*CONSTCOND*/0) #define PMDK_LIST_INSERT_BEFORE(listelm, elm, field) do { \ (elm)->field.le_prev = (listelm)->field.le_prev; \ (elm)->field.le_next = (listelm); \ *(listelm)->field.le_prev = (elm); \ (listelm)->field.le_prev = &(elm)->field.le_next; \ } while (/*CONSTCOND*/0) #define PMDK_LIST_INSERT_HEAD(head, elm, field) do { \ if (((elm)->field.le_next = (head)->lh_first) != NULL) \ (head)->lh_first->field.le_prev = &(elm)->field.le_next;\ (head)->lh_first = (elm); \ (elm)->field.le_prev = &(head)->lh_first; \ } while (/*CONSTCOND*/0) #define PMDK_LIST_REMOVE(elm, field) do { \ ANALYZER_ASSERT((elm) != NULL); \ if ((elm)->field.le_next != NULL) \ (elm)->field.le_next->field.le_prev = \ (elm)->field.le_prev; \ *(elm)->field.le_prev = (elm)->field.le_next; \ } while (/*CONSTCOND*/0) #define PMDK_LIST_FOREACH(var, head, field) \ for ((var) = ((head)->lh_first); \ (var); \ (var) = ((var)->field.le_next)) /* * List access methods. */ #define PMDK_LIST_EMPTY(head) ((head)->lh_first == NULL) #define PMDK_LIST_FIRST(head) ((head)->lh_first) #define PMDK_LIST_NEXT(elm, field) ((elm)->field.le_next) /* * Singly-linked List definitions. */ #define PMDK_SLIST_HEAD(name, type) \ struct name { \ struct type *slh_first; /* first element */ \ } #define PMDK_SLIST_HEAD_INITIALIZER(head) \ { NULL } #define PMDK_SLIST_ENTRY(type) \ struct { \ struct type *sle_next; /* next element */ \ } /* * Singly-linked List functions. */ #define PMDK_SLIST_INIT(head) do { \ (head)->slh_first = NULL; \ } while (/*CONSTCOND*/0) #define PMDK_SLIST_INSERT_AFTER(slistelm, elm, field) do { \ (elm)->field.sle_next = (slistelm)->field.sle_next; \ (slistelm)->field.sle_next = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_SLIST_INSERT_HEAD(head, elm, field) do { \ (elm)->field.sle_next = (head)->slh_first; \ (head)->slh_first = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_SLIST_REMOVE_HEAD(head, field) do { \ (head)->slh_first = (head)->slh_first->field.sle_next; \ } while (/*CONSTCOND*/0) #define PMDK_SLIST_REMOVE(head, elm, type, field) do { \ if ((head)->slh_first == (elm)) { \ PMDK_SLIST_REMOVE_HEAD((head), field); \ } \ else { \ struct type *curelm = (head)->slh_first; \ while(curelm->field.sle_next != (elm)) \ curelm = curelm->field.sle_next; \ curelm->field.sle_next = \ curelm->field.sle_next->field.sle_next; \ } \ } while (/*CONSTCOND*/0) #define PMDK_SLIST_FOREACH(var, head, field) \ for((var) = (head)->slh_first; (var); (var) = (var)->field.sle_next) /* * Singly-linked List access methods. */ #define PMDK_SLIST_EMPTY(head) ((head)->slh_first == NULL) #define PMDK_SLIST_FIRST(head) ((head)->slh_first) #define PMDK_SLIST_NEXT(elm, field) ((elm)->field.sle_next) /* * Singly-linked Tail queue declarations. */ #define PMDK_STAILQ_HEAD(name, type) \ struct name { \ struct type *stqh_first; /* first element */ \ struct type **stqh_last; /* addr of last next element */ \ } #define PMDK_STAILQ_HEAD_INITIALIZER(head) \ { NULL, &(head).stqh_first } #define PMDK_STAILQ_ENTRY(type) \ struct { \ struct type *stqe_next; /* next element */ \ } /* * Singly-linked Tail queue functions. */ #define PMDK_STAILQ_INIT(head) do { \ (head)->stqh_first = NULL; \ (head)->stqh_last = &(head)->stqh_first; \ } while (/*CONSTCOND*/0) #define PMDK_STAILQ_INSERT_HEAD(head, elm, field) do { \ if (((elm)->field.stqe_next = (head)->stqh_first) == NULL) \ (head)->stqh_last = &(elm)->field.stqe_next; \ (head)->stqh_first = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_STAILQ_INSERT_TAIL(head, elm, field) do { \ (elm)->field.stqe_next = NULL; \ *(head)->stqh_last = (elm); \ (head)->stqh_last = &(elm)->field.stqe_next; \ } while (/*CONSTCOND*/0) #define PMDK_STAILQ_INSERT_AFTER(head, listelm, elm, field) do { \ if (((elm)->field.stqe_next = (listelm)->field.stqe_next) == NULL)\ (head)->stqh_last = &(elm)->field.stqe_next; \ (listelm)->field.stqe_next = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_STAILQ_REMOVE_HEAD(head, field) do { \ if (((head)->stqh_first = (head)->stqh_first->field.stqe_next) == NULL) \ (head)->stqh_last = &(head)->stqh_first; \ } while (/*CONSTCOND*/0) #define PMDK_STAILQ_REMOVE(head, elm, type, field) do { \ if ((head)->stqh_first == (elm)) { \ PMDK_STAILQ_REMOVE_HEAD((head), field); \ } else { \ struct type *curelm = (head)->stqh_first; \ while (curelm->field.stqe_next != (elm)) \ curelm = curelm->field.stqe_next; \ if ((curelm->field.stqe_next = \ curelm->field.stqe_next->field.stqe_next) == NULL) \ (head)->stqh_last = &(curelm)->field.stqe_next; \ } \ } while (/*CONSTCOND*/0) #define PMDK_STAILQ_FOREACH(var, head, field) \ for ((var) = ((head)->stqh_first); \ (var); \ (var) = ((var)->field.stqe_next)) #define PMDK_STAILQ_CONCAT(head1, head2) do { \ if (!PMDK_STAILQ_EMPTY((head2))) { \ *(head1)->stqh_last = (head2)->stqh_first; \ (head1)->stqh_last = (head2)->stqh_last; \ PMDK_STAILQ_INIT((head2)); \ } \ } while (/*CONSTCOND*/0) /* * Singly-linked Tail queue access methods. */ #define PMDK_STAILQ_EMPTY(head) ((head)->stqh_first == NULL) #define PMDK_STAILQ_FIRST(head) ((head)->stqh_first) #define PMDK_STAILQ_NEXT(elm, field) ((elm)->field.stqe_next) /* * Simple queue definitions. */ #define PMDK_SIMPLEQ_HEAD(name, type) \ struct name { \ struct type *sqh_first; /* first element */ \ struct type **sqh_last; /* addr of last next element */ \ } #define PMDK_SIMPLEQ_HEAD_INITIALIZER(head) \ { NULL, &(head).sqh_first } #define PMDK_SIMPLEQ_ENTRY(type) \ struct { \ struct type *sqe_next; /* next element */ \ } /* * Simple queue functions. */ #define PMDK_SIMPLEQ_INIT(head) do { \ (head)->sqh_first = NULL; \ (head)->sqh_last = &(head)->sqh_first; \ } while (/*CONSTCOND*/0) #define PMDK_SIMPLEQ_INSERT_HEAD(head, elm, field) do { \ if (((elm)->field.sqe_next = (head)->sqh_first) == NULL) \ (head)->sqh_last = &(elm)->field.sqe_next; \ (head)->sqh_first = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_SIMPLEQ_INSERT_TAIL(head, elm, field) do { \ (elm)->field.sqe_next = NULL; \ *(head)->sqh_last = (elm); \ (head)->sqh_last = &(elm)->field.sqe_next; \ } while (/*CONSTCOND*/0) #define PMDK_SIMPLEQ_INSERT_AFTER(head, listelm, elm, field) do { \ if (((elm)->field.sqe_next = (listelm)->field.sqe_next) == NULL)\ (head)->sqh_last = &(elm)->field.sqe_next; \ (listelm)->field.sqe_next = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_SIMPLEQ_REMOVE_HEAD(head, field) do { \ if (((head)->sqh_first = (head)->sqh_first->field.sqe_next) == NULL) \ (head)->sqh_last = &(head)->sqh_first; \ } while (/*CONSTCOND*/0) #define PMDK_SIMPLEQ_REMOVE(head, elm, type, field) do { \ if ((head)->sqh_first == (elm)) { \ PMDK_SIMPLEQ_REMOVE_HEAD((head), field); \ } else { \ struct type *curelm = (head)->sqh_first; \ while (curelm->field.sqe_next != (elm)) \ curelm = curelm->field.sqe_next; \ if ((curelm->field.sqe_next = \ curelm->field.sqe_next->field.sqe_next) == NULL) \ (head)->sqh_last = &(curelm)->field.sqe_next; \ } \ } while (/*CONSTCOND*/0) #define PMDK_SIMPLEQ_FOREACH(var, head, field) \ for ((var) = ((head)->sqh_first); \ (var); \ (var) = ((var)->field.sqe_next)) /* * Simple queue access methods. */ #define PMDK_SIMPLEQ_EMPTY(head) ((head)->sqh_first == NULL) #define PMDK_SIMPLEQ_FIRST(head) ((head)->sqh_first) #define PMDK_SIMPLEQ_NEXT(elm, field) ((elm)->field.sqe_next) /* * Tail queue definitions. */ #define PMDK__TAILQ_HEAD(name, type, qual) \ struct name { \ qual type *tqh_first; /* first element */ \ qual type *qual *tqh_last; /* addr of last next element */ \ } #define PMDK_TAILQ_HEAD(name, type) PMDK__TAILQ_HEAD(name, struct type,) #define PMDK_TAILQ_HEAD_INITIALIZER(head) \ { NULL, &(head).tqh_first } #define PMDK__TAILQ_ENTRY(type, qual) \ struct { \ qual type *tqe_next; /* next element */ \ qual type *qual *tqe_prev; /* address of previous next element */\ } #define PMDK_TAILQ_ENTRY(type) PMDK__TAILQ_ENTRY(struct type,) /* * Tail queue functions. */ #define PMDK_TAILQ_INIT(head) do { \ (head)->tqh_first = NULL; \ (head)->tqh_last = &(head)->tqh_first; \ } while (/*CONSTCOND*/0) #define PMDK_TAILQ_INSERT_HEAD(head, elm, field) do { \ if (((elm)->field.tqe_next = (head)->tqh_first) != NULL) \ (head)->tqh_first->field.tqe_prev = \ &(elm)->field.tqe_next; \ else \ (head)->tqh_last = &(elm)->field.tqe_next; \ (head)->tqh_first = (elm); \ (elm)->field.tqe_prev = &(head)->tqh_first; \ } while (/*CONSTCOND*/0) #define PMDK_TAILQ_INSERT_TAIL(head, elm, field) do { \ (elm)->field.tqe_next = NULL; \ (elm)->field.tqe_prev = (head)->tqh_last; \ *(head)->tqh_last = (elm); \ (head)->tqh_last = &(elm)->field.tqe_next; \ } while (/*CONSTCOND*/0) #define PMDK_TAILQ_INSERT_AFTER(head, listelm, elm, field) do { \ if (((elm)->field.tqe_next = (listelm)->field.tqe_next) != NULL)\ (elm)->field.tqe_next->field.tqe_prev = \ &(elm)->field.tqe_next; \ else \ (head)->tqh_last = &(elm)->field.tqe_next; \ (listelm)->field.tqe_next = (elm); \ (elm)->field.tqe_prev = &(listelm)->field.tqe_next; \ } while (/*CONSTCOND*/0) #define PMDK_TAILQ_INSERT_BEFORE(listelm, elm, field) do { \ (elm)->field.tqe_prev = (listelm)->field.tqe_prev; \ (elm)->field.tqe_next = (listelm); \ *(listelm)->field.tqe_prev = (elm); \ (listelm)->field.tqe_prev = &(elm)->field.tqe_next; \ } while (/*CONSTCOND*/0) #define PMDK_TAILQ_REMOVE(head, elm, field) do { \ ANALYZER_ASSERT((elm) != NULL); \ if (((elm)->field.tqe_next) != NULL) \ (elm)->field.tqe_next->field.tqe_prev = \ (elm)->field.tqe_prev; \ else \ (head)->tqh_last = (elm)->field.tqe_prev; \ *(elm)->field.tqe_prev = (elm)->field.tqe_next; \ } while (/*CONSTCOND*/0) #define PMDK_TAILQ_FOREACH(var, head, field) \ for ((var) = ((head)->tqh_first); \ (var); \ (var) = ((var)->field.tqe_next)) #define PMDK_TAILQ_FOREACH_REVERSE(var, head, headname, field) \ for ((var) = (*(((struct headname *)((head)->tqh_last))->tqh_last)); \ (var); \ (var) = (*(((struct headname *)((var)->field.tqe_prev))->tqh_last))) #define PMDK_TAILQ_CONCAT(head1, head2, field) do { \ if (!PMDK_TAILQ_EMPTY(head2)) { \ *(head1)->tqh_last = (head2)->tqh_first; \ (head2)->tqh_first->field.tqe_prev = (head1)->tqh_last; \ (head1)->tqh_last = (head2)->tqh_last; \ PMDK_TAILQ_INIT((head2)); \ } \ } while (/*CONSTCOND*/0) /* * Tail queue access methods. */ #define PMDK_TAILQ_EMPTY(head) ((head)->tqh_first == NULL) #define PMDK_TAILQ_FIRST(head) ((head)->tqh_first) #define PMDK_TAILQ_NEXT(elm, field) ((elm)->field.tqe_next) #define PMDK_TAILQ_LAST(head, headname) \ (*(((struct headname *)((head)->tqh_last))->tqh_last)) #define PMDK_TAILQ_PREV(elm, headname, field) \ (*(((struct headname *)((elm)->field.tqe_prev))->tqh_last)) /* * Circular queue definitions. */ #define PMDK_CIRCLEQ_HEAD(name, type) \ struct name { \ struct type *cqh_first; /* first element */ \ struct type *cqh_last; /* last element */ \ } #define PMDK_CIRCLEQ_HEAD_INITIALIZER(head) \ { (void *)&(head), (void *)&(head) } #define PMDK_CIRCLEQ_ENTRY(type) \ struct { \ struct type *cqe_next; /* next element */ \ struct type *cqe_prev; /* previous element */ \ } /* * Circular queue functions. */ #define PMDK_CIRCLEQ_INIT(head) do { \ PMDK__CAST_AND_ASSIGN((head)->cqh_first, (head)); \ PMDK__CAST_AND_ASSIGN((head)->cqh_last, (head)); \ } while (/*CONSTCOND*/0) #define PMDK_CIRCLEQ_INSERT_AFTER(head, listelm, elm, field) do { \ (elm)->field.cqe_next = (listelm)->field.cqe_next; \ (elm)->field.cqe_prev = (listelm); \ if ((listelm)->field.cqe_next == (void *)(head)) \ (head)->cqh_last = (elm); \ else \ (listelm)->field.cqe_next->field.cqe_prev = (elm); \ (listelm)->field.cqe_next = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_CIRCLEQ_INSERT_BEFORE(head, listelm, elm, field) do { \ (elm)->field.cqe_next = (listelm); \ (elm)->field.cqe_prev = (listelm)->field.cqe_prev; \ if ((listelm)->field.cqe_prev == (void *)(head)) \ (head)->cqh_first = (elm); \ else \ (listelm)->field.cqe_prev->field.cqe_next = (elm); \ (listelm)->field.cqe_prev = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_CIRCLEQ_INSERT_HEAD(head, elm, field) do { \ (elm)->field.cqe_next = (head)->cqh_first; \ (elm)->field.cqe_prev = (void *)(head); \ if ((head)->cqh_last == (void *)(head)) \ (head)->cqh_last = (elm); \ else \ (head)->cqh_first->field.cqe_prev = (elm); \ (head)->cqh_first = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_CIRCLEQ_INSERT_TAIL(head, elm, field) do { \ PMDK__CAST_AND_ASSIGN((elm)->field.cqe_next, (head)); \ (elm)->field.cqe_prev = (head)->cqh_last; \ if ((head)->cqh_first == (void *)(head)) \ (head)->cqh_first = (elm); \ else \ (head)->cqh_last->field.cqe_next = (elm); \ (head)->cqh_last = (elm); \ } while (/*CONSTCOND*/0) #define PMDK_CIRCLEQ_REMOVE(head, elm, field) do { \ if ((elm)->field.cqe_next == (void *)(head)) \ (head)->cqh_last = (elm)->field.cqe_prev; \ else \ (elm)->field.cqe_next->field.cqe_prev = \ (elm)->field.cqe_prev; \ if ((elm)->field.cqe_prev == (void *)(head)) \ (head)->cqh_first = (elm)->field.cqe_next; \ else \ (elm)->field.cqe_prev->field.cqe_next = \ (elm)->field.cqe_next; \ } while (/*CONSTCOND*/0) #define PMDK_CIRCLEQ_FOREACH(var, head, field) \ for ((var) = ((head)->cqh_first); \ (var) != (const void *)(head); \ (var) = ((var)->field.cqe_next)) #define PMDK_CIRCLEQ_FOREACH_REVERSE(var, head, field) \ for ((var) = ((head)->cqh_last); \ (var) != (const void *)(head); \ (var) = ((var)->field.cqe_prev)) /* * Circular queue access methods. */ #define PMDK_CIRCLEQ_EMPTY(head) ((head)->cqh_first == (void *)(head)) #define PMDK_CIRCLEQ_FIRST(head) ((head)->cqh_first) #define PMDK_CIRCLEQ_LAST(head) ((head)->cqh_last) #define PMDK_CIRCLEQ_NEXT(elm, field) ((elm)->field.cqe_next) #define PMDK_CIRCLEQ_PREV(elm, field) ((elm)->field.cqe_prev) #define PMDK_CIRCLEQ_LOOP_NEXT(head, elm, field) \ (((elm)->field.cqe_next == (void *)(head)) \ ? ((head)->cqh_first) \ : ((elm)->field.cqe_next)) #define PMDK_CIRCLEQ_LOOP_PREV(head, elm, field) \ (((elm)->field.cqe_prev == (void *)(head)) \ ? ((head)->cqh_last) \ : ((elm)->field.cqe_prev)) /* * Sorted queue functions. */ #define PMDK_SORTEDQ_HEAD(name, type) PMDK_CIRCLEQ_HEAD(name, type) #define PMDK_SORTEDQ_HEAD_INITIALIZER(head) PMDK_CIRCLEQ_HEAD_INITIALIZER(head) #define PMDK_SORTEDQ_ENTRY(type) PMDK_CIRCLEQ_ENTRY(type) #define PMDK_SORTEDQ_INIT(head) PMDK_CIRCLEQ_INIT(head) #define PMDK_SORTEDQ_INSERT(head, elm, field, type, comparer) { \ type *_elm_it; \ for (_elm_it = (head)->cqh_first; \ ((_elm_it != (void *)(head)) && \ (comparer(_elm_it, (elm)) < 0)); \ _elm_it = _elm_it->field.cqe_next) \ /*NOTHING*/; \ if (_elm_it == (void *)(head)) \ PMDK_CIRCLEQ_INSERT_TAIL(head, elm, field); \ else \ PMDK_CIRCLEQ_INSERT_BEFORE(head, _elm_it, elm, field); \ } #define PMDK_SORTEDQ_REMOVE(head, elm, field) PMDK_CIRCLEQ_REMOVE(head, elm, field) #define PMDK_SORTEDQ_FOREACH(var, head, field) PMDK_CIRCLEQ_FOREACH(var, head, field) #define PMDK_SORTEDQ_FOREACH_REVERSE(var, head, field) \ PMDK_CIRCLEQ_FOREACH_REVERSE(var, head, field) /* * Sorted queue access methods. */ #define PMDK_SORTEDQ_EMPTY(head) PMDK_CIRCLEQ_EMPTY(head) #define PMDK_SORTEDQ_FIRST(head) PMDK_CIRCLEQ_FIRST(head) #define PMDK_SORTEDQ_LAST(head) PMDK_CIRCLEQ_LAST(head) #define PMDK_SORTEDQ_NEXT(elm, field) PMDK_CIRCLEQ_NEXT(elm, field) #define PMDK_SORTEDQ_PREV(elm, field) PMDK_CIRCLEQ_PREV(elm, field) #endif /* sys/queue.h */ pmdk-1.8/src/common/pmemcommon.inc0000664000000000000000000000460313615011243015735 0ustar rootroot# Copyright 2017-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/pmemcommon.inc -- common SOURCE definitions for PMDK libraries # SOURCE =\ $(COMMON)/alloc.c\ $(COMMON)/badblock_$(OS_DIMM).c\ $(COMMON)/badblock.c\ $(COMMON)/ctl.c\ $(COMMON)/ctl_prefault.c\ $(COMMON)/ctl_sds.c\ $(COMMON)/ctl_fallocate.c\ $(COMMON)/ctl_cow.c\ $(call osdep, $(COMMON)/extent,.c)\ $(COMMON)/file.c\ $(COMMON)/file_posix.c\ $(COMMON)/fs_posix.c\ $(COMMON)/mmap.c\ $(COMMON)/mmap_posix.c\ $(COMMON)/os_posix.c\ $(COMMON)/os_thread_posix.c\ $(COMMON)/os_dimm_$(OS_DIMM).c\ $(COMMON)/os_deep_linux.c\ $(COMMON)/out.c\ $(COMMON)/pool_hdr.c\ $(COMMON)/rand.c\ $(COMMON)/set.c\ $(COMMON)/shutdown_state.c\ $(COMMON)/util.c\ $(COMMON)/util_posix.c\ $(COMMON)/uuid.c\ $(call osdep, $(COMMON)/uuid,.c) ifeq ($(OS_KERNEL_NAME),Linux) SOURCE += $(COMMON)/os_auto_flush_linux.c else SOURCE += $(COMMON)/os_auto_flush_none.c endif pmdk-1.8/src/common/os_badblock.h0000664000000000000000000000643213615011243015510 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_badblock.h -- linux bad block API */ #ifndef PMDK_BADBLOCK_H #define PMDK_BADBLOCK_H 1 #include #include #ifdef __cplusplus extern "C" { #endif #define B2SEC(n) ((n) >> 9) /* convert bytes to sectors */ #define SEC2B(n) ((n) << 9) /* convert sectors to bytes */ #define NO_HEALTHY_REPLICA ((int)(-1)) #define BB_NOT_SUPP \ "checking bad blocks is not supported on this OS, please switch off the CHECK_BAD_BLOCKS compat feature using 'pmempool-feature'" /* * 'struct badblock' is already defined in ndctl/libndctl.h, * so we cannot use this name. * * libndctl returns offset relative to the beginning of the region, * but in this structure we save offset relative to the beginning of: * - namespace (before os_badblocks_get()) * and * - file (before sync_recalc_badblocks()) * and * - pool (after sync_recalc_badblocks()) */ struct bad_block { /* * offset in bytes relative to the beginning of * - namespace (before os_badblocks_get()) * and * - file (before sync_recalc_badblocks()) * and * - pool (after sync_recalc_badblocks()) */ unsigned long long offset; /* length in bytes */ unsigned length; /* number of healthy replica to fix this bad block */ int nhealthy; }; struct badblocks { unsigned long long ns_resource; /* address of the namespace */ unsigned bb_cnt; /* number of bad blocks */ struct bad_block *bbv; /* array of bad blocks */ }; long os_badblocks_count(const char *path); int os_badblocks_get(const char *file, struct badblocks *bbs); int os_badblocks_clear(const char *path, struct badblocks *bbs); int os_badblocks_clear_all(const char *file); int os_badblocks_check_file(const char *path); #ifdef __cplusplus } #endif #endif /* PMDK_BADBLOCK_H */ pmdk-1.8/src/common/extent.h0000664000000000000000000000412113615011243014546 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * extent.h -- fs extent query API */ #ifndef PMDK_EXTENT_H #define PMDK_EXTENT_H 1 #include #include #ifdef __cplusplus extern "C" { #endif struct extent { uint64_t offset_physical; uint64_t offset_logical; uint64_t length; }; struct extents { uint64_t blksize; uint32_t extents_count; struct extent *extents; }; long os_extents_count(const char *path, struct extents *exts); int os_extents_get(const char *path, struct extents *exts); #ifdef __cplusplus } #endif #endif /* PMDK_EXTENT_H */ pmdk-1.8/src/common/out.c0000664000000000000000000003253513615011243014053 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * out.c -- support for logging, tracing, and assertion output * * Macros like LOG(), OUT, ASSERT(), etc. end up here. */ #include #include #include #include #include #include #include #include "out.h" #include "os.h" #include "os_thread.h" #include "valgrind_internal.h" #include "util.h" /* XXX - modify Linux makefiles to generate srcversion.h and remove #ifdef */ #ifdef _WIN32 #include "srcversion.h" #endif static const char *Log_prefix; static int Log_level; static FILE *Out_fp; static unsigned Log_alignment; #ifndef NO_LIBPTHREAD #define MAXPRINT 8192 /* maximum expected log line */ #else #define MAXPRINT 256 /* maximum expected log line for libpmem */ #endif struct errormsg { char msg[MAXPRINT]; #ifdef _WIN32 wchar_t wmsg[MAXPRINT]; #endif }; #ifndef NO_LIBPTHREAD static os_once_t Last_errormsg_key_once = OS_ONCE_INIT; static os_tls_key_t Last_errormsg_key; static void _Last_errormsg_key_alloc(void) { int pth_ret = os_tls_key_create(&Last_errormsg_key, free); if (pth_ret) FATAL("!os_thread_key_create"); VALGRIND_ANNOTATE_HAPPENS_BEFORE(&Last_errormsg_key_once); } static void Last_errormsg_key_alloc(void) { os_once(&Last_errormsg_key_once, _Last_errormsg_key_alloc); /* * Workaround Helgrind's bug: * https://bugs.kde.org/show_bug.cgi?id=337735 */ VALGRIND_ANNOTATE_HAPPENS_AFTER(&Last_errormsg_key_once); } static inline void Last_errormsg_fini(void) { void *p = os_tls_get(Last_errormsg_key); if (p) { free(p); (void) os_tls_set(Last_errormsg_key, NULL); } (void) os_tls_key_delete(Last_errormsg_key); } static inline struct errormsg * Last_errormsg_get(void) { Last_errormsg_key_alloc(); struct errormsg *errormsg = os_tls_get(Last_errormsg_key); if (errormsg == NULL) { errormsg = malloc(sizeof(struct errormsg)); if (errormsg == NULL) FATAL("!malloc"); /* make sure it contains empty string initially */ errormsg->msg[0] = '\0'; int ret = os_tls_set(Last_errormsg_key, errormsg); if (ret) FATAL("!os_tls_set"); } return errormsg; } #else /* * We don't want libpmem to depend on libpthread. Instead of using pthread * API to dynamically allocate thread-specific error message buffer, we put * it into TLS. However, keeping a pretty large static buffer (8K) in TLS * may lead to some issues, so the maximum message length is reduced. * Fortunately, it looks like the longest error message in libpmem should * not be longer than about 90 chars (in case of pmem_check_version()). */ static __thread struct errormsg Last_errormsg; static inline void Last_errormsg_key_alloc(void) { } static inline void Last_errormsg_fini(void) { } static inline const struct errormsg * Last_errormsg_get(void) { return &Last_errormsg.msg[0]; } #endif /* NO_LIBPTHREAD */ /* * out_init -- initialize the log * * This is called from the library initialization code. */ void out_init(const char *log_prefix, const char *log_level_var, const char *log_file_var, int major_version, int minor_version) { static int once; /* only need to initialize the out module once */ if (once) return; once++; Log_prefix = log_prefix; #ifdef DEBUG char *log_level; char *log_file; if ((log_level = os_getenv(log_level_var)) != NULL) { Log_level = atoi(log_level); if (Log_level < 0) { Log_level = 0; } } if ((log_file = os_getenv(log_file_var)) != NULL && log_file[0] != '\0') { /* reserve more than enough space for a PID + '\0' */ char log_file_pid[PATH_MAX]; size_t len = strlen(log_file); if (len > 0 && log_file[len - 1] == '-') { int ret = snprintf(log_file_pid, PATH_MAX, "%s%d", log_file, getpid()); if (ret < 0 || ret >= PATH_MAX) { ERR("snprintf: %d", ret); abort(); } log_file = log_file_pid; } if ((Out_fp = os_fopen(log_file, "w")) == NULL) { char buff[UTIL_MAX_ERR_MSG]; util_strerror(errno, buff, UTIL_MAX_ERR_MSG); fprintf(stderr, "Error (%s): %s=%s: %s\n", log_prefix, log_file_var, log_file, buff); abort(); } } #endif /* DEBUG */ char *log_alignment = os_getenv("PMDK_LOG_ALIGN"); if (log_alignment) { int align = atoi(log_alignment); if (align > 0) Log_alignment = (unsigned)align; } if (Out_fp == NULL) Out_fp = stderr; else setlinebuf(Out_fp); #ifdef DEBUG static char namepath[PATH_MAX]; LOG(1, "pid %d: program: %s", getpid(), util_getexecname(namepath, PATH_MAX)); #endif LOG(1, "%s version %d.%d", log_prefix, major_version, minor_version); static __attribute__((used)) const char *version_msg = "src version: " SRCVERSION; LOG(1, "%s", version_msg); #if VG_PMEMCHECK_ENABLED /* * Attribute "used" to prevent compiler from optimizing out the variable * when LOG expands to no code (!DEBUG) */ static __attribute__((used)) const char *pmemcheck_msg = "compiled with support for Valgrind pmemcheck"; LOG(1, "%s", pmemcheck_msg); #endif /* VG_PMEMCHECK_ENABLED */ #if VG_HELGRIND_ENABLED static __attribute__((used)) const char *helgrind_msg = "compiled with support for Valgrind helgrind"; LOG(1, "%s", helgrind_msg); #endif /* VG_HELGRIND_ENABLED */ #if VG_MEMCHECK_ENABLED static __attribute__((used)) const char *memcheck_msg = "compiled with support for Valgrind memcheck"; LOG(1, "%s", memcheck_msg); #endif /* VG_MEMCHECK_ENABLED */ #if VG_DRD_ENABLED static __attribute__((used)) const char *drd_msg = "compiled with support for Valgrind drd"; LOG(1, "%s", drd_msg); #endif /* VG_DRD_ENABLED */ #if SDS_ENABLED static __attribute__((used)) const char *shutdown_state_msg = "compiled with support for shutdown state"; LOG(1, "%s", shutdown_state_msg); #endif #if NDCTL_ENABLED static __attribute__((used)) const char *ndctl_ge_63_msg = "compiled with libndctl 63+"; LOG(1, "%s", ndctl_ge_63_msg); #endif Last_errormsg_key_alloc(); } /* * out_fini -- close the log file * * This is called to close log file before process stop. */ void out_fini(void) { if (Out_fp != NULL && Out_fp != stderr) { fclose(Out_fp); Out_fp = stderr; } Last_errormsg_fini(); } /* * out_print_func -- default print_func, goes to stderr or Out_fp */ static void out_print_func(const char *s) { /* to suppress drd false-positive */ /* XXX: confirm real nature of this issue: pmem/issues#863 */ #ifdef SUPPRESS_FPUTS_DRD_ERROR VALGRIND_ANNOTATE_IGNORE_READS_BEGIN(); VALGRIND_ANNOTATE_IGNORE_WRITES_BEGIN(); #endif fputs(s, Out_fp); #ifdef SUPPRESS_FPUTS_DRD_ERROR VALGRIND_ANNOTATE_IGNORE_READS_END(); VALGRIND_ANNOTATE_IGNORE_WRITES_END(); #endif } /* * calling Print(s) calls the current print_func... */ typedef void (*Print_func)(const char *s); typedef int (*Vsnprintf_func)(char *str, size_t size, const char *format, va_list ap); static Print_func Print = out_print_func; static Vsnprintf_func Vsnprintf = vsnprintf; /* * out_set_print_func -- allow override of print_func used by out module */ void out_set_print_func(void (*print_func)(const char *s)) { LOG(3, "print %p", print_func); Print = (print_func == NULL) ? out_print_func : print_func; } /* * out_set_vsnprintf_func -- allow override of vsnprintf_func used by out module */ void out_set_vsnprintf_func(int (*vsnprintf_func)(char *str, size_t size, const char *format, va_list ap)) { LOG(3, "vsnprintf %p", vsnprintf_func); Vsnprintf = (vsnprintf_func == NULL) ? vsnprintf : vsnprintf_func; } /* * out_snprintf -- (internal) custom snprintf implementation */ FORMAT_PRINTF(3, 4) static int out_snprintf(char *str, size_t size, const char *format, ...) { int ret; va_list ap; va_start(ap, format); ret = Vsnprintf(str, size, format, ap); va_end(ap); return (ret); } /* * out_common -- common output code, all output goes through here */ static void out_common(const char *file, int line, const char *func, int level, const char *suffix, const char *fmt, va_list ap) { int oerrno = errno; char buf[MAXPRINT]; unsigned cc = 0; int ret; const char *sep = ""; char errstr[UTIL_MAX_ERR_MSG] = ""; if (file) { char *f = strrchr(file, OS_DIR_SEPARATOR); if (f) file = f + 1; ret = out_snprintf(&buf[cc], MAXPRINT - cc, "<%s>: <%d> [%s:%d %s] ", Log_prefix, level, file, line, func); if (ret < 0) { Print("out_snprintf failed"); goto end; } cc += (unsigned)ret; if (cc < Log_alignment) { memset(buf + cc, ' ', Log_alignment - cc); cc = Log_alignment; } } if (fmt) { if (*fmt == '!') { fmt++; sep = ": "; util_strerror(errno, errstr, UTIL_MAX_ERR_MSG); } ret = Vsnprintf(&buf[cc], MAXPRINT - cc, fmt, ap); if (ret < 0) { Print("Vsnprintf failed"); goto end; } cc += (unsigned)ret; } out_snprintf(&buf[cc], MAXPRINT - cc, "%s%s%s", sep, errstr, suffix); Print(buf); end: errno = oerrno; } /* * out_error -- common error output code, all error messages go through here */ static void out_error(const char *file, int line, const char *func, const char *suffix, const char *fmt, va_list ap) { int oerrno = errno; unsigned cc = 0; int ret; const char *sep = ""; char errstr[UTIL_MAX_ERR_MSG] = ""; char *errormsg = (char *)out_get_errormsg(); if (fmt) { if (*fmt == '!') { sep = ": "; fmt++; if (*fmt == '!') { fmt++; /* it will abort on non Windows OS */ util_strwinerror(errstr, UTIL_MAX_ERR_MSG); } else { util_strerror(errno, errstr, UTIL_MAX_ERR_MSG); } } ret = Vsnprintf(&errormsg[cc], MAXPRINT, fmt, ap); if (ret < 0) { strcpy(errormsg, "Vsnprintf failed"); goto end; } cc += (unsigned)ret; out_snprintf(&errormsg[cc], MAXPRINT - cc, "%s%s", sep, errstr); } #ifdef DEBUG if (Log_level >= 1) { char buf[MAXPRINT]; cc = 0; if (file) { char *f = strrchr(file, OS_DIR_SEPARATOR); if (f) file = f + 1; ret = out_snprintf(&buf[cc], MAXPRINT, "<%s>: <1> [%s:%d %s] ", Log_prefix, file, line, func); if (ret < 0) { Print("out_snprintf failed"); goto end; } cc += (unsigned)ret; if (cc < Log_alignment) { memset(buf + cc, ' ', Log_alignment - cc); cc = Log_alignment; } } out_snprintf(&buf[cc], MAXPRINT - cc, "%s%s", errormsg, suffix); Print(buf); } #endif end: errno = oerrno; } /* * out -- output a line, newline added automatically */ void out(const char *fmt, ...) { va_list ap; va_start(ap, fmt); out_common(NULL, 0, NULL, 0, "\n", fmt, ap); va_end(ap); } /* * out_nonl -- output a line, no newline added automatically */ void out_nonl(int level, const char *fmt, ...) { va_list ap; if (Log_level < level) return; va_start(ap, fmt); out_common(NULL, 0, NULL, level, "", fmt, ap); va_end(ap); } /* * out_log -- output a log line if Log_level >= level */ void out_log(const char *file, int line, const char *func, int level, const char *fmt, ...) { va_list ap; if (Log_level < level) return; va_start(ap, fmt); out_common(file, line, func, level, "\n", fmt, ap); va_end(ap); } /* * out_fatal -- output a fatal error & die (i.e. assertion failure) */ void out_fatal(const char *file, int line, const char *func, const char *fmt, ...) { va_list ap; va_start(ap, fmt); out_common(file, line, func, 1, "\n", fmt, ap); va_end(ap); abort(); } /* * out_err -- output an error message */ void out_err(const char *file, int line, const char *func, const char *fmt, ...) { va_list ap; va_start(ap, fmt); out_error(file, line, func, "\n", fmt, ap); va_end(ap); } /* * out_get_errormsg -- get the last error message */ const char * out_get_errormsg(void) { const struct errormsg *errormsg = Last_errormsg_get(); return &errormsg->msg[0]; } #ifdef _WIN32 /* * out_get_errormsgW -- get the last error message in wchar_t */ const wchar_t * out_get_errormsgW(void) { struct errormsg *errormsg = Last_errormsg_get(); const char *utf8 = &errormsg->msg[0]; wchar_t *utf16 = &errormsg->wmsg[0]; if (util_toUTF16_buff(utf8, utf16, sizeof(errormsg->wmsg)) != 0) FATAL("!Failed to convert string"); return (const wchar_t *)utf16; } #endif pmdk-1.8/src/common/fs.h0000664000000000000000000000444613615011243013661 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fs.h -- file system traversal abstraction layer */ #ifndef PMDK_FS_H #define PMDK_FS_H 1 #include #ifdef __cplusplus extern "C" { #endif struct fs; enum fs_entry_type { FS_ENTRY_FILE, FS_ENTRY_DIRECTORY, FS_ENTRY_SYMLINK, FS_ENTRY_OTHER, MAX_FS_ENTRY_TYPES }; struct fs_entry { enum fs_entry_type type; const char *name; size_t namelen; const char *path; size_t pathlen; /* the depth of the traversal */ /* XXX long on FreeBSD. Linux uses short. No harm in it being bigger */ long level; }; struct fs *fs_new(const char *path); void fs_delete(struct fs *f); /* this call invalidates the previous entry */ struct fs_entry *fs_read(struct fs *f); #ifdef __cplusplus } #endif #endif /* PMDK_FS_H */ pmdk-1.8/src/common/os_auto_flush_linux.c0000664000000000000000000001334113615011243017327 0ustar rootroot/* * Copyright 2018-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_auto_flush_linux.c -- Linux abstraction layer for auto flush detection */ #define _GNU_SOURCE #include #include #include #include #include #include "out.h" #include "os.h" #include "fs.h" #include "os_auto_flush.h" #define BUS_DEVICE_PATH "/sys/bus/nd/devices" #define PERSISTENCE_DOMAIN "persistence_domain" #define DOMAIN_VALUE_LEN 32 /* * check_cpu_cache -- (internal) check if file contains "cpu_cache" entry */ static int check_cpu_cache(const char *domain_path) { LOG(3, "domain_path: %s", domain_path); char domain_value[DOMAIN_VALUE_LEN]; int domain_fd; int cpu_cache = 0; if ((domain_fd = os_open(domain_path, O_RDONLY)) < 0) { LOG(1, "!open(\"%s\", O_RDONLY)", domain_path); goto end; } ssize_t len = read(domain_fd, domain_value, DOMAIN_VALUE_LEN); if (len < 0) { ERR("!read(%d, %p, %d)", domain_fd, domain_value, DOMAIN_VALUE_LEN); cpu_cache = -1; goto end; } else if (len == 0) { errno = EIO; ERR("read(%d, %p, %d) empty string", domain_fd, domain_value, DOMAIN_VALUE_LEN); cpu_cache = -1; goto end; } else if (domain_value[len - 1] != '\n') { ERR("!read(%d, %p, %d) invalid format", domain_fd, domain_value, DOMAIN_VALUE_LEN); cpu_cache = -1; goto end; } domain_value[len - 1] = '\0'; LOG(15, "detected persistent_domain: %s", domain_value); if (strcmp(domain_value, "cpu_cache") == 0) { LOG(15, "cpu_cache in persistent_domain: %s", domain_path); cpu_cache = 1; } else { LOG(15, "cpu_cache not in persistent_domain: %s", domain_path); cpu_cache = 0; } end: if (domain_fd >= 0) os_close(domain_fd); return cpu_cache; } /* * check_domain_in_region -- (internal) check if region * contains persistence_domain file */ static int check_domain_in_region(const char *region_path) { LOG(3, "region_path: %s", region_path); struct fs *reg = NULL; struct fs_entry *reg_entry; char domain_path[PATH_MAX]; int cpu_cache = 0; reg = fs_new(region_path); if (reg == NULL) { ERR("!fs_new: \"%s\"", region_path); cpu_cache = -1; goto end; } while ((reg_entry = fs_read(reg)) != NULL) { /* * persistence_domain has to be a file type entry * and it has to be first level child for region; * there is no need to run into deeper levels */ if (reg_entry->type != FS_ENTRY_FILE || strcmp(reg_entry->name, PERSISTENCE_DOMAIN) != 0 || reg_entry->level != 1) continue; int ret = snprintf(domain_path, PATH_MAX, "%s/"PERSISTENCE_DOMAIN, region_path); if (ret < 0) { ERR("snprintf(%p, %d," "%s/"PERSISTENCE_DOMAIN", %s): %d", domain_path, PATH_MAX, region_path, region_path, ret); cpu_cache = -1; goto end; } cpu_cache = check_cpu_cache(domain_path); } end: if (reg) fs_delete(reg); return cpu_cache; } /* * os_auto_flush -- check if platform supports auto flush for all regions * * Traverse "/sys/bus/nd/devices" path to find all the nvdimm regions, * then for each region checks if "persistence_domain" file exists and * contains "cpu_cache" string. * If for any region "persistence_domain" entry does not exists, or its * context is not as expected, assume eADR is not available on this platform. */ int os_auto_flush(void) { LOG(15, NULL); char *device_path; int cpu_cache = 0; device_path = BUS_DEVICE_PATH; os_stat_t sdev; if (os_stat(device_path, &sdev) != 0 || S_ISDIR(sdev.st_mode) == 0) { LOG(3, "eADR not supported"); return cpu_cache; } struct fs *dev = fs_new(device_path); if (dev == NULL) { ERR("!fs_new: \"%s\"", device_path); return -1; } struct fs_entry *dev_entry; while ((dev_entry = fs_read(dev)) != NULL) { /* * Skip if not a symlink, because we expect that * region on sysfs path is a symlink. * Skip if depth is different than 1, because region * we are interested in should be the first level * child for device. */ if ((dev_entry->type != FS_ENTRY_SYMLINK) || !strstr(dev_entry->name, "region") || dev_entry->level != 1) continue; LOG(15, "Start traversing region: %s", dev_entry->path); cpu_cache = check_domain_in_region(dev_entry->path); if (cpu_cache != 1) goto end; } end: fs_delete(dev); return cpu_cache; } pmdk-1.8/src/common/set.c0000664000000000000000000032564213615011243014043 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * set.c -- pool set utilities */ #ifndef _GNU_SOURCE #define _GNU_SOURCE #endif #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "libpmem.h" #include "librpmem.h" #include "set.h" #include "file.h" #include "os.h" #include "mmap.h" #include "util.h" #include "out.h" #include "dlsym.h" #include "valgrind_internal.h" #include "sys_util.h" #include "util_pmem.h" #include "fs.h" #include "os_deep.h" #include "badblock.h" #define LIBRARY_REMOTE "librpmem.so.1" #define SIZE_AUTODETECT_STR "AUTO" #define PMEM_EXT ".pmem" #define PMEM_EXT_LEN sizeof(PMEM_EXT) #define PMEM_FILE_PADDING 6 #define PMEM_FILE_NAME_MAX_LEN 20 #define PMEM_FILE_MAX_LEN (PMEM_FILE_NAME_MAX_LEN + PMEM_FILE_PADDING) static RPMEMpool *(*Rpmem_create)(const char *target, const char *pool_set_name, void *pool_addr, size_t pool_size, unsigned *nlanes, const struct rpmem_pool_attr *rpmem_attr); static RPMEMpool *(*Rpmem_open)(const char *target, const char *pool_set_name, void *pool_addr, size_t pool_size, unsigned *nlanes, struct rpmem_pool_attr *rpmem_attr); int (*Rpmem_close)(RPMEMpool *rpp); int (*Rpmem_persist)(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane, unsigned flags); int (*Rpmem_deep_persist)(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane); int (*Rpmem_read)(RPMEMpool *rpp, void *buff, size_t offset, size_t length, unsigned lane); int (*Rpmem_remove)(const char *target, const char *pool_set_name, int flags); int (*Rpmem_set_attr)(RPMEMpool *rpp, const struct rpmem_pool_attr *rattr); static int Remote_replication_available; static os_mutex_t Remote_lock; static void *Rpmem_handle_remote; int Prefault_at_open = 0; int Prefault_at_create = 0; int SDS_at_create = POOL_FEAT_INCOMPAT_DEFAULT & POOL_E_FEAT_SDS ? 1 : 0; int Fallocate_at_create = 1; int COW_at_open = 0; /* list of pool set option names and flags */ static const struct pool_set_option Options[] = { { "SINGLEHDR", OPTION_SINGLEHDR }, #ifndef _WIN32 { "NOHDRS", OPTION_NOHDRS }, #endif { NULL, OPTION_UNKNOWN } }; /* * util_remote_init -- initialize remote replication */ void util_remote_init(void) { LOG(3, NULL); /* XXX Is duplicate initialization really okay? */ if (!Remote_replication_available) { util_mutex_init(&Remote_lock); Remote_replication_available = 1; } } /* * util_remote_fini -- finalize remote replication */ void util_remote_fini(void) { LOG(3, NULL); util_remote_unload(); /* XXX Okay to be here if not initialized? */ if (Remote_replication_available) { Remote_replication_available = 0; util_mutex_destroy(&Remote_lock); } } /* * util_dl_check_error -- check libdl error */ static int util_dl_check_error(void *handle, const char *func) { LOG(15, "handle %p func %s", handle, func); if (handle == NULL) { char *errstr = util_dlerror(); if (errstr) ERR("%s(): %s", func, errstr); errno = ELIBACC; return -1; } return 0; } /* * util_remote_unload_core -- (internal) unload remote library (core function) */ static void util_remote_unload_core(void) { if (Rpmem_handle_remote != NULL) { util_dlclose(Rpmem_handle_remote); Rpmem_handle_remote = NULL; } Rpmem_create = NULL; Rpmem_open = NULL; Rpmem_close = NULL; Rpmem_persist = NULL; Rpmem_deep_persist = NULL; Rpmem_read = NULL; Rpmem_remove = NULL; Rpmem_set_attr = NULL; } /* * util_remote_unload -- unload remote library */ void util_remote_unload(void) { LOG(3, NULL); if (!Remote_replication_available) return; util_mutex_lock(&Remote_lock); util_remote_unload_core(); util_mutex_unlock(&Remote_lock); } /* * util_remote_load -- load remote library */ int util_remote_load(void) { LOG(3, NULL); if (!Remote_replication_available) { ERR("remote replication is not available"); return -1; } CHECK_FUNC_COMPATIBLE(rpmem_create, *Rpmem_create); CHECK_FUNC_COMPATIBLE(rpmem_open, *Rpmem_open); CHECK_FUNC_COMPATIBLE(rpmem_close, *Rpmem_close); CHECK_FUNC_COMPATIBLE(rpmem_persist, *Rpmem_persist); CHECK_FUNC_COMPATIBLE(rpmem_deep_persist, *Rpmem_deep_persist); CHECK_FUNC_COMPATIBLE(rpmem_read, *Rpmem_read); CHECK_FUNC_COMPATIBLE(rpmem_remove, *Rpmem_remove); util_mutex_lock(&Remote_lock); if (Rpmem_handle_remote) goto end; Rpmem_handle_remote = util_dlopen(LIBRARY_REMOTE); if (util_dl_check_error(Rpmem_handle_remote, "dlopen")) { ERR("the pool set requires a remote replica, " "but the '%s' library cannot be loaded", LIBRARY_REMOTE); goto err; } Rpmem_create = util_dlsym(Rpmem_handle_remote, "rpmem_create"); if (util_dl_check_error(Rpmem_create, "dlsym")) { ERR("symbol 'rpmem_create' not found"); goto err; } Rpmem_open = util_dlsym(Rpmem_handle_remote, "rpmem_open"); if (util_dl_check_error(Rpmem_open, "dlsym")) { ERR("symbol 'rpmem_open' not found"); goto err; } Rpmem_close = util_dlsym(Rpmem_handle_remote, "rpmem_close"); if (util_dl_check_error(Rpmem_close, "dlsym")) { ERR("symbol 'rpmem_close' not found"); goto err; } Rpmem_persist = util_dlsym(Rpmem_handle_remote, "rpmem_persist"); if (util_dl_check_error(Rpmem_persist, "dlsym")) { ERR("symbol 'rpmem_persist' not found"); goto err; } Rpmem_deep_persist = util_dlsym(Rpmem_handle_remote, "rpmem_deep_persist"); if (util_dl_check_error(Rpmem_deep_persist, "dlsym")) { ERR("symbol 'rpmem_deep_persist' not found"); goto err; } Rpmem_read = util_dlsym(Rpmem_handle_remote, "rpmem_read"); if (util_dl_check_error(Rpmem_read, "dlsym")) { ERR("symbol 'rpmem_read' not found"); goto err; } Rpmem_remove = util_dlsym(Rpmem_handle_remote, "rpmem_remove"); if (util_dl_check_error(Rpmem_remove, "dlsym")) { ERR("symbol 'rpmem_remove' not found"); goto err; } Rpmem_set_attr = util_dlsym(Rpmem_handle_remote, "rpmem_set_attr"); if (util_dl_check_error(Rpmem_set_attr, "dlsym")) { ERR("symbol 'rpmem_set_attr' not found"); goto err; } end: util_mutex_unlock(&Remote_lock); return 0; err: LOG(4, "error clean up"); util_remote_unload_core(); util_mutex_unlock(&Remote_lock); return -1; } /* reserve space for size, path and some whitespace and/or comment */ enum parser_codes { PARSER_CONTINUE = 0, PARSER_PMEMPOOLSET, PARSER_REPLICA, PARSER_INVALID_TOKEN, PARSER_REMOTE_REPLICA_EXPECTED, PARSER_WRONG_SIZE, PARSER_CANNOT_READ_SIZE, PARSER_ABSOLUTE_PATH_EXPECTED, PARSER_RELATIVE_PATH_EXPECTED, PARSER_SET_NO_PARTS, PARSER_REP_NO_PARTS, PARSER_REMOTE_REP_UNEXPECTED_PARTS, PARSER_SIZE_MISMATCH, PARSER_OUT_OF_MEMORY, PARSER_OPTION_UNKNOWN, PARSER_OPTION_EXPECTED, PARSER_FORMAT_OK, PARSER_MAX_CODE }; static const char *parser_errstr[PARSER_MAX_CODE] = { "", /* parsing */ "the first line must be exactly 'PMEMPOOLSET'", "exactly 'REPLICA' expected", "invalid token found in the current line", "address of remote node and descriptor of remote pool set expected", "incorrect format of size", "cannot determine size of a part", "incorrect path (must be an absolute one)", "incorrect descriptor (must be a relative path)", "no pool set parts", "no replica parts", "unexpected parts for remote replica", "sizes of pool set and replica mismatch", "allocating memory failed", "unknown option", "missing option name", "" /* format correct */ }; /* * util_replica_force_page_allocation - (internal) forces page allocation for * replica */ static void util_replica_force_page_allocation(struct pool_replica *rep) { volatile char *cur_addr = rep->part[0].addr; char *addr_end = (char *)cur_addr + rep->resvsize; for (; cur_addr < addr_end; cur_addr += Pagesize) { *cur_addr = *cur_addr; VALGRIND_SET_CLEAN(cur_addr, 1); } } /* * util_map_hdr -- map a header of a pool set */ int util_map_hdr(struct pool_set_part *part, int flags, int rdonly) { LOG(3, "part %p flags %d", part, flags); COMPILE_ERROR_ON(POOL_HDR_SIZE == 0); ASSERTeq(POOL_HDR_SIZE % Pagesize, 0); /* * Workaround for Device DAX not allowing to map a portion * of the device if offset/length are not aligned to the internal * device alignment (page size). I.e. if the device alignment * is 2M, we cannot map the 4K header, but need to align the mapping * length to 2M. * * According to mmap(2), system should automatically align mapping * length to be a multiple of the underlying page size, but it's * not true for Device DAX. */ size_t hdrsize = part->alignment > POOL_HDR_SIZE ? part->alignment : POOL_HDR_SIZE; void *addr = NULL; #if VG_MEMCHECK_ENABLED if (On_valgrind) { /* this is required only for Device DAX & memcheck */ addr = util_map_hint(hdrsize, hdrsize); if (addr == MAP_FAILED) { LOG(1, "cannot find a contiguous region of given size"); /* there's nothing we can do */ return -1; } } #endif int prot = rdonly ? PROT_READ : PROT_READ|PROT_WRITE; void *hdrp = util_map_sync(addr, hdrsize, prot, flags, part->fd, 0, &part->hdr_map_sync); if (hdrp == MAP_FAILED) { ERR("!mmap: %s", part->path); return -1; } part->hdrsize = hdrsize; part->hdr = hdrp; VALGRIND_REGISTER_PMEM_MAPPING(part->hdr, part->hdrsize); VALGRIND_REGISTER_PMEM_FILE(part->fd, part->hdr, part->hdrsize, 0); return 0; } /* * util_unmap_hdr -- unmap pool set part header */ void util_unmap_hdr(struct pool_set_part *part) { if (part->hdr == NULL || part->hdrsize == 0) return; LOG(4, "munmap: addr %p size %zu", part->hdr, part->hdrsize); VALGRIND_REMOVE_PMEM_MAPPING(part->hdr, part->hdrsize); if (munmap(part->hdr, part->hdrsize) != 0) /* this means there's a bug on the caller side */ FATAL("!munmap: %s", part->path); part->hdr = NULL; part->hdrsize = 0; } /* * util_map_part -- map a part of a pool set */ int util_map_part(struct pool_set_part *part, void *addr, size_t size, size_t offset, int flags, int rdonly) { LOG(3, "part %p addr %p size %zu offset %zu flags %d", part, addr, size, offset, flags); ASSERTeq((uintptr_t)addr % Mmap_align, 0); ASSERTeq(offset % Mmap_align, 0); ASSERTeq(size % Mmap_align, 0); ASSERT(((os_off_t)offset) >= 0); ASSERTeq(offset % part->alignment, 0); ASSERT(offset < part->filesize); if (!size) size = (part->filesize - offset) & ~(part->alignment - 1); else size = roundup(size, part->alignment); int prot = rdonly ? PROT_READ : PROT_READ | PROT_WRITE; void *addrp = util_map_sync(addr, size, prot, flags, part->fd, (os_off_t)offset, &part->map_sync); if (addrp == MAP_FAILED) { ERR("!mmap: %s", part->path); return -1; } if (addr != NULL && (flags & MAP_FIXED) && addrp != addr) { ERR("unable to map at requested address %p", addr); munmap(addrp, size); return -1; } part->addr = addrp; part->size = size; VALGRIND_REGISTER_PMEM_MAPPING(part->addr, part->size); VALGRIND_REGISTER_PMEM_FILE(part->fd, part->addr, part->size, offset); return 0; } /* * util_unmap_part -- unmap a part of a pool set */ int util_unmap_part(struct pool_set_part *part) { LOG(3, "part %p", part); if (part->addr != NULL && part->size != 0) { LOG(4, "munmap: addr %p size %zu", part->addr, part->size); VALGRIND_REMOVE_PMEM_MAPPING(part->addr, part->size); if (munmap(part->addr, part->size) != 0) { ERR("!munmap: %s", part->path); } part->addr = NULL; part->size = 0; } return 0; } /* * util_unmap_parts -- unmap parts from start_index to the end_index */ int util_unmap_parts(struct pool_replica *rep, unsigned start_index, unsigned end_index) { LOG(3, "rep: %p, start_index: %u, end_index: %u", rep, start_index, end_index); for (unsigned p = start_index; p <= end_index; p++) util_unmap_part(&rep->part[p]); return 0; } /* * util_poolset_free -- free pool set info */ void util_poolset_free(struct pool_set *set) { LOG(3, "set %p", set); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; if (rep->remote == NULL) { /* only local replicas have paths */ for (unsigned p = 0; p < rep->nallocated; p++) { Free((void *)(rep->part[p].path)); } } else { /* remote replica */ ASSERTeq(rep->nparts, 1); Free(rep->remote->node_addr); Free(rep->remote->pool_desc); Free(rep->remote); } struct pool_set_directory *d; VEC_FOREACH_BY_PTR(d, &rep->directory) { Free((void *)d->path); } VEC_DELETE(&rep->directory); Free(set->replica[r]); } Free(set->path); Free(set); } /* * util_poolset_open -- open all replicas from a poolset */ int util_poolset_open(struct pool_set *set) { for (unsigned r = 0; r < set->nreplicas; ++r) { if (util_replica_open(set, r, MAP_SHARED)) { LOG(2, "replica open failed: replica %u", r); errno = EINVAL; return -1; } } return 0; } /* * util_replica_close_local -- close local replica, optionally delete the * replica's parts */ int util_replica_close_local(struct pool_replica *rep, unsigned repn, enum del_parts_mode del) { for (unsigned p = 0; p < rep->nparts; p++) { if (rep->part[p].fd != -1) (void) os_close(rep->part[p].fd); if ((del == DELETE_CREATED_PARTS && rep->part[p].created) || del == DELETE_ALL_PARTS) { LOG(4, "unlink %s", rep->part[p].path); int olderrno = errno; if (util_unlink(rep->part[p].path) && errno != ENOENT) { ERR("!unlink %s failed (part %u, replica %u)", rep->part[p].path, p, repn); return -1; } errno = olderrno; } } return 0; } /* * util_replica_close_remote -- close remote replica, optionally delete the * replica */ int util_replica_close_remote(struct pool_replica *rep, unsigned repn, enum del_parts_mode del) { if (!rep->remote) return 0; if (rep->remote->rpp) { LOG(4, "closing remote replica #%u", repn); Rpmem_close(rep->remote->rpp); rep->remote->rpp = NULL; } if ((del == DELETE_CREATED_PARTS && rep->part[0].created) || del == DELETE_ALL_PARTS) { LOG(4, "removing remote replica #%u", repn); int ret = Rpmem_remove(rep->remote->node_addr, rep->remote->pool_desc, 0); if (ret) { LOG(1, "!removing remote replica #%u failed", repn); return -1; } } return 0; } /* * util_poolset_close -- unmap and close all the parts of the pool set, * optionally delete parts */ void util_poolset_close(struct pool_set *set, enum del_parts_mode del) { LOG(3, "set %p del %d", set, del); int oerrno = errno; for (unsigned r = 0; r < set->nreplicas; r++) { util_replica_close(set, r); struct pool_replica *rep = set->replica[r]; if (!rep->remote) (void) util_replica_close_local(rep, r, del); else (void) util_replica_close_remote(rep, r, del); } /* * XXX On FreeBSD, mmap()ing a file does not increment the flock() * reference count, so we had to keep the files open until now. */ #ifdef __FreeBSD__ util_poolset_fdclose_always(set); #endif util_poolset_free(set); errno = oerrno; } /* * util_poolset_chmod -- change mode for all created files related to pool set */ int util_poolset_chmod(struct pool_set *set, mode_t mode) { LOG(3, "set %p mode %o", set, mode); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; /* skip remote replicas */ if (rep->remote != NULL) continue; for (unsigned p = 0; p < rep->nparts; p++) { struct pool_set_part *part = &rep->part[p]; /* skip not created or closed parts */ if (!part->created || part->fd == -1) continue; os_stat_t stbuf; if (os_fstat(part->fd, &stbuf) != 0) { ERR("!fstat %d %s", part->fd, part->path); return -1; } if (stbuf.st_mode & ~(unsigned)S_IFMT) { LOG(1, "file permissions changed during pool " "initialization, file: %s (%o)", part->path, stbuf.st_mode & ~(unsigned)S_IFMT); } if (os_chmod(part->path, mode)) { ERR("!chmod %u/%u/%s", r, p, part->path); return -1; } } } return 0; } /* * util_poolset_fdclose_always -- close file descriptors related to pool set */ void util_poolset_fdclose_always(struct pool_set *set) { LOG(3, "set %p", set); for (unsigned r = 0; r < set->nreplicas; r++) util_replica_fdclose(set->replica[r]); } /* * util_poolset_fdclose -- close pool set file descriptors if not FreeBSD * * XXX On FreeBSD, mmap()ing a file does not increment the flock() * reference count, so we need to keep the files open. */ void util_poolset_fdclose(struct pool_set *set) { #ifdef __FreeBSD__ LOG(3, "set %p: holding open", set); #else util_poolset_fdclose_always(set); #endif } /* * util_autodetect_size -- (internal) retrieves size of an existing file */ static ssize_t util_autodetect_size(const char *path) { enum file_type type = util_file_get_type(path); if (type < 0) return -1; if (type == TYPE_NORMAL) { ERR("size autodetection is supported only for device dax"); return -1; } return util_file_get_size(path); } /* * parser_read_line -- (internal) read line and validate size and path * from a pool set file */ static enum parser_codes parser_read_line(char *line, size_t *size, char **path) { int ret; char *size_str; char *path_str; char *rest_str; char *saveptr = NULL; /* must be NULL initialized on Windows */ size_str = strtok_r(line, " \t", &saveptr); path_str = strtok_r(NULL, " \t", &saveptr); rest_str = strtok_r(NULL, " \t", &saveptr); if (!size_str || !path_str || rest_str) return PARSER_INVALID_TOKEN; LOG(10, "size '%s' path '%s'", size_str, path_str); /* * A format of the size is checked in detail. As regards the path, * it is checked only if the read path is an absolute path. * The rest should be checked during creating/opening the file. */ /* check if the read path is an absolute path */ if (!util_is_absolute_path(path_str)) return PARSER_ABSOLUTE_PATH_EXPECTED; *path = Strdup(path_str); if (!(*path)) { ERR("!Strdup"); return PARSER_OUT_OF_MEMORY; } if (strcmp(SIZE_AUTODETECT_STR, size_str) == 0) { /* * XXX: this should be done after the parsing completes, but * currently this operation is performed in simply too many * places in the code to move this someplace else. */ ssize_t s = util_autodetect_size(path_str); if (s < 0) { Free(*path); *path = NULL; return PARSER_CANNOT_READ_SIZE; } *size = (size_t)s; return PARSER_CONTINUE; } ret = util_parse_size(size_str, size); if (ret != 0 || *size == 0) { Free(*path); *path = NULL; return PARSER_WRONG_SIZE; } return PARSER_CONTINUE; } /* * parser_read_replica -- (internal) read line and validate remote replica * from a pool set file */ static enum parser_codes parser_read_replica(char *line, char **node_addr, char **pool_desc) { char *addr_str; char *desc_str; char *rest_str; char *saveptr = NULL; /* must be NULL initialized on Windows */ addr_str = strtok_r(line, " \t", &saveptr); desc_str = strtok_r(NULL, " \t", &saveptr); rest_str = strtok_r(NULL, " \t", &saveptr); if (!addr_str || !desc_str) return PARSER_REMOTE_REPLICA_EXPECTED; if (rest_str) return PARSER_INVALID_TOKEN; LOG(10, "node address '%s' pool set descriptor '%s'", addr_str, desc_str); /* check if the descriptor is a relative path */ if (util_is_absolute_path(desc_str)) return PARSER_RELATIVE_PATH_EXPECTED; *node_addr = Strdup(addr_str); *pool_desc = Strdup(desc_str); if (!(*node_addr) || !(*pool_desc)) { ERR("!Strdup"); if (*node_addr) Free(*node_addr); if (*pool_desc) Free(*pool_desc); return PARSER_OUT_OF_MEMORY; } return PARSER_CONTINUE; } /* * parser_read_options -- (internal) read line and validate options */ static enum parser_codes parser_read_options(char *line, unsigned *options) { LOG(3, "line '%s'", line); int opt_cnt = 0; char *saveptr = NULL; /* must be NULL initialized on Windows */ char *opt_str = strtok_r(line, " \t", &saveptr); while (opt_str != NULL) { LOG(4, "option '%s'", opt_str); int i = 0; while (Options[i].name && strcmp(opt_str, Options[i].name) != 0) i++; if (Options[i].name == NULL) { LOG(4, "unknown option '%s'", opt_str); return PARSER_OPTION_UNKNOWN; } if (*options & Options[i].flag) LOG(4, "duplicated option '%s'", opt_str); *options |= Options[i].flag; opt_cnt++; opt_str = strtok_r(NULL, " \t", &saveptr); } if (opt_cnt == 0) return PARSER_OPTION_EXPECTED; return PARSER_CONTINUE; } /* * util_replica_reserve -- reserves part slots capacity in a replica */ static int util_replica_reserve(struct pool_replica **repp, unsigned n) { LOG(3, "replica %p n %u", *repp, n); struct pool_replica *rep = *repp; if (rep->nallocated >= n) return 0; rep = Realloc(rep, sizeof(struct pool_replica) + (n) * sizeof(struct pool_set_part)); if (rep == NULL) { ERR("!Realloc"); return -1; } size_t nsize = sizeof(struct pool_set_part) * (n - rep->nallocated); memset(rep->part + rep->nallocated, 0, nsize); rep->nallocated = n; *repp = rep; return 0; } /* * util_replica_add_part_by_idx -- (internal) allocates, initializes and adds a * part structure at the provided location in the replica info */ static int util_replica_add_part_by_idx(struct pool_replica **repp, const char *path, size_t filesize, unsigned p) { LOG(3, "replica %p path %s filesize %zu", *repp, path, filesize); if (util_replica_reserve(repp, p + 1) != 0) return -1; struct pool_replica *rep = *repp; ASSERTne(rep, NULL); int is_dev_dax = 0; if (path != NULL) { enum file_type type = util_file_get_type(path); if (type == OTHER_ERROR) return -1; is_dev_dax = type == TYPE_DEVDAX; } rep->part[p].path = path; rep->part[p].filesize = filesize; rep->part[p].fd = -1; rep->part[p].is_dev_dax = is_dev_dax; rep->part[p].created = 0; rep->part[p].hdr = NULL; rep->part[p].addr = NULL; rep->part[p].remote_hdr = NULL; rep->part[p].has_bad_blocks = 0; if (is_dev_dax) rep->part[p].alignment = util_file_device_dax_alignment(path); else rep->part[p].alignment = Mmap_align; ASSERTne(rep->part[p].alignment, 0); rep->nparts += 1; return 0; } /* * util_replica_add_part -- adds a next part in replica info */ static int util_replica_add_part(struct pool_replica **repp, const char *path, size_t filesize) { LOG(3, "replica %p path \"%s\" filesize %zu", *repp, path, filesize); return util_replica_add_part_by_idx(repp, path, filesize, (*repp)->nparts); } /* * util_parse_add_part -- (internal) add a new part file to the replica info */ static int util_parse_add_part(struct pool_set *set, const char *path, size_t filesize) { LOG(3, "set %p path %s filesize %zu", set, path, filesize); ASSERTne(set, NULL); if (set->directory_based) { ERR("cannot mix directories and files in a set"); errno = EINVAL; return -1; } return util_replica_add_part(&set->replica[set->nreplicas - 1], path, filesize); } /* * util_parse_add_directory -- * (internal) add a new directory to the replica info */ static int util_parse_add_directory(struct pool_set *set, const char *path, size_t filesize) { LOG(3, "set %p path %s filesize %zu", set, path, filesize); ASSERTne(set, NULL); struct pool_replica *rep = set->replica[set->nreplicas - 1]; ASSERTne(rep, NULL); if (set->directory_based == 0) { if (rep->nparts > 0 || set->nreplicas > 1) { ERR("cannot mix directories and files in a set"); errno = EINVAL; return -1; } set->directory_based = 1; } char *rpath = util_part_realpath(path); if (rpath == NULL) { ERR("cannot resolve realpath of new directory"); return -1; } for (unsigned i = 0; i < set->nreplicas; ++i) { struct pool_replica *r = set->replica[i]; struct pool_set_directory *dir; char *dpath = NULL; VEC_FOREACH_BY_PTR(dir, &r->directory) { dpath = util_part_realpath(dir->path); ASSERTne(dpath, NULL); /* must have been resolved */ if (strcmp(rpath, dpath) == 0) { ERR("cannot use the same directory twice"); errno = EEXIST; free(dpath); free(rpath); return -1; } free(dpath); } } free(rpath); struct pool_set_directory d; d.path = path; d.resvsize = filesize; if (VEC_PUSH_BACK(&rep->directory, d) != 0) return -1; rep->resvsize += filesize; return 0; } /* * util_parse_add_element -- * (internal) add a new element to the replica info */ static int util_parse_add_element(struct pool_set *set, const char *path, size_t filesize) { LOG(3, "set %p path %s filesize %zu", set, path, filesize); os_stat_t stat; int olderrno = errno; if (os_stat(path, &stat) == 0 && S_ISDIR(stat.st_mode)) return util_parse_add_directory(set, path, filesize); errno = olderrno; return util_parse_add_part(set, path, filesize); } /* * util_parse_add_replica -- (internal) add a new replica to the pool set info */ static int util_parse_add_replica(struct pool_set **setp) { LOG(3, "setp %p", setp); ASSERTne(setp, NULL); struct pool_set *set = *setp; ASSERTne(set, NULL); set = Realloc(set, sizeof(struct pool_set) + (set->nreplicas + 1) * sizeof(struct pool_replica *)); if (set == NULL) { ERR("!Realloc"); return -1; } *setp = set; struct pool_replica *rep; rep = Zalloc(sizeof(struct pool_replica)); if (rep == NULL) { ERR("!Zalloc"); return -1; } VEC_INIT(&rep->directory); unsigned r = set->nreplicas++; set->replica[r] = rep; return 0; } /* * util_replica_check_map_sync -- (internal) check MAP_SYNC restrictions */ static int util_replica_check_map_sync(struct pool_set *set, unsigned repidx, int check_hdr) { LOG(3, "set %p repidx %u", set, repidx); struct pool_replica *rep = set->replica[repidx]; int map_sync = rep->part[0].map_sync; for (unsigned p = 1; p < rep->nparts; p++) { if (map_sync != rep->part[p].map_sync) { ERR("replica #%u part %u %smapped with MAP_SYNC", repidx, p, rep->part[p].map_sync ? "" : "not"); return -1; } } if (check_hdr) { for (unsigned p = 0; p < rep->nhdrs; p++) { if (map_sync != rep->part[p].hdr_map_sync) { ERR("replica #%u part %u header %smapped " "with MAP_SYNC", repidx, p, rep->part[p].hdr_map_sync ? "" : "not"); return -1; } } } return 0; } /* * util_poolset_check_devdax -- (internal) check Device DAX restrictions */ static int util_poolset_check_devdax(struct pool_set *set) { LOG(3, "set %p", set); if (set->directory_based) return 0; for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; int is_dev_dax = rep->part[0].is_dev_dax; for (unsigned p = 0; p < rep->nparts; p++) { if (rep->part[p].is_dev_dax != is_dev_dax) { ERR( "either all the parts must be Device DAX or none"); return -1; } if (is_dev_dax && rep->nparts > 1 && (set->options & (OPTION_SINGLEHDR | OPTION_NOHDRS)) == 0 && util_file_device_dax_alignment(rep->part[p].path) != Pagesize) { ERR( "Multiple DAX devices with alignment other than 4KB. Use the SINGLEHDR poolset option."); return -1; } } } return 0; } /* * util_poolset_check_options -- (internal) check if poolset options are * admissible */ static int util_poolset_check_options(struct pool_set *set) { LOG(3, "set %p", set); if ((set->options & OPTION_SINGLEHDR) && (set->options & OPTION_NOHDRS)) { ERR( "both SINGLEHDR and NOHDR poolset options used at the same time"); return -1; } return 0; } /* * util_poolset_set_size -- (internal) calculate pool size */ static void util_poolset_set_size(struct pool_set *set) { LOG(3, "set %p", set); set->poolsize = SIZE_MAX; set->resvsize = SIZE_MAX; for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; if (set->options & OPTION_SINGLEHDR) rep->nhdrs = 1; else if (set->options & OPTION_NOHDRS) rep->nhdrs = 0; else rep->nhdrs = rep->nparts; rep->repsize = 0; for (unsigned p = 0; p < rep->nparts; p++) { rep->repsize += (rep->part[p].filesize & ~(Mmap_align - 1)); } if (rep->nhdrs > 0) rep->repsize -= (rep->nhdrs - 1) * Mmap_align; if (rep->resvsize == 0) rep->resvsize = rep->repsize; /* * Calculate pool size - choose the smallest replica size. * Ignore remote replicas. */ if (rep->remote == NULL && rep->repsize < set->poolsize) set->poolsize = rep->repsize; if (rep->remote == NULL && rep->resvsize < set->resvsize) set->resvsize = rep->resvsize; } LOG(3, "pool size set to %zu", set->poolsize); } /* * util_parse_add_remote_replica -- (internal) add a new remote replica * to the pool set info */ static int util_parse_add_remote_replica(struct pool_set **setp, char *node_addr, char *pool_desc) { LOG(3, "setp %p node_addr %s pool_desc %s", setp, node_addr, pool_desc); ASSERTne(setp, NULL); ASSERTne(node_addr, NULL); ASSERTne(pool_desc, NULL); int ret = util_parse_add_replica(setp); if (ret != 0) return ret; /* * A remote replica has one fake part of size equal twice pool header * size for storing pool header and pool descriptor. */ ret = util_parse_add_part(*setp, NULL, 2 * POOL_HDR_SIZE); if (ret != 0) return ret; struct pool_set *set = *setp; struct pool_replica *rep = set->replica[set->nreplicas - 1]; ASSERTne(rep, NULL); rep->remote = Zalloc(sizeof(struct remote_replica)); if (rep->remote == NULL) { ERR("!Malloc"); return -1; } rep->remote->node_addr = node_addr; rep->remote->pool_desc = pool_desc; set->remote = 1; return 0; } /* * util_part_idx_by_file_name -- (internal) retrieves the part index from a * name of the file that is an element of a directory poolset */ static long util_part_idx_by_file_name(const char *filename) { LOG(3, "filename \"%s\"", filename); int olderrno = errno; errno = 0; long part_idx = strtol(filename, NULL, 10); if (errno != 0) return -1; errno = olderrno; return part_idx; } /* * util_poolset_directory_load -- (internal) loads and initializes all * existing parts in a single directory */ static int util_poolset_directory_load(struct pool_replica **repp, const char *directory) { LOG(3, "rep %p dir \"%s\"", *repp, directory); struct fs *f = fs_new(directory); if (f == NULL) { ERR("!fs_new: \"%s\"", directory); return -1; } int nparts = 0; char *path = NULL; struct fs_entry *entry; while ((entry = fs_read(f)) != NULL) { if (entry->level != 1) continue; if (entry->type != FS_ENTRY_FILE) continue; if (entry->namelen < PMEM_EXT_LEN) continue; const char *ext = entry->path + entry->pathlen - PMEM_EXT_LEN + 1; if (strcmp(PMEM_EXT, ext) != 0) continue; long part_idx = util_part_idx_by_file_name(entry->name); if (part_idx < 0) continue; ssize_t size = util_file_get_size(entry->path); if (size < 0) { LOG(2, "cannot read size of file (%s) in a poolset directory", entry->path); goto err; } if ((path = Strdup(entry->path)) == NULL) { ERR("!Strdup"); goto err; } if (util_replica_add_part_by_idx(repp, path, (size_t)size, (unsigned)part_idx) != 0) { ERR("unable to load part %s", entry->path); goto err; } nparts++; } fs_delete(f); return nparts; err: fs_delete(f); return -1; } /* * util_poolset_directories_load -- (internal) loads and initializes all * existing parts in the poolset directories */ static int util_poolset_directories_load(struct pool_set *set) { LOG(3, "set %p", set); if (!set->directory_based) return 0; unsigned next_part_id = 0; unsigned max_parts_rep = 0; for (unsigned r = 0; r < set->nreplicas; r++) { next_part_id = 0; struct pool_set_directory *d; int nparts = 0; int prev_nparts = 0; VEC_FOREACH_BY_PTR(d, &set->replica[r]->directory) { prev_nparts = nparts; nparts = util_poolset_directory_load(&set->replica[r], d->path); if (nparts < 0) { ERR("failed to load parts from directory %s", d->path); return -1; } next_part_id += (unsigned)nparts; /* always try to evenly spread files across dirs */ if (r == 0 && prev_nparts > nparts) set->next_directory_id++; } if (next_part_id > set->replica[max_parts_rep]->nparts) max_parts_rep = r; if (r == 0) set->next_id = next_part_id; } /* * In order to maintain the same semantics of poolset parsing for * regular poolsets and directory poolsets, we need to speculatively * recreate the information regarding any missing parts in replicas. */ struct pool_replica *rep; struct pool_replica *mrep = set->replica[max_parts_rep]; for (unsigned r = 0; r < set->nreplicas; r++) { if (set->replica[r]->nparts == mrep->nparts) continue; if (VEC_SIZE(&set->replica[r]->directory) == 0) { errno = ENOENT; ERR("!no directories in replica"); return -1; } if (util_replica_reserve(&set->replica[r], mrep->nparts) != 0) return -1; rep = set->replica[r]; struct pool_set_directory *d = VEC_GET(&rep->directory, 0); for (unsigned pidx = 0; pidx < rep->nallocated; ++pidx) { struct pool_set_part *p = &rep->part[pidx]; *p = mrep->part[pidx]; size_t path_len = strlen(d->path) + PMEM_FILE_MAX_LEN; if ((p->path = Malloc(path_len)) == NULL) { ERR("!Malloc"); return -1; } snprintf((char *)p->path, path_len, "%s" OS_DIR_SEP_STR "%0*u%s", d->path, PMEM_FILE_PADDING, pidx, PMEM_EXT); } rep->nparts = mrep->nparts; } return 0; } /* * util_poolset_parse -- parse pool set config file * * Returns 0 if the file is a valid poolset config file, * and -1 in case of any error. * * XXX: use memory mapped file */ int util_poolset_parse(struct pool_set **setp, const char *path, int fd) { LOG(3, "setp %p path %s fd %d", setp, path, fd); struct pool_set *set = NULL; enum parser_codes result; char *line; char *ppath; char *pool_desc; char *node_addr; char *cp; size_t psize; FILE *fs; int oerrno; if (os_lseek(fd, 0, SEEK_SET) != 0) { ERR("!lseek %d", fd); return -1; } fd = dup(fd); if (fd < 0) { ERR("!dup"); return -1; } /* associate a stream with the file descriptor */ if ((fs = os_fdopen(fd, "r")) == NULL) { ERR("!fdopen %d", fd); os_close(fd); return -1; } unsigned nlines = 0; unsigned nparts = 0; /* number of parts in current replica */ /* read the first line */ line = util_readline(fs); if (line == NULL) { ERR("!Reading poolset file"); goto err; } nlines++; set = Zalloc(sizeof(struct pool_set)); if (set == NULL) { ERR("!Malloc for pool set"); goto err; } set->path = Strdup(path); if (set->path == NULL) { ERR("!Strdup"); goto err; } /* check also if the last character is '\n' */ if (strncmp(line, POOLSET_HDR_SIG, POOLSET_HDR_SIG_LEN) == 0 && line[POOLSET_HDR_SIG_LEN] == '\n') { /* 'PMEMPOOLSET' signature detected */ LOG(10, "PMEMPOOLSET"); int ret = util_parse_add_replica(&set); if (ret != 0) goto err; nparts = 0; result = PARSER_CONTINUE; } else { result = PARSER_PMEMPOOLSET; } while (result == PARSER_CONTINUE) { Free(line); /* read next line */ line = util_readline(fs); nlines++; if (line) { /* chop off newline and comments */ if ((cp = strchr(line, '\n')) != NULL) *cp = '\0'; if (cp != line && (cp = strchr(line, '#')) != NULL) *cp = '\0'; /* skip comments and blank lines */ if (cp == line) continue; } if (!line) { if (nparts >= 1) { result = PARSER_FORMAT_OK; } else { if (set->nreplicas == 1) result = PARSER_SET_NO_PARTS; else result = PARSER_REP_NO_PARTS; } } else if (strncmp(line, POOLSET_OPTION_SIG, POOLSET_OPTION_SIG_LEN) == 0) { result = parser_read_options( line + POOLSET_OPTION_SIG_LEN, &set->options); if (result == PARSER_CONTINUE) { LOG(10, "OPTIONS: %x", set->options); } } else if (strncmp(line, POOLSET_REPLICA_SIG, POOLSET_REPLICA_SIG_LEN) == 0) { if (line[POOLSET_REPLICA_SIG_LEN] != '\0') { /* something more than 'REPLICA' */ char c = line[POOLSET_REPLICA_SIG_LEN]; if (!isblank((unsigned char)c)) { result = PARSER_REPLICA; continue; } /* check if it is a remote replica */ result = parser_read_replica( line + POOLSET_REPLICA_SIG_LEN, &node_addr, &pool_desc); if (result == PARSER_CONTINUE) { /* remote REPLICA */ LOG(10, "REMOTE REPLICA " "node address '%s' " "pool set descriptor '%s'", node_addr, pool_desc); if (util_parse_add_remote_replica(&set, node_addr, pool_desc)) goto err; } } else if (nparts >= 1) { /* 'REPLICA' signature detected */ LOG(10, "REPLICA"); int ret = util_parse_add_replica(&set); if (ret != 0) goto err; nparts = 0; result = PARSER_CONTINUE; } else { if (set->nreplicas == 1) result = PARSER_SET_NO_PARTS; else result = PARSER_REP_NO_PARTS; } } else { /* there could be no parts for remote replicas */ if (set->replica[set->nreplicas - 1]->remote) { result = PARSER_REMOTE_REP_UNEXPECTED_PARTS; continue; } /* read size and path */ result = parser_read_line(line, &psize, &ppath); if (result == PARSER_CONTINUE) { /* add a new pool's part to the list */ int ret = util_parse_add_element(set, ppath, psize); if (ret != 0) { Free(ppath); goto err; } nparts++; } } } if (result != PARSER_FORMAT_OK) { ERR("%s [%s:%d]", path, parser_errstr[result], nlines); switch (result) { case PARSER_CANNOT_READ_SIZE: case PARSER_OUT_OF_MEMORY: /* do not overwrite errno */ break; default: errno = EINVAL; } goto err; } if (util_poolset_check_devdax(set) != 0) { errno = EINVAL; goto err; } if (util_poolset_directories_load(set) != 0) { ERR("cannot load part files from directories"); goto err; } LOG(4, "set file format correct (%s)", path); (void) os_fclose(fs); Free(line); util_poolset_check_options(set); util_poolset_set_size(set); *setp = set; return 0; err: oerrno = errno; Free(line); (void) os_fclose(fs); if (set) util_poolset_free(set); errno = oerrno; return -1; } /* * util_poolset_single -- (internal) create a one-part pool set * * On success returns a pointer to a newly allocated and initialized * pool set structure. Otherwise, NULL is returned. */ static struct pool_set * util_poolset_single(const char *path, size_t filesize, int create, int ignore_sds) { LOG(3, "path %s filesize %zu create %d", path, filesize, create); enum file_type type = util_file_get_type(path); if (type == OTHER_ERROR) return NULL; struct pool_set *set; set = Zalloc(sizeof(struct pool_set) + sizeof(struct pool_replica *)); if (set == NULL) { ERR("!Malloc for pool set"); return NULL; } set->path = Strdup(path); if (set->path == NULL) { ERR("!Strdup"); Free(set); return NULL; } struct pool_replica *rep; rep = Zalloc(sizeof(struct pool_replica) + sizeof(struct pool_set_part)); if (rep == NULL) { ERR("!Malloc for pool set replica"); Free(set->path); Free(set); return NULL; } VEC_INIT(&rep->directory); set->replica[0] = rep; rep->part[0].filesize = filesize; rep->part[0].path = Strdup(path); rep->part[0].fd = -1; /* will be filled out by util_poolset_file() */ rep->part[0].is_dev_dax = type == TYPE_DEVDAX; rep->part[0].created = create; rep->part[0].hdr = NULL; rep->part[0].addr = NULL; rep->part[0].has_bad_blocks = 0; if (rep->part[0].is_dev_dax) rep->part[0].alignment = util_file_device_dax_alignment(path); else rep->part[0].alignment = Mmap_align; ASSERTne(rep->part[0].alignment, 0); rep->nallocated = 1; rep->nparts = 1; rep->nhdrs = 1; /* it does not have a remote replica */ rep->remote = NULL; set->remote = 0; /* round down to the nearest mapping alignment boundary */ rep->repsize = rep->part[0].filesize & ~(rep->part[0].alignment - 1); rep->resvsize = rep->repsize; set->poolsize = rep->repsize; set->resvsize = rep->resvsize; set->nreplicas = 1; set->ignore_sds = ignore_sds || (set->options & OPTION_NOHDRS); return set; } /* * util_part_open -- open or create a single part file */ int util_part_open(struct pool_set_part *part, size_t minsize, int create_part) { LOG(3, "part %p minsize %zu create %d", part, minsize, create_part); int exists = util_file_exists(part->path); if (exists < 0) return -1; int create_file = create_part; if (exists) create_file = 0; part->created = 0; if (create_file) { part->fd = util_file_create(part->path, part->filesize, minsize); if (part->fd == -1) { LOG(2, "failed to create file: %s", part->path); return -1; } part->created = 1; } else { size_t size = 0; int flags = O_RDWR; part->fd = util_file_open(part->path, &size, minsize, flags); if (part->fd == -1) { LOG(2, "failed to open file: %s", part->path); return -1; } if (Fallocate_at_create && create_part && !part->is_dev_dax) { int ret = os_posix_fallocate(part->fd, 0, (os_off_t)size); if (ret != 0) { errno = ret; ERR("!posix_fallocate \"%s\", %zu", part->path, size); return -1; } } /* check if filesize matches */ if (part->filesize != size) { ERR("file size does not match config: %s, %zu != %zu", part->path, size, part->filesize); errno = EINVAL; return -1; } } return 0; } /* * util_part_fdclose -- close part file */ void util_part_fdclose(struct pool_set_part *part) { LOG(3, "part %p", part); if (part->fd != -1) { (void) os_close(part->fd); part->fd = -1; } } /* * util_set_rpmem_attr -- (internal) overwrite existing pool attributes * * does not set uuid, next_part_uuid, prev_part_uuid */ static void util_set_rpmem_attr(struct pool_hdr *hdrp, const struct rpmem_pool_attr *rattr) { LOG(5, "hdrp %p rattr %p", hdrp, rattr); memcpy(hdrp->signature, rattr->signature, POOL_HDR_SIG_LEN); hdrp->major = rattr->major; hdrp->features.compat = rattr->compat_features; hdrp->features.incompat = rattr->incompat_features; hdrp->features.ro_compat = rattr->ro_compat_features; memcpy(hdrp->poolset_uuid, rattr->poolset_uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->next_repl_uuid, rattr->next_uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->prev_repl_uuid, rattr->prev_uuid, POOL_HDR_UUID_LEN); memcpy(&hdrp->arch_flags, rattr->user_flags, sizeof(struct arch_flags)); } /* * util_get_rpmem_attr -- (internal) get attributes for remote replica header */ static void util_get_rpmem_attr(struct rpmem_pool_attr *rattr, const struct pool_hdr *hdrp) { LOG(5, "rpmem_attr %p hdrp %p", rattr, hdrp); ASSERTne(rattr, NULL); memcpy(rattr->signature, hdrp->signature, POOL_HDR_SIG_LEN); rattr->major = hdrp->major; rattr->compat_features = hdrp->features.compat; rattr->incompat_features = hdrp->features.incompat; rattr->ro_compat_features = hdrp->features.ro_compat; memcpy(rattr->poolset_uuid, hdrp->poolset_uuid, POOL_HDR_UUID_LEN); memcpy(rattr->uuid, hdrp->uuid, POOL_HDR_UUID_LEN); memcpy(rattr->next_uuid, hdrp->next_repl_uuid, POOL_HDR_UUID_LEN); memcpy(rattr->prev_uuid, hdrp->prev_repl_uuid, POOL_HDR_UUID_LEN); memcpy(rattr->user_flags, &hdrp->arch_flags, sizeof(struct arch_flags)); } /* * util_remote_store_attr -- (internal) store attributes read from remote * replica in the local volatile pool header */ static void util_remote_store_attr(struct pool_hdr *hdrp, const struct rpmem_pool_attr *rattr) { LOG(4, "hdrp %p rpmem_attr %p", hdrp, rattr); util_set_rpmem_attr(hdrp, rattr); memcpy(hdrp->uuid, rattr->uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->next_part_uuid, rattr->uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->prev_part_uuid, rattr->uuid, POOL_HDR_UUID_LEN); } /* * util_update_remote_header -- update attributes of a remote replica; * the remote replica must be open */ int util_update_remote_header(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); ASSERTne(REP(set, repn)->remote, NULL); ASSERTne(REP(set, repn)->remote->rpp, NULL); struct pool_replica *rep = REP(set, repn); struct pool_hdr *hdr = HDR(rep, 0); /* get attributes from the local pool header */ struct rpmem_pool_attr attributes; util_get_rpmem_attr(&attributes, hdr); /* push the attributes to the remote replica */ RPMEMpool *rpp = rep->remote->rpp; int ret = Rpmem_set_attr(rpp, &attributes); if (ret) { ERR("!Rpmem_set_attr"); return -1; } return 0; } /* * util_pool_close_remote -- close a remote replica */ int util_pool_close_remote(RPMEMpool *rpp) { LOG(3, "rpp %p", rpp); return Rpmem_close(rpp); } /* * util_poolset_remote_open -- open or create a remote replica */ int util_poolset_remote_open(struct pool_replica *rep, unsigned repidx, size_t minsize, int create, void *pool_addr, size_t pool_size, unsigned *nlanes) { LOG(3, "rep %p repidx %u minsize %zu create %d " "pool_addr %p pool_size %zu nlanes %p", rep, repidx, minsize, create, pool_addr, pool_size, nlanes); ASSERTne(nlanes, NULL); if (!Rpmem_handle_remote) { return -1; } unsigned remote_nlanes = *nlanes; if (create) { struct rpmem_pool_attr rpmem_attr_create; util_get_rpmem_attr(&rpmem_attr_create, rep->part[0].hdr); rep->remote->rpp = Rpmem_create(rep->remote->node_addr, rep->remote->pool_desc, pool_addr, pool_size, &remote_nlanes, &rpmem_attr_create); if (rep->remote->rpp == NULL) { ERR("creating remote replica #%u failed", repidx); return -1; } rep->part[0].created = 1; } else { /* open */ struct rpmem_pool_attr rpmem_attr_open; rep->remote->rpp = Rpmem_open(rep->remote->node_addr, rep->remote->pool_desc, pool_addr, pool_size, &remote_nlanes, &rpmem_attr_open); if (rep->remote->rpp == NULL) { ERR("opening remote replica #%u failed", repidx); return -1; } util_remote_store_attr(rep->part[0].hdr, &rpmem_attr_open); } if (remote_nlanes < *nlanes) *nlanes = remote_nlanes; return 0; } /* * util_poolset_files_local -- (internal) open or create all the local * part files of a pool set and replica sets */ static int util_poolset_files_local(struct pool_set *set, size_t minpartsize, int create) { LOG(3, "set %p minpartsize %zu create %d", set, minpartsize, create); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; if (!rep->remote) { for (unsigned p = 0; p < rep->nparts; p++) { if (util_part_open(&rep->part[p], minpartsize, create)) return -1; } } } return 0; } /* * util_poolset_remote_replica_open -- open remote replica */ int util_poolset_remote_replica_open(struct pool_set *set, unsigned repidx, size_t minsize, int create, unsigned *nlanes) { #ifndef _WIN32 /* * This is a workaround for an issue with using device dax with * libibverbs. To handle fork() function calls correctly libfabric use * ibv_fork_init(3) which makes all registered memory being madvised * with MADV_DONTFORK flag. In libpmemobj the remote replication is * performed without pool header (first 4k). In such case the address * passed to madvise(2) is aligned to 4k, but device dax can require * different alignment (default is 2MB). This workaround madvises the * entire memory region before registering it by fi_mr_reg(3). * * The librpmem client requires fork() support to work correctly. */ if (set->replica[0]->part[0].is_dev_dax) { int ret = os_madvise(set->replica[0]->part[0].addr, set->replica[0]->part[0].filesize, MADV_DONTFORK); if (ret) { ERR("!madvise"); return ret; } } #endif void *pool_addr = (void *)((uintptr_t)set->replica[0]->part[0].addr); return util_poolset_remote_open(set->replica[repidx], repidx, minsize, create, pool_addr, set->poolsize, nlanes); } /* * util_poolset_files_remote -- (internal) open or create all the remote * part files of a pool set and replica sets */ static int util_poolset_files_remote(struct pool_set *set, size_t minsize, unsigned *nlanes, int create) { LOG(3, "set %p minsize %zu nlanes %p create %d", set, minsize, nlanes, create); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; if (rep->remote) { if (util_poolset_remote_replica_open(set, r, minsize, create, nlanes)) return -1; } } return 0; } /* * util_poolset_read -- read memory pool set file * * On success returns 0 and a pointer to a newly allocated structure * containing the info of all the parts of the pool set and replicas. */ int util_poolset_read(struct pool_set **setp, const char *path) { LOG(3, "setp %p path %s", setp, path); int oerrno; int ret = 0; int fd; if ((fd = os_open(path, O_RDONLY)) < 0) { ERR("!open: path \"%s\"", path); return -1; } ret = util_poolset_parse(setp, path, fd); oerrno = errno; (void) os_close(fd); errno = oerrno; return ret; } /* * util_poolset_create_set -- create a new pool set structure * * On success returns 0 and a pointer to a newly allocated structure * containing the info of all the parts of the pool set and replicas. */ int util_poolset_create_set(struct pool_set **setp, const char *path, size_t poolsize, size_t minsize, int ignore_sds) { LOG(3, "setp %p path %s poolsize %zu minsize %zu", setp, path, poolsize, minsize); int oerrno; int ret = 0; int fd; size_t size = 0; enum file_type type = util_file_get_type(path); if (type == OTHER_ERROR) return -1; if (poolsize != 0) { if (type == TYPE_DEVDAX) { ERR("size must be zero for device dax"); return -1; } *setp = util_poolset_single(path, poolsize, 1, ignore_sds); if (*setp == NULL) return -1; return 0; } /* do not check minsize */ if ((fd = util_file_open(path, &size, 0, O_RDONLY)) == -1) return -1; char signature[POOLSET_HDR_SIG_LEN]; if (type == TYPE_NORMAL) { /* * read returns ssize_t, but we know it will return value * between -1 and POOLSET_HDR_SIG_LEN (11), so we can safely * cast it to int */ ret = (int)read(fd, signature, POOLSET_HDR_SIG_LEN); if (ret < 0) { ERR("!read %d", fd); goto err; } } if (type == TYPE_DEVDAX || ret < POOLSET_HDR_SIG_LEN || strncmp(signature, POOLSET_HDR_SIG, POOLSET_HDR_SIG_LEN)) { LOG(4, "not a pool set header"); (void) os_close(fd); if (size < minsize) { ERR("file is not a poolset file and its size (%zu)" " is smaller than %zu", size, minsize); errno = EINVAL; return -1; } *setp = util_poolset_single(path, size, 0, ignore_sds); if (*setp == NULL) return -1; return 0; } ret = util_poolset_parse(setp, path, fd); if (ret) goto err; (*setp)->ignore_sds = ignore_sds || ((*setp)->options & OPTION_NOHDRS); #ifdef _WIN32 /* remote replication is not supported on Windows */ if ((*setp)->remote) { util_poolset_free(*setp); ERR("remote replication is not supported on Windows"); errno = ENOTSUP; ret = -1; goto err; } #endif /* _WIN32 */ err: oerrno = errno; (void) os_close(fd); errno = oerrno; return ret; } /* * util_poolset_check_header_options -- (internal) check if poolset options * match given flags */ static int util_poolset_check_header_options(struct pool_set *set, uint32_t incompat) { LOG(3, "set %p, incompat %#x", set, incompat); if (((set->options & OPTION_SINGLEHDR) == 0) != ((incompat & POOL_FEAT_SINGLEHDR) == 0)) { ERR( "poolset file options (%u) do not match incompat feature flags (%#x)", set->options, incompat); errno = EINVAL; return -1; } return 0; } /* * util_header_create -- create header of a single pool set file */ int util_header_create(struct pool_set *set, unsigned repidx, unsigned partidx, const struct pool_attr *attr, int overwrite) { LOG(3, "set %p repidx %u partidx %u attr %p overwrite %d", set, repidx, partidx, attr, overwrite); ASSERTne(attr, NULL); struct pool_replica *rep = set->replica[repidx]; /* opaque info lives at the beginning of mapped memory pool */ struct pool_hdr *hdrp = rep->part[partidx].hdr; /* check if the pool header is all zeros */ if (!util_is_zeroed(hdrp, sizeof(*hdrp)) && !overwrite) { ERR("Non-empty file detected"); errno = EEXIST; return -1; } /* create pool's header */ util_pool_attr2hdr(hdrp, attr); if (set->options & OPTION_SINGLEHDR) hdrp->features.incompat |= POOL_FEAT_SINGLEHDR; memcpy(hdrp->poolset_uuid, set->uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->uuid, PART(rep, partidx)->uuid, POOL_HDR_UUID_LEN); /* link parts */ if (set->options & OPTION_SINGLEHDR) { /* next/prev part point to part #0 */ ASSERTeq(partidx, 0); memcpy(hdrp->prev_part_uuid, PART(rep, 0)->uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->next_part_uuid, PART(rep, 0)->uuid, POOL_HDR_UUID_LEN); } else { memcpy(hdrp->prev_part_uuid, PARTP(rep, partidx)->uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->next_part_uuid, PARTN(rep, partidx)->uuid, POOL_HDR_UUID_LEN); } /* link replicas */ if (!util_is_zeroed(attr->prev_repl_uuid, POOL_HDR_UUID_LEN)) { memcpy(hdrp->prev_repl_uuid, attr->prev_repl_uuid, POOL_HDR_UUID_LEN); } else { memcpy(hdrp->prev_repl_uuid, PART(REPP(set, repidx), 0)->uuid, POOL_HDR_UUID_LEN); } if (!util_is_zeroed(attr->next_repl_uuid, POOL_HDR_UUID_LEN)) { memcpy(hdrp->next_repl_uuid, attr->next_repl_uuid, POOL_HDR_UUID_LEN); } else { memcpy(hdrp->next_repl_uuid, PART(REPN(set, repidx), 0)->uuid, POOL_HDR_UUID_LEN); } if (!rep->remote) { os_stat_t stbuf; if (os_fstat(rep->part[partidx].fd, &stbuf) != 0) { ERR("!fstat"); return -1; } ASSERT(stbuf.st_ctime); hdrp->crtime = (uint64_t)stbuf.st_ctime; } int arch_is_zeroed = util_is_zeroed(attr->arch_flags, POOL_HDR_ARCH_LEN); if (arch_is_zeroed) util_get_arch_flags(&hdrp->arch_flags); util_convert2le_hdr(hdrp); if (!arch_is_zeroed) { memcpy(&hdrp->arch_flags, attr->arch_flags, POOL_HDR_ARCH_LEN); } if (!set->ignore_sds && partidx == 0 && !rep->remote) { shutdown_state_init(&hdrp->sds, rep); for (unsigned p = 0; p < rep->nparts; p++) { if (shutdown_state_add_part(&hdrp->sds, PART(rep, p)->path, rep)) return -1; } shutdown_state_set_dirty(&hdrp->sds, rep); } util_checksum(hdrp, sizeof(*hdrp), &hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(hdrp)); /* store pool's header */ util_persist_auto(rep->is_pmem, hdrp, sizeof(*hdrp)); return 0; } /* * util_header_check -- (internal) validate header of a single pool set file */ static int util_header_check(struct pool_set *set, unsigned repidx, unsigned partidx, const struct pool_attr *attr) { LOG(3, "set %p repidx %u partidx %u attr %p", set, repidx, partidx, attr); ASSERTne(attr, NULL); struct pool_replica *rep = set->replica[repidx]; /* opaque info lives at the beginning of mapped memory pool */ struct pool_hdr *hdrp = rep->part[partidx].hdr; struct pool_hdr hdr; memcpy(&hdr, hdrp, sizeof(hdr)); /* local copy of a remote header does not need to be converted */ if (rep->remote == NULL) util_convert2h_hdr_nocheck(&hdr); /* to be valid, a header must have a major version of at least 1 */ if (hdr.major == 0) { ERR("invalid major version (0)"); errno = EINVAL; return -1; } /* check signature */ if (memcmp(hdr.signature, attr->signature, POOL_HDR_SIG_LEN)) { ERR("wrong pool type: \"%.8s\"", hdr.signature); errno = EINVAL; return -1; } /* check format version number */ if (hdr.major != attr->major) { ERR("pool version %d (library expects %d)", hdr.major, attr->major); if (hdr.major < attr->major) ERR( "Please run the pmdk-convert utility to upgrade the pool."); errno = EINVAL; return -1; } rep->part[partidx].rdonly = 0; int retval = util_feature_check(&hdr, attr->features); if (retval < 0) return -1; if (retval == 0) rep->part[partidx].rdonly = 1; if (rep->remote == NULL) { /* * and to be valid, the fields must checksum correctly * * NOTE: checksum validation is performed after format version * and feature check, because if POOL_FEAT_CKSUM_2K flag is set, * we want to report it as incompatible feature, rather than * invalid checksum. */ if (!util_checksum(&hdr, sizeof(hdr), &hdr.checksum, 0, POOL_HDR_CSUM_END_OFF(&hdr))) { ERR("invalid checksum of pool header"); errno = EINVAL; return -1; } LOG(3, "valid header, signature \"%.8s\"", hdr.signature); } if (util_check_arch_flags(&hdr.arch_flags)) { ERR("wrong architecture flags"); errno = EINVAL; return -1; } /* check pool set UUID */ if (memcmp(HDR(REP(set, 0), 0)->poolset_uuid, hdr.poolset_uuid, POOL_HDR_UUID_LEN)) { ERR("wrong pool set UUID"); errno = EINVAL; return -1; } /* check pool set linkage */ if (memcmp(HDRP(rep, partidx)->uuid, hdr.prev_part_uuid, POOL_HDR_UUID_LEN) || memcmp(HDRN(rep, partidx)->uuid, hdr.next_part_uuid, POOL_HDR_UUID_LEN)) { ERR("wrong part UUID"); errno = EINVAL; return -1; } /* check format version */ if (HDR(rep, 0)->major != hdrp->major) { ERR("incompatible pool format"); errno = EINVAL; return -1; } /* check compatibility features */ if (HDR(rep, 0)->features.compat != hdrp->features.compat || HDR(rep, 0)->features.incompat != hdrp->features.incompat || HDR(rep, 0)->features.ro_compat != hdrp->features.ro_compat) { ERR("incompatible feature flags"); errno = EINVAL; return -1; } /* check poolset options */ if (util_poolset_check_header_options(set, HDR(rep, 0)->features.incompat)) return -1; return 0; } /* * util_header_check_remote -- (internal) validate header of a remote * pool set file */ static int util_header_check_remote(struct pool_set *set, unsigned partidx) { LOG(3, "set %p partidx %u ", set, partidx); /* there is only one replica in remote poolset */ struct pool_replica *rep = set->replica[0]; /* opaque info lives at the beginning of mapped memory pool */ struct pool_hdr *hdrp = rep->part[partidx].hdr; struct pool_hdr hdr; if (util_is_zeroed(hdrp, sizeof(*hdrp))) { ERR("pool header zeroed"); errno = EINVAL; return -1; } memcpy(&hdr, hdrp, sizeof(hdr)); util_convert2h_hdr_nocheck(&hdr); /* valid header found */ if (memcmp(HDR(rep, 0)->signature, hdrp->signature, POOL_HDR_SIG_LEN)) { ERR("pool signature mismatch in part %d", partidx); errno = EINVAL; return -1; } /* check format version */ if (HDR(rep, 0)->major != hdrp->major) { ERR("pool version mismatch in part %d", partidx); errno = EINVAL; return -1; } /* check compatibility features */ if (HDR(rep, 0)->features.compat != hdrp->features.compat) { ERR("'may have' compatibility flags mismatch in part %d", partidx); errno = EINVAL; return -1; } if (HDR(rep, 0)->features.incompat != hdrp->features.incompat) { ERR("'must support' compatibility flags mismatch in part %d", partidx); errno = EINVAL; return -1; } if (HDR(rep, 0)->features.ro_compat != hdrp->features.ro_compat) { ERR("'force read-only' compatibility flags mismatch in part %d", partidx); errno = EINVAL; return -1; } /* * and to be valid, the fields must checksum correctly * * NOTE: checksum validation is performed after format version and * feature check, because if POOL_FEAT_CKSUM_2K flag is set, * we want to report it as incompatible feature, rather than invalid * checksum. */ if (!util_checksum(&hdr, sizeof(hdr), &hdr.checksum, 0, POOL_HDR_CSUM_END_OFF(&hdr))) { ERR("invalid checksum of pool header"); return -1; } LOG(3, "valid header, signature \"%.8s\"", hdr.signature); /* check pool set UUID */ if (memcmp(HDR(rep, 0)->poolset_uuid, hdrp->poolset_uuid, POOL_HDR_UUID_LEN)) { ERR("wrong pool set UUID in part %d", partidx); errno = EINVAL; return -1; } /* check previous replica UUID */ if (memcmp(HDR(rep, 0)->prev_repl_uuid, hdrp->prev_repl_uuid, POOL_HDR_UUID_LEN)) { ERR("wrong previous replica UUID in part %d", partidx); errno = EINVAL; return -1; } /* check next replica UUID */ if (memcmp(HDR(rep, 0)->next_repl_uuid, hdrp->next_repl_uuid, POOL_HDR_UUID_LEN)) { ERR("wrong next replica UUID in part %d", partidx); errno = EINVAL; return -1; } if (memcmp(&HDR(rep, 0)->arch_flags, &hdrp->arch_flags, sizeof(hdrp->arch_flags))) { ERR("wrong architecture flags"); errno = EINVAL; return -1; } /* check pool set linkage */ if (memcmp(HDRP(rep, partidx)->uuid, hdrp->prev_part_uuid, POOL_HDR_UUID_LEN) || memcmp(HDRN(rep, partidx)->uuid, hdrp->next_part_uuid, POOL_HDR_UUID_LEN)) { ERR("wrong part UUID in part %d", partidx); errno = EINVAL; return -1; } /* read shutdown state toggle from header */ set->ignore_sds |= IGNORE_SDS(HDR(rep, 0)); if (!set->ignore_sds && partidx == 0) { struct shutdown_state sds; shutdown_state_init(&sds, NULL); for (unsigned p = 0; p < rep->nparts; p++) { if (shutdown_state_add_part(&sds, PART(rep, p)->path, NULL)) return -1; } if (shutdown_state_check(&sds, &hdrp->sds, rep)) { errno = EINVAL; return -1; } shutdown_state_set_dirty(&hdrp->sds, rep); } rep->part[partidx].rdonly = 0; return 0; } /* * util_replica_set_is_pmem -- sets per-replica is_pmem flag * * The replica is PMEM if: * - all parts are on device dax, or * - all parts are mapped with MAP_SYNC. * * It's enough to check only first part because it's already verified * that either all or none parts are device dax or mapped with MAP_SYNC. */ static inline void util_replica_set_is_pmem(struct pool_replica *rep) { rep->is_pmem = rep->part[0].is_dev_dax || rep->part[0].map_sync || pmem_is_pmem(rep->part[0].addr, rep->resvsize); } /* * util_replica_map_local -- (internal) map memory pool for local replica */ static int util_replica_map_local(struct pool_set *set, unsigned repidx, int flags) { LOG(3, "set %p repidx %u flags %d", set, repidx, flags); /* * XXX: Like we reserve space for all parts in this replica when we map * the first part, we need to reserve the space for all replicas * upfront. It is not necessary that the replicas are contiguous but * that way we would not fragment the memory much. I think we should * leave this to MM, but let's have a note as per our collective minds. */ #ifndef _WIN32 int remaining_retries = 0; #else int remaining_retries = 10; #endif int retry_for_contiguous_addr; size_t mapsize; /* header size for all headers but the first one */ size_t hdrsize = (set->options & (OPTION_SINGLEHDR | OPTION_NOHDRS)) ? 0 : Mmap_align; void *addr; struct pool_replica *rep = set->replica[repidx]; ASSERTeq(rep->remote, NULL); ASSERTne(rep->part, NULL); do { retry_for_contiguous_addr = 0; mapsize = rep->part[0].filesize & ~(Mmap_align - 1); /* determine a hint address for mmap() */ addr = util_map_hint(rep->resvsize, 0); if (addr == MAP_FAILED) { LOG(1, "cannot find a contiguous region of given size"); return -1; } /* map the first part and reserve space for remaining parts */ if (util_map_part(&rep->part[0], addr, rep->resvsize, 0, flags, 0) != 0) { LOG(2, "pool mapping failed - replica #%u part #0", repidx); return -1; } VALGRIND_REGISTER_PMEM_MAPPING(rep->part[0].addr, rep->part[0].size); VALGRIND_REGISTER_PMEM_FILE(rep->part[0].fd, rep->part[0].addr, rep->part[0].size, 0); set->zeroed &= rep->part[0].created; addr = (char *)rep->part[0].addr + mapsize; /* * map the remaining parts of the usable pool space * (aligned to memory mapping granularity) */ for (unsigned p = 1; p < rep->nparts; p++) { /* map data part */ if (util_map_part(&rep->part[p], addr, 0, hdrsize, flags | MAP_FIXED, 0) != 0) { /* * if we can't map the part at the address we * asked for, unmap all the parts that are * mapped and remap at a different address. */ if ((errno == EINVAL) && (remaining_retries > 0)) { LOG(2, "usable space mapping failed - " "part #%d - retrying", p); retry_for_contiguous_addr = 1; remaining_retries--; util_unmap_parts(rep, 0, p - 1); /* release rest of the VA reserved */ ASSERTne(addr, NULL); ASSERTne(addr, MAP_FAILED); munmap(addr, rep->resvsize - mapsize); break; } LOG(2, "usable space mapping failed - part #%d", p); goto err; } VALGRIND_REGISTER_PMEM_FILE(rep->part[p].fd, rep->part[p].addr, rep->part[p].size, hdrsize); mapsize += rep->part[p].size; set->zeroed &= rep->part[p].created; addr = (char *)addr + rep->part[p].size; } } while (retry_for_contiguous_addr); /* * Initially part[0].size is the size of address space * reservation for all parts from given replica. After * mapping that space we need to overwrite part[0].size * with its actual size to be consistent - size for each * part should be the actual mapping size of this part * only - it simplifies future calculations. */ rep->part[0].size = rep->part[0].filesize & ~(Mmap_align - 1); if (util_replica_check_map_sync(set, repidx, 0)) goto err; util_replica_set_is_pmem(rep); if (Prefault_at_create) util_replica_force_page_allocation(rep); ASSERTeq(mapsize, rep->repsize); LOG(3, "replica #%u addr %p", repidx, rep->part[0].addr); return 0; err: LOG(4, "error clean up"); int oerrno = errno; if (mapsize < rep->repsize) { ASSERTne(rep->part[0].addr, NULL); ASSERTne(rep->part[0].addr, MAP_FAILED); munmap(rep->part[0].addr, rep->resvsize - mapsize); } for (unsigned p = 0; p < rep->nparts; p++) { util_unmap_part(&rep->part[p]); } errno = oerrno; return -1; } /* * util_replica_init_headers_local -- (internal) initialize pool headers */ static int util_replica_init_headers_local(struct pool_set *set, unsigned repidx, int flags, const struct pool_attr *attr) { LOG(3, "set %p repidx %u flags %d attr %p", set, repidx, flags, attr); struct pool_replica *rep = set->replica[repidx]; /* map all headers - don't care about the address */ for (unsigned p = 0; p < rep->nhdrs; p++) { if (util_map_hdr(&rep->part[p], flags, 0) != 0) { LOG(2, "header mapping failed - part #%d", p); goto err; } } /* create headers, set UUID's */ for (unsigned p = 0; p < rep->nhdrs; p++) { if (util_header_create(set, repidx, p, attr, 0) != 0) { LOG(2, "header creation failed - part #%d", p); goto err; } } /* unmap all headers */ for (unsigned p = 0; p < rep->nhdrs; p++) util_unmap_hdr(&rep->part[p]); return 0; err: LOG(4, "error clean up"); int oerrno = errno; for (unsigned p = 0; p < rep->nhdrs; p++) { util_unmap_hdr(&rep->part[p]); } errno = oerrno; return -1; } /* * util_replica_create_local -- (internal) create a new memory pool for local * replica */ static int util_replica_create_local(struct pool_set *set, unsigned repidx, int flags, const struct pool_attr *attr) { LOG(3, "set %p repidx %u flags %d attr %p", set, repidx, flags, attr); /* * the first replica has to be mapped prior to remote ones so if * a replica is already mapped skip mapping creation */ if (PART(REP(set, repidx), 0)->addr == NULL) { if (util_replica_map_local(set, repidx, flags) != 0) { LOG(2, "replica #%u map failed", repidx); return -1; } } if (attr == NULL) return 0; if (util_replica_init_headers_local(set, repidx, flags, attr) != 0) { LOG(2, "replica #%u headers initialization failed", repidx); return -1; } return 0; } /* * util_replica_create_remote -- (internal) create a new memory pool * for remote replica */ static int util_replica_create_remote(struct pool_set *set, unsigned repidx, int flags, const struct pool_attr *attr) { LOG(3, "set %p repidx %u flags %d attr %p", set, repidx, flags, attr); struct pool_replica *rep = set->replica[repidx]; ASSERTne(rep->remote, NULL); ASSERTne(rep->part, NULL); ASSERTeq(rep->nparts, 1); ASSERTeq(rep->nhdrs, 1); ASSERTne(attr, NULL); struct pool_set_part *part = rep->part; /* * A remote replica has one fake part of size equal twice pool header * size for storing pool header and pool descriptor. */ part->size = rep->repsize; ASSERT(IS_PAGE_ALIGNED(part->size)); part->remote_hdr = Zalloc(part->size + Pagesize); if (!part->remote_hdr) { ERR("!Zalloc"); return -1; } part->hdr = PAGE_ALIGN_UP(part->remote_hdr); part->addr = PAGE_ALIGN_UP(part->remote_hdr); part->hdrsize = POOL_HDR_SIZE; /* create header, set UUID's */ if (util_header_create(set, repidx, 0, attr, 0) != 0) { LOG(2, "header creation failed - part #0"); Free(part->remote_hdr); return -1; } LOG(3, "replica #%u addr %p", repidx, rep->part[0].addr); return 0; } /* * util_replica_close -- close a memory pool replica * * This function unmaps all mapped memory regions. */ int util_replica_close(struct pool_set *set, unsigned repidx) { LOG(3, "set %p repidx %u", set, repidx); struct pool_replica *rep = set->replica[repidx]; if (rep->remote == NULL) { struct pool_set_part *part = PART(rep, 0); if (!set->ignore_sds && part->addr != NULL && part->size != 0) { struct pool_hdr *hdr = part->addr; RANGE_RW(hdr, sizeof(*hdr), part->is_dev_dax); /* * deep drain will call msync on one page in each * part in replica to trigger WPQ flush. * This pages may have been marked as * undefined/inaccessible, but msyncing such memory * is not a bug, so as a workaround temporarily * disable error reporting. */ VALGRIND_DO_DISABLE_ERROR_REPORTING; util_replica_deep_drain(part->addr, rep->repsize, set, repidx); VALGRIND_DO_ENABLE_ERROR_REPORTING; shutdown_state_clear_dirty(&hdr->sds, rep); } for (unsigned p = 0; p < rep->nhdrs; p++) util_unmap_hdr(&rep->part[p]); rep->part[0].size = rep->resvsize; util_unmap_part(&rep->part[0]); } else { LOG(4, "freeing volatile header of remote replica #%u", repidx); Free(rep->part[0].remote_hdr); rep->part[0].remote_hdr = NULL; rep->part[0].hdr = NULL; rep->part[0].hdrsize = 0; rep->part[0].addr = NULL; rep->part[0].size = 0; } return 0; } /* * util_poolset_append_new_part -- (internal) creates a new part in each replica * of the poolset */ static int util_poolset_append_new_part(struct pool_set *set, size_t size) { LOG(3, "set %p size %zu", set, size); if (!set->directory_based) return -1; struct pool_set_directory *d; size_t directory_id; char *path; size_t path_len; unsigned r; for (r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; directory_id = set->next_directory_id % VEC_SIZE(&rep->directory); d = VEC_GET(&rep->directory, directory_id); path_len = strlen(d->path) + PMEM_FILE_MAX_LEN; if ((path = Malloc(path_len)) == NULL) { ERR("!Malloc"); goto err_part_init; } snprintf(path, path_len, "%s" OS_DIR_SEP_STR "%0*u%s", d->path, PMEM_FILE_PADDING, set->next_id, PMEM_EXT); if (util_replica_add_part(&set->replica[r], path, size) != 0) FATAL("cannot add a new part to the replica info"); } set->next_directory_id += 1; set->next_id += 1; util_poolset_set_size(set); return 0; err_part_init: /* for each replica 0..r-1 remove the last part */ for (unsigned rn = 0; rn < r; ++rn) { struct pool_replica *rep = set->replica[rn]; unsigned pidx = rep->nparts - 1; Free((void *)(rep->part[pidx].path)); rep->part[pidx].path = NULL; rep->nparts--; } return -1; } /* * util_pool_extend -- extends the poolset by the provided size */ void * util_pool_extend(struct pool_set *set, size_t *size, size_t minpartsize) { LOG(3, "set %p size %zu minpartsize %zu", set, *size, minpartsize); if (*size == 0) { ERR("cannot extend pool by 0 bytes"); return NULL; } if ((set->options & OPTION_SINGLEHDR) == 0) { ERR( "extending the pool by appending parts with headers is not supported!"); return NULL; } if (set->poolsize + *size > set->resvsize) { *size = set->resvsize - set->poolsize; if (*size < minpartsize) { ERR("exceeded reservation size"); return NULL; } LOG(4, "extend size adjusted to not exceed reservation size"); } size_t old_poolsize = set->poolsize; if (util_poolset_append_new_part(set, *size) != 0) { ERR("unable to append a new part to the pool"); return NULL; } size_t hdrsize = (set->options & OPTION_SINGLEHDR) ? 0 : Mmap_align; void *addr = NULL; void *addr_base = NULL; unsigned r; for (r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; unsigned pidx = rep->nparts - 1; struct pool_set_part *p = &rep->part[pidx]; if (util_part_open(p, 0, 1 /* create */) != 0) { ERR("cannot open the new part"); goto err; } addr = (char *)rep->part[0].addr + old_poolsize; if (addr_base == NULL) addr_base = addr; if (util_map_part(p, addr, 0, hdrsize, MAP_SHARED | MAP_FIXED, 0) != 0) { ERR("cannot map the new part"); goto err; } /* * new part must be mapped the same way as all the rest * within a replica */ if (p->map_sync != rep->part[0].map_sync) { if (p->map_sync) ERR("new part cannot be mapped with MAP_SYNC"); else ERR("new part mapped with MAP_SYNC"); goto err; } } /* XXX: mode should be the same as for pmemxxx_create() */ if (util_poolset_chmod(set, S_IWUSR | S_IRUSR)) goto err; util_poolset_fdclose(set); return addr_base; err: for (unsigned rn = 0; rn <= r; ++rn) { struct pool_replica *rep = set->replica[r]; unsigned pidx = rep->nparts - 1; struct pool_set_part *p = &rep->part[pidx]; rep->nparts--; if (p->fd != 0) (void) os_close(p->fd); if (p->created) os_unlink(p->path); Free((void *)p->path); p->path = NULL; } util_poolset_set_size(set); return NULL; } /* * util_print_bad_files_cb -- (internal) callback printing names of pool files * containing bad blocks */ static int util_print_bad_files_cb(struct part_file *pf, void *arg) { if (!pf->is_remote && pf->part && pf->part->has_bad_blocks) ERR("file contains bad blocks -- '%s'", pf->part->path); return 0; } /* * util_pool_create_uuids -- create a new memory pool (set or a single file) * with given uuids * * On success returns 0 and a pointer to a newly allocated structure * containing the info of all the parts of the pool set and replicas. */ int util_pool_create_uuids(struct pool_set **setp, const char *path, size_t poolsize, size_t minsize, size_t minpartsize, const struct pool_attr *attr, unsigned *nlanes, int can_have_rep, int remote) { LOG(3, "setp %p path %s poolsize %zu minsize %zu minpartsize %zu " "pattr %p nlanes %p can_have_rep %i remote %i", setp, path, poolsize, minsize, minpartsize, attr, nlanes, can_have_rep, remote); /* attributes cannot be NULL for local replicas */ ASSERT(remote || attr != NULL); int flags = MAP_SHARED; int oerrno; int exists = util_file_exists(path); if (exists < 0) return -1; /* check if file exists */ if (poolsize > 0 && exists) { ERR("file %s already exists", path); errno = EEXIST; return -1; } int ret = util_poolset_create_set(setp, path, poolsize, minsize, IGNORE_SDS(attr)); if (ret < 0) { LOG(2, "cannot create pool set -- '%s'", path); return -1; } struct pool_set *set = *setp; ASSERT(set->nreplicas > 0); if (!remote && (set->options & OPTION_NOHDRS)) { ERR( "the NOHDRS poolset option is not supported for local poolsets"); errno = EINVAL; goto err_poolset_free; } if ((attr == NULL) != ((set->options & OPTION_NOHDRS) != 0)) { ERR( "pool attributes are not supported for poolsets without headers (with the NOHDRS option)"); errno = EINVAL; goto err_poolset_free; } if (set->directory_based && ((set->options & OPTION_SINGLEHDR) == 0)) { ERR( "directory based pools are not supported for poolsets with headers (without SINGLEHDR option)"); errno = EINVAL; goto err_poolset_free; } if (set->resvsize < minsize) { ERR("reservation pool size %zu smaller than %zu", set->resvsize, minsize); errno = EINVAL; goto err_poolset_free; } if (set->directory_based && set->poolsize == 0 && util_poolset_append_new_part(set, minsize) != 0) { ERR("cannot create a new part in provided directories"); goto err_poolset_free; } if (attr != NULL && (attr->features.compat & POOL_FEAT_CHECK_BAD_BLOCKS)) { int bbs = badblocks_check_poolset(set, 1 /* create */); if (bbs < 0) { LOG(1, "failed to check pool set for bad blocks -- '%s'", path); goto err_poolset_free; } if (bbs > 0) { util_poolset_foreach_part_struct(set, util_print_bad_files_cb, NULL); ERR( "pool set contains bad blocks and cannot be created, run 'pmempool create --clear-bad-blocks' utility to clear bad blocks and create a pool"); errno = EIO; goto err_poolset_free; } } if (set->poolsize < minsize) { ERR("net pool size %zu smaller than %zu", set->poolsize, minsize); errno = EINVAL; goto err_poolset_free; } if (remote) { /* it is a remote replica - it cannot have replicas */ if (set->nreplicas > 1) { LOG(2, "remote pool set cannot have replicas"); errno = EINVAL; goto err_poolset_free; } /* check if poolset options match remote pool attributes */ if (attr != NULL && ((set->options & OPTION_SINGLEHDR) == 0) != ((attr->features.incompat & POOL_FEAT_SINGLEHDR) == 0)) { ERR( "pool incompat feature flags and remote poolset options do not match"); errno = EINVAL; goto err_poolset_free; } } if (!can_have_rep && set->nreplicas > 1) { ERR("replication not supported"); errno = ENOTSUP; goto err_poolset_free; } if (set->remote && util_remote_load()) { ERR( "the pool set requires a remote replica, but the '%s' library cannot be loaded", LIBRARY_REMOTE); goto err_poolset_free; } set->zeroed = 1; if (attr != NULL) { if (!util_is_zeroed(attr->poolset_uuid, POOL_HDR_UUID_LEN)) { memcpy(set->uuid, attr->poolset_uuid, POOL_HDR_UUID_LEN); } else { /* generate pool set UUID */ ret = util_uuid_generate(set->uuid); if (ret < 0) { LOG(2, "cannot generate pool set UUID"); goto err_poolset; } } /* generate UUID's for all the parts */ for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; for (unsigned i = 0; i < rep->nhdrs; i++) { ret = util_uuid_generate(rep->part[i].uuid); if (ret < 0) { LOG(2, "cannot generate pool set part UUID"); goto err_poolset; } } } /* overwrite UUID of the first part if given */ if (!util_is_zeroed(attr->first_part_uuid, POOL_HDR_UUID_LEN)) { memcpy(set->replica[0]->part[0].uuid, attr->first_part_uuid, POOL_HDR_UUID_LEN); } } ret = util_poolset_files_local(set, minpartsize, 1); if (ret != 0) goto err_poolset; /* map first local replica - it has to exist prior to remote ones */ ret = util_replica_map_local(set, 0, flags); if (ret != 0) goto err_poolset; /* prepare remote replicas first */ if (set->remote) { for (unsigned r = 0; r < set->nreplicas; r++) { if (REP(set, r)->remote == NULL) { continue; } if (util_replica_create_remote(set, r, flags, attr) != 0) { LOG(2, "replica #%u creation failed", r); goto err_create; } } ret = util_poolset_files_remote(set, minsize, nlanes, 1 /* create */); if (ret != 0) goto err_create; } /* prepare local replicas */ if (remote) { if (util_replica_create_local(set, 0, flags, attr) != 0) { LOG(2, "replica #0 creation failed"); goto err_create; } } else { for (unsigned r = 0; r < set->nreplicas; r++) { if (REP(set, r)->remote != NULL) { continue; } if (util_replica_create_local(set, r, flags, attr) != 0) { LOG(2, "replica #%u creation failed", r); goto err_create; } } } return 0; err_create: oerrno = errno; for (unsigned r = 0; r < set->nreplicas; r++) util_replica_close(set, r); errno = oerrno; err_poolset: oerrno = errno; util_poolset_close(set, DELETE_CREATED_PARTS); errno = oerrno; return -1; err_poolset_free: oerrno = errno; util_poolset_free(set); errno = oerrno; return -1; } /* * util_pool_create -- create a new memory pool (set or a single file) * * On success returns 0 and a pointer to a newly allocated structure * containing the info of all the parts of the pool set and replicas. */ int util_pool_create(struct pool_set **setp, const char *path, size_t poolsize, size_t minsize, size_t minpartsize, const struct pool_attr *attr, unsigned *nlanes, int can_have_rep) { LOG(3, "setp %p path %s poolsize %zu minsize %zu minpartsize %zu " "attr %p nlanes %p can_have_rep %i", setp, path, poolsize, minsize, minpartsize, attr, nlanes, can_have_rep); return util_pool_create_uuids(setp, path, poolsize, minsize, minpartsize, attr, nlanes, can_have_rep, POOL_LOCAL); } /* * util_replica_open_local -- (internal) open a memory pool local replica */ static int util_replica_open_local(struct pool_set *set, unsigned repidx, int flags) { LOG(3, "set %p repidx %u flags %d", set, repidx, flags); int remaining_retries = 10; int retry_for_contiguous_addr; size_t mapsize; size_t hdrsize = (set->options & (OPTION_SINGLEHDR | OPTION_NOHDRS)) ? 0 : Mmap_align; struct pool_replica *rep = set->replica[repidx]; void *addr = NULL; do { retry_for_contiguous_addr = 0; /* determine a hint address for mmap() if not specified */ if (addr == NULL) addr = util_map_hint(rep->resvsize, 0); if (addr == MAP_FAILED) { LOG(1, "cannot find a contiguous region of given size"); return -1; } mapsize = rep->part[0].filesize & ~(Mmap_align - 1); /* map the first part and reserve space for remaining parts */ if (util_map_part(&rep->part[0], addr, rep->resvsize, 0, flags, 0) != 0) { LOG(2, "pool mapping failed - replica #%u part #0", repidx); return -1; } VALGRIND_REGISTER_PMEM_MAPPING(rep->part[0].addr, rep->resvsize); VALGRIND_REGISTER_PMEM_FILE(rep->part[0].fd, rep->part[0].addr, rep->resvsize, 0); /* map all headers - don't care about the address */ for (unsigned p = 0; p < rep->nhdrs; p++) { if (util_map_hdr(&rep->part[p], flags, 0) != 0) { LOG(2, "header mapping failed - part #%d", p); goto err; } } addr = (char *)rep->part[0].addr + mapsize; /* * map the remaining parts of the usable pool space * (aligned to memory mapping granularity) */ for (unsigned p = 1; p < rep->nparts; p++) { struct pool_set_part *part = &rep->part[p]; size_t targetsize = mapsize + ALIGN_DOWN(part->filesize - hdrsize, part->alignment); if (targetsize > rep->resvsize) { ERR( "pool mapping failed - address space reservation too small"); errno = EINVAL; goto err; } /* map data part */ if (util_map_part(part, addr, 0, hdrsize, flags | MAP_FIXED, 0) != 0) { /* * if we can't map the part at the address we * asked for, unmap all the parts that are * mapped and remap at a different address. */ if ((errno == EINVAL) && (remaining_retries > 0)) { LOG(2, "usable space mapping failed - " "part #%d - retrying", p); retry_for_contiguous_addr = 1; remaining_retries--; util_unmap_parts(rep, 0, p - 1); /* release rest of the VA reserved */ munmap(rep->part[0].addr, rep->resvsize); break; } LOG(2, "usable space mapping failed - part #%d", p); goto err; } VALGRIND_REGISTER_PMEM_FILE(part->fd, part->addr, part->size, hdrsize); mapsize += part->size; addr = (char *)addr + part->size; } } while (retry_for_contiguous_addr); /* * Initially part[0].size is the size of address space * reservation for all parts from given replica. After * mapping that space we need to overwrite part[0].size * with its actual size to be consistent - size for each * part should be the actual mapping size of this part * only - it simplifies future calculations. */ rep->part[0].size = rep->part[0].filesize & ~(Mmap_align - 1); if (util_replica_check_map_sync(set, repidx, 1)) goto err; util_replica_set_is_pmem(rep); if (Prefault_at_open) util_replica_force_page_allocation(rep); ASSERTeq(mapsize, rep->repsize); /* calculate pool size - choose the smallest replica size */ if (rep->repsize < set->poolsize) set->poolsize = rep->repsize; LOG(3, "replica addr %p", rep->part[0].addr); return 0; err: LOG(4, "error clean up"); int oerrno = errno; if (mapsize < rep->repsize) { ASSERTne(rep->part[0].addr, NULL); ASSERTne(rep->part[0].addr, MAP_FAILED); munmap(rep->part[0].addr, rep->resvsize - mapsize); } for (unsigned p = 0; p < rep->nhdrs; p++) util_unmap_hdr(&rep->part[p]); for (unsigned p = 0; p < rep->nparts; p++) util_unmap_part(&rep->part[p]); errno = oerrno; return -1; } /* * util_replica_open_remote -- open a memory pool for remote replica */ int util_replica_open_remote(struct pool_set *set, unsigned repidx, int flags) { LOG(3, "set %p repidx %u flags %d", set, repidx, flags); struct pool_replica *rep = set->replica[repidx]; ASSERTne(rep->remote, NULL); ASSERTne(rep->part, NULL); ASSERTeq(rep->nparts, 1); ASSERTeq(rep->nhdrs, 1); struct pool_set_part *part = rep->part; part->size = rep->repsize; ASSERT(IS_PAGE_ALIGNED(part->size)); part->remote_hdr = Zalloc(part->size + Pagesize); if (!part->remote_hdr) { ERR("!Zalloc"); return -1; } part->hdr = PAGE_ALIGN_UP(part->remote_hdr); part->addr = PAGE_ALIGN_UP(part->remote_hdr); part->hdrsize = POOL_HDR_SIZE; LOG(3, "replica #%u addr %p", repidx, rep->part[0].addr); return 0; } /* * util_replica_open -- open a memory pool replica */ int util_replica_open(struct pool_set *set, unsigned repidx, int flags) { LOG(3, "set %p repidx %u flags %d", set, repidx, flags); if (set->replica[repidx]->remote) return util_replica_open_remote(set, repidx, flags); return util_replica_open_local(set, repidx, flags); } /* * util_replica_set_attr -- overwrite existing replica attributes */ int util_replica_set_attr(struct pool_replica *rep, const struct rpmem_pool_attr *rattr) { LOG(3, "rep %p, rattr %p", rep, rattr); ASSERT(rattr != NULL || rep->nhdrs == 0); if (rattr != NULL && rep->nhdrs == 0) { ERR( "cannot set pool attributes for a replica without headers (with the NOHDRS option)"); errno = EINVAL; return -1; } int flags = MAP_SHARED; /* map all headers - don't care about the address */ for (unsigned p = 0; p < rep->nparts; p++) { if (util_map_hdr(&rep->part[p], flags, 0) != 0) { LOG(2, "header mapping failed - part #%d", p); goto err; } } for (unsigned p = 0; p < rep->nhdrs; p++) { ASSERTne(rattr, NULL); struct pool_hdr *hdrp = HDR(rep, p); ASSERTne(hdrp, NULL); util_convert2h_hdr_nocheck(hdrp); util_set_rpmem_attr(hdrp, rattr); if (hdrp == HDR(rep, 0)) memcpy(hdrp->uuid, rattr->uuid, POOL_HDR_UUID_LEN); if (hdrp == HDRP(rep, 0)) memcpy(hdrp->next_part_uuid, rattr->uuid, POOL_HDR_UUID_LEN); if (hdrp == HDRN(rep, 0)) memcpy(hdrp->prev_part_uuid, rattr->uuid, POOL_HDR_UUID_LEN); util_convert2le_hdr(hdrp); util_checksum(hdrp, sizeof(*hdrp), &hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(hdrp)); /* store pool's header */ util_persist_auto(rep->is_pmem, hdrp, sizeof(*hdrp)); } /* unmap all headers */ for (unsigned p = 0; p < rep->nhdrs; p++) util_unmap_hdr(&rep->part[p]); return 0; err: for (unsigned p = 0; p < rep->nhdrs; p++) { util_unmap_hdr(&rep->part[p]); } return -1; } /* * util_get_attr_from_header -- get pool attributes from a pool header */ void util_pool_hdr2attr(struct pool_attr *attr, struct pool_hdr *hdr) { LOG(3, "attr %p, hdr %p", attr, hdr); ASSERTne(attr, NULL); ASSERTne(hdr, NULL); memset(attr, 0, sizeof(*attr)); memcpy(attr->signature, hdr->signature, POOL_HDR_SIG_LEN); attr->major = hdr->major; attr->features.compat = hdr->features.compat; attr->features.incompat = hdr->features.incompat; attr->features.ro_compat = hdr->features.ro_compat; memcpy(attr->poolset_uuid, hdr->poolset_uuid, POOL_HDR_UUID_LEN); } /* * util_copy_attr_to_header -- copy pool attributes into pool header */ void util_pool_attr2hdr(struct pool_hdr *hdr, const struct pool_attr *attr) { LOG(3, "hdr %p, attr %p", hdr, attr); ASSERTne(hdr, NULL); ASSERTne(attr, NULL); memcpy(hdr->signature, attr->signature, POOL_HDR_SIG_LEN); hdr->major = attr->major; hdr->features.compat = attr->features.compat; hdr->features.incompat = attr->features.incompat; hdr->features.ro_compat = attr->features.ro_compat; } /* * util_unmap_all_hdrs -- unmap all pool set headers */ static void util_unmap_all_hdrs(struct pool_set *set) { LOG(3, "set %p", set); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; if (rep->remote == NULL) { for (unsigned p = 0; p < rep->nhdrs; p++) util_unmap_hdr(&rep->part[p]); } else { /* * hdr & hdrsize were set only for util_header_check(), * they will not be used any more. The memory will be * freed by util_replica_close() */ rep->part[0].hdr = NULL; rep->part[0].hdrsize = 0; } } } /* * util_replica_check -- check headers, check UUID's, check replicas linkage */ static int util_replica_check(struct pool_set *set, const struct pool_attr *attr) { LOG(3, "set %p attr %p", set, attr); /* read shutdown state toggle from header */ set->ignore_sds |= IGNORE_SDS(HDR(REP(set, 0), 0)); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; for (unsigned p = 0; p < rep->nhdrs; p++) { if (util_header_check(set, r, p, attr) != 0) { LOG(2, "header check failed - part #%d", p); return -1; } set->rdonly |= rep->part[p].rdonly; } if (memcmp(HDR(REPP(set, r), 0)->uuid, HDR(REP(set, r), 0)->prev_repl_uuid, POOL_HDR_UUID_LEN) || memcmp(HDR(REPN(set, r), 0)->uuid, HDR(REP(set, r), 0)->next_repl_uuid, POOL_HDR_UUID_LEN)) { ERR("wrong replica UUID"); errno = EINVAL; return -1; } if (!set->ignore_sds && !rep->remote && rep->nhdrs) { struct shutdown_state sds; shutdown_state_init(&sds, NULL); for (unsigned p = 0; p < rep->nparts; p++) { if (shutdown_state_add_part(&sds, PART(rep, p)->path, NULL)) return -1; } ASSERTne(rep->nhdrs, 0); ASSERTne(rep->nparts, 0); if (shutdown_state_check(&sds, &HDR(rep, 0)->sds, rep)) { LOG(2, "ADR failure detected"); errno = EINVAL; return -1; } shutdown_state_set_dirty(&HDR(rep, 0)->sds, rep); } } return 0; } /* * util_pool_has_device_dax -- (internal) check if poolset has any device dax */ int util_pool_has_device_dax(struct pool_set *set) { for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); /* either all the parts must be Device DAX or none */ if (PART(rep, 0)->is_dev_dax) return 1; } return 0; } /* * util_pool_open_nocheck -- open a memory pool (set or a single file) * * This function opens a pool set without checking the header values. */ int util_pool_open_nocheck(struct pool_set *set, unsigned flags) { LOG(3, "set %p flags 0x%x", set, flags); int cow = flags & POOL_OPEN_COW; if (cow && util_pool_has_device_dax(set)) { ERR("device dax cannot be mapped privately"); errno = ENOTSUP; return -1; } int mmap_flags = cow ? MAP_PRIVATE|MAP_NORESERVE : MAP_SHARED; int oerrno; ASSERTne(set, NULL); ASSERT(set->nreplicas > 0); if (flags & POOL_OPEN_CHECK_BAD_BLOCKS) { /* check if any bad block recovery file exists */ int bfe = badblocks_recovery_file_exists(set); if (bfe > 0) { ERR( "error: a bad block recovery file exists, run 'pmempool sync --bad-blocks' utility to try to recover the pool"); errno = EINVAL; return -1; } if (bfe < 0) { LOG(1, "an error occurred when checking whether recovery file exists."); return -1; } int bbs = badblocks_check_poolset(set, 0 /* not create */); if (bbs < 0) { LOG(1, "failed to check pool set for bad blocks"); return -1; } if (bbs > 0) { if (flags & POOL_OPEN_IGNORE_BAD_BLOCKS) { LOG(1, "WARNING: pool set contains bad blocks, ignoring"); } else { ERR( "pool set contains bad blocks and cannot be opened, run 'pmempool sync --bad-blocks' utility to try to recover the pool"); errno = EIO; return -1; } } } if (set->remote && util_remote_load()) { ERR("the pool set requires a remote replica, " "but the '%s' library cannot be loaded", LIBRARY_REMOTE); return -1; } int ret = util_poolset_files_local(set, 0 /* minpartsize */, 0); if (ret != 0) goto err_poolset; set->rdonly = 0; for (unsigned r = 0; r < set->nreplicas; r++) { if (util_replica_open(set, r, mmap_flags) != 0) { LOG(2, "replica #%u open failed", r); goto err_replica; } } if (set->remote) { ret = util_poolset_files_remote(set, 0, NULL, 0); if (ret != 0) goto err_replica; } util_unmap_all_hdrs(set); return 0; err_replica: LOG(4, "error clean up"); oerrno = errno; for (unsigned r = 0; r < set->nreplicas; r++) util_replica_close(set, r); errno = oerrno; err_poolset: oerrno = errno; util_poolset_close(set, DO_NOT_DELETE_PARTS); errno = oerrno; return -1; } /* * util_read_compat_features -- (internal) read compat features from the header */ static int util_read_compat_features(struct pool_set *set, uint32_t *compat_features) { LOG(3, "set %p pcompat_features %p", set, compat_features); *compat_features = 0; for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; if (rep->remote) continue; for (unsigned p = 0; p < rep->nparts; p++) { struct pool_set_part *part = &rep->part[p]; if (util_part_open(part, 0, 0 /* create */)) { LOG(1, "!cannot open the part -- \"%s\"", part->path); /* try to open the next part */ continue; } if (util_map_hdr(part, MAP_SHARED, 0) != 0) { LOG(1, "header mapping failed -- \"%s\"", part->path); util_part_fdclose(part); return -1; } struct pool_hdr *hdrp = part->hdr; *compat_features = hdrp->features.compat; util_unmap_hdr(part); util_part_fdclose(part); /* exit on the first successfully opened part */ return 0; } } return 0; } /* * unlink_remote_replicas -- removes remote replicas from poolset * * It is necessary when COW flag is set because remote replicas * cannot be mapped privately */ static int unlink_remote_replicas(struct pool_set *set) { unsigned i = 0; while (i < set->nreplicas) { if (set->replica[i]->remote == NULL) { i++; continue; } util_replica_close(set, i); int ret = util_replica_close_remote(set->replica[i], i, DO_NOT_DELETE_PARTS); if (ret != 0) return ret; size_t size = sizeof(set->replica[i]) * (set->nreplicas - i - 1); memmove(&set->replica[i], &set->replica[i + 1], size); set->nreplicas--; } set->remote = 0; return 0; } /* * util_pool_open -- open a memory pool (set or a single file) * * This routine does all the work, but takes a rdonly flag so internal * calls can map a read-only pool if required. */ int util_pool_open(struct pool_set **setp, const char *path, size_t minpartsize, const struct pool_attr *attr, unsigned *nlanes, void *addr, unsigned flags) { LOG(3, "setp %p path %s minpartsize %zu attr %p nlanes %p " "addr %p flags 0x%x ", setp, path, minpartsize, attr, nlanes, addr, flags); int cow = flags & POOL_OPEN_COW; int mmap_flags = cow ? MAP_PRIVATE|MAP_NORESERVE : MAP_SHARED; int oerrno; /* do not check minsize */ int ret = util_poolset_create_set(setp, path, 0, 0, flags & POOL_OPEN_IGNORE_SDS); if (ret < 0) { LOG(2, "cannot open pool set -- '%s'", path); return -1; } if ((*setp)->replica[0]->nparts == 0) { errno = ENOENT; ERR("!no parts in replicas"); goto err_poolset_free; } if (cow && (*setp)->replica[0]->part[0].is_dev_dax) { ERR("device dax cannot be mapped privately"); errno = ENOTSUP; goto err_poolset_free; } struct pool_set *set = *setp; ASSERT(set->nreplicas > 0); uint32_t compat_features; if (util_read_compat_features(set, &compat_features)) { LOG(1, "reading compat features failed"); goto err_poolset_free; } if (compat_features & POOL_FEAT_CHECK_BAD_BLOCKS) { /* check if any bad block recovery file exists */ int bfe = badblocks_recovery_file_exists(set); if (bfe > 0) { ERR( "error: a bad block recovery file exists, run 'pmempool sync --bad-blocks' utility to try to recover the pool"); errno = EINVAL; goto err_poolset_free; } if (bfe < 0) { LOG(1, "an error occurred when checking whether recovery file exists."); goto err_poolset_free; } int bbs = badblocks_check_poolset(set, 0 /* not create */); if (bbs < 0) { LOG(1, "failed to check pool set for bad blocks -- '%s'", path); goto err_poolset_free; } if (bbs > 0) { if (flags & POOL_OPEN_IGNORE_BAD_BLOCKS) { LOG(1, "WARNING: pool set contains bad blocks, ignoring -- '%s'", path); } else { ERR( "pool set contains bad blocks and cannot be opened, run 'pmempool sync --bad-blocks' utility to try to recover the pool -- '%s'", path); errno = EIO; goto err_poolset_free; } } } if (set->remote && util_remote_load()) { ERR( "the pool set requires a remote replica, but the '%s' library cannot be loaded", LIBRARY_REMOTE); goto err_poolset_free; } ret = util_poolset_files_local(set, minpartsize, 0); if (ret != 0) goto err_poolset; for (unsigned r = 0; r < set->nreplicas; r++) { if (util_replica_open(set, r, mmap_flags) != 0) { LOG(2, "replica #%u open failed", r); goto err_replica; } } if (set->remote) { /* do not check minsize */ ret = util_poolset_files_remote(set, 0, nlanes, 0); if (ret != 0) goto err_replica; } /* check headers, check UUID's, check replicas linkage */ if (attr != NULL && util_replica_check(set, attr)) goto err_replica; /* unmap all headers */ util_unmap_all_hdrs(set); /* remove all remote replicas from poolset when cow */ if (cow && set->remote) { ret = unlink_remote_replicas(set); if (ret != 0) goto err_replica; } return 0; err_replica: LOG(4, "error clean up"); oerrno = errno; for (unsigned r = 0; r < set->nreplicas; r++) util_replica_close(set, r); errno = oerrno; err_poolset: oerrno = errno; util_poolset_close(set, DO_NOT_DELETE_PARTS); errno = oerrno; return -1; err_poolset_free: oerrno = errno; util_poolset_free(*setp); errno = oerrno; return -1; } /* * util_pool_open_remote -- open a remote pool set file * * This routine does all the work, but takes a rdonly flag so internal * calls can map a read-only pool if required. */ int util_pool_open_remote(struct pool_set **setp, const char *path, int cow, size_t minpartsize, struct rpmem_pool_attr *rattr) { LOG(3, "setp %p path %s cow %d minpartsize %zu rattr %p", setp, path, cow, minpartsize, rattr); int flags = cow ? MAP_PRIVATE|MAP_NORESERVE : MAP_SHARED; int oerrno; /* do not check minsize */ int ret = util_poolset_create_set(setp, path, 0, 0, 0); if (ret < 0) { LOG(2, "cannot open pool set -- '%s'", path); return -1; } if (cow && (*setp)->replica[0]->part[0].is_dev_dax) { ERR("device dax cannot be mapped privately"); errno = ENOTSUP; return -1; } struct pool_set *set = *setp; if (set->nreplicas > 1) { LOG(2, "remote pool set cannot have replicas"); goto err_poolset; } uint32_t compat_features; if (util_read_compat_features(set, &compat_features)) { LOG(1, "reading compat features failed"); goto err_poolset; } if (compat_features & POOL_FEAT_CHECK_BAD_BLOCKS) { /* check if there are any bad blocks */ int bbs = badblocks_check_poolset(set, 0 /* not create */); if (bbs < 0) { LOG(1, "failed to check the remote replica for bad blocks -- '%s'", path); goto err_poolset; } if (bbs > 0) { ERR( "remote replica contains bad blocks and cannot be opened, run 'pmempool sync --bad-blocks' utility to recreate it -- '%s'", path); errno = EIO; goto err_poolset; } } ret = util_poolset_files_local(set, minpartsize, 0); if (ret != 0) goto err_poolset; if (util_replica_open(set, 0, flags) != 0) { LOG(2, "replica open failed"); goto err_replica; } struct pool_replica *rep = set->replica[0]; set->rdonly |= rep->part[0].rdonly; /* check headers, check UUID's, check replicas linkage */ for (unsigned p = 0; p < rep->nhdrs; p++) { if (util_header_check_remote(set, p) != 0) { LOG(2, "header check failed - part #%d", p); goto err_replica; } set->rdonly |= rep->part[p].rdonly; } if (rep->nhdrs > 0) { /* header exists, copy pool attributes */ struct pool_hdr *hdr = rep->part[0].hdr; util_get_rpmem_attr(rattr, hdr); } else { /* header does not exist, zero pool attributes */ memset(rattr, 0, sizeof(*rattr)); } /* unmap all headers */ for (unsigned p = 0; p < rep->nhdrs; p++) util_unmap_hdr(&rep->part[p]); return 0; err_replica: LOG(4, "error clean up"); oerrno = errno; util_replica_close(set, 0); errno = oerrno; err_poolset: oerrno = errno; util_poolset_close(set, DO_NOT_DELETE_PARTS); errno = oerrno; return -1; } /* * util_is_poolset_file -- check if specified file is a poolset file * * Return value: * -1 - error * 0 - not a poolset * 1 - is a poolset */ int util_is_poolset_file(const char *path) { enum file_type type = util_file_get_type(path); if (type < 0) return -1; if (type == TYPE_DEVDAX) return 0; int fd = util_file_open(path, NULL, 0, O_RDONLY); if (fd < 0) return -1; int ret = 0; ssize_t sret; char signature[POOLSET_HDR_SIG_LEN]; size_t rd = 0; do { sret = util_read(fd, &signature[rd], sizeof(signature) - rd); if (sret > 0) rd += (size_t)sret; } while (sret > 0); if (sret < 0) { ERR("!read"); ret = -1; goto out; } else if (rd != sizeof(signature)) { ret = 0; goto out; } if (memcmp(signature, POOLSET_HDR_SIG, POOLSET_HDR_SIG_LEN) == 0) ret = 1; out: os_close(fd); return ret; } /* * util_poolset_foreach_part_struct -- walk through all poolset file parts * of the given set * * Stops processing if callback returns non-zero value. * The value returned by callback is returned to the caller. */ int util_poolset_foreach_part_struct(struct pool_set *set, int (*callback)(struct part_file *pf, void *arg), void *arg) { LOG(3, "set %p callback %p arg %p", set, callback, arg); ASSERTne(callback, NULL); int ret; for (unsigned r = 0; r < set->nreplicas; r++) { struct part_file cbdata; if (set->replica[r]->remote) { cbdata.is_remote = 1; cbdata.remote = set->replica[r]->remote; cbdata.part = NULL; ret = (*callback)(&cbdata, arg); if (ret) return ret; } else { cbdata.is_remote = 0; cbdata.remote = NULL; for (unsigned p = 0; p < set->replica[r]->nparts; p++) { cbdata.part = &set->replica[r]->part[p]; ret = (*callback)(&cbdata, arg); if (ret) return ret; } } } return 0; } /* * util_poolset_foreach_part -- walk through all poolset file parts * * Stops processing if callback returns non-zero value. * The value returned by callback is returned to the caller. * * Return value: * 0 - all part files have been processed * -1 - parsing poolset file error */ int util_poolset_foreach_part(const char *path, int (*callback)(struct part_file *pf, void *arg), void *arg) { LOG(3, "path %s callback %p arg %p", path, callback, arg); ASSERTne(callback, NULL); int fd = os_open(path, O_RDONLY); if (fd < 0) { ERR("!open: path \"%s\"", path); return -1; } struct pool_set *set; int ret = util_poolset_parse(&set, path, fd); if (ret) { ERR("util_poolset_parse failed -- '%s'", path); ret = -1; goto err_close; } ret = util_poolset_foreach_part_struct(set, callback, arg); /* * Make sure callback does not return -1, * because this value is reserved for parsing * error. */ ASSERTne(ret, -1); util_poolset_free(set); err_close: os_close(fd); return ret; } /* * util_poolset_size -- get size of poolset, returns 0 on error */ size_t util_poolset_size(const char *path) { int fd = os_open(path, O_RDONLY); if (fd < 0) return 0; size_t size = 0; struct pool_set *set; if (util_poolset_parse(&set, path, fd)) goto err_close; size = set->poolsize; util_poolset_free(set); err_close: os_close(fd); return size; } /* * util_replica_fdclose -- close all parts of given replica */ void util_replica_fdclose(struct pool_replica *rep) { for (unsigned p = 0; p < rep->nparts; p++) { struct pool_set_part *part = &rep->part[p]; util_part_fdclose(part); } } /* * util_replica_deep_common -- performs common calculations * on all parts from replica to define intersection ranges * for final flushing operations that take place in * os_part_deep_common function. */ int util_replica_deep_common(const void *addr, size_t len, struct pool_set *set, unsigned replica_id, int flush) { LOG(3, "addr %p len %zu set %p replica_id %u flush %d", addr, len, set, replica_id, flush); struct pool_replica *rep = set->replica[replica_id]; uintptr_t rep_start = (uintptr_t)rep->part[0].addr; uintptr_t rep_end = rep_start + rep->repsize; uintptr_t start = (uintptr_t)addr; uintptr_t end = start + len; ASSERT(start >= rep_start); ASSERT(end <= rep_end); for (unsigned p = 0; p < rep->nparts; p++) { struct pool_set_part *part = &rep->part[p]; uintptr_t part_start = (uintptr_t)part->addr; uintptr_t part_end = part_start + part->size; /* init intersection start and end addresses */ uintptr_t range_start = start; uintptr_t range_end = end; if (part_start > end || part_end < start) continue; /* recalculate intersection addresses */ if (part_start > start) range_start = part_start; if (part_end < end) range_end = part_end; size_t range_len = range_end - range_start; LOG(15, "perform deep flushing for replica %u " "part %p, addr %p, len %lu", replica_id, part, (void *)range_start, range_len); if (os_part_deep_common(rep, p, (void *)range_start, range_len, flush)) { LOG(1, "os_part_deep_common(%p, %p, %lu)", part, (void *)range_start, range_len); return -1; } } return 0; } /* * util_replica_deep_persist -- wrapper for util_replica_deep_common * Calling the target precedes initialization of function that * partly defines way of deep replica flushing. */ int util_replica_deep_persist(const void *addr, size_t len, struct pool_set *set, unsigned replica_id) { LOG(3, "addr %p len %zu set %p replica_id %u", addr, len, set, replica_id); int flush = 1; return util_replica_deep_common(addr, len, set, replica_id, flush); } /* * util_replica_deep_drain -- wrapper for util_replica_deep_common * Calling the target precedes initialization of function that * partly defines way of deep replica flushing. */ int util_replica_deep_drain(const void *addr, size_t len, struct pool_set *set, unsigned replica_id) { LOG(3, "addr %p len %zu set %p replica_id %u", addr, len, set, replica_id); int flush = 0; return util_replica_deep_common(addr, len, set, replica_id, flush); } pmdk-1.8/src/common/uuid_freebsd.c0000664000000000000000000000370313615011243015677 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * uuid_freebsd.c -- FreeBSD-specific implementation for UUID generation */ #include "uuid.h" /* XXX Can't include because it also defines uuid_t */ void uuid_generate(uuid_t); /* * util_uuid_generate -- generate a uuid * * Uses the available FreeBSD uuid_generate library function. */ int util_uuid_generate(uuid_t uuid) { uuid_generate(uuid); return 0; } pmdk-1.8/src/common/badblock_ndctl.c0000664000000000000000000002110613615011243016161 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * badblock_ndctl.c - implementation of the bad block API using ndctl library */ #define _GNU_SOURCE #include #include #include "file.h" #include "os.h" #include "out.h" #include "extent.h" #include "os_dimm.h" #include "os_badblock.h" #include "badblock.h" #include "vec.h" /* * os_badblocks_get -- returns 0 and bad blocks in the 'bbs' array * (that has to be pre-allocated) * or -1 in case of an error */ int os_badblocks_get(const char *file, struct badblocks *bbs) { LOG(3, "file %s badblocks %p", file, bbs); ASSERTne(bbs, NULL); VEC(bbsvec, struct bad_block) bbv = VEC_INITIALIZER; struct extents *exts = NULL; long extents = 0; unsigned long long bb_beg; unsigned long long bb_end; unsigned long long bb_len; unsigned long long bb_off; unsigned long long ext_beg; unsigned long long ext_end; unsigned long long not_block_aligned; int bb_found = -1; /* -1 means an error */ memset(bbs, 0, sizeof(*bbs)); if (os_dimm_files_namespace_badblocks(file, bbs)) { LOG(1, "checking the file for bad blocks failed -- '%s'", file); goto error_free_all; } if (bbs->bb_cnt == 0) { bb_found = 0; goto exit_free_all; } exts = Zalloc(sizeof(struct extents)); if (exts == NULL) { ERR("!Zalloc"); goto error_free_all; } extents = os_extents_count(file, exts); if (extents < 0) { LOG(1, "counting file's extents failed -- '%s'", file); goto error_free_all; } if (extents == 0) { /* dax device has no extents */ bb_found = (int)bbs->bb_cnt; for (unsigned b = 0; b < bbs->bb_cnt; b++) { LOG(4, "bad block found: offset: %llu, length: %u", bbs->bbv[b].offset, bbs->bbv[b].length); } goto exit_free_all; } exts->extents = Zalloc(exts->extents_count * sizeof(struct extent)); if (exts->extents == NULL) { ERR("!Zalloc"); goto error_free_all; } if (os_extents_get(file, exts)) { LOG(1, "getting file's extents failed -- '%s'", file); goto error_free_all; } bb_found = 0; for (unsigned b = 0; b < bbs->bb_cnt; b++) { bb_beg = bbs->bbv[b].offset; bb_end = bb_beg + bbs->bbv[b].length - 1; for (unsigned e = 0; e < exts->extents_count; e++) { ext_beg = exts->extents[e].offset_physical; ext_end = ext_beg + exts->extents[e].length - 1; /* check if the bad block overlaps with file's extent */ if (bb_beg > ext_end || ext_beg > bb_end) continue; bb_found++; bb_beg = (bb_beg > ext_beg) ? bb_beg : ext_beg; bb_end = (bb_end < ext_end) ? bb_end : ext_end; bb_len = bb_end - bb_beg + 1; bb_off = bb_beg + exts->extents[e].offset_logical - exts->extents[e].offset_physical; LOG(10, "bad block found: physical offset: %llu, length: %llu", bb_beg, bb_len); /* check if offset is block-aligned */ not_block_aligned = bb_off & (exts->blksize - 1); if (not_block_aligned) { bb_off -= not_block_aligned; bb_len += not_block_aligned; } /* check if length is block-aligned */ bb_len = ALIGN_UP(bb_len, exts->blksize); LOG(4, "bad block found: logical offset: %llu, length: %llu", bb_off, bb_len); /* * Form a new bad block structure with offset and length * expressed in bytes and offset relative * to the beginning of the file. */ struct bad_block bb; bb.offset = bb_off; bb.length = (unsigned)(bb_len); /* unknown healthy replica */ bb.nhealthy = NO_HEALTHY_REPLICA; /* add the new bad block to the vector */ if (VEC_PUSH_BACK(&bbv, bb)) { VEC_DELETE(&bbv); bb_found = -1; goto error_free_all; } } } error_free_all: Free(bbs->bbv); bbs->bbv = NULL; bbs->bb_cnt = 0; exit_free_all: if (exts) { Free(exts->extents); Free(exts); } if (extents > 0 && bb_found > 0) { bbs->bbv = VEC_ARR(&bbv); bbs->bb_cnt = (unsigned)VEC_SIZE(&bbv); LOG(10, "number of bad blocks detected: %u", bbs->bb_cnt); /* sanity check */ ASSERTeq((unsigned)bb_found, bbs->bb_cnt); } return (bb_found >= 0) ? 0 : -1; } /* * os_badblocks_count -- returns number of bad blocks in the file * or -1 in case of an error */ long os_badblocks_count(const char *file) { LOG(3, "file %s", file); struct badblocks *bbs = badblocks_new(); if (bbs == NULL) return -1; int ret = os_badblocks_get(file, bbs); long count = (ret == 0) ? (long)bbs->bb_cnt : -1; badblocks_delete(bbs); return count; } /* * os_badblocks_check_file -- check if the file contains bad blocks * * Return value: * -1 : an error * 0 : no bad blocks * 1 : bad blocks detected */ int os_badblocks_check_file(const char *file) { LOG(3, "file %s", file); long bbsc = os_badblocks_count(file); if (bbsc < 0) { LOG(1, "counting bad blocks failed -- '%s'", file); return -1; } if (bbsc > 0) { LOG(1, "pool file '%s' contains %li bad block(s)", file, bbsc); return 1; } return 0; } /* * os_badblocks_clear_file -- clear the given bad blocks in the regular file * (not in a dax device) */ static int os_badblocks_clear_file(const char *file, struct badblocks *bbs) { LOG(3, "file %s badblocks %p", file, bbs); ASSERTne(bbs, NULL); int ret = 0; int fd; if ((fd = os_open(file, O_RDWR)) < 0) { ERR("!open: %s", file); return -1; } for (unsigned b = 0; b < bbs->bb_cnt; b++) { off_t offset = (off_t)bbs->bbv[b].offset; off_t length = (off_t)bbs->bbv[b].length; LOG(10, "clearing bad block: logical offset %li length %li (in 512B sectors) -- '%s'", B2SEC(offset), B2SEC(length), file); /* deallocate bad blocks */ if (fallocate(fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, offset, length)) { ERR("!fallocate"); ret = -1; break; } /* allocate new blocks */ if (fallocate(fd, FALLOC_FL_KEEP_SIZE, offset, length)) { ERR("!fallocate"); ret = -1; break; } } os_close(fd); return ret; } /* * os_badblocks_clear -- clears the given bad blocks in a file * (regular file or dax device) */ int os_badblocks_clear(const char *file, struct badblocks *bbs) { LOG(3, "file %s badblocks %p", file, bbs); ASSERTne(bbs, NULL); enum file_type type = util_file_get_type(file); if (type < 0) return -1; if (type == TYPE_DEVDAX) return os_dimm_devdax_clear_badblocks(file, bbs); return os_badblocks_clear_file(file, bbs); } /* * os_badblocks_clear_all -- clears all bad blocks in a file * (regular file or dax device) */ int os_badblocks_clear_all(const char *file) { LOG(3, "file %s", file); enum file_type type = util_file_get_type(file); if (type < 0) return -1; if (type == TYPE_DEVDAX) return os_dimm_devdax_clear_badblocks_all(file); struct badblocks *bbs = badblocks_new(); if (bbs == NULL) return -1; int ret = os_badblocks_get(file, bbs); if (ret) { LOG(1, "checking bad blocks in the file failed -- '%s'", file); goto error_free_all; } if (bbs->bb_cnt > 0) { ret = os_badblocks_clear_file(file, bbs); if (ret < 0) { LOG(1, "clearing bad blocks in the file failed -- '%s'", file); goto error_free_all; } } error_free_all: badblocks_delete(bbs); return ret; } pmdk-1.8/src/common/mmap.c0000664000000000000000000003031513615011243014170 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * mmap.c -- mmap utilities */ #include #include #include #include #include #include #include #include "file.h" #include "queue.h" #include "mmap.h" #include "sys_util.h" #include "os.h" #include "alloc.h" int Mmap_no_random; void *Mmap_hint; static os_rwlock_t Mmap_list_lock; static PMDK_SORTEDQ_HEAD(map_list_head, map_tracker) Mmap_list = PMDK_SORTEDQ_HEAD_INITIALIZER(Mmap_list); /* * util_mmap_init -- initialize the mmap utils * * This is called from the library initialization code. */ void util_mmap_init(void) { LOG(3, NULL); util_rwlock_init(&Mmap_list_lock); /* * For testing, allow overriding the default mmap() hint address. * If hint address is defined, it also disables address randomization. */ char *e = os_getenv("PMEM_MMAP_HINT"); if (e) { char *endp; errno = 0; unsigned long long val = strtoull(e, &endp, 16); if (errno || endp == e) { LOG(2, "Invalid PMEM_MMAP_HINT"); } else if (os_access(OS_MAPFILE, R_OK)) { LOG(2, "No /proc, PMEM_MMAP_HINT ignored"); } else { Mmap_hint = (void *)val; Mmap_no_random = 1; LOG(3, "PMEM_MMAP_HINT set to %p", Mmap_hint); } } } /* * util_mmap_fini -- clean up the mmap utils * * This is called before process stop. */ void util_mmap_fini(void) { LOG(3, NULL); util_rwlock_destroy(&Mmap_list_lock); } /* * util_map -- memory map a file * * This is just a convenience function that calls mmap() with the * appropriate arguments and includes our trace points. */ void * util_map(int fd, os_off_t off, size_t len, int flags, int rdonly, size_t req_align, int *map_sync) { LOG(3, "fd %d len %zu flags %d rdonly %d req_align %zu map_sync %p", fd, len, flags, rdonly, req_align, map_sync); void *base; void *addr = util_map_hint(len, req_align); if (addr == MAP_FAILED) { LOG(1, "cannot find a contiguous region of given size"); return NULL; } if (req_align) ASSERTeq((uintptr_t)addr % req_align, 0); int proto = rdonly ? PROT_READ : PROT_READ|PROT_WRITE; base = util_map_sync(addr, len, proto, flags, fd, off, map_sync); if (base == MAP_FAILED) { ERR("!mmap %zu bytes", len); return NULL; } LOG(3, "mapped at %p", base); return base; } /* * util_unmap -- unmap a file * * This is just a convenience function that calls munmap() with the * appropriate arguments and includes our trace points. */ int util_unmap(void *addr, size_t len) { LOG(3, "addr %p len %zu", addr, len); /* * XXX Workaround for https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=169608 */ #ifdef __FreeBSD__ if (!IS_PAGE_ALIGNED((uintptr_t)addr)) { errno = EINVAL; ERR("!munmap"); return -1; } #endif int retval = munmap(addr, len); if (retval < 0) ERR("!munmap"); return retval; } /* * util_range_ro -- set a memory range read-only */ int util_range_ro(void *addr, size_t len) { LOG(3, "addr %p len %zu", addr, len); uintptr_t uptr; int retval; /* * mprotect requires addr to be a multiple of pagesize, so * adjust addr and len to represent the full 4k chunks * covering the given range. */ /* increase len by the amount we gain when we round addr down */ len += (uintptr_t)addr & (Pagesize - 1); /* round addr down to page boundary */ uptr = (uintptr_t)addr & ~(Pagesize - 1); if ((retval = mprotect((void *)uptr, len, PROT_READ)) < 0) ERR("!mprotect: PROT_READ"); return retval; } /* * util_range_rw -- set a memory range read-write */ int util_range_rw(void *addr, size_t len) { LOG(3, "addr %p len %zu", addr, len); uintptr_t uptr; int retval; /* * mprotect requires addr to be a multiple of pagesize, so * adjust addr and len to represent the full 4k chunks * covering the given range. */ /* increase len by the amount we gain when we round addr down */ len += (uintptr_t)addr & (Pagesize - 1); /* round addr down to page boundary */ uptr = (uintptr_t)addr & ~(Pagesize - 1); if ((retval = mprotect((void *)uptr, len, PROT_READ|PROT_WRITE)) < 0) ERR("!mprotect: PROT_READ|PROT_WRITE"); return retval; } /* * util_range_none -- set a memory range for no access allowed */ int util_range_none(void *addr, size_t len) { LOG(3, "addr %p len %zu", addr, len); uintptr_t uptr; int retval; /* * mprotect requires addr to be a multiple of pagesize, so * adjust addr and len to represent the full 4k chunks * covering the given range. */ /* increase len by the amount we gain when we round addr down */ len += (uintptr_t)addr & (Pagesize - 1); /* round addr down to page boundary */ uptr = (uintptr_t)addr & ~(Pagesize - 1); if ((retval = mprotect((void *)uptr, len, PROT_NONE)) < 0) ERR("!mprotect: PROT_NONE"); return retval; } /* * util_range_comparer -- (internal) compares the two mapping trackers */ static intptr_t util_range_comparer(struct map_tracker *a, struct map_tracker *b) { return ((intptr_t)a->base_addr - (intptr_t)b->base_addr); } /* * util_range_find_unlocked -- (internal) find the map tracker * for given address range * * Returns the first entry at least partially overlapping given range. * It's up to the caller to check whether the entry exactly matches the range, * or if the range spans multiple entries. */ static struct map_tracker * util_range_find_unlocked(uintptr_t addr, size_t len) { LOG(10, "addr 0x%016" PRIxPTR " len %zu", addr, len); uintptr_t end = addr + len; struct map_tracker *mt; PMDK_SORTEDQ_FOREACH(mt, &Mmap_list, entry) { if (addr < mt->end_addr && (addr >= mt->base_addr || end > mt->base_addr)) goto out; /* break if there is no chance to find matching entry */ if (addr < mt->base_addr) break; } mt = NULL; out: return mt; } /* * util_range_find -- find the map tracker for given address range * the same as util_range_find_unlocked but locked */ struct map_tracker * util_range_find(uintptr_t addr, size_t len) { LOG(10, "addr 0x%016" PRIxPTR " len %zu", addr, len); util_rwlock_rdlock(&Mmap_list_lock); struct map_tracker *mt = util_range_find_unlocked(addr, len); util_rwlock_unlock(&Mmap_list_lock); return mt; } /* * util_range_register -- add a memory range into a map tracking list */ int util_range_register(const void *addr, size_t len, const char *path, enum pmem_map_type type) { LOG(3, "addr %p len %zu path %s type %d", addr, len, path, type); /* check if not tracked already */ if (util_range_find((uintptr_t)addr, len) != NULL) { ERR( "duplicated persistent memory range; presumably unmapped with munmap() instead of pmem_unmap(): addr %p len %zu", addr, len); errno = ENOMEM; return -1; } struct map_tracker *mt; mt = Malloc(sizeof(struct map_tracker)); if (mt == NULL) { ERR("!Malloc"); return -1; } mt->base_addr = (uintptr_t)addr; mt->end_addr = mt->base_addr + len; mt->type = type; if (type == PMEM_DEV_DAX) mt->region_id = util_ddax_region_find(path); util_rwlock_wrlock(&Mmap_list_lock); PMDK_SORTEDQ_INSERT(&Mmap_list, mt, entry, struct map_tracker, util_range_comparer); util_rwlock_unlock(&Mmap_list_lock); return 0; } /* * util_range_split -- (internal) remove or split a map tracking entry */ static int util_range_split(struct map_tracker *mt, const void *addrp, const void *endp) { LOG(3, "begin %p end %p", addrp, endp); uintptr_t addr = (uintptr_t)addrp; uintptr_t end = (uintptr_t)endp; ASSERTne(mt, NULL); if (addr == end || addr % Mmap_align != 0 || end % Mmap_align != 0) { ERR( "invalid munmap length, must be non-zero and page aligned"); return -1; } struct map_tracker *mtb = NULL; struct map_tracker *mte = NULL; /* * 1) b e b e * xxxxxxxxxxxxx => xxx.......xxxx - mtb+mte * 2) b e b e * xxxxxxxxxxxxx => xxxxxxx....... - mtb * 3) b e b e * xxxxxxxxxxxxx => ........xxxxxx - mte * 4) b e b e * xxxxxxxxxxxxx => .............. - */ if (addr > mt->base_addr) { /* case #1/2 */ /* new mapping at the beginning */ mtb = Malloc(sizeof(struct map_tracker)); if (mtb == NULL) { ERR("!Malloc"); goto err; } mtb->base_addr = mt->base_addr; mtb->end_addr = addr; mtb->region_id = mt->region_id; mtb->type = mt->type; } if (end < mt->end_addr) { /* case #1/3 */ /* new mapping at the end */ mte = Malloc(sizeof(struct map_tracker)); if (mte == NULL) { ERR("!Malloc"); goto err; } mte->base_addr = end; mte->end_addr = mt->end_addr; mte->region_id = mt->region_id; mte->type = mt->type; } PMDK_SORTEDQ_REMOVE(&Mmap_list, mt, entry); if (mtb) { PMDK_SORTEDQ_INSERT(&Mmap_list, mtb, entry, struct map_tracker, util_range_comparer); } if (mte) { PMDK_SORTEDQ_INSERT(&Mmap_list, mte, entry, struct map_tracker, util_range_comparer); } /* free entry for the original mapping */ Free(mt); return 0; err: Free(mtb); Free(mte); return -1; } /* * util_range_unregister -- remove a memory range * from map tracking list * * Remove the region between [begin,end]. If it's in a middle of the existing * mapping, it results in two new map trackers. */ int util_range_unregister(const void *addr, size_t len) { LOG(3, "addr %p len %zu", addr, len); int ret = 0; util_rwlock_wrlock(&Mmap_list_lock); /* * Changes in the map tracker list must match the underlying behavior. * * $ man 2 mmap: * The address addr must be a multiple of the page size (but length * need not be). All pages containing a part of the indicated range * are unmapped. * * This means that we must align the length to the page size. */ len = PAGE_ALIGNED_UP_SIZE(len); void *end = (char *)addr + len; /* XXX optimize the loop */ struct map_tracker *mt; while ((mt = util_range_find_unlocked((uintptr_t)addr, len)) != NULL) { if (util_range_split(mt, addr, end) != 0) { ret = -1; break; } } util_rwlock_unlock(&Mmap_list_lock); return ret; } /* * util_range_is_pmem -- return true if entire range * is persistent memory */ int util_range_is_pmem(const void *addrp, size_t len) { LOG(10, "addr %p len %zu", addrp, len); uintptr_t addr = (uintptr_t)addrp; int retval = 1; util_rwlock_rdlock(&Mmap_list_lock); do { struct map_tracker *mt = util_range_find(addr, len); if (mt == NULL) { LOG(4, "address not found 0x%016" PRIxPTR, addr); retval = 0; break; } LOG(10, "range found - begin 0x%016" PRIxPTR " end 0x%016" PRIxPTR, mt->base_addr, mt->end_addr); if (mt->base_addr > addr) { LOG(10, "base address doesn't match: " "0x%" PRIxPTR " > 0x%" PRIxPTR, mt->base_addr, addr); retval = 0; break; } uintptr_t map_len = mt->end_addr - addr; if (map_len > len) map_len = len; len -= map_len; addr += map_len; } while (len > 0); util_rwlock_unlock(&Mmap_list_lock); return retval; } pmdk-1.8/src/common/os_auto_flush_windows.h0000664000000000000000000000525613615011243017675 0ustar rootroot/* * Copyright 2018-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef PMDK_OS_AUTO_FLUSH_WINDOWS_H #define PMDK_OS_AUTO_FLUSH_WINDOWS_H 1 #define ACPI_SIGNATURE 0x41435049 /* hex value of ACPI signature */ #define NFIT_REV_SIGNATURE 0x5449464e /* hex value of htonl(NFIT) signature */ #define NFIT_STR_SIGNATURE "NFIT" #define NFIT_SIGNATURE_LEN 4 #define NFIT_OEM_ID_LEN 6 #define NFIT_OEM_TABLE_ID_LEN 8 #define NFIT_MAX_STRUCTURES 8 #define PCS_RESERVED 3 #define PCS_RESERVED_2 4 #define PCS_TYPE_NUMBER 7 /* check if bit on 'bit' position in number 'num' is set */ #define CHECK_BIT(num, bit) (((num) >> (bit)) & 1) /* * sets alignment of members of structure */ #pragma pack(1) struct platform_capabilities { uint16_t type; uint16_t length; uint8_t highest_valid; uint8_t reserved[PCS_RESERVED]; uint32_t capabilities; uint8_t reserved2[PCS_RESERVED_2]; }; struct nfit_header { uint8_t signature[NFIT_SIGNATURE_LEN]; uint32_t length; uint8_t revision; uint8_t checksum; uint8_t oem_id[NFIT_OEM_ID_LEN]; uint8_t oem_table_id[NFIT_OEM_TABLE_ID_LEN]; uint32_t oem_revision; uint8_t creator_id[4]; uint32_t creator_revision; uint32_t reserved; }; #pragma pack() #endif pmdk-1.8/src/common/extent_freebsd.c0000664000000000000000000000442513615011243016242 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * extent_freebsd.c - implementation of the FreeBSD fs extent query API * XXX THIS IS CURRENTLY A DUMMY MODULE. */ #include #include #include #include "file.h" #include "out.h" #include "extent.h" /* * os_extents_count -- get number of extents of the given file * (and optionally read its block size) */ long os_extents_count(const char *path, struct extents *exts) { LOG(3, "path %s extents %p", path, exts); return -1; } /* * os_extents_get -- get extents of the given file * (and optionally read its block size) */ int os_extents_get(const char *path, struct extents *exts) { LOG(3, "path %s extents %p", path, exts); return -1; } pmdk-1.8/src/common/common.rc0000664000000000000000000000647513615011243014722 0ustar rootroot/* * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * common.rc -- common part of PMDK rc files */ #include #include "srcversion.h" #define VERSION(major, minor, build, revision) major, minor, build, revision #ifdef _DEBUG #define VERSION_DEBUG VS_FF_DEBUG #else #define VERSION_DEBUG 0 #endif #ifdef PRERELEASE #define VERSION_PRERELEASE VS_FF_PRERELEASE #else #define VERSION_PRERELEASE 0 #endif #ifdef BUGFIX #define VERSION_PATCHED VS_FF_PATCHED #else #define VERSION_PATCHED 0 #endif #ifdef PRIVATE #define VERSION_PRIVATE VS_FF_PRIVATE #else #define VERSION_PRIVATE 0 #endif #ifdef CUSTOM #define VERSION_SPECIAL VS_FF_SPECIALBUILD #else #define VERSION_SPECIAL 0 #endif #define VERSION_PRIVATEBUILD VS_FF_PRIVATEBUILD #define VER_PATCHED VS_FF_PATCHED VS_VERSION_INFO VERSIONINFO FILEVERSION VERSION(MAJOR, MINOR, BUILD, REVISION) PRODUCTVERSION VERSION(MAJOR, MINOR, BUILD, REVISION) FILEFLAGSMASK VS_FFI_FILEFLAGSMASK FILEFLAGS (VERSION_PRIVATEBUILD | VERSION_PRERELEASE | VERSION_DEBUG | VERSION_SPECIAL | VERSION_PATCHED) FILEOS VOS__WINDOWS32 FILETYPE TYPE FILESUBTYPE VFT2_UNKNOWN BEGIN BLOCK "StringFileInfo" BEGIN BLOCK "040904b0" BEGIN VALUE "CompanyName", "Intel" VALUE "FileDescription", DESCRIPTION VALUE "FileVersion", SRCVERSION VALUE "InternalName", "PMDK" VALUE "LegalCopyright", "Copyright 2014-2017, Intel Corporation" VALUE "OriginalFilename", FILE_NAME VALUE "ProductName", "Persistent Memory Development Kit" VALUE "ProductVersion", SRCVERSION #if VERSION_SPECIAL == VS_FF_SPECIALBUILD VALUE "SpecialBuild", VERSION_CUSTOM_MSG #endif #if VERSION_PRIVATEBUILD == VS_FF_SPECIALBUILD VALUE "PrivateBuild", "Not a release build" #endif END END BLOCK "VarFileInfo" BEGIN /* XXX: Update to UNICODE */ VALUE "Translation", 0x409, 0 END END pmdk-1.8/src/common/fs_windows.c0000664000000000000000000000742613615011243015427 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fs_windows.c -- file system traversal windows implementation */ #include #include "util.h" #include "out.h" #include "vec.h" #include "fs.h" struct fs { size_t dirlen; WIN32_FIND_DATAW ffd; HANDLE hFind; int first_done; const char *dir; struct fs_entry entry; }; /* * fs_new -- creates fs traversal instance */ struct fs * fs_new(const char *path) { size_t pathlen = strlen(path); char *search_path = Malloc(strlen(path) + sizeof("\\*\0")); if (search_path == NULL) goto error_spath_alloc; strcpy(search_path, path); strcpy(search_path + pathlen, "\\*\0"); wchar_t *pathw = util_toUTF16(search_path); if (pathw == NULL) goto error_path_alloc; struct fs *f = Zalloc(sizeof(*f)); if (f == NULL) goto error_fs_alloc; f->first_done = 0; f->hFind = FindFirstFileW(pathw, &f->ffd); if (f->hFind == INVALID_HANDLE_VALUE) goto error_fff; f->dir = path; f->dirlen = pathlen; util_free_UTF16(pathw); Free(search_path); return f; error_fff: Free(f); error_fs_alloc: util_free_UTF16(pathw); error_path_alloc: Free(search_path); error_spath_alloc: return NULL; } /* * fs_read -- reads an entry from the fs path */ struct fs_entry * fs_read(struct fs *f) { util_free_UTF8((char *)f->entry.name); Free((char *)f->entry.path); f->entry.name = NULL; f->entry.path = NULL; if (f->first_done) { if (FindNextFileW(f->hFind, &f->ffd) == 0) return NULL; } else { f->first_done = 1; } if (f->ffd.dwFileAttributes & FILE_ATTRIBUTE_DIRECTORY) f->entry.type = FS_ENTRY_DIRECTORY; else f->entry.type = FS_ENTRY_FILE; f->entry.name = util_toUTF8(f->ffd.cFileName); if (f->entry.name == NULL) return NULL; f->entry.namelen = strnlen(f->entry.name, MAX_PATH); f->entry.pathlen = f->dirlen + f->entry.namelen + 1; char *path = Zalloc(f->entry.pathlen + 1); if (path == NULL) { util_free_UTF8((char *)f->entry.name); return NULL; } strcpy(path, f->dir); path[f->dirlen] = '\\'; strcpy(path + f->dirlen + 1, f->entry.name); f->entry.path = path; f->entry.level = 1; return &f->entry; } /* * fs_delete -- deletes a fs traversal instance */ void fs_delete(struct fs *f) { util_free_UTF8((char *)f->entry.name); Free((char *)f->entry.path); FindClose(f->hFind); Free(f); } pmdk-1.8/src/common/ctl_cow.c0000664000000000000000000000501313615011243014665 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctl_cow.c -- implementation of the CTL copy on write namespace */ #include "ctl.h" #include "set.h" #include "out.h" #include "ctl_global.h" /* * CTL_READ_HANDLER(at_open) -- returns at_open field */ static int CTL_READ_HANDLER(at_open)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int *arg_out = arg; *arg_out = COW_at_open; return 0; } /* * CTL_WRITE_HANDLER(at_open) -- sets the at_open field in copy_on_write */ static int CTL_WRITE_HANDLER(at_open)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int arg_in = *(int *)arg; COW_at_open = arg_in; return 0; } static struct ctl_argument CTL_ARG(at_open) = CTL_ARG_BOOLEAN; static const struct ctl_node CTL_NODE(copy_on_write)[] = { CTL_LEAF_RW(at_open), CTL_NODE_END }; /* * cow_ctl_register -- registers ctl nodes for "copy_on_write" module */ void ctl_cow_register(void) { CTL_REGISTER_MODULE(NULL, copy_on_write); } pmdk-1.8/src/common/page_size.h0000664000000000000000000000354113615011243015212 0ustar rootroot/* * Copyright 2019, IBM Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef PMDK_PAGE_SIZE_H #define PMDK_PAGE_SIZE_H #if defined(__x86_64) || defined(_M_X64) || defined(__aarch64__) #define PMEM_PAGESIZE 4096 #elif defined(__PPC64__) #define PMEM_PAGESIZE 65536 #else #error unable to recognize ISA at compile time #endif #endif /* PMDK_PAGE_SIZE_H */ pmdk-1.8/src/common/sys_util.h0000664000000000000000000001733613615011243015126 0ustar rootroot/* * Copyright 2016-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * sys_util.h -- internal utility wrappers around system functions */ #ifndef PMDK_SYS_UTIL_H #define PMDK_SYS_UTIL_H 1 #include #include "os_thread.h" #include "out.h" #ifdef __cplusplus extern "C" { #endif /* * util_mutex_init -- os_mutex_init variant that never fails from * caller perspective. If os_mutex_init failed, this function aborts * the program. */ static inline void util_mutex_init(os_mutex_t *m) { int tmp = os_mutex_init(m); if (tmp) { errno = tmp; FATAL("!os_mutex_init"); } } /* * util_mutex_destroy -- os_mutex_destroy variant that never fails from * caller perspective. If os_mutex_destroy failed, this function aborts * the program. */ static inline void util_mutex_destroy(os_mutex_t *m) { int tmp = os_mutex_destroy(m); if (tmp) { errno = tmp; FATAL("!os_mutex_destroy"); } } /* * util_mutex_lock -- os_mutex_lock variant that never fails from * caller perspective. If os_mutex_lock failed, this function aborts * the program. */ static inline void util_mutex_lock(os_mutex_t *m) { int tmp = os_mutex_lock(m); if (tmp) { errno = tmp; FATAL("!os_mutex_lock"); } } /* * util_mutex_trylock -- os_mutex_trylock variant that never fails from * caller perspective (other than EBUSY). If util_mutex_trylock failed, this * function aborts the program. * Returns 0 if locked successfully, otherwise returns EBUSY. */ static inline int util_mutex_trylock(os_mutex_t *m) { int tmp = os_mutex_trylock(m); if (tmp && tmp != EBUSY) { errno = tmp; FATAL("!os_mutex_trylock"); } return tmp; } /* * util_mutex_unlock -- os_mutex_unlock variant that never fails from * caller perspective. If os_mutex_unlock failed, this function aborts * the program. */ static inline void util_mutex_unlock(os_mutex_t *m) { int tmp = os_mutex_unlock(m); if (tmp) { errno = tmp; FATAL("!os_mutex_unlock"); } } /* * util_rwlock_init -- os_rwlock_init variant that never fails from * caller perspective. If os_rwlock_init failed, this function aborts * the program. */ static inline void util_rwlock_init(os_rwlock_t *m) { int tmp = os_rwlock_init(m); if (tmp) { errno = tmp; FATAL("!os_rwlock_init"); } } /* * util_rwlock_rdlock -- os_rwlock_rdlock variant that never fails from * caller perspective. If os_rwlock_rdlock failed, this function aborts * the program. */ static inline void util_rwlock_rdlock(os_rwlock_t *m) { int tmp = os_rwlock_rdlock(m); if (tmp) { errno = tmp; FATAL("!os_rwlock_rdlock"); } } /* * util_rwlock_wrlock -- os_rwlock_wrlock variant that never fails from * caller perspective. If os_rwlock_wrlock failed, this function aborts * the program. */ static inline void util_rwlock_wrlock(os_rwlock_t *m) { int tmp = os_rwlock_wrlock(m); if (tmp) { errno = tmp; FATAL("!os_rwlock_wrlock"); } } /* * util_rwlock_unlock -- os_rwlock_unlock variant that never fails from * caller perspective. If os_rwlock_unlock failed, this function aborts * the program. */ static inline void util_rwlock_unlock(os_rwlock_t *m) { int tmp = os_rwlock_unlock(m); if (tmp) { errno = tmp; FATAL("!os_rwlock_unlock"); } } /* * util_rwlock_destroy -- os_rwlock_destroy variant that never fails from * caller perspective. If os_rwlock_destroy failed, this function aborts * the program. */ static inline void util_rwlock_destroy(os_rwlock_t *m) { int tmp = os_rwlock_destroy(m); if (tmp) { errno = tmp; FATAL("!os_rwlock_destroy"); } } /* * util_spin_init -- os_spin_init variant that logs on fail and sets errno. */ static inline int util_spin_init(os_spinlock_t *lock, int pshared) { int tmp = os_spin_init(lock, pshared); if (tmp) { errno = tmp; ERR("!os_spin_init"); } return tmp; } /* * util_spin_destroy -- os_spin_destroy variant that never fails from * caller perspective. If os_spin_destroy failed, this function aborts * the program. */ static inline void util_spin_destroy(os_spinlock_t *lock) { int tmp = os_spin_destroy(lock); if (tmp) { errno = tmp; FATAL("!os_spin_destroy"); } } /* * util_spin_lock -- os_spin_lock variant that never fails from caller * perspective. If os_spin_lock failed, this function aborts the program. */ static inline void util_spin_lock(os_spinlock_t *lock) { int tmp = os_spin_lock(lock); if (tmp) { errno = tmp; FATAL("!os_spin_lock"); } } /* * util_spin_unlock -- os_spin_unlock variant that never fails * from caller perspective. If os_spin_unlock failed, * this function aborts the program. */ static inline void util_spin_unlock(os_spinlock_t *lock) { int tmp = os_spin_unlock(lock); if (tmp) { errno = tmp; FATAL("!os_spin_unlock"); } } /* * util_semaphore_init -- os_semaphore_init variant that never fails * from caller perspective. If os_semaphore_init failed, * this function aborts the program. */ static inline void util_semaphore_init(os_semaphore_t *sem, unsigned value) { if (os_semaphore_init(sem, value)) FATAL("!os_semaphore_init"); } /* * util_semaphore_destroy -- deletes a semaphore instance */ static inline void util_semaphore_destroy(os_semaphore_t *sem) { if (os_semaphore_destroy(sem) != 0) FATAL("!os_semaphore_destroy"); } /* * util_semaphore_wait -- decreases the value of the semaphore */ static inline void util_semaphore_wait(os_semaphore_t *sem) { errno = 0; int ret; do { ret = os_semaphore_wait(sem); } while (errno == EINTR); /* signal interrupt */ if (ret != 0) FATAL("!os_semaphore_wait"); } /* * util_semaphore_trywait -- tries to decrease the value of the semaphore */ static inline int util_semaphore_trywait(os_semaphore_t *sem) { errno = 0; int ret; do { ret = os_semaphore_trywait(sem); } while (errno == EINTR); /* signal interrupt */ if (ret != 0 && errno != EAGAIN) FATAL("!os_semaphore_trywait"); return ret; } /* * util_semaphore_post -- increases the value of the semaphore */ static inline void util_semaphore_post(os_semaphore_t *sem) { if (os_semaphore_post(sem) != 0) FATAL("!os_semaphore_post"); } static inline void util_cond_init(os_cond_t *__restrict cond) { if (os_cond_init(cond)) FATAL("!os_cond_init"); } static inline void util_cond_destroy(os_cond_t *__restrict cond) { if (os_cond_destroy(cond)) FATAL("!os_cond_destroy"); } #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/util_pmem.h0000664000000000000000000000453613615011243015244 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * util_pmem.h -- internal definitions for pmem utils */ #ifndef PMDK_UTIL_PMEM_H #define PMDK_UTIL_PMEM_H 1 #include "libpmem.h" #include "out.h" #ifdef __cplusplus extern "C" { #endif /* * util_persist -- flush to persistence */ static inline void util_persist(int is_pmem, const void *addr, size_t len) { LOG(3, "is_pmem %d, addr %p, len %zu", is_pmem, addr, len); if (is_pmem) pmem_persist(addr, len); else if (pmem_msync(addr, len)) FATAL("!pmem_msync"); } /* * util_persist_auto -- flush to persistence */ static inline void util_persist_auto(int is_pmem, const void *addr, size_t len) { LOG(3, "is_pmem %d, addr %p, len %zu", is_pmem, addr, len); util_persist(is_pmem || pmem_is_pmem(addr, len), addr, len); } #ifdef __cplusplus } #endif #endif /* util_pmem.h */ pmdk-1.8/src/common/util.h0000664000000000000000000003777213615011243014236 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * util.h -- internal definitions for util module */ #ifndef PMDK_UTIL_H #define PMDK_UTIL_H 1 #include #include #include #include #include #ifdef _MSC_VER #include /* popcnt, bitscan */ #endif #include #ifdef __cplusplus extern "C" { #endif extern unsigned long long Pagesize; extern unsigned long long Mmap_align; #if defined(__x86_64) || defined(_M_X64) || defined(__aarch64__) #define CACHELINE_SIZE 64ULL #elif defined(__PPC64__) #define CACHELINE_SIZE 128ULL #else #error unable to recognize architecture at compile time #endif #define PAGE_ALIGNED_DOWN_SIZE(size) ((size) & ~(Pagesize - 1)) #define PAGE_ALIGNED_UP_SIZE(size)\ PAGE_ALIGNED_DOWN_SIZE((size) + (Pagesize - 1)) #define IS_PAGE_ALIGNED(size) (((size) & (Pagesize - 1)) == 0) #define PAGE_ALIGN_UP(addr) ((void *)PAGE_ALIGNED_UP_SIZE((uintptr_t)(addr))) #define ALIGN_UP(size, align) (((size) + (align) - 1) & ~((align) - 1)) #define ALIGN_DOWN(size, align) ((size) & ~((align) - 1)) #define ADDR_SUM(vp, lp) ((void *)((char *)(vp) + (lp))) #define util_alignof(t) offsetof(struct {char _util_c; t _util_m; }, _util_m) #define FORMAT_PRINTF(a, b) __attribute__((__format__(__printf__, (a), (b)))) void util_init(void); int util_is_zeroed(const void *addr, size_t len); uint64_t util_checksum_compute(void *addr, size_t len, uint64_t *csump, size_t skip_off); int util_checksum(void *addr, size_t len, uint64_t *csump, int insert, size_t skip_off); uint64_t util_checksum_seq(const void *addr, size_t len, uint64_t csum); int util_parse_size(const char *str, size_t *sizep); char *util_fgets(char *buffer, int max, FILE *stream); char *util_getexecname(char *path, size_t pathlen); char *util_part_realpath(const char *path); int util_compare_file_inodes(const char *path1, const char *path2); void *util_aligned_malloc(size_t alignment, size_t size); void util_aligned_free(void *ptr); struct tm *util_localtime(const time_t *timep); int util_safe_strcpy(char *dst, const char *src, size_t max_length); void util_emit_log(const char *lib, const char *func, int order); char *util_readline(FILE *fh); #ifdef _WIN32 char *util_toUTF8(const wchar_t *wstr); wchar_t *util_toUTF16(const char *wstr); void util_free_UTF8(char *str); void util_free_UTF16(wchar_t *str); int util_toUTF16_buff(const char *in, wchar_t *out, size_t out_size); int util_toUTF8_buff(const wchar_t *in, char *out, size_t out_size); void util_suppress_errmsg(void); #endif #define UTIL_MAX_ERR_MSG 128 void util_strerror(int errnum, char *buff, size_t bufflen); void util_strwinerror(char *buff, size_t bufflen); void util_set_alloc_funcs( void *(*malloc_func)(size_t size), void (*free_func)(void *ptr), void *(*realloc_func)(void *ptr, size_t size), char *(*strdup_func)(const char *s)); /* * Macro calculates number of elements in given table */ #ifndef ARRAY_SIZE #define ARRAY_SIZE(x) (sizeof(x) / sizeof((x)[0])) #endif #ifdef _MSC_VER #define force_inline inline __forceinline #define NORETURN __declspec(noreturn) #else #define force_inline __attribute__((always_inline)) inline #define NORETURN __attribute__((noreturn)) #endif #define util_get_not_masked_bits(x, mask) ((x) & ~(mask)) /* * util_setbit -- setbit macro substitution which properly deals with types */ static inline void util_setbit(uint8_t *b, uint32_t i) { b[i / 8] = (uint8_t)(b[i / 8] | (uint8_t)(1 << (i % 8))); } /* * util_clrbit -- clrbit macro substitution which properly deals with types */ static inline void util_clrbit(uint8_t *b, uint32_t i) { b[i / 8] = (uint8_t)(b[i / 8] & (uint8_t)(~(1 << (i % 8)))); } #define util_isset(a, i) isset(a, i) #define util_isclr(a, i) isclr(a, i) #define util_flag_isset(a, f) ((a) & (f)) #define util_flag_isclr(a, f) (((a) & (f)) == 0) /* * util_is_pow2 -- returns !0 when there's only 1 bit set in v, 0 otherwise */ static force_inline int util_is_pow2(uint64_t v) { return v && !(v & (v - 1)); } /* * util_div_ceil -- divides a by b and rounds up the result */ static force_inline unsigned util_div_ceil(unsigned a, unsigned b) { return (unsigned)(((unsigned long)a + b - 1) / b); } /* * util_bool_compare_and_swap -- perform an atomic compare and swap * util_fetch_and_* -- perform an operation atomically, return old value * util_synchronize -- issue a full memory barrier * util_popcount -- count number of set bits * util_lssb_index -- return index of least significant set bit, * undefined on zero * util_mssb_index -- return index of most significant set bit * undefined on zero * * XXX assertions needed on (value != 0) in both versions of bitscans * */ #ifndef _MSC_VER /* * ISO C11 -- 7.17.1.4 * memory_order - an enumerated type whose enumerators identify memory ordering * constraints. */ typedef enum { memory_order_relaxed = __ATOMIC_RELAXED, memory_order_consume = __ATOMIC_CONSUME, memory_order_acquire = __ATOMIC_ACQUIRE, memory_order_release = __ATOMIC_RELEASE, memory_order_acq_rel = __ATOMIC_ACQ_REL, memory_order_seq_cst = __ATOMIC_SEQ_CST } memory_order; /* * ISO C11 -- 7.17.7.2 The atomic_load generic functions * Integer width specific versions as supplement for: * * * #include * C atomic_load(volatile A *object); * C atomic_load_explicit(volatile A *object, memory_order order); * * The atomic_load interface doesn't return the loaded value, but instead * copies it to a specified address -- see comments at the MSVC version. * * Also, instead of generic functions, two versions are available: * for 32 bit fundamental integers, and for 64 bit ones. */ #define util_atomic_load_explicit32 __atomic_load #define util_atomic_load_explicit64 __atomic_load /* * ISO C11 -- 7.17.7.1 The atomic_store generic functions * Integer width specific versions as supplement for: * * #include * void atomic_store(volatile A *object, C desired); * void atomic_store_explicit(volatile A *object, C desired, * memory_order order); */ #define util_atomic_store_explicit32 __atomic_store_n #define util_atomic_store_explicit64 __atomic_store_n /* * https://gcc.gnu.org/onlinedocs/gcc/_005f_005fsync-Builtins.html * https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html * https://clang.llvm.org/docs/LanguageExtensions.html#builtin-functions */ #define util_bool_compare_and_swap32 __sync_bool_compare_and_swap #define util_bool_compare_and_swap64 __sync_bool_compare_and_swap #define util_fetch_and_add32 __sync_fetch_and_add #define util_fetch_and_add64 __sync_fetch_and_add #define util_fetch_and_sub32 __sync_fetch_and_sub #define util_fetch_and_sub64 __sync_fetch_and_sub #define util_fetch_and_and32 __sync_fetch_and_and #define util_fetch_and_and64 __sync_fetch_and_and #define util_fetch_and_or32 __sync_fetch_and_or #define util_fetch_and_or64 __sync_fetch_and_or #define util_synchronize __sync_synchronize #define util_popcount(value) ((unsigned char)__builtin_popcount(value)) #define util_popcount64(value) ((unsigned char)__builtin_popcountll(value)) #define util_lssb_index(value) ((unsigned char)__builtin_ctz(value)) #define util_lssb_index64(value) ((unsigned char)__builtin_ctzll(value)) #define util_mssb_index(value) ((unsigned char)(31 - __builtin_clz(value))) #define util_mssb_index64(value) ((unsigned char)(63 - __builtin_clzll(value))) #else /* ISO C11 -- 7.17.1.4 */ typedef enum { memory_order_relaxed, memory_order_consume, memory_order_acquire, memory_order_release, memory_order_acq_rel, memory_order_seq_cst } memory_order; /* * ISO C11 -- 7.17.7.2 The atomic_load generic functions * Integer width specific versions as supplement for: * * * #include * C atomic_load(volatile A *object); * C atomic_load_explicit(volatile A *object, memory_order order); * * The atomic_load interface doesn't return the loaded value, but instead * copies it to a specified address. * The MSVC specific implementation needs to trigger a barrier (at least * compiler barrier) after the load from the volatile value. The actual load * from the volatile value itself is expected to be atomic. * * The actual isnterface here: * #include "util.h" * void util_atomic_load32(volatile A *object, A *destination); * void util_atomic_load64(volatile A *object, A *destination); * void util_atomic_load_explicit32(volatile A *object, A *destination, * memory_order order); * void util_atomic_load_explicit64(volatile A *object, A *destination, * memory_order order); */ #ifndef _M_X64 #error MSVC ports of util_atomic_ only work on X86_64 #endif #if _MSC_VER >= 2000 #error util_atomic_ utility functions not tested with this version of VC++ #error These utility functions are not future proof, as they are not #error based on publicly available documentation. #endif #define util_atomic_load_explicit(object, dest, order)\ do {\ COMPILE_ERROR_ON(order != memory_order_seq_cst &&\ order != memory_order_consume &&\ order != memory_order_acquire &&\ order != memory_order_relaxed);\ *dest = *object;\ if (order == memory_order_seq_cst ||\ order == memory_order_consume ||\ order == memory_order_acquire)\ _ReadWriteBarrier();\ } while (0) #define util_atomic_load_explicit32 util_atomic_load_explicit #define util_atomic_load_explicit64 util_atomic_load_explicit /* ISO C11 -- 7.17.7.1 The atomic_store generic functions */ #define util_atomic_store_explicit64(object, desired, order)\ do {\ COMPILE_ERROR_ON(order != memory_order_seq_cst &&\ order != memory_order_release &&\ order != memory_order_relaxed);\ if (order == memory_order_seq_cst) {\ _InterlockedExchange64(\ (volatile long long *)object, desired);\ } else {\ if (order == memory_order_release)\ _ReadWriteBarrier();\ *object = desired;\ }\ } while (0) #define util_atomic_store_explicit32(object, desired, order)\ do {\ COMPILE_ERROR_ON(order != memory_order_seq_cst &&\ order != memory_order_release &&\ order != memory_order_relaxed);\ if (order == memory_order_seq_cst) {\ _InterlockedExchange(\ (volatile long *)object, desired);\ } else {\ if (order == memory_order_release)\ _ReadWriteBarrier();\ *object = desired;\ }\ } while (0) /* * https://msdn.microsoft.com/en-us/library/hh977022.aspx */ static __inline int bool_compare_and_swap32_VC(volatile LONG *ptr, LONG oldval, LONG newval) { LONG old = InterlockedCompareExchange(ptr, newval, oldval); return (old == oldval); } static __inline int bool_compare_and_swap64_VC(volatile LONG64 *ptr, LONG64 oldval, LONG64 newval) { LONG64 old = InterlockedCompareExchange64(ptr, newval, oldval); return (old == oldval); } #define util_bool_compare_and_swap32(p, o, n)\ bool_compare_and_swap32_VC((LONG *)(p), (LONG)(o), (LONG)(n)) #define util_bool_compare_and_swap64(p, o, n)\ bool_compare_and_swap64_VC((LONG64 *)(p), (LONG64)(o), (LONG64)(n)) #define util_fetch_and_add32(ptr, value)\ InterlockedExchangeAdd((LONG *)(ptr), value) #define util_fetch_and_add64(ptr, value)\ InterlockedExchangeAdd64((LONG64 *)(ptr), value) #define util_fetch_and_sub32(ptr, value)\ InterlockedExchangeSubtract((LONG *)(ptr), value) #define util_fetch_and_sub64(ptr, value)\ InterlockedExchangeAdd64((LONG64 *)(ptr), -((LONG64)(value))) #define util_fetch_and_and32(ptr, value)\ InterlockedAnd((LONG *)(ptr), value) #define util_fetch_and_and64(ptr, value)\ InterlockedAnd64((LONG64 *)(ptr), value) #define util_fetch_and_or32(ptr, value)\ InterlockedOr((LONG *)(ptr), value) #define util_fetch_and_or64(ptr, value)\ InterlockedOr64((LONG64 *)(ptr), value) static __inline void util_synchronize(void) { MemoryBarrier(); } #define util_popcount(value) (unsigned char)__popcnt(value) #define util_popcount64(value) (unsigned char)__popcnt64(value) static __inline unsigned char util_lssb_index(int value) { unsigned long ret; _BitScanForward(&ret, value); return (unsigned char)ret; } static __inline unsigned char util_lssb_index64(long long value) { unsigned long ret; _BitScanForward64(&ret, value); return (unsigned char)ret; } static __inline unsigned char util_mssb_index(int value) { unsigned long ret; _BitScanReverse(&ret, value); return (unsigned char)ret; } static __inline unsigned char util_mssb_index64(long long value) { unsigned long ret; _BitScanReverse64(&ret, value); return (unsigned char)ret; } #endif /* ISO C11 -- 7.17.7 Operations on atomic types */ #define util_atomic_load32(object, dest)\ util_atomic_load_explicit32(object, dest, memory_order_seq_cst) #define util_atomic_load64(object, dest)\ util_atomic_load_explicit64(object, dest, memory_order_seq_cst) #define util_atomic_store32(object, desired)\ util_atomic_store_explicit32(object, desired, memory_order_seq_cst) #define util_atomic_store64(object, desired)\ util_atomic_store_explicit64(object, desired, memory_order_seq_cst) /* * util_get_printable_ascii -- convert non-printable ascii to dot '.' */ static inline char util_get_printable_ascii(char c) { return isprint((unsigned char)c) ? c : '.'; } char *util_concat_str(const char *s1, const char *s2); #if !defined(likely) #if defined(__GNUC__) #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) #else #define likely(x) (!!(x)) #define unlikely(x) (!!(x)) #endif #endif #if defined(__CHECKER__) #define COMPILE_ERROR_ON(cond) #define ASSERT_COMPILE_ERROR_ON(cond) #elif defined(_MSC_VER) #define COMPILE_ERROR_ON(cond) C_ASSERT(!(cond)) /* XXX - can't be done with C_ASSERT() unless we have __builtin_constant_p() */ #define ASSERT_COMPILE_ERROR_ON(cond) do {} while (0) #else #define COMPILE_ERROR_ON(cond) ((void)sizeof(char[(cond) ? -1 : 1])) #define ASSERT_COMPILE_ERROR_ON(cond) COMPILE_ERROR_ON(cond) #endif #ifndef _MSC_VER #define ATTR_CONSTRUCTOR __attribute__((constructor)) static #define ATTR_DESTRUCTOR __attribute__((destructor)) static #else #define ATTR_CONSTRUCTOR #define ATTR_DESTRUCTOR #endif #ifndef _MSC_VER #define CONSTRUCTOR(fun) ATTR_CONSTRUCTOR #else #ifdef __cplusplus #define CONSTRUCTOR(fun) \ void fun(); \ struct _##fun { \ _##fun() { \ fun(); \ } \ }; static _##fun foo; \ static #else #define CONSTRUCTOR(fun) \ MSVC_CONSTR(fun) \ static #endif #endif #ifdef __GNUC__ #define CHECK_FUNC_COMPATIBLE(func1, func2)\ COMPILE_ERROR_ON(!__builtin_types_compatible_p(typeof(func1),\ typeof(func2))) #else #define CHECK_FUNC_COMPATIBLE(func1, func2) do {} while (0) #endif /* __GNUC__ */ #ifdef __cplusplus } #endif #endif /* util.h */ pmdk-1.8/src/common/file.c0000664000000000000000000003543613615011243014166 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * file.c -- file utilities */ #include #include #include #include #include #include #include #include #include #if !defined(_WIN32) && !defined(__FreeBSD__) #include #endif #include "file.h" #include "os.h" #include "out.h" #include "mmap.h" #define MAX_SIZE_LENGTH 64 #define DEVICE_DAX_ZERO_LEN (2 * MEGABYTE) #ifndef _WIN32 /* * device_dax_stat_size -- (internal) checks the size of a given * dax device from given stat structure */ static ssize_t device_dax_stat_size(os_stat_t *st) { char spath[PATH_MAX]; snprintf(spath, PATH_MAX, "/sys/dev/char/%u:%u/size", os_major(st->st_rdev), os_minor(st->st_rdev)); LOG(4, "device size path \"%s\"", spath); int fd = os_open(spath, O_RDONLY); if (fd < 0) { ERR("!open \"%s\"", spath); return -1; } ssize_t size = -1; char sizebuf[MAX_SIZE_LENGTH + 1]; ssize_t nread; if ((nread = read(fd, sizebuf, MAX_SIZE_LENGTH)) < 0) { ERR("!read"); goto out; } sizebuf[nread] = 0; /* null termination */ char *endptr; int olderrno = errno; errno = 0; size = strtoll(sizebuf, &endptr, 0); if (endptr == sizebuf || *endptr != '\n' || ((size == LLONG_MAX || size == LLONG_MIN) && errno == ERANGE)) { ERR("invalid device size %s", sizebuf); size = -1; goto out; } errno = olderrno; out: olderrno = errno; (void) os_close(fd); errno = olderrno; LOG(4, "device size %zu", size); return size; } /* * device_dax_file_size -- (internal) checks the size of a given dax device */ static ssize_t device_dax_file_size(const char *path) { LOG(3, "path \"%s\"", path); os_stat_t st; if (os_stat(path, &st) < 0) { ERR("!stat \"%s\"", path); return -1; } return device_dax_stat_size(&st); } #endif /* * util_file_exists -- checks whether file exists */ int util_file_exists(const char *path) { LOG(3, "path \"%s\"", path); if (os_access(path, F_OK) == 0) return 1; if (errno != ENOENT) { ERR("!os_access \"%s\"", path); return -1; } /* * ENOENT means that some component of a pathname does not exists. * * XXX - we should also call os_access on parent directory and * if this also results in ENOENT -1 should be returned. * * The problem is that we would need to use realpath, which fails * if file does not exist. */ return 0; } /* * util_stat_get_type -- checks whether stat structure describes * device dax or a normal file */ enum file_type util_stat_get_type(const os_stat_t *st) { #ifdef _WIN32 return TYPE_NORMAL; #else if (!S_ISCHR(st->st_mode)) { LOG(4, "not a character device"); return TYPE_NORMAL; } char spath[PATH_MAX]; snprintf(spath, PATH_MAX, "/sys/dev/char/%u:%u/subsystem", os_major(st->st_rdev), os_minor(st->st_rdev)); LOG(4, "device subsystem path \"%s\"", spath); char npath[PATH_MAX]; char *rpath = realpath(spath, npath); if (rpath == NULL) { ERR("!realpath \"%s\"", spath); return OTHER_ERROR; } char *basename = strrchr(rpath, '/'); if (!basename || strcmp("dax", basename + 1) != 0) { LOG(3, "%s path does not match device dax prefix path", rpath); errno = EINVAL; return OTHER_ERROR; } return TYPE_DEVDAX; #endif } /* * util_fd_get_type -- checks whether a file descriptor is associated * with a device dax or a normal file */ enum file_type util_fd_get_type(int fd) { LOG(3, "fd %d", fd); #ifdef _WIN32 return TYPE_NORMAL; #else os_stat_t st; if (os_fstat(fd, &st) < 0) { ERR("!fstat"); return OTHER_ERROR; } return util_stat_get_type(&st); #endif } /* * util_file_get_type -- checks whether the path points to a device dax, * normal file or non-existent file */ enum file_type util_file_get_type(const char *path) { LOG(3, "path \"%s\"", path); if (path == NULL) { ERR("invalid (NULL) path"); errno = EINVAL; return OTHER_ERROR; } int exists = util_file_exists(path); if (exists < 0) return OTHER_ERROR; if (!exists) return NOT_EXISTS; #ifdef _WIN32 return TYPE_NORMAL; #else os_stat_t st; if (os_stat(path, &st) < 0) { ERR("!stat"); return OTHER_ERROR; } return util_stat_get_type(&st); #endif } /* * util_file_get_size -- returns size of a file */ ssize_t util_file_get_size(const char *path) { LOG(3, "path \"%s\"", path); int file_type = util_file_get_type(path); if (file_type < 0) return -1; #ifndef _WIN32 if (file_type == TYPE_DEVDAX) { return device_dax_file_size(path); } #endif os_stat_t stbuf; if (os_stat(path, &stbuf) < 0) { ERR("!stat \"%s\"", path); return -1; } LOG(4, "file length %zu", stbuf.st_size); return stbuf.st_size; } /* * util_fd_get_size -- returns size of a file behind a given file descriptor */ ssize_t util_fd_get_size(int fd) { LOG(3, "fd %d", fd); os_stat_t st; if (os_fstat(fd, &st) < 0) { ERR("!fstat"); return OTHER_ERROR; } int file_type = util_stat_get_type(&st); if (file_type < 0) return -1; #ifndef _WIN32 if (file_type == TYPE_DEVDAX) { return device_dax_stat_size(&st); } #endif LOG(4, "file length %zu", st.st_size); return st.st_size; } /* * util_file_map_whole -- maps the entire file into memory */ void * util_file_map_whole(const char *path) { LOG(3, "path \"%s\"", path); int fd; int olderrno; void *addr = NULL; int flags = O_RDWR; #ifdef _WIN32 flags |= O_BINARY; #endif if ((fd = os_open(path, flags)) < 0) { ERR("!open \"%s\"", path); return NULL; } ssize_t size = util_file_get_size(path); if (size < 0) { LOG(2, "cannot determine file length \"%s\"", path); goto out; } addr = util_map(fd, 0, (size_t)size, MAP_SHARED, 0, 0, NULL); if (addr == NULL) { LOG(2, "failed to map entire file \"%s\"", path); goto out; } out: olderrno = errno; (void) os_close(fd); errno = olderrno; return addr; } /* * util_file_zero -- zeroes the specified region of the file */ int util_file_zero(const char *path, os_off_t off, size_t len) { LOG(3, "path \"%s\" off %ju len %zu", path, off, len); int fd; int olderrno; int ret = 0; int flags = O_RDWR; #ifdef _WIN32 flags |= O_BINARY; #endif if ((fd = os_open(path, flags)) < 0) { ERR("!open \"%s\"", path); return -1; } ssize_t size = util_file_get_size(path); if (size < 0) { LOG(2, "cannot determine file length \"%s\"", path); ret = -1; goto out; } if (off > size) { LOG(2, "offset beyond file length, %ju > %ju", off, size); ret = -1; goto out; } if ((size_t)off + len > (size_t)size) { LOG(2, "requested size of write goes beyond the file length, " "%zu > %zu", (size_t)off + len, size); LOG(4, "adjusting len to %zu", size - off); len = (size_t)(size - off); } void *addr = util_map(fd, 0, (size_t)size, MAP_SHARED, 0, 0, NULL); if (addr == NULL) { LOG(2, "failed to map entire file \"%s\"", path); ret = -1; goto out; } /* zero initialize the specified region */ memset((char *)addr + off, 0, len); util_unmap(addr, (size_t)size); out: olderrno = errno; (void) os_close(fd); errno = olderrno; return ret; } /* * util_file_pwrite -- writes to a file with an offset */ ssize_t util_file_pwrite(const char *path, const void *buffer, size_t size, os_off_t offset) { LOG(3, "path \"%s\" buffer %p size %zu offset %ju", path, buffer, size, offset); enum file_type type = util_file_get_type(path); if (type < 0) return -1; if (type == TYPE_NORMAL) { int fd = util_file_open(path, NULL, 0, O_RDWR); if (fd < 0) { LOG(2, "failed to open file \"%s\"", path); return -1; } ssize_t write_len = pwrite(fd, buffer, size, offset); int olderrno = errno; (void) os_close(fd); errno = olderrno; return write_len; } ssize_t file_size = util_file_get_size(path); if (file_size < 0) { LOG(2, "cannot determine file length \"%s\"", path); return -1; } size_t max_size = (size_t)(file_size - offset); if (size > max_size) { LOG(2, "requested size of write goes beyond the file length, " "%zu > %zu", size, max_size); LOG(4, "adjusting size to %zu", max_size); size = max_size; } void *addr = util_file_map_whole(path); if (addr == NULL) { LOG(2, "failed to map entire file \"%s\"", path); return -1; } memcpy(ADDR_SUM(addr, offset), buffer, size); util_unmap(addr, (size_t)file_size); return (ssize_t)size; } /* * util_file_pread -- reads from a file with an offset */ ssize_t util_file_pread(const char *path, void *buffer, size_t size, os_off_t offset) { LOG(3, "path \"%s\" buffer %p size %zu offset %ju", path, buffer, size, offset); enum file_type type = util_file_get_type(path); if (type < 0) return -1; if (type == TYPE_NORMAL) { int fd = util_file_open(path, NULL, 0, O_RDONLY); if (fd < 0) { LOG(2, "failed to open file \"%s\"", path); return -1; } ssize_t read_len = pread(fd, buffer, size, offset); int olderrno = errno; (void) os_close(fd); errno = olderrno; return read_len; } ssize_t file_size = util_file_get_size(path); if (file_size < 0) { LOG(2, "cannot determine file length \"%s\"", path); return -1; } size_t max_size = (size_t)(file_size - offset); if (size > max_size) { LOG(2, "requested size of read goes beyond the file length, " "%zu > %zu", size, max_size); LOG(4, "adjusting size to %zu", max_size); size = max_size; } void *addr = util_file_map_whole(path); if (addr == NULL) { LOG(2, "failed to map entire file \"%s\"", path); return -1; } memcpy(buffer, ADDR_SUM(addr, offset), size); util_unmap(addr, (size_t)file_size); return (ssize_t)size; } /* * util_file_create -- create a new memory pool file */ int util_file_create(const char *path, size_t size, size_t minsize) { LOG(3, "path \"%s\" size %zu minsize %zu", path, size, minsize); ASSERTne(size, 0); if (size < minsize) { ERR("size %zu smaller than %zu", size, minsize); errno = EINVAL; return -1; } if (((os_off_t)size) < 0) { ERR("invalid size (%zu) for os_off_t", size); errno = EFBIG; return -1; } int fd; int mode; int flags = O_RDWR | O_CREAT | O_EXCL; #ifndef _WIN32 mode = 0; #else mode = S_IWRITE | S_IREAD; flags |= O_BINARY; #endif /* * Create file without any permission. It will be granted once * initialization completes. */ if ((fd = os_open(path, flags, mode)) < 0) { ERR("!open \"%s\"", path); return -1; } if ((errno = os_posix_fallocate(fd, 0, (os_off_t)size)) != 0) { ERR("!posix_fallocate \"%s\", %zu", path, size); goto err; } /* for windows we can't flock until after we fallocate */ if (os_flock(fd, OS_LOCK_EX | OS_LOCK_NB) < 0) { ERR("!flock \"%s\"", path); goto err; } return fd; err: LOG(4, "error clean up"); int oerrno = errno; if (fd != -1) (void) os_close(fd); os_unlink(path); errno = oerrno; return -1; } /* * util_file_open -- open a memory pool file */ int util_file_open(const char *path, size_t *size, size_t minsize, int flags) { LOG(3, "path \"%s\" size %p minsize %zu flags %d", path, size, minsize, flags); int oerrno; int fd; #ifdef _WIN32 flags |= O_BINARY; #endif if ((fd = os_open(path, flags)) < 0) { ERR("!open \"%s\"", path); return -1; } if (os_flock(fd, OS_LOCK_EX | OS_LOCK_NB) < 0) { ERR("!flock \"%s\"", path); (void) os_close(fd); return -1; } if (size || minsize) { if (size) ASSERTeq(*size, 0); ssize_t actual_size = util_file_get_size(path); if (actual_size < 0) { ERR("stat \"%s\": negative size", path); errno = EINVAL; goto err; } if ((size_t)actual_size < minsize) { ERR("size %zu smaller than %zu", (size_t)actual_size, minsize); errno = EINVAL; goto err; } if (size) { *size = (size_t)actual_size; LOG(4, "actual file size %zu", *size); } } return fd; err: oerrno = errno; if (os_flock(fd, OS_LOCK_UN)) ERR("!flock unlock"); (void) os_close(fd); errno = oerrno; return -1; } /* * util_unlink -- unlinks a file or zeroes a device dax */ int util_unlink(const char *path) { LOG(3, "path \"%s\"", path); enum file_type type = util_file_get_type(path); if (type < 0) return -1; if (type == TYPE_DEVDAX) { return util_file_zero(path, 0, DEVICE_DAX_ZERO_LEN); } else { #ifdef _WIN32 /* on Windows we can not unlink Read-Only files */ if (os_chmod(path, S_IREAD | S_IWRITE) == -1) { ERR("!chmod \"%s\"", path); return -1; } #endif return os_unlink(path); } } /* * util_unlink_flock -- flocks the file and unlinks it * * The unlink(2) call on a file which is opened and locked using flock(2) * by different process works on linux. Thus in order to forbid removing a * pool when in use by different process we need to flock(2) the pool files * first before unlinking. */ int util_unlink_flock(const char *path) { LOG(3, "path \"%s\"", path); #ifdef WIN32 /* * On Windows it is not possible to unlink the * file if it is flocked. */ return util_unlink(path); #else int fd = util_file_open(path, NULL, 0, O_RDONLY); if (fd < 0) { LOG(2, "failed to open file \"%s\"", path); return -1; } int ret = util_unlink(path); (void) os_close(fd); return ret; #endif } /* * util_write_all -- a wrapper for util_write * * writes exactly count bytes from buf to file referred to by fd * returns -1 on error, 0 otherwise */ int util_write_all(int fd, const char *buf, size_t count) { ssize_t n_wrote = 0; size_t total = 0; while (count > total) { n_wrote = util_write(fd, buf, count - total); if (n_wrote <= 0) return -1; buf += (size_t)n_wrote; total += (size_t)n_wrote; } return 0; } pmdk-1.8/src/common/os_thread.h0000664000000000000000000001331013615011243015207 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_thread.h -- os thread abstraction layer */ #ifndef OS_THREAD_H #define OS_THREAD_H 1 #include #include #ifdef __cplusplus extern "C" { #endif typedef union { long long align; char padding[44]; /* linux: 40 windows: 44 */ } os_mutex_t; typedef union { long long align; char padding[56]; /* linux: 56 windows: 13 */ } os_rwlock_t; typedef union { long long align; char padding[48]; /* linux: 48 windows: 12 */ } os_cond_t; typedef union { long long align; char padding[32]; /* linux: 8 windows: 32 */ } os_thread_t; typedef union { long long align; /* linux: long windows: 8 FreeBSD: 12 */ char padding[16]; /* 16 to be safe */ } os_once_t; #define OS_ONCE_INIT { .padding = {0} } typedef unsigned os_tls_key_t; typedef union { long long align; char padding[56]; /* linux: 56 windows: 8 */ } os_semaphore_t; typedef union { long long align; char padding[56]; /* linux: 56 windows: 8 */ } os_thread_attr_t; typedef union { long long align; char padding[512]; } os_cpu_set_t; #ifdef __FreeBSD__ #define cpu_set_t cpuset_t typedef uintptr_t os_spinlock_t; #else typedef volatile int os_spinlock_t; /* XXX: not implemented on windows */ #endif void os_cpu_zero(os_cpu_set_t *set); void os_cpu_set(size_t cpu, os_cpu_set_t *set); #ifndef _WIN32 #define _When_(...) #endif int os_once(os_once_t *o, void (*func)(void)); int os_tls_key_create(os_tls_key_t *key, void (*destructor)(void *)); int os_tls_key_delete(os_tls_key_t key); int os_tls_set(os_tls_key_t key, const void *value); void *os_tls_get(os_tls_key_t key); int os_mutex_init(os_mutex_t *__restrict mutex); int os_mutex_destroy(os_mutex_t *__restrict mutex); _When_(return == 0, _Acquires_lock_(mutex->lock)) int os_mutex_lock(os_mutex_t *__restrict mutex); _When_(return == 0, _Acquires_lock_(mutex->lock)) int os_mutex_trylock(os_mutex_t *__restrict mutex); int os_mutex_unlock(os_mutex_t *__restrict mutex); /* XXX - non POSIX */ int os_mutex_timedlock(os_mutex_t *__restrict mutex, const struct timespec *abstime); int os_rwlock_init(os_rwlock_t *__restrict rwlock); int os_rwlock_destroy(os_rwlock_t *__restrict rwlock); int os_rwlock_rdlock(os_rwlock_t *__restrict rwlock); int os_rwlock_wrlock(os_rwlock_t *__restrict rwlock); int os_rwlock_tryrdlock(os_rwlock_t *__restrict rwlock); _When_(return == 0, _Acquires_exclusive_lock_(rwlock->lock)) int os_rwlock_trywrlock(os_rwlock_t *__restrict rwlock); _When_(rwlock->is_write != 0, _Requires_exclusive_lock_held_(rwlock->lock)) _When_(rwlock->is_write == 0, _Requires_shared_lock_held_(rwlock->lock)) int os_rwlock_unlock(os_rwlock_t *__restrict rwlock); int os_rwlock_timedrdlock(os_rwlock_t *__restrict rwlock, const struct timespec *abstime); int os_rwlock_timedwrlock(os_rwlock_t *__restrict rwlock, const struct timespec *abstime); int os_spin_init(os_spinlock_t *lock, int pshared); int os_spin_destroy(os_spinlock_t *lock); int os_spin_lock(os_spinlock_t *lock); int os_spin_unlock(os_spinlock_t *lock); int os_spin_trylock(os_spinlock_t *lock); int os_cond_init(os_cond_t *__restrict cond); int os_cond_destroy(os_cond_t *__restrict cond); int os_cond_broadcast(os_cond_t *__restrict cond); int os_cond_signal(os_cond_t *__restrict cond); int os_cond_timedwait(os_cond_t *__restrict cond, os_mutex_t *__restrict mutex, const struct timespec *abstime); int os_cond_wait(os_cond_t *__restrict cond, os_mutex_t *__restrict mutex); /* threading */ int os_thread_create(os_thread_t *thread, const os_thread_attr_t *attr, void *(*start_routine)(void *), void *arg); int os_thread_join(os_thread_t *thread, void **result); void os_thread_self(os_thread_t *thread); /* thread affinity */ int os_thread_setaffinity_np(os_thread_t *thread, size_t set_size, const os_cpu_set_t *set); int os_thread_atfork(void (*prepare)(void), void (*parent)(void), void (*child)(void)); int os_semaphore_init(os_semaphore_t *sem, unsigned value); int os_semaphore_destroy(os_semaphore_t *sem); int os_semaphore_wait(os_semaphore_t *sem); int os_semaphore_trywait(os_semaphore_t *sem); int os_semaphore_post(os_semaphore_t *sem); #ifdef __cplusplus } #endif #endif /* OS_THREAD_H */ pmdk-1.8/src/common/ctl_sds.c0000664000000000000000000000445613615011243014700 0ustar rootroot/* * Copyright 2018-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctl_sds.c -- implementation of the sds CTL namespace */ #include "ctl.h" #include "set.h" #include "out.h" #include "ctl_global.h" static int CTL_READ_HANDLER(at_create)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int *arg_out = arg; *arg_out = SDS_at_create; return 0; } static int CTL_WRITE_HANDLER(at_create)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int arg_in = *(int *)arg; SDS_at_create = arg_in; return 0; } static const struct ctl_argument CTL_ARG(at_create) = CTL_ARG_BOOLEAN; static const struct ctl_node CTL_NODE(sds)[] = { CTL_LEAF_RW(at_create), CTL_NODE_END }; void ctl_sds_register(void) { CTL_REGISTER_MODULE(NULL, sds); } pmdk-1.8/src/common/ctl_prefault.c0000664000000000000000000000545113615011243015725 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctl_prefault.c -- implementation of the prefault CTL namespace */ #include "ctl.h" #include "set.h" #include "out.h" #include "ctl_global.h" static int CTL_READ_HANDLER(at_create)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int *arg_out = arg; *arg_out = Prefault_at_create; return 0; } static int CTL_WRITE_HANDLER(at_create)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int arg_in = *(int *)arg; Prefault_at_create = arg_in; return 0; } static int CTL_READ_HANDLER(at_open)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int *arg_out = arg; *arg_out = Prefault_at_open; return 0; } static int CTL_WRITE_HANDLER(at_open)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int arg_in = *(int *)arg; Prefault_at_open = arg_in; return 0; } static const struct ctl_argument CTL_ARG(at_create) = CTL_ARG_BOOLEAN; static const struct ctl_argument CTL_ARG(at_open) = CTL_ARG_BOOLEAN; static const struct ctl_node CTL_NODE(prefault)[] = { CTL_LEAF_RW(at_create), CTL_LEAF_RW(at_open), CTL_NODE_END }; void ctl_prefault_register(void) { CTL_REGISTER_MODULE(NULL, prefault); } pmdk-1.8/src/common/os_dimm_ndctl.c0000664000000000000000000005010113615011243016044 0ustar rootroot/* * Copyright 2017-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_dimm_ndctl.c -- implementation of DIMMs API based on the ndctl library */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include #include "file.h" #include "out.h" #include "os.h" #include "os_dimm.h" #include "os_badblock.h" #include "badblock.h" #include "vec.h" #define FOREACH_BUS_REGION_NAMESPACE(ctx, bus, region, ndns) \ ndctl_bus_foreach(ctx, bus) \ ndctl_region_foreach(bus, region) \ ndctl_namespace_foreach(region, ndns) \ /* * os_dimm_match_devdax -- (internal) returns 1 if the devdax matches * with the given file, 0 if it doesn't match, * and -1 in case of error. */ static int os_dimm_match_devdax(const os_stat_t *st, const char *devname) { LOG(3, "st %p devname %s", st, devname); if (*devname == '\0') return 0; char path[PATH_MAX]; os_stat_t stat; int ret; if ((ret = snprintf(path, PATH_MAX, "/dev/%s", devname)) < 0) { ERR("snprintf: %d", ret); return -1; } if (os_stat(path, &stat)) { ERR("!stat %s", path); return -1; } if (st->st_rdev == stat.st_rdev) { LOG(4, "found matching device: %s", path); return 1; } LOG(10, "skipping not matching device: %s", path); return 0; } #define BUFF_LENGTH 64 /* * os_dimm_match_fsdax -- (internal) returns 1 if the device matches * with the given file, 0 if it doesn't match, * and -1 in case of error. */ static int os_dimm_match_fsdax(const os_stat_t *st, const char *devname) { LOG(3, "st %p devname %s", st, devname); if (*devname == '\0') return 0; char path[PATH_MAX]; char dev_id[BUFF_LENGTH]; int ret; ret = snprintf(path, PATH_MAX, "/sys/block/%s/dev", devname); if (ret < 0) { ERR("snprintf: %d", ret); return -1; } ret = snprintf(dev_id, BUFF_LENGTH, "%d:%d", major(st->st_dev), minor(st->st_dev)); if (ret < 0) { ERR("snprintf: %d", ret); return -1; } int fd = os_open(path, O_RDONLY); if (fd < 0) { ERR("!open \"%s\"", path); return -1; } char buff[BUFF_LENGTH]; ssize_t nread = read(fd, buff, BUFF_LENGTH); if (nread < 0) { ERR("!read"); os_close(fd); return -1; } os_close(fd); if (nread == 0) { ERR("%s is empty", path); return -1; } if (buff[nread - 1] != '\n') { ERR("%s doesn't end with new line", path); return -1; } buff[nread - 1] = '\0'; if (strcmp(buff, dev_id) == 0) { LOG(4, "found matching device: %s", path); return 1; } LOG(10, "skipping not matching device: %s", path); return 0; } /* * os_dimm_region_namespace -- (internal) returns the region * (and optionally the namespace) * where the given file is located */ static int os_dimm_region_namespace(struct ndctl_ctx *ctx, const os_stat_t *st, struct ndctl_region **pregion, struct ndctl_namespace **pndns) { LOG(3, "ctx %p stat %p pregion %p pnamespace %p", ctx, st, pregion, pndns); struct ndctl_bus *bus; struct ndctl_region *region; struct ndctl_namespace *ndns; ASSERTne(pregion, NULL); *pregion = NULL; if (pndns) *pndns = NULL; enum file_type type = util_stat_get_type(st); if (type == OTHER_ERROR) return -1; FOREACH_BUS_REGION_NAMESPACE(ctx, bus, region, ndns) { struct ndctl_btt *btt; struct ndctl_dax *dax = NULL; struct ndctl_pfn *pfn; const char *devname; if ((dax = ndctl_namespace_get_dax(ndns))) { if (type == TYPE_NORMAL) continue; ASSERTeq(type, TYPE_DEVDAX); struct daxctl_region *dax_region; dax_region = ndctl_dax_get_daxctl_region(dax); if (!dax_region) { ERR("!cannot find dax region"); return -1; } struct daxctl_dev *dev; daxctl_dev_foreach(dax_region, dev) { devname = daxctl_dev_get_devname(dev); int ret = os_dimm_match_devdax(st, devname); if (ret < 0) return ret; if (ret) { *pregion = region; if (pndns) *pndns = ndns; return 0; } } } else { if (type == TYPE_DEVDAX) continue; ASSERTeq(type, TYPE_NORMAL); if ((btt = ndctl_namespace_get_btt(ndns))) { devname = ndctl_btt_get_block_device(btt); } else if ((pfn = ndctl_namespace_get_pfn(ndns))) { devname = ndctl_pfn_get_block_device(pfn); } else { devname = ndctl_namespace_get_block_device(ndns); } int ret = os_dimm_match_fsdax(st, devname); if (ret < 0) return ret; if (ret) { *pregion = region; if (pndns) *pndns = ndns; return 0; } } } LOG(10, "did not found any matching device"); return 0; } /* * os_dimm_interleave_set -- (internal) returns set of dimms * where the pool file is located */ static struct ndctl_interleave_set * os_dimm_interleave_set(struct ndctl_ctx *ctx, const os_stat_t *st) { LOG(3, "ctx %p stat %p", ctx, st); struct ndctl_region *region = NULL; if (os_dimm_region_namespace(ctx, st, ®ion, NULL)) return NULL; return region ? ndctl_region_get_interleave_set(region) : NULL; } /* * os_dimm_uid -- returns a file uid based on dimms uids * * if uid == null then function will return required buffer size */ int os_dimm_uid(const char *path, char *uid, size_t *buff_len) { LOG(3, "path %s, uid %p, len %lu", path, uid, *buff_len); os_stat_t st; struct ndctl_ctx *ctx; struct ndctl_interleave_set *set; struct ndctl_dimm *dimm; int ret = 0; if (os_stat(path, &st)) { ERR("!stat %s", path); return -1; } if (ndctl_new(&ctx)) { ERR("!ndctl_new"); return -1; } if (uid == NULL) { *buff_len = 1; /* '\0' */ } set = os_dimm_interleave_set(ctx, &st); if (set == NULL) goto end; if (uid == NULL) { ndctl_dimm_foreach_in_interleave_set(set, dimm) { *buff_len += strlen(ndctl_dimm_get_unique_id(dimm)); } goto end; } size_t len = 1; ndctl_dimm_foreach_in_interleave_set(set, dimm) { const char *dimm_uid = ndctl_dimm_get_unique_id(dimm); len += strlen(dimm_uid); if (len > *buff_len) { ret = -1; goto end; } strncat(uid, dimm_uid, *buff_len); } end: ndctl_unref(ctx); return ret; } static long long os_dimm_usc_dimm(struct ndctl_dimm *dimm) { long long ret = ndctl_dimm_get_dirty_shutdown(dimm); if (ret < 0) ERR( "Cannot read unsafe shutdown count. For more information please check https://github.com/pmem/issues/issues/1039"); return ret; } /* * os_dimm_usc -- returns unsafe shutdown count */ int os_dimm_usc(const char *path, uint64_t *usc) { LOG(3, "path %s, uid %p", path, usc); os_stat_t st; struct ndctl_ctx *ctx; int ret = -1; *usc = 0; if (os_stat(path, &st)) { ERR("!stat %s", path); return -1; } if (ndctl_new(&ctx)) { ERR("!ndctl_new"); return -1; } struct ndctl_interleave_set *iset = os_dimm_interleave_set(ctx, &st); if (iset == NULL) goto out; struct ndctl_dimm *dimm; ndctl_dimm_foreach_in_interleave_set(iset, dimm) { long long dimm_usc = os_dimm_usc_dimm(dimm); if (dimm_usc < 0) goto err; *usc += (unsigned long long)dimm_usc; } out: ret = 0; err: ndctl_unref(ctx); return ret; } /* * os_dimm_get_namespace_bounds -- (internal) returns the bounds * (offset and size) of the given namespace * relative to the beginning of its region */ static int os_dimm_get_namespace_bounds(struct ndctl_region *region, struct ndctl_namespace *ndns, unsigned long long *ns_offset, unsigned long long *ns_size) { LOG(3, "region %p namespace %p ns_offset %p ns_size %p", region, ndns, ns_offset, ns_size); struct ndctl_pfn *pfn = ndctl_namespace_get_pfn(ndns); struct ndctl_dax *dax = ndctl_namespace_get_dax(ndns); ASSERTne(ns_offset, NULL); ASSERTne(ns_size, NULL); if (pfn) { *ns_offset = ndctl_pfn_get_resource(pfn); if (*ns_offset == ULLONG_MAX) { ERR("!(pfn) cannot read offset of the namespace"); return -1; } *ns_size = ndctl_pfn_get_size(pfn); if (*ns_size == ULLONG_MAX) { ERR("!(pfn) cannot read size of the namespace"); return -1; } LOG(10, "(pfn) ns_offset 0x%llx ns_size %llu", *ns_offset, *ns_size); } else if (dax) { *ns_offset = ndctl_dax_get_resource(dax); if (*ns_offset == ULLONG_MAX) { ERR("!(dax) cannot read offset of the namespace"); return -1; } *ns_size = ndctl_dax_get_size(dax); if (*ns_size == ULLONG_MAX) { ERR("!(dax) cannot read size of the namespace"); return -1; } LOG(10, "(dax) ns_offset 0x%llx ns_size %llu", *ns_offset, *ns_size); } else { /* raw or btt */ *ns_offset = ndctl_namespace_get_resource(ndns); if (*ns_offset == ULLONG_MAX) { ERR("!(raw/btt) cannot read offset of the namespace"); return -1; } *ns_size = ndctl_namespace_get_size(ndns); if (*ns_size == ULLONG_MAX) { ERR("!(raw/btt) cannot read size of the namespace"); return -1; } LOG(10, "(raw/btt) ns_offset 0x%llx ns_size %llu", *ns_offset, *ns_size); } unsigned long long region_offset = ndctl_region_get_resource(region); if (region_offset == ULLONG_MAX) { ERR("!cannot read offset of the region"); return -1; } LOG(10, "region_offset 0x%llx", region_offset); *ns_offset -= region_offset; return 0; } /* * os_dimm_namespace_get_badblocks_by_region -- (internal) returns bad blocks * in the given namespace using the * universal region interface. * * This function works for all types of namespaces, but requires read access to * privileged device information. */ static int os_dimm_namespace_get_badblocks_by_region(struct ndctl_region *region, struct ndctl_namespace *ndns, struct badblocks *bbs) { LOG(3, "region %p, namespace %p", region, ndns); ASSERTne(bbs, NULL); unsigned long long ns_beg, ns_size, ns_end; unsigned long long bb_beg, bb_end; unsigned long long beg, end; VEC(bbsvec, struct bad_block) bbv = VEC_INITIALIZER; bbs->ns_resource = 0; bbs->bb_cnt = 0; bbs->bbv = NULL; if (os_dimm_get_namespace_bounds(region, ndns, &ns_beg, &ns_size)) { LOG(1, "cannot read namespace's bounds"); return -1; } ns_end = ns_beg + ns_size - 1; LOG(10, "namespace: begin %llu, end %llu size %llu (in 512B sectors)", B2SEC(ns_beg), B2SEC(ns_end + 1) - 1, B2SEC(ns_size)); struct badblock *bb; ndctl_region_badblock_foreach(region, bb) { /* * libndctl returns offset and length of a bad block * both expressed in 512B sectors and offset is relative * to the beginning of the region. */ bb_beg = SEC2B(bb->offset); bb_end = bb_beg + SEC2B(bb->len) - 1; LOG(10, "region bad block: begin %llu end %llu length %u (in 512B sectors)", bb->offset, bb->offset + bb->len - 1, bb->len); if (bb_beg > ns_end || ns_beg > bb_end) continue; beg = (bb_beg > ns_beg) ? bb_beg : ns_beg; end = (bb_end < ns_end) ? bb_end : ns_end; /* * Form a new bad block structure with offset and length * expressed in bytes and offset relative to the beginning * of the namespace. */ struct bad_block bbn; bbn.offset = beg - ns_beg; bbn.length = (unsigned)(end - beg + 1); bbn.nhealthy = NO_HEALTHY_REPLICA; /* unknown healthy replica */ /* add the new bad block to the vector */ if (VEC_PUSH_BACK(&bbv, bbn)) { VEC_DELETE(&bbv); return -1; } LOG(4, "namespace bad block: begin %llu end %llu length %llu (in 512B sectors)", B2SEC(beg - ns_beg), B2SEC(end - ns_beg), B2SEC(end - beg) + 1); } bbs->bb_cnt = (unsigned)VEC_SIZE(&bbv); bbs->bbv = VEC_ARR(&bbv); bbs->ns_resource = ns_beg + ndctl_region_get_resource(region); LOG(4, "number of bad blocks detected: %u", bbs->bb_cnt); return 0; } /* * os_dimm_namespace_get_badblocks_by_namespace -- (internal) returns bad blocks * in the given namespace using the * block device badblocks interface. * * This function works only for fsdax, but does not require any special * permissions. */ static int os_dimm_namespace_get_badblocks_by_namespace(struct ndctl_namespace *ndns, struct badblocks *bbs) { ASSERTeq(ndctl_namespace_get_mode(ndns), NDCTL_NS_MODE_FSDAX); VEC(bbsvec, struct bad_block) bbv = VEC_INITIALIZER; struct badblock *bb; ndctl_namespace_badblock_foreach(ndns, bb) { struct bad_block bbn; bbn.offset = SEC2B(bb->offset); bbn.length = (unsigned)SEC2B(bb->len); bbn.nhealthy = NO_HEALTHY_REPLICA; /* unknown healthy replica */ if (VEC_PUSH_BACK(&bbv, bbn)) { VEC_DELETE(&bbv); return -1; } } bbs->bb_cnt = (unsigned)VEC_SIZE(&bbv); bbs->bbv = VEC_ARR(&bbv); bbs->ns_resource = 0; return 0; } /* * os_dimm_namespace_get_badblocks -- (internal) returns bad blocks in the given * namespace, using the least privileged * path. */ static int os_dimm_namespace_get_badblocks(struct ndctl_region *region, struct ndctl_namespace *ndns, struct badblocks *bbs) { /* * Only the new NDCTL versions have the namespace badblock iterator, * when compiled with older versions, the library needs to rely on the * old region interface. */ if (ndctl_namespace_get_mode(ndns) == NDCTL_NS_MODE_FSDAX) return os_dimm_namespace_get_badblocks_by_namespace(ndns, bbs); return os_dimm_namespace_get_badblocks_by_region(region, ndns, bbs); } /* * os_dimm_files_namespace_bus -- (internal) returns bus where the given * file is located */ static int os_dimm_files_namespace_bus(struct ndctl_ctx *ctx, const char *path, struct ndctl_bus **pbus) { LOG(3, "ctx %p path %s pbus %p", ctx, path, pbus); ASSERTne(pbus, NULL); struct ndctl_region *region; struct ndctl_namespace *ndns; os_stat_t st; if (os_stat(path, &st)) { ERR("!stat %s", path); return -1; } int rv = os_dimm_region_namespace(ctx, &st, ®ion, &ndns); if (rv) { LOG(1, "getting region and namespace failed"); return -1; } if (!region) { ERR("region unknown"); return -1; } *pbus = ndctl_region_get_bus(region); return 0; } /* * os_dimm_files_namespace_badblocks_bus -- (internal) returns badblocks * in the namespace where the given * file is located * (optionally returns also the bus) */ static int os_dimm_files_namespace_badblocks_bus(struct ndctl_ctx *ctx, const char *path, struct ndctl_bus **pbus, struct badblocks *bbs) { LOG(3, "ctx %p path %s pbus %p badblocks %p", ctx, path, pbus, bbs); struct ndctl_region *region; struct ndctl_namespace *ndns; os_stat_t st; if (os_stat(path, &st)) { ERR("!stat %s", path); return -1; } int rv = os_dimm_region_namespace(ctx, &st, ®ion, &ndns); if (rv) { LOG(1, "getting region and namespace failed"); return -1; } memset(bbs, 0, sizeof(*bbs)); if (region == NULL || ndns == NULL) return 0; if (pbus) *pbus = ndctl_region_get_bus(region); return os_dimm_namespace_get_badblocks(region, ndns, bbs); } /* * os_dimm_files_namespace_badblocks -- returns badblocks in the namespace * where the given file is located */ int os_dimm_files_namespace_badblocks(const char *path, struct badblocks *bbs) { LOG(3, "path %s", path); struct ndctl_ctx *ctx; if (ndctl_new(&ctx)) { ERR("!ndctl_new"); return -1; } int ret = os_dimm_files_namespace_badblocks_bus(ctx, path, NULL, bbs); ndctl_unref(ctx); return ret; } /* * os_dimm_devdax_clear_one_badblock -- (internal) clear one bad block * in the dax device */ static int os_dimm_devdax_clear_one_badblock(struct ndctl_bus *bus, unsigned long long address, unsigned long long length) { LOG(3, "bus %p address 0x%llx length %llu (bytes)", bus, address, length); int ret = 0; struct ndctl_cmd *cmd_ars_cap = ndctl_bus_cmd_new_ars_cap(bus, address, length); if (cmd_ars_cap == NULL) { ERR("failed to create cmd (bus '%s')", ndctl_bus_get_provider(bus)); return -1; } if ((ret = ndctl_cmd_submit(cmd_ars_cap)) < 0) { ERR("failed to submit cmd (bus '%s')", ndctl_bus_get_provider(bus)); goto out_ars_cap; } struct ndctl_range range; if (ndctl_cmd_ars_cap_get_range(cmd_ars_cap, &range)) { ERR("failed to get ars_cap range\n"); goto out_ars_cap; } struct ndctl_cmd *cmd_clear_error = ndctl_bus_cmd_new_clear_error( range.address, range.length, cmd_ars_cap); if ((ret = ndctl_cmd_submit(cmd_clear_error)) < 0) { ERR("failed to submit cmd (bus '%s')", ndctl_bus_get_provider(bus)); goto out_clear_error; } size_t cleared = ndctl_cmd_clear_error_get_cleared(cmd_clear_error); LOG(4, "cleared %zu out of %llu bad blocks", cleared, length); ret = cleared == length ? 0 : -1; out_clear_error: ndctl_cmd_unref(cmd_clear_error); out_ars_cap: ndctl_cmd_unref(cmd_ars_cap); return ret; } /* * os_dimm_devdax_clear_badblocks -- clear the given bad blocks in the dax * device (or all of them if 'pbbs' is not set) */ int os_dimm_devdax_clear_badblocks(const char *path, struct badblocks *pbbs) { LOG(3, "path %s badblocks %p", path, pbbs); struct ndctl_ctx *ctx; struct ndctl_bus *bus; int ret = -1; if (ndctl_new(&ctx)) { ERR("!ndctl_new"); return -1; } struct badblocks *bbs = badblocks_new(); if (bbs == NULL) goto exit_free_all; if (pbbs) { ret = os_dimm_files_namespace_bus(ctx, path, &bus); if (ret) { LOG(1, "getting bad blocks' bus failed -- %s", path); goto exit_free_all; } badblocks_delete(bbs); bbs = pbbs; } else { ret = os_dimm_files_namespace_badblocks_bus(ctx, path, &bus, bbs); if (ret) { LOG(1, "getting bad blocks for the file failed -- %s", path); goto exit_free_all; } } if (bbs->bb_cnt == 0 || bbs->bbv == NULL) /* OK - no bad blocks found */ goto exit_free_all; LOG(4, "clearing %u bad block(s)...", bbs->bb_cnt); unsigned b; for (b = 0; b < bbs->bb_cnt; b++) { LOG(4, "clearing bad block: offset %llu length %u (in 512B sectors)", B2SEC(bbs->bbv[b].offset), B2SEC(bbs->bbv[b].length)); ret = os_dimm_devdax_clear_one_badblock(bus, bbs->bbv[b].offset + bbs->ns_resource, bbs->bbv[b].length); if (ret) { LOG(1, "failed to clear bad block: offset %llu length %u (in 512B sectors)", B2SEC(bbs->bbv[b].offset), B2SEC(bbs->bbv[b].length)); goto exit_free_all; } } exit_free_all: if (!pbbs) badblocks_delete(bbs); ndctl_unref(ctx); return ret; } /* * os_dimm_devdax_clear_badblocks_all -- clear all bad blocks in the dax device */ int os_dimm_devdax_clear_badblocks_all(const char *path) { LOG(3, "path %s", path); return os_dimm_devdax_clear_badblocks(path, NULL); } pmdk-1.8/src/common/vecq.h0000664000000000000000000001022613615011243014200 0ustar rootroot/* * Copyright 2018-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * vecq.h -- vector queue (FIFO) interface */ #ifndef PMDK_VECQ_H #define PMDK_VECQ_H 1 #include #include "util.h" #include "out.h" #include "alloc.h" #ifdef __cplusplus extern "C" { #endif #define VECQ_INIT_SIZE (64) #define VECQ(name, type)\ struct name {\ type *buffer;\ size_t capacity;\ size_t front;\ size_t back;\ } #define VECQ_INIT(vec) do {\ (vec)->buffer = NULL;\ (vec)->capacity = 0;\ (vec)->front = 0;\ (vec)->back = 0;\ } while (0) #define VECQ_REINIT(vec) do {\ VALGRIND_ANNOTATE_NEW_MEMORY((vec), sizeof(*vec));\ VALGRIND_ANNOTATE_NEW_MEMORY((vec)->buffer,\ (sizeof(*(vec)->buffer) * ((vec)->capacity)));\ (vec)->front = 0;\ (vec)->back = 0;\ } while (0) #define VECQ_FRONT_POS(vec)\ ((vec)->front & ((vec)->capacity - 1)) #define VECQ_BACK_POS(vec)\ ((vec)->back & ((vec)->capacity - 1)) #define VECQ_FRONT(vec)\ (vec)->buffer[VECQ_FRONT_POS(vec)] #define VECQ_BACK(vec)\ (vec)->buffer[VECQ_BACK_POS(vec)] #define VECQ_DEQUEUE(vec)\ ((vec)->buffer[(((vec)->front++) & ((vec)->capacity - 1))]) #define VECQ_SIZE(vec)\ ((vec)->back - (vec)->front) static inline int realloc_set(void **buf, size_t s) { void *tbuf = Realloc(*buf, s); if (tbuf == NULL) { ERR("!Realloc"); return -1; } *buf = tbuf; return 0; } #define VECQ_NCAPACITY(vec)\ ((vec)->capacity == 0 ? VECQ_INIT_SIZE : (vec)->capacity * 2) #define VECQ_GROW(vec)\ (realloc_set((void **)&(vec)->buffer,\ VECQ_NCAPACITY(vec) * sizeof(*(vec)->buffer)) ? -1 :\ (memcpy((vec)->buffer + (vec)->capacity, (vec)->buffer,\ VECQ_FRONT_POS(vec) * sizeof(*(vec)->buffer)),\ (vec)->front = VECQ_FRONT_POS(vec),\ (vec)->back = (vec)->front + (vec)->capacity,\ (vec)->capacity = VECQ_NCAPACITY(vec),\ 0\ )) #define VECQ_INSERT(vec, element)\ (VECQ_BACK(vec) = element, (vec)->back += 1, 0) #define VECQ_ENQUEUE(vec, element)\ ((vec)->capacity == VECQ_SIZE(vec) ?\ (VECQ_GROW(vec) == 0 ? VECQ_INSERT(vec, element) : -1) :\ VECQ_INSERT(vec, element)) #define VECQ_CAPACITY(vec)\ ((vec)->capacity) #define VECQ_FOREACH(el, vec)\ for (size_t _vec_i = 0;\ _vec_i < VECQ_SIZE(vec) &&\ (((el) = (vec)->buffer[_vec_i & ((vec)->capacity - 1)]), 1);\ ++_vec_i) #define VECQ_FOREACH_REVERSE(el, vec)\ for (size_t _vec_i = VECQ_SIZE(vec);\ _vec_i > 0 &&\ (((el) = (vec)->buffer[(_vec_i - 1) & ((vec)->capacity - 1)]), 1);\ --_vec_i) #define VECQ_CLEAR(vec) do {\ (vec)->front = 0;\ (vec)->back = 0;\ } while (0) #define VECQ_DELETE(vec) do {\ Free((vec)->buffer);\ (vec)->buffer = NULL;\ (vec)->capacity = 0;\ (vec)->front = 0;\ (vec)->back = 0;\ } while (0) #ifdef __cplusplus } #endif #endif /* PMDK_VECQ_H */ pmdk-1.8/src/common/ctl.h0000664000000000000000000001476213615011243014035 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctl.h -- internal declaration of statistics and control related structures */ #ifndef PMDK_CTL_H #define PMDK_CTL_H 1 #include "queue.h" #include "errno.h" #include "out.h" #ifdef __cplusplus extern "C" { #endif struct ctl; struct ctl_index { const char *name; long value; PMDK_SLIST_ENTRY(ctl_index) entry; }; PMDK_SLIST_HEAD(ctl_indexes, ctl_index); enum ctl_query_source { CTL_UNKNOWN_QUERY_SOURCE, /* query executed directly from the program */ CTL_QUERY_PROGRAMMATIC, /* query executed from the config file */ CTL_QUERY_CONFIG_INPUT, MAX_CTL_QUERY_SOURCE }; enum ctl_query_type { CTL_QUERY_READ, CTL_QUERY_WRITE, CTL_QUERY_RUNNABLE, MAX_CTL_QUERY_TYPE }; typedef int (*node_callback)(void *ctx, enum ctl_query_source type, void *arg, struct ctl_indexes *indexes); enum ctl_node_type { CTL_NODE_UNKNOWN, CTL_NODE_NAMED, CTL_NODE_LEAF, CTL_NODE_INDEXED, MAX_CTL_NODE }; typedef int (*ctl_arg_parser)(const void *arg, void *dest, size_t dest_size); struct ctl_argument_parser { size_t dest_offset; /* offset of the field inside of the argument */ size_t dest_size; /* size of the field inside of the argument */ ctl_arg_parser parser; }; struct ctl_argument { size_t dest_size; /* sizeof the entire argument */ struct ctl_argument_parser parsers[]; /* array of 'fields' in arg */ }; #define sizeof_member(t, m) sizeof(((t *)0)->m) #define CTL_ARG_PARSER(t, p)\ {0, sizeof(t), p} #define CTL_ARG_PARSER_STRUCT(t, m, p)\ {offsetof(t, m), sizeof_member(t, m), p} #define CTL_ARG_PARSER_END {0, 0, NULL} /* * CTL Tree node structure, do not use directly. All the necessary functionality * is provided by the included macros. */ struct ctl_node { const char *name; enum ctl_node_type type; node_callback cb[MAX_CTL_QUERY_TYPE]; const struct ctl_argument *arg; const struct ctl_node *children; }; struct ctl *ctl_new(void); void ctl_delete(struct ctl *stats); int ctl_load_config_from_string(struct ctl *ctl, void *ctx, const char *cfg_string); int ctl_load_config_from_file(struct ctl *ctl, void *ctx, const char *cfg_file); /* Use through CTL_REGISTER_MODULE, never directly */ void ctl_register_module_node(struct ctl *c, const char *name, struct ctl_node *n); int ctl_arg_boolean(const void *arg, void *dest, size_t dest_size); #define CTL_ARG_BOOLEAN {sizeof(int),\ {{0, sizeof(int), ctl_arg_boolean},\ CTL_ARG_PARSER_END}}; int ctl_arg_integer(const void *arg, void *dest, size_t dest_size); #define CTL_ARG_INT {sizeof(int),\ {{0, sizeof(int), ctl_arg_integer},\ CTL_ARG_PARSER_END}}; #define CTL_ARG_LONG_LONG {sizeof(long long),\ {{0, sizeof(long long), ctl_arg_integer},\ CTL_ARG_PARSER_END}}; int ctl_arg_string(const void *arg, void *dest, size_t dest_size); #define CTL_ARG_STRING(len) {len,\ {{0, len, ctl_arg_string},\ CTL_ARG_PARSER_END}}; #define CTL_STR(name) #name #define CTL_NODE_END {NULL, CTL_NODE_UNKNOWN, {NULL, NULL, NULL}, NULL, NULL} #define CTL_NODE(name, ...)\ ctl_node_##__VA_ARGS__##_##name int ctl_query(struct ctl *ctl, void *ctx, enum ctl_query_source source, const char *name, enum ctl_query_type type, void *arg); /* Declaration of a new child node */ #define CTL_CHILD(name, ...)\ {CTL_STR(name), CTL_NODE_NAMED, {NULL, NULL, NULL}, NULL,\ (struct ctl_node *)CTL_NODE(name, __VA_ARGS__)} /* Declaration of a new indexed node */ #define CTL_INDEXED(name, ...)\ {CTL_STR(name), CTL_NODE_INDEXED, {NULL, NULL, NULL}, NULL,\ (struct ctl_node *)CTL_NODE(name, __VA_ARGS__)} #define CTL_READ_HANDLER(name, ...)\ ctl_##__VA_ARGS__##_##name##_read #define CTL_WRITE_HANDLER(name, ...)\ ctl_##__VA_ARGS__##_##name##_write #define CTL_RUNNABLE_HANDLER(name, ...)\ ctl_##__VA_ARGS__##_##name##_runnable #define CTL_ARG(name)\ ctl_arg_##name /* * Declaration of a new read-only leaf. If used the corresponding read function * must be declared by CTL_READ_HANDLER macro. */ #define CTL_LEAF_RO(name, ...)\ {CTL_STR(name), CTL_NODE_LEAF, \ {CTL_READ_HANDLER(name, __VA_ARGS__), NULL, NULL}, NULL, NULL} /* * Declaration of a new write-only leaf. If used the corresponding write * function must be declared by CTL_WRITE_HANDLER macro. */ #define CTL_LEAF_WO(name, ...)\ {CTL_STR(name), CTL_NODE_LEAF, \ {NULL, CTL_WRITE_HANDLER(name, __VA_ARGS__), NULL},\ &CTL_ARG(name), NULL} /* * Declaration of a new runnable leaf. If used the corresponding run * function must be declared by CTL_RUNNABLE_HANDLER macro. */ #define CTL_LEAF_RUNNABLE(name, ...)\ {CTL_STR(name), CTL_NODE_LEAF, \ {NULL, NULL, CTL_RUNNABLE_HANDLER(name, __VA_ARGS__)},\ NULL, NULL} /* * Declaration of a new read-write leaf. If used both read and write function * must be declared by CTL_READ_HANDLER and CTL_WRITE_HANDLER macros. */ #define CTL_LEAF_RW(name)\ {CTL_STR(name), CTL_NODE_LEAF,\ {CTL_READ_HANDLER(name), CTL_WRITE_HANDLER(name), NULL},\ &CTL_ARG(name), NULL} #define CTL_REGISTER_MODULE(_ctl, name)\ ctl_register_module_node((_ctl), CTL_STR(name),\ (struct ctl_node *)CTL_NODE(name)) #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/mmap_posix.c0000664000000000000000000001545113615011243015416 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * mmap_posix.c -- memory-mapped files for Posix */ #include #include #include #include "mmap.h" #include "out.h" #include "os.h" #define PROCMAXLEN 2048 /* maximum expected line length in /proc files */ char *Mmap_mapfile = OS_MAPFILE; /* Should be modified only for testing */ #ifdef __FreeBSD__ static const char * const sscanf_os = "%p %p"; #else static const char * const sscanf_os = "%p-%p"; #endif /* * util_map_hint_unused -- use /proc to determine a hint address for mmap() * * This is a helper function for util_map_hint(). * It opens up /proc/self/maps and looks for the first unused address * in the process address space that is: * - greater or equal 'minaddr' argument, * - large enough to hold range of given length, * - aligned to the specified unit. * * Asking for aligned address like this will allow the DAX code to use large * mappings. It is not an error if mmap() ignores the hint and chooses * different address. */ char * util_map_hint_unused(void *minaddr, size_t len, size_t align) { LOG(3, "minaddr %p len %zu align %zu", minaddr, len, align); ASSERT(align > 0); FILE *fp; if ((fp = os_fopen(Mmap_mapfile, "r")) == NULL) { ERR("!%s", Mmap_mapfile); return MAP_FAILED; } char line[PROCMAXLEN]; /* for fgets() */ char *lo = NULL; /* beginning of current range in maps file */ char *hi = NULL; /* end of current range in maps file */ char *raddr = minaddr; /* ignore regions below 'minaddr' */ if (raddr == NULL) raddr += Pagesize; raddr = (char *)roundup((uintptr_t)raddr, align); while (fgets(line, PROCMAXLEN, fp) != NULL) { /* check for range line */ if (sscanf(line, sscanf_os, &lo, &hi) == 2) { LOG(4, "%p-%p", lo, hi); if (lo > raddr) { if ((uintptr_t)(lo - raddr) >= len) { LOG(4, "unused region of size %zu " "found at %p", lo - raddr, raddr); break; } else { LOG(4, "region is too small: %zu < %zu", lo - raddr, len); } } if (hi > raddr) { raddr = (char *)roundup((uintptr_t)hi, align); LOG(4, "nearest aligned addr %p", raddr); } if (raddr == NULL) { LOG(4, "end of address space reached"); break; } } } /* * Check for a case when this is the last unused range in the address * space, but is not large enough. (very unlikely) */ if ((raddr != NULL) && (UINTPTR_MAX - (uintptr_t)raddr < len)) { ERR("end of address space reached"); raddr = MAP_FAILED; } fclose(fp); LOG(3, "returning %p", raddr); return raddr; } /* * util_map_hint -- determine hint address for mmap() * * If PMEM_MMAP_HINT environment variable is not set, we let the system to pick * the randomized mapping address. Otherwise, a user-defined hint address * is used. * * ALSR in 64-bit Linux kernel uses 28-bit of randomness for mmap * (bit positions 12-39), which means the base mapping address is randomized * within [0..1024GB] range, with 4KB granularity. Assuming additional * 1GB alignment, it results in 1024 possible locations. * * Configuring the hint address via PMEM_MMAP_HINT environment variable * disables address randomization. In such case, the function will search for * the first unused, properly aligned region of given size, above the specified * address. */ char * util_map_hint(size_t len, size_t req_align) { LOG(3, "len %zu req_align %zu", len, req_align); char *hint_addr = MAP_FAILED; /* choose the desired alignment based on the requested length */ size_t align = util_map_hint_align(len, req_align); if (Mmap_no_random) { LOG(4, "user-defined hint %p", Mmap_hint); hint_addr = util_map_hint_unused(Mmap_hint, len, align); } else { /* * Create dummy mapping to find an unused region of given size. * Request for increased size for later address alignment. * Use MAP_PRIVATE with read-only access to simulate * zero cost for overcommit accounting. Note: MAP_NORESERVE * flag is ignored if overcommit is disabled (mode 2). */ char *addr = mmap(NULL, len + align, PROT_READ, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0); if (addr == MAP_FAILED) { ERR("!mmap MAP_ANONYMOUS"); } else { LOG(4, "system choice %p", addr); hint_addr = (char *)roundup((uintptr_t)addr, align); munmap(addr, len + align); } } LOG(4, "hint %p", hint_addr); return hint_addr; } /* * util_map_sync -- memory map given file into memory, if MAP_SHARED flag is * provided it attempts to use MAP_SYNC flag. Otherwise it fallbacks to * mmap(2). */ void * util_map_sync(void *addr, size_t len, int proto, int flags, int fd, os_off_t offset, int *map_sync) { LOG(15, "addr %p len %zu proto %x flags %x fd %d offset %ld " "map_sync %p", addr, len, proto, flags, fd, offset, map_sync); if (map_sync) *map_sync = 0; /* if map_sync is NULL do not even try to mmap with MAP_SYNC flag */ if (!map_sync || flags & MAP_PRIVATE) return mmap(addr, len, proto, flags, fd, offset); /* MAP_SHARED */ void *ret = mmap(addr, len, proto, flags | MAP_SHARED_VALIDATE | MAP_SYNC, fd, offset); if (ret != MAP_FAILED) { LOG(4, "mmap with MAP_SYNC succeeded"); *map_sync = 1; return ret; } if (errno == EINVAL || errno == ENOTSUP) { LOG(4, "mmap with MAP_SYNC not supported"); return mmap(addr, len, proto, flags, fd, offset); } /* other error */ return MAP_FAILED; } pmdk-1.8/src/common/uuid_linux.c0000664000000000000000000000476613615011243015436 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * uuid_linux.c -- pool set utilities with OS-specific implementation */ #include #include #include #include "uuid.h" #include "os.h" #include "out.h" /* * util_uuid_generate -- generate a uuid * * This function reads the uuid string from /proc/sys/kernel/random/uuid * It converts this string into the binary uuid format as specified in * https://www.ietf.org/rfc/rfc4122.txt */ int util_uuid_generate(uuid_t uuid) { char uu[POOL_HDR_UUID_STR_LEN]; int fd = os_open(POOL_HDR_UUID_GEN_FILE, O_RDONLY); if (fd < 0) { /* Fatal error */ LOG(2, "!open(uuid)"); return -1; } ssize_t num = read(fd, uu, POOL_HDR_UUID_STR_LEN); if (num < POOL_HDR_UUID_STR_LEN) { /* Fatal error */ LOG(2, "!read(uuid)"); os_close(fd); return -1; } os_close(fd); uu[POOL_HDR_UUID_STR_LEN - 1] = '\0'; int ret = util_uuid_from_string(uu, (struct uuid *)uuid); if (ret < 0) return ret; return 0; } pmdk-1.8/src/common/ctl_global.h0000664000000000000000000000406613615011243015351 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctl_global.h -- definitions for the global CTL namespace */ #ifndef PMDK_CTL_GLOBAL_H #define PMDK_CTL_GLOBAL_H 1 #ifdef __cplusplus extern "C" { #endif extern void ctl_prefault_register(void); extern void ctl_sds_register(void); extern void ctl_fallocate_register(void); extern void ctl_cow_register(void); static inline void ctl_global_register(void) { ctl_prefault_register(); ctl_sds_register(); ctl_fallocate_register(); ctl_cow_register(); } #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/ctl.c0000664000000000000000000003361513615011243014026 0ustar rootroot/* * Copyright 2016-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctl.c -- implementation of the interface for examination and modification of * the library's internal state */ #include "ctl.h" #include "os.h" #include "alloc.h" #define CTL_MAX_ENTRIES 100 #define MAX_CONFIG_FILE_LEN (1 << 20) /* 1 megabyte */ #define CTL_STRING_QUERY_SEPARATOR ";" #define CTL_NAME_VALUE_SEPARATOR "=" #define CTL_QUERY_NODE_SEPARATOR "." #define CTL_VALUE_ARG_SEPARATOR "," static int ctl_global_first_free = 0; static struct ctl_node CTL_NODE(global)[CTL_MAX_ENTRIES]; /* * This is the top level node of the ctl tree structure. Each node can contain * children and leaf nodes. * * Internal nodes simply create a new path in the tree whereas child nodes are * the ones providing the read/write functionality by the means of callbacks. * * Each tree node must be NULL-terminated, CTL_NODE_END macro is provided for * convenience. */ struct ctl { struct ctl_node root[CTL_MAX_ENTRIES]; int first_free; }; /* * ctl_find_node -- (internal) searches for a matching entry point in the * provided nodes * * The caller is responsible for freeing all of the allocated indexes, * regardless of the return value. */ static const struct ctl_node * ctl_find_node(const struct ctl_node *nodes, const char *name, struct ctl_indexes *indexes) { LOG(3, "nodes %p name %s indexes %p", nodes, name, indexes); const struct ctl_node *n = NULL; char *sptr = NULL; char *parse_str = Strdup(name); if (parse_str == NULL) return NULL; char *node_name = strtok_r(parse_str, CTL_QUERY_NODE_SEPARATOR, &sptr); /* * Go through the string and separate tokens that correspond to nodes * in the main ctl tree. */ while (node_name != NULL) { char *endptr; /* * Ignore errno from strtol: FreeBSD returns EINVAL if no * conversion is performed. Linux does not, but endptr * check is valid in both cases. */ int tmp_errno = errno; long index_value = strtol(node_name, &endptr, 0); errno = tmp_errno; struct ctl_index *index_entry = NULL; if (endptr != node_name) { /* a valid index */ index_entry = Malloc(sizeof(*index_entry)); if (index_entry == NULL) goto error; index_entry->value = index_value; PMDK_SLIST_INSERT_HEAD(indexes, index_entry, entry); } for (n = &nodes[0]; n->name != NULL; ++n) { if (index_entry && n->type == CTL_NODE_INDEXED) break; else if (strcmp(n->name, node_name) == 0) break; } if (n->name == NULL) goto error; if (index_entry) index_entry->name = n->name; nodes = n->children; node_name = strtok_r(NULL, CTL_QUERY_NODE_SEPARATOR, &sptr); } Free(parse_str); return n; error: Free(parse_str); return NULL; } /* * ctl_delete_indexes -- * (internal) removes and frees all entries on the index list */ static void ctl_delete_indexes(struct ctl_indexes *indexes) { while (!PMDK_SLIST_EMPTY(indexes)) { struct ctl_index *index = PMDK_SLIST_FIRST(indexes); PMDK_SLIST_REMOVE_HEAD(indexes, entry); Free(index); } } /* * ctl_parse_args -- (internal) parses a string argument based on the node * structure */ static void * ctl_parse_args(const struct ctl_argument *arg_proto, char *arg) { ASSERTne(arg, NULL); char *dest_arg = Malloc(arg_proto->dest_size); if (dest_arg == NULL) { ERR("!Malloc"); return NULL; } char *sptr = NULL; char *arg_sep = strtok_r(arg, CTL_VALUE_ARG_SEPARATOR, &sptr); for (const struct ctl_argument_parser *p = arg_proto->parsers; p->parser != NULL; ++p) { ASSERT(p->dest_offset + p->dest_size <= arg_proto->dest_size); if (arg_sep == NULL) { ERR("!strtok_r"); goto error_parsing; } if (p->parser(arg_sep, dest_arg + p->dest_offset, p->dest_size) != 0) goto error_parsing; arg_sep = strtok_r(NULL, CTL_VALUE_ARG_SEPARATOR, &sptr); } return dest_arg; error_parsing: Free(dest_arg); return NULL; } /* * ctl_query_get_real_args -- (internal) returns a pointer with actual argument * structure as required by the node callback */ static void * ctl_query_get_real_args(const struct ctl_node *n, void *write_arg, enum ctl_query_source source) { void *real_arg = NULL; switch (source) { case CTL_QUERY_CONFIG_INPUT: real_arg = ctl_parse_args(n->arg, write_arg); break; case CTL_QUERY_PROGRAMMATIC: real_arg = write_arg; break; default: ASSERT(0); break; } return real_arg; } /* * ctl_query_cleanup_real_args -- (internal) cleanups relevant argument * structures allocated as a result of the get_real_args call */ static void ctl_query_cleanup_real_args(const struct ctl_node *n, void *real_arg, enum ctl_query_source source) { switch (source) { case CTL_QUERY_CONFIG_INPUT: Free(real_arg); break; case CTL_QUERY_PROGRAMMATIC: break; default: ASSERT(0); break; } } /* * ctl_exec_query_read -- (internal) calls the read callback of a node */ static int ctl_exec_query_read(void *ctx, const struct ctl_node *n, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { if (arg == NULL) { ERR("read queries require non-NULL argument"); errno = EINVAL; return -1; } return n->cb[CTL_QUERY_READ](ctx, source, arg, indexes); } /* * ctl_exec_query_write -- (internal) calls the write callback of a node */ static int ctl_exec_query_write(void *ctx, const struct ctl_node *n, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { if (arg == NULL) { ERR("write queries require non-NULL argument"); errno = EINVAL; return -1; } void *real_arg = ctl_query_get_real_args(n, arg, source); if (real_arg == NULL) { LOG(1, "Invalid arguments"); return -1; } int ret = n->cb[CTL_QUERY_WRITE](ctx, source, real_arg, indexes); ctl_query_cleanup_real_args(n, real_arg, source); return ret; } /* * ctl_exec_query_runnable -- (internal) calls the run callback of a node */ static int ctl_exec_query_runnable(void *ctx, const struct ctl_node *n, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { return n->cb[CTL_QUERY_RUNNABLE](ctx, source, arg, indexes); } static int (*ctl_exec_query[MAX_CTL_QUERY_TYPE])(void *ctx, const struct ctl_node *n, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) = { ctl_exec_query_read, ctl_exec_query_write, ctl_exec_query_runnable, }; /* * ctl_query -- (internal) parses the name and calls the appropriate methods * from the ctl tree */ int ctl_query(struct ctl *ctl, void *ctx, enum ctl_query_source source, const char *name, enum ctl_query_type type, void *arg) { LOG(3, "ctl %p ctx %p source %d name %s type %d arg %p", ctl, ctx, source, name, type, arg); if (name == NULL) { ERR("invalid query"); errno = EINVAL; return -1; } /* * All of the indexes are put on this list so that the handlers can * easily retrieve the index values. The list is cleared once the ctl * query has been handled. */ struct ctl_indexes indexes; PMDK_SLIST_INIT(&indexes); int ret = -1; const struct ctl_node *n = ctl_find_node(CTL_NODE(global), name, &indexes); if (n == NULL && ctl) { ctl_delete_indexes(&indexes); n = ctl_find_node(ctl->root, name, &indexes); } if (n == NULL || n->type != CTL_NODE_LEAF || n->cb[type] == NULL) { ERR("invalid query entry point %s", name); errno = EINVAL; goto out; } ret = ctl_exec_query[type](ctx, n, source, arg, &indexes); out: ctl_delete_indexes(&indexes); return ret; } /* * ctl_register_module_node -- adds a new node to the CTL tree root. */ void ctl_register_module_node(struct ctl *c, const char *name, struct ctl_node *n) { struct ctl_node *nnode = c == NULL ? &CTL_NODE(global)[ctl_global_first_free++] : &c->root[c->first_free++]; nnode->children = n; nnode->type = CTL_NODE_NAMED; nnode->name = name; } /* * ctl_parse_query -- (internal) splits an entire query string * into name and value */ static int ctl_parse_query(char *qbuf, char **name, char **value) { if (qbuf == NULL) return -1; char *sptr; *name = strtok_r(qbuf, CTL_NAME_VALUE_SEPARATOR, &sptr); if (*name == NULL) return -1; *value = strtok_r(NULL, CTL_NAME_VALUE_SEPARATOR, &sptr); if (*value == NULL) return -1; /* the value itself mustn't include CTL_NAME_VALUE_SEPARATOR */ char *extra = strtok_r(NULL, CTL_NAME_VALUE_SEPARATOR, &sptr); if (extra != NULL) return -1; return 0; } /* * ctl_load_config -- executes the entire query collection from a provider */ static int ctl_load_config(struct ctl *ctl, void *ctx, char *buf) { int r = 0; char *sptr = NULL; /* for internal use of strtok */ char *name; char *value; ASSERTne(buf, NULL); char *qbuf = strtok_r(buf, CTL_STRING_QUERY_SEPARATOR, &sptr); while (qbuf != NULL) { r = ctl_parse_query(qbuf, &name, &value); if (r != 0) { ERR("failed to parse query %s", qbuf); return -1; } r = ctl_query(ctl, ctx, CTL_QUERY_CONFIG_INPUT, name, CTL_QUERY_WRITE, value); if (r < 0 && ctx != NULL) return -1; qbuf = strtok_r(NULL, CTL_STRING_QUERY_SEPARATOR, &sptr); } return 0; } /* * ctl_load_config_from_string -- loads obj configuration from string */ int ctl_load_config_from_string(struct ctl *ctl, void *ctx, const char *cfg_string) { LOG(3, "ctl %p ctx %p cfg_string \"%s\"", ctl, ctx, cfg_string); char *buf = Strdup(cfg_string); if (buf == NULL) { ERR("!Strdup"); return -1; } int ret = ctl_load_config(ctl, ctx, buf); Free(buf); return ret; } /* * ctl_load_config_from_file -- loads obj configuration from file * * This function opens up the config file, allocates a buffer of size equal to * the size of the file, reads its content and sanitizes it for ctl_load_config. */ int ctl_load_config_from_file(struct ctl *ctl, void *ctx, const char *cfg_file) { LOG(3, "ctl %p ctx %p cfg_file \"%s\"", ctl, ctx, cfg_file); int ret = -1; FILE *fp = os_fopen(cfg_file, "r"); if (fp == NULL) return ret; int err; if ((err = fseek(fp, 0, SEEK_END)) != 0) goto error_file_parse; long fsize = ftell(fp); if (fsize == -1) goto error_file_parse; if (fsize > MAX_CONFIG_FILE_LEN) { ERR("Config file too large"); goto error_file_parse; } if ((err = fseek(fp, 0, SEEK_SET)) != 0) goto error_file_parse; char *buf = Zalloc((size_t)fsize + 1); /* +1 for NULL-termination */ if (buf == NULL) { ERR("!Zalloc"); goto error_file_parse; } size_t bufpos = 0; int c; int is_comment_section = 0; while ((c = fgetc(fp)) != EOF) { if (c == '#') is_comment_section = 1; else if (c == '\n') is_comment_section = 0; else if (!is_comment_section && !isspace(c)) buf[bufpos++] = (char)c; } ret = ctl_load_config(ctl, ctx, buf); Free(buf); error_file_parse: (void) fclose(fp); return ret; } /* * ctl_new -- allocates and initializes ctl data structures */ struct ctl * ctl_new(void) { struct ctl *c = Zalloc(sizeof(struct ctl)); if (c == NULL) { ERR("!Zalloc"); return NULL; } c->first_free = 0; return c; } /* * ctl_delete -- deletes ctl */ void ctl_delete(struct ctl *c) { Free(c); } /* * ctl_parse_ll -- (internal) parses and returns a long long signed integer */ static long long ctl_parse_ll(const char *str) { char *endptr; int olderrno = errno; errno = 0; long long val = strtoll(str, &endptr, 0); if (endptr == str || errno != 0) return LLONG_MIN; errno = olderrno; return val; } /* * ctl_arg_boolean -- checks whether the provided argument contains * either a 1 or y or Y. */ int ctl_arg_boolean(const void *arg, void *dest, size_t dest_size) { int *intp = dest; char in = ((char *)arg)[0]; if (tolower(in) == 'y' || in == '1') { *intp = 1; return 0; } else if (tolower(in) == 'n' || in == '0') { *intp = 0; return 0; } return -1; } /* * ctl_arg_integer -- parses signed integer argument */ int ctl_arg_integer(const void *arg, void *dest, size_t dest_size) { long long val = ctl_parse_ll(arg); if (val == LLONG_MIN) return -1; switch (dest_size) { case sizeof(int): if (val > INT_MAX || val < INT_MIN) return -1; *(int *)dest = (int)val; break; case sizeof(long long): *(long long *)dest = val; break; case sizeof(uint8_t): if (val > UINT8_MAX || val < 0) return -1; *(uint8_t *)dest = (uint8_t)val; break; default: ERR("invalid destination size %zu", dest_size); errno = EINVAL; return -1; } return 0; } /* * ctl_arg_string -- verifies length and copies a string argument into a zeroed * buffer */ int ctl_arg_string(const void *arg, void *dest, size_t dest_size) { /* check if the incoming string is longer or equal to dest_size */ if (strnlen(arg, dest_size) == dest_size) return -1; strncpy(dest, arg, dest_size); return 0; } pmdk-1.8/src/common/fs_posix.c0000664000000000000000000000560213615011243015071 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fs_posix.c -- file system traversal Posix implementation */ #include #include "util.h" #include "out.h" #include "vec.h" #include "fs.h" struct fs { FTS *ft; struct fs_entry entry; }; /* * fs_new -- creates fs traversal instance */ struct fs * fs_new(const char *path) { struct fs *f = Zalloc(sizeof(*f)); if (f == NULL) goto error_fs_alloc; const char *paths[2] = {path, NULL}; f->ft = fts_open((char * const *)paths, FTS_COMFOLLOW | FTS_XDEV, NULL); if (f->ft == NULL) goto error_fts_open; return f; error_fts_open: Free(f); error_fs_alloc: return NULL; } /* * fs_read -- reads an entry from the fs path */ struct fs_entry * fs_read(struct fs *f) { FTSENT *entry = fts_read(f->ft); if (entry == NULL) return NULL; switch (entry->fts_info) { case FTS_D: f->entry.type = FS_ENTRY_DIRECTORY; break; case FTS_F: f->entry.type = FS_ENTRY_FILE; break; case FTS_SL: f->entry.type = FS_ENTRY_SYMLINK; break; default: f->entry.type = FS_ENTRY_OTHER; break; } f->entry.name = entry->fts_name; f->entry.namelen = entry->fts_namelen; f->entry.path = entry->fts_path; f->entry.pathlen = entry->fts_pathlen; f->entry.level = entry->fts_level; return &f->entry; } /* * fs_delete -- deletes a fs traversal instance */ void fs_delete(struct fs *f) { fts_close(f->ft); Free(f); } pmdk-1.8/src/common/fault_injection.h0000664000000000000000000000416413615011243016423 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef COMMON_FAULT_INJECTION #define COMMON_FAULT_INJECTION #include #ifdef __cplusplus extern "C" { #endif enum pmem_allocation_type { PMEM_MALLOC, PMEM_REALLOC }; #if FAULT_INJECTION void common_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at); int common_fault_injection_enabled(void); #else static inline void common_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { abort(); } static inline int common_fault_injection_enabled(void) { return 0; } #endif #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/libpmemcommon.vcxproj0000664000000000000000000001607213615011243017351 0ustar rootroot Debug x64 Release x64 {492BAA3D-0D5D-478E-9765-500463AE69AA} Win32Proj libpmemcommon 10.0.16299.0 StaticLibrary true v140 NotSet StaticLibrary true v140 NotSet true .lib $(SolutionDir)\include;$(SolutionDir)\windows\include;$(VC_IncludePath);$(WindowsSDK_IncludePath); true .lib $(SolutionDir)\include;$(SolutionDir)\windows\include;$(VC_IncludePath);$(WindowsSDK_IncludePath); NotUsing Level3 PMDK_UTF8_API;NTDDI_VERSION=NTDDI_WIN10_RS1;_DEBUG;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions) platform.h CompileAsC MultiThreadedDebugDLL false true Console true ntdll.lib;%(AdditionalDependencies) true NotUsing Level3 PMDK_UTF8_API;NTDDI_VERSION=NTDDI_WIN10_RS1;_DEBUG;_CRT_SECURE_NO_WARNINGS;%(PreprocessorDefinitions) platform.h CompileAsC MaxSpeed MultiThreadedDLL Default false ProgramDatabase true Console true ntdll.lib;%(AdditionalDependencies) true pmdk-1.8/src/common/badblock_none.c0000664000000000000000000000567413615011243016030 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * badblock_none.c - fake bad block API */ #include #include "out.h" #include "os_badblock.h" /* * os_badblocks_check_file -- check if the file contains bad blocks * * Return value: * -1 : an error * 0 : no bad blocks * 1 : bad blocks detected */ int os_badblocks_check_file(const char *file) { LOG(3, "file %s", file); /* not supported */ errno = ENOTSUP; return -1; } /* * os_badblocks_count -- returns number of bad blocks in the file * or -1 in case of an error */ long os_badblocks_count(const char *file) { LOG(3, "file %s", file); /* not supported */ errno = ENOTSUP; return -1; } /* * os_badblocks_get -- returns list of bad blocks in the file */ int os_badblocks_get(const char *file, struct badblocks *bbs) { LOG(3, "file %s", file); /* not supported */ errno = ENOTSUP; return -1; } /* * os_badblocks_clear -- clears the given bad blocks in a file * (regular file or dax device) */ int os_badblocks_clear(const char *file, struct badblocks *bbs) { LOG(3, "file %s badblocks %p", file, bbs); /* not supported */ errno = ENOTSUP; return -1; } /* * os_badblocks_clear_all -- clears all bad blocks in a file * (regular file or dax device) */ int os_badblocks_clear_all(const char *file) { LOG(3, "file %s", file); /* not supported */ errno = ENOTSUP; return -1; } pmdk-1.8/src/common/dlsym.h0000664000000000000000000000567013615011243014401 0ustar rootroot/* * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * dlsym.h -- dynamic linking utilities with library-specific implementation */ #ifndef PMDK_DLSYM_H #define PMDK_DLSYM_H 1 #include "out.h" #if defined(USE_LIBDL) && !defined(_WIN32) #include /* * util_dlopen -- calls real dlopen() */ static inline void * util_dlopen(const char *filename) { LOG(3, "filename %s", filename); return dlopen(filename, RTLD_NOW); } /* * util_dlerror -- calls real dlerror() */ static inline char * util_dlerror(void) { return dlerror(); } /* * util_dlsym -- calls real dlsym() */ static inline void * util_dlsym(void *handle, const char *symbol) { LOG(3, "handle %p symbol %s", handle, symbol); return dlsym(handle, symbol); } /* * util_dlclose -- calls real dlclose() */ static inline int util_dlclose(void *handle) { LOG(3, "handle %p", handle); return dlclose(handle); } #else /* empty functions */ /* * util_dlopen -- empty function */ static inline void * util_dlopen(const char *filename) { errno = ENOSYS; return NULL; } /* * util_dlerror -- empty function */ static inline char * util_dlerror(void) { errno = ENOSYS; return NULL; } /* * util_dlsym -- empty function */ static inline void * util_dlsym(void *handle, const char *symbol) { errno = ENOSYS; return NULL; } /* * util_dlclose -- empty function */ static inline int util_dlclose(void *handle) { errno = ENOSYS; return 0; } #endif #endif pmdk-1.8/src/common/os_deep_linux.c0000664000000000000000000001356613615011243016104 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_deep_linux.c -- Linux abstraction layer */ #define _GNU_SOURCE #include #include #include #include "out.h" #include "os.h" #include "mmap.h" #include "file.h" #include "libpmem.h" #include "os_deep.h" /* * os_deep_flush_write -- (internal) perform write to deep_flush file * on given region_id */ static int os_deep_flush_write(int region_id) { LOG(3, "region_id %d", region_id); char deep_flush_path[PATH_MAX]; int deep_flush_fd; snprintf(deep_flush_path, PATH_MAX, "/sys/bus/nd/devices/region%d/deep_flush", region_id); if ((deep_flush_fd = os_open(deep_flush_path, O_WRONLY)) < 0) { LOG(1, "!os_open(\"%s\", O_WRONLY)", deep_flush_path); return -1; } if (write(deep_flush_fd, "1", 1) != 1) { LOG(1, "!write(%d, \"1\")", deep_flush_fd); int oerrno = errno; os_close(deep_flush_fd); errno = oerrno; return -1; } os_close(deep_flush_fd); return 0; } /* * os_deep_type -- (internal) perform deep operation based on a pmem * mapping type */ static int os_deep_type(const struct map_tracker *mt, void *addr, size_t len) { LOG(15, "mt %p addr %p len %zu", mt, addr, len); switch (mt->type) { case PMEM_DEV_DAX: pmem_drain(); if (os_deep_flush_write(mt->region_id) < 0) { if (errno == ENOENT) { errno = ENOTSUP; LOG(1, "!deep_flush not supported"); } else { LOG(2, "cannot write to deep_flush" "in region %d", mt->region_id); } return -1; } return 0; case PMEM_MAP_SYNC: return pmem_msync(addr, len); default: ASSERT(0); return -1; } } /* * os_range_deep_common -- perform deep action of given address range */ int os_range_deep_common(uintptr_t addr, size_t len) { LOG(3, "addr 0x%016" PRIxPTR " len %zu", addr, len); while (len != 0) { const struct map_tracker *mt = util_range_find(addr, len); /* no more overlapping track regions or NOT a device DAX */ if (mt == NULL) { LOG(15, "pmem_msync addr %p, len %lu", (void *)addr, len); return pmem_msync((void *)addr, len); } /* * For range that intersects with the found mapping * write to (Device DAX) deep_flush file. * Call msync for the non-intersecting part. */ if (mt->base_addr > addr) { size_t curr_len = mt->base_addr - addr; if (curr_len > len) curr_len = len; if (pmem_msync((void *)addr, curr_len) != 0) return -1; len -= curr_len; if (len == 0) return 0; addr = mt->base_addr; } size_t mt_in_len = mt->end_addr - addr; size_t persist_len = MIN(len, mt_in_len); if (os_deep_type(mt, (void *)addr, persist_len)) return -1; if (mt->end_addr >= addr + len) return 0; len -= mt_in_len; addr = mt->end_addr; } return 0; } /* * os_part_deep_common -- common function to handle both * deep_persist and deep_drain part flush cases. */ int os_part_deep_common(struct pool_replica *rep, unsigned partidx, void *addr, size_t len, int flush) { LOG(3, "part %p part %d addr %p len %lu flush %d", rep, partidx, addr, len, flush); if (!rep->is_pmem) { /* * In case of part on non-pmem call msync on the range * to deep flush the data. Deep drain is empty as all * data is msynced to persistence. */ if (!flush) return 0; if (pmem_msync(addr, len)) { LOG(1, "pmem_msync(%p, %lu)", addr, len); return -1; } return 0; } struct pool_set_part part = rep->part[partidx]; /* Call deep flush if it was requested */ if (flush) { LOG(15, "pmem_deep_flush addr %p, len %lu", addr, len); pmem_deep_flush(addr, len); } /* * Before deep drain call normal drain to ensure that data * is at least in WPQ. */ pmem_drain(); if (part.is_dev_dax) { /* * During deep_drain for part on device DAX search for * device region id, and perform WPQ flush on found * device DAX region. */ int region_id = util_ddax_region_find(part.path); if (region_id < 0) { if (errno == ENOENT) { errno = ENOTSUP; LOG(1, "!deep_flush not supported"); } else { LOG(1, "invalid dax_region id %d", region_id); } return -1; } if (os_deep_flush_write(region_id)) { LOG(1, "ddax_deep_flush_write(%d)", region_id); return -1; } } else { /* * For deep_drain on normal pmem it is enough to * call msync on one page. */ if (pmem_msync(addr, MIN(Pagesize, len))) { LOG(1, "pmem_msync(%p, %lu)", addr, len); return -1; } } return 0; } pmdk-1.8/src/common/extent_linux.c0000664000000000000000000001267613615011243015776 0ustar rootroot/* * Copyright 2018-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * extent_linux.c - implementation of the linux fs extent query API */ #include #include #include #include #include #include "file.h" #include "out.h" #include "extent.h" #include "alloc.h" /* * os_extents_common -- (internal) common part of getting extents * of the given file * * Returns: number of extents the file consists of or -1 in case of an error. * Sets: file descriptor, struct fiemap and struct extents. */ static long os_extents_common(const char *path, struct extents *exts, int *pfd, struct fiemap **pfmap) { LOG(3, "path %s exts %p pfd %p pfmap %p", path, exts, pfd, pfmap); int fd = open(path, O_RDONLY); if (fd == -1) { ERR("!open %s", path); return -1; } enum file_type type = util_fd_get_type(fd); if (type < 0) goto error_close; struct stat st; if (fstat(fd, &st) < 0) { ERR("!fstat %d", fd); goto error_close; } if (exts->extents_count == 0) { LOG(10, "%s: block size: %li", path, (long int)st.st_blksize); exts->blksize = (uint64_t)st.st_blksize; } /* devdax does not have any extents */ if (type == TYPE_DEVDAX) { close(fd); return 0; } struct fiemap *fmap = Zalloc(sizeof(struct fiemap)); if (fmap == NULL) { ERR("!malloc"); goto error_close; } fmap->fm_start = 0; fmap->fm_length = (size_t)st.st_size; fmap->fm_flags = 0; fmap->fm_extent_count = 0; if (ioctl(fd, FS_IOC_FIEMAP, fmap) != 0) { ERR("!ioctl %d", fd); goto error_free; } if (exts->extents_count == 0) { exts->extents_count = fmap->fm_mapped_extents; LOG(4, "%s: number of extents: %u", path, exts->extents_count); } else if (exts->extents_count != fmap->fm_mapped_extents) { ERR("number of extents differs (was: %u, is: %u)", exts->extents_count, fmap->fm_mapped_extents); goto error_free; } *pfd = fd; *pfmap = fmap; return exts->extents_count; error_free: Free(fmap); error_close: close(fd); return -1; } /* * os_extents_count -- get number of extents of the given file * (and optionally read its block size) */ long os_extents_count(const char *path, struct extents *exts) { LOG(3, "path %s extents %p", path, exts); struct fiemap *fmap = NULL; int fd = -1; ASSERTne(exts, NULL); memset(exts, 0, sizeof(*exts)); long ret = os_extents_common(path, exts, &fd, &fmap); Free(fmap); if (fd != -1) close(fd); return ret; } /* * os_extents_get -- get extents of the given file * (and optionally read its block size) */ int os_extents_get(const char *path, struct extents *exts) { LOG(3, "path %s extents %p", path, exts); struct fiemap *fmap = NULL; int fd = -1; int ret = -1; ASSERTne(exts, NULL); if (exts->extents_count == 0) return 0; ASSERTne(exts->extents, NULL); if (os_extents_common(path, exts, &fd, &fmap) <= 0) goto error_free; struct fiemap *newfmap = Realloc(fmap, sizeof(struct fiemap) + fmap->fm_mapped_extents * sizeof(struct fiemap_extent)); if (newfmap == NULL) { ERR("!Realloc"); goto error_free; } fmap = newfmap; fmap->fm_extent_count = fmap->fm_mapped_extents; memset(fmap->fm_extents, 0, fmap->fm_mapped_extents * sizeof(struct fiemap_extent)); if (ioctl(fd, FS_IOC_FIEMAP, fmap) != 0) { ERR("!ioctl %d", fd); goto error_free; } if (fmap->fm_extent_count > 0) { LOG(10, "file %s has %u extents:", path, fmap->fm_extent_count); } unsigned e; for (e = 0; e < fmap->fm_extent_count; e++) { exts->extents[e].offset_physical = fmap->fm_extents[e].fe_physical; exts->extents[e].offset_logical = fmap->fm_extents[e].fe_logical; exts->extents[e].length = fmap->fm_extents[e].fe_length; LOG(10, " #%u: off_phy: %lu off_log: %lu len: %lu", e, exts->extents[e].offset_physical, exts->extents[e].offset_logical, exts->extents[e].length); } ret = 0; error_free: Free(fmap); if (fd != -1) close(fd); return ret; } pmdk-1.8/src/common/pool_hdr.c0000664000000000000000000002401013615011243015037 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pool_hdr.c -- pool header utilities */ #include #include #include #include #include "out.h" #include "pool_hdr.h" /* Determine ISA for which PMDK is currently compiled */ #if defined(__x86_64) || defined(_M_X64) /* x86 -- 64 bit */ #define PMDK_MACHINE PMDK_MACHINE_X86_64 #define PMDK_MACHINE_CLASS PMDK_MACHINE_CLASS_64 #elif defined(__aarch64__) /* 64 bit ARM not supported yet */ #define PMDK_MACHINE PMDK_MACHINE_AARCH64 #define PMDK_MACHINE_CLASS PMDK_MACHINE_CLASS_64 #elif defined(__PPC64__) #define PMDK_MACHINE PMDK_MACHINE_PPC64 #define PMDK_MACHINE_CLASS PMDK_MACHINE_CLASS_64 #else /* add appropriate definitions here when porting PMDK to another ISA */ #error unable to recognize ISA at compile time #endif /* * arch_machine -- (internal) determine endianness */ static uint8_t arch_data(void) { uint16_t word = (PMDK_DATA_BE << 8) + PMDK_DATA_LE; return ((uint8_t *)&word)[0]; } /* * util_get_arch_flags -- get architecture identification flags */ void util_get_arch_flags(struct arch_flags *arch_flags) { memset(arch_flags, 0, sizeof(*arch_flags)); arch_flags->machine = PMDK_MACHINE; arch_flags->machine_class = PMDK_MACHINE_CLASS; arch_flags->data = arch_data(); arch_flags->alignment_desc = alignment_desc(); } /* * util_convert2le_hdr -- convert pool_hdr into little-endian byte order */ void util_convert2le_hdr(struct pool_hdr *hdrp) { hdrp->major = htole32(hdrp->major); hdrp->features.compat = htole32(hdrp->features.compat); hdrp->features.incompat = htole32(hdrp->features.incompat); hdrp->features.ro_compat = htole32(hdrp->features.ro_compat); hdrp->arch_flags.alignment_desc = htole64(hdrp->arch_flags.alignment_desc); hdrp->arch_flags.machine = htole16(hdrp->arch_flags.machine); hdrp->crtime = htole64(hdrp->crtime); hdrp->checksum = htole64(hdrp->checksum); } /* * util_convert2h_hdr_nocheck -- convert pool_hdr into host byte order */ void util_convert2h_hdr_nocheck(struct pool_hdr *hdrp) { hdrp->major = le32toh(hdrp->major); hdrp->features.compat = le32toh(hdrp->features.compat); hdrp->features.incompat = le32toh(hdrp->features.incompat); hdrp->features.ro_compat = le32toh(hdrp->features.ro_compat); hdrp->crtime = le64toh(hdrp->crtime); hdrp->arch_flags.machine = le16toh(hdrp->arch_flags.machine); hdrp->arch_flags.alignment_desc = le64toh(hdrp->arch_flags.alignment_desc); hdrp->checksum = le64toh(hdrp->checksum); } /* * util_arch_flags_check -- validates arch_flags */ int util_check_arch_flags(const struct arch_flags *arch_flags) { struct arch_flags cur_af; int ret = 0; util_get_arch_flags(&cur_af); if (!util_is_zeroed(&arch_flags->reserved, sizeof(arch_flags->reserved))) { ERR("invalid reserved values"); ret = -1; } if (arch_flags->machine != cur_af.machine) { ERR("invalid machine value"); ret = -1; } if (arch_flags->data != cur_af.data) { ERR("invalid data value"); ret = -1; } if (arch_flags->machine_class != cur_af.machine_class) { ERR("invalid machine_class value"); ret = -1; } if (arch_flags->alignment_desc != cur_af.alignment_desc) { ERR("invalid alignment_desc value"); ret = -1; } return ret; } /* * util_get_unknown_features -- filter out unknown features flags */ features_t util_get_unknown_features(features_t features, features_t known) { features_t unknown; unknown.compat = util_get_not_masked_bits( features.compat, known.compat); unknown.incompat = util_get_not_masked_bits( features.incompat, known.incompat); unknown.ro_compat = util_get_not_masked_bits( features.ro_compat, known.ro_compat); return unknown; } /* * util_feature_check -- check features masks */ int util_feature_check(struct pool_hdr *hdrp, features_t known) { LOG(3, "hdrp %p features {incompat %#x ro_compat %#x compat %#x}", hdrp, known.incompat, known.ro_compat, known.compat); features_t unknown = util_get_unknown_features(hdrp->features, known); /* check incompatible ("must support") features */ if (unknown.incompat) { ERR("unsafe to continue due to unknown incompat "\ "features: %#x", unknown.incompat); errno = EINVAL; return -1; } /* check RO-compatible features (force RO if unsupported) */ if (unknown.ro_compat) { ERR("switching to read-only mode due to unknown ro_compat "\ "features: %#x", unknown.ro_compat); return 0; } /* check compatible ("may") features */ if (unknown.compat) { LOG(3, "ignoring unknown compat features: %#x", unknown.compat); } return 1; } /* * util_feature_cmp -- compares features with reference * * returns 1 if features and reference match and 0 otherwise */ int util_feature_cmp(features_t features, features_t ref) { LOG(3, "features {incompat %#x ro_compat %#x compat %#x} " "ref {incompat %#x ro_compat %#x compat %#x}", features.incompat, features.ro_compat, features.compat, ref.incompat, ref.ro_compat, ref.compat); return features.compat == ref.compat && features.incompat == ref.incompat && features.ro_compat == ref.ro_compat; } /* * util_feature_is_zero -- check if features flags are zeroed * * returns 1 if features is zeroed and 0 otherwise */ int util_feature_is_zero(features_t features) { const uint32_t bits = features.compat | features.incompat | features.ro_compat; return bits ? 0 : 1; } /* * util_feature_is_set -- check if feature flag is set in features * * returns 1 if feature flag is set and 0 otherwise */ int util_feature_is_set(features_t features, features_t flag) { uint32_t bits = 0; bits |= features.compat & flag.compat; bits |= features.incompat & flag.incompat; bits |= features.ro_compat & flag.ro_compat; return bits ? 1 : 0; } /* * util_feature_enable -- enable feature */ void util_feature_enable(features_t *features, features_t new_feature) { #define FEATURE_ENABLE(flags, X) \ (flags) |= (X) FEATURE_ENABLE(features->compat, new_feature.compat); FEATURE_ENABLE(features->incompat, new_feature.incompat); FEATURE_ENABLE(features->ro_compat, new_feature.ro_compat); #undef FEATURE_ENABLE } /* * util_feature_disable -- (internal) disable feature */ void util_feature_disable(features_t *features, features_t old_feature) { #define FEATURE_DISABLE(flags, X) \ (flags) &= ~(X) FEATURE_DISABLE(features->compat, old_feature.compat); FEATURE_DISABLE(features->incompat, old_feature.incompat); FEATURE_DISABLE(features->ro_compat, old_feature.ro_compat); #undef FEATURE_DISABLE } static const features_t feature_2_pmempool_feature_map[] = { FEAT_INCOMPAT(SINGLEHDR), /* PMEMPOOL_FEAT_SINGLEHDR */ FEAT_INCOMPAT(CKSUM_2K), /* PMEMPOOL_FEAT_CKSUM_2K */ FEAT_INCOMPAT(SDS), /* PMEMPOOL_FEAT_SHUTDOWN_STATE */ FEAT_COMPAT(CHECK_BAD_BLOCKS), /* PMEMPOOL_FEAT_CHECK_BAD_BLOCKS */ }; #define FEAT_2_PMEMPOOL_FEATURE_MAP_SIZE \ ARRAY_SIZE(feature_2_pmempool_feature_map) static const char *str_2_pmempool_feature_map[] = { "SINGLEHDR", "CKSUM_2K", "SHUTDOWN_STATE", "CHECK_BAD_BLOCKS", }; #define PMEMPOOL_FEATURE_2_STR_MAP_SIZE ARRAY_SIZE(str_2_pmempool_feature_map) /* * util_str2feature -- convert string to feat_flags value */ features_t util_str2feature(const char *str) { /* all features have to be named in incompat_features_str array */ COMPILE_ERROR_ON(FEAT_2_PMEMPOOL_FEATURE_MAP_SIZE != PMEMPOOL_FEATURE_2_STR_MAP_SIZE); for (uint32_t f = 0; f < PMEMPOOL_FEATURE_2_STR_MAP_SIZE; ++f) { if (strcmp(str, str_2_pmempool_feature_map[f]) == 0) { return feature_2_pmempool_feature_map[f]; } } return features_zero; } /* * util_feature2pmempool_feature -- convert feature to pmempool_feature */ uint32_t util_feature2pmempool_feature(features_t feat) { for (uint32_t pf = 0; pf < FEAT_2_PMEMPOOL_FEATURE_MAP_SIZE; ++pf) { const features_t *record = &feature_2_pmempool_feature_map[pf]; if (util_feature_cmp(feat, *record)) { return pf; } } return UINT32_MAX; } /* * util_str2pmempool_feature -- convert string to uint32_t enum pmempool_feature * equivalent */ uint32_t util_str2pmempool_feature(const char *str) { features_t fval = util_str2feature(str); if (util_feature_is_zero(fval)) return UINT32_MAX; return util_feature2pmempool_feature(fval); } /* * util_feature2str -- convert uint32_t feature to string */ const char * util_feature2str(features_t features, features_t *found) { for (uint32_t i = 0; i < FEAT_2_PMEMPOOL_FEATURE_MAP_SIZE; ++i) { const features_t *record = &feature_2_pmempool_feature_map[i]; if (util_feature_is_set(features, *record)) { if (found) memcpy(found, record, sizeof(features_t)); return str_2_pmempool_feature_map[i]; } } return NULL; } pmdk-1.8/src/common/badblock.c0000664000000000000000000001607513615011243015006 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * badblock.c - common part of implementation of bad blocks API */ #define _GNU_SOURCE #include #include #include #include "file.h" #include "os.h" #include "out.h" #include "extent.h" #include "os_badblock.h" #include "badblock.h" /* * badblocks_new -- zalloc bad blocks structure */ struct badblocks * badblocks_new(void) { LOG(3, " "); struct badblocks *bbs = Zalloc(sizeof(struct badblocks)); if (bbs == NULL) { ERR("!Zalloc"); } return bbs; } /* * badblocks_delete -- free bad blocks structure */ void badblocks_delete(struct badblocks *bbs) { LOG(3, "badblocks %p", bbs); if (bbs == NULL) return; Free(bbs->bbv); Free(bbs); } /* helper structure for badblocks_check_file_cb() */ struct check_file_cb { int n_files_bbs; /* number of files with bad blocks */ int create; /* poolset is just being created */ }; /* * badblocks_check_file_cb -- (internal) callback checking bad blocks * in the given file */ static int badblocks_check_file_cb(struct part_file *pf, void *arg) { LOG(3, "part_file %p arg %p", pf, arg); struct check_file_cb *pcfcb = arg; if (pf->is_remote) { /* * Remote replicas are checked for bad blocks * while opening in util_pool_open_remote(). */ return 0; } int exists = util_file_exists(pf->part->path); if (exists < 0) return -1; if (!exists) /* the part does not exist, so it has no bad blocks */ return 0; int ret = os_badblocks_check_file(pf->part->path); if (ret < 0) { ERR("checking the pool file for bad blocks failed -- '%s'", pf->part->path); return -1; } if (ret > 0) { ERR("part file contains bad blocks -- '%s'", pf->part->path); pcfcb->n_files_bbs++; pf->part->has_bad_blocks = 1; } return 0; } /* * badblocks_check_poolset -- checks if the pool set contains bad blocks * * Return value: * -1 error * 0 pool set does not contain bad blocks * 1 pool set contains bad blocks */ int badblocks_check_poolset(struct pool_set *set, int create) { LOG(3, "set %p create %i", set, create); struct check_file_cb cfcb; cfcb.n_files_bbs = 0; cfcb.create = create; if (util_poolset_foreach_part_struct(set, badblocks_check_file_cb, &cfcb)) { return -1; } if (cfcb.n_files_bbs) { LOG(1, "%i pool file(s) contain bad blocks", cfcb.n_files_bbs); set->has_bad_blocks = 1; } return (cfcb.n_files_bbs > 0); } /* * badblocks_clear_poolset_cb -- (internal) callback clearing bad blocks * in the given file */ static int badblocks_clear_poolset_cb(struct part_file *pf, void *arg) { LOG(3, "part_file %p arg %p", pf, arg); int *create = arg; if (pf->is_remote) { /* XXX not supported yet */ LOG(1, "WARNING: clearing bad blocks in remote replicas is not supported yet -- '%s:%s'", pf->remote->node_addr, pf->remote->pool_desc); return 0; } if (*create) { /* * Poolset is just being created - check if file exists * and if we can read it. */ int exists = util_file_exists(pf->part->path); if (exists < 0) return -1; if (!exists) return 0; } int ret = os_badblocks_clear_all(pf->part->path); if (ret < 0) { ERR("clearing bad blocks in the pool file failed -- '%s'", pf->part->path); errno = EIO; return -1; } pf->part->has_bad_blocks = 0; return 0; } /* * badblocks_clear_poolset -- clears bad blocks in the pool set */ int badblocks_clear_poolset(struct pool_set *set, int create) { LOG(3, "set %p create %i", set, create); if (util_poolset_foreach_part_struct(set, badblocks_clear_poolset_cb, &create)) { return -1; } set->has_bad_blocks = 0; return 0; } /* * badblocks_recovery_file_alloc -- allocate name of bad block recovery file, * the allocated name has to be freed * using Free() */ char * badblocks_recovery_file_alloc(const char *file, unsigned rep, unsigned part) { LOG(3, "file %s rep %u part %u", file, rep, part); char bbs_suffix[64]; char *path; sprintf(bbs_suffix, "_r%u_p%u_badblocks.txt", rep, part); size_t len_file = strlen(file); size_t len_bbs_suffix = strlen(bbs_suffix); size_t len_path = len_file + len_bbs_suffix; path = Malloc(len_path + 1); if (path == NULL) { ERR("!Malloc"); return NULL; } strcpy(path, file); strcat(path, bbs_suffix); return path; } /* * badblocks_recovery_file_exists -- check if any bad block recovery file exists * * Returns: * 0 when there are no bad block recovery files and * 1 when there is at least one bad block recovery file. */ int badblocks_recovery_file_exists(struct pool_set *set) { LOG(3, "set %p", set); int recovery_file_exists = 0; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; /* XXX: not supported yet */ if (rep->remote) continue; for (unsigned p = 0; p < rep->nparts; ++p) { const char *path = PART(rep, p)->path; int exists = util_file_exists(path); if (exists < 0) return -1; if (!exists) { /* part file does not exist - skip it */ continue; } char *rec_file = badblocks_recovery_file_alloc(set->path, r, p); if (rec_file == NULL) { LOG(1, "allocating name of bad block recovery file failed"); return -1; } exists = util_file_exists(rec_file); if (exists < 0) { Free(rec_file); return -1; } if (exists) { LOG(3, "bad block recovery file exists: %s", rec_file); recovery_file_exists = 1; } Free(rec_file); if (recovery_file_exists) return 1; } } return 0; } pmdk-1.8/src/common/os_auto_flush_none.c0000664000000000000000000000336313615011243017132 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include "os_auto_flush.h" #include "out.h" /* * os_auto_flush -- check if platform supports auto flush for all regions */ int os_auto_flush(void) { LOG(15, NULL); return 0; } pmdk-1.8/src/common/rand.c0000664000000000000000000000730113615011243014161 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rand.c -- random utils */ #include #include #include #include #include #include "rand.h" #ifdef _WIN32 #include #include #else #include #endif /* * hash64 -- a u64 -> u64 hash */ uint64_t hash64(uint64_t x) { x += 0x9e3779b97f4a7c15; x = (x ^ (x >> 30)) * 0xbf58476d1ce4e5b9; x = (x ^ (x >> 27)) * 0x94d049bb133111eb; return x ^ (x >> 31); } /* * xoshiro256** random generator * * Fastest available good PRNG as of 2018 (sub-nanosecond per entry), produces * much better output than old stuff like rand() or Mersenne's Twister. * * By David Blackman and Sebastiano Vigna; PD/CC0 2018. * * It has a period of 2²⁵⁶-1, excluding all-zero state; it must always get * initialized to avoid that zero. */ static inline uint64_t rotl(const uint64_t x, int k) { /* optimized to a single instruction on x86 */ return (x << k) | (x >> (64 - k)); } /* * rnd64_r -- return 64-bits of randomness */ uint64_t rnd64_r(rng_t *state) { uint64_t *s = (void *)state; const uint64_t result = rotl(s[1] * 5, 7) * 9; const uint64_t t = s[1] << 17; s[2] ^= s[0]; s[3] ^= s[1]; s[1] ^= s[2]; s[0] ^= s[3]; s[2] ^= t; s[3] = rotl(s[3], 45); return result; } /* * randomize_r -- initialize random generator * * Seed of 0 means random. */ void randomize_r(rng_t *state, uint64_t seed) { if (!seed) { #ifdef SYS_getrandom /* We want getentropy() but ancient Red Hat lacks it. */ if (!syscall(SYS_getrandom, state, sizeof(rng_t), 0)) return; /* nofail, but ENOSYS on kernel < 3.16 */ #elif _WIN32 #pragma comment(lib, "Bcrypt.lib") if (BCryptGenRandom(NULL, (PUCHAR)state, sizeof(rng_t), BCRYPT_USE_SYSTEM_PREFERRED_RNG)) { return; } #endif seed = (uint64_t)getpid(); } uint64_t *s = (void *)state; s[0] = hash64(seed); s[1] = hash64(s[0]); s[2] = hash64(s[1]); s[3] = hash64(s[2]); } static rng_t global_rng; /* * rnd64 -- global state version of rnd64_t */ uint64_t rnd64(void) { return rnd64_r(&global_rng); } /* * randomize -- initialize global RNG */ void randomize(uint64_t seed) { randomize_r(&global_rng, seed); } pmdk-1.8/src/common/os_auto_flush.h0000664000000000000000000000346713615011243016125 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_auto_flush.h -- abstraction layer for auto flush detection functionality */ #ifndef PMDK_OS_AUTO_FLUSH_H #define PMDK_OS_AUTO_FLUSH_H 1 #ifdef __cplusplus extern "C" { #endif int os_auto_flush(void); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/mmap.h0000664000000000000000000001130013615011243014166 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * mmap.h -- internal definitions for mmap module */ #ifndef PMDK_MMAP_H #define PMDK_MMAP_H 1 #include #include #include #include #include #include #include "out.h" #include "queue.h" #include "os.h" #ifdef __cplusplus extern "C" { #endif extern int Mmap_no_random; extern void *Mmap_hint; extern char *Mmap_mapfile; void *util_map_sync(void *addr, size_t len, int proto, int flags, int fd, os_off_t offset, int *map_sync); void *util_map(int fd, os_off_t off, size_t len, int flags, int rdonly, size_t req_align, int *map_sync); int util_unmap(void *addr, size_t len); #ifdef __FreeBSD__ #define MAP_NORESERVE 0 #define OS_MAPFILE "/proc/curproc/map" #else #define OS_MAPFILE "/proc/self/maps" #endif #ifndef MAP_SYNC #define MAP_SYNC 0x80000 #endif #ifndef MAP_SHARED_VALIDATE #define MAP_SHARED_VALIDATE 0x03 #endif /* * macros for micromanaging range protections for the debug version */ #ifdef DEBUG #define RANGE(addr, len, is_dev_dax, type) do {\ if (!is_dev_dax) ASSERT(util_range_##type(addr, len) >= 0);\ } while (0) #else #define RANGE(addr, len, is_dev_dax, type) do {} while (0) #endif #define RANGE_RO(addr, len, is_dev_dax) RANGE(addr, len, is_dev_dax, ro) #define RANGE_RW(addr, len, is_dev_dax) RANGE(addr, len, is_dev_dax, rw) #define RANGE_NONE(addr, len, is_dev_dax) RANGE(addr, len, is_dev_dax, none) /* pmem mapping type */ enum pmem_map_type { PMEM_DEV_DAX, /* device dax */ PMEM_MAP_SYNC, /* mapping with MAP_SYNC flag on dax fs */ MAX_PMEM_TYPE }; /* * this structure tracks the file mappings outstanding per file handle */ struct map_tracker { PMDK_SORTEDQ_ENTRY(map_tracker) entry; uintptr_t base_addr; uintptr_t end_addr; int region_id; enum pmem_map_type type; #ifdef _WIN32 /* Windows-specific data */ HANDLE FileHandle; HANDLE FileMappingHandle; DWORD Access; os_off_t Offset; size_t FileLen; #endif }; void util_mmap_init(void); void util_mmap_fini(void); int util_range_ro(void *addr, size_t len); int util_range_rw(void *addr, size_t len); int util_range_none(void *addr, size_t len); char *util_map_hint_unused(void *minaddr, size_t len, size_t align); char *util_map_hint(size_t len, size_t req_align); #define MEGABYTE ((uintptr_t)1 << 20) #define GIGABYTE ((uintptr_t)1 << 30) /* * util_map_hint_align -- choose the desired mapping alignment * * The smallest supported alignment is 2 megabytes because of the object * alignment requirements. Changing this value to 4 kilobytes constitues a * layout change. * * Use 1GB page alignment only if the mapping length is at least * twice as big as the page size. */ static inline size_t util_map_hint_align(size_t len, size_t req_align) { size_t align = 2 * MEGABYTE; if (req_align) align = req_align; else if (len >= 2 * GIGABYTE) align = GIGABYTE; return align; } int util_range_register(const void *addr, size_t len, const char *path, enum pmem_map_type type); int util_range_unregister(const void *addr, size_t len); struct map_tracker *util_range_find(uintptr_t addr, size_t len); int util_range_is_pmem(const void *addr, size_t len); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/file_posix.c0000664000000000000000000002161013615011243015375 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * file_posix.c -- Posix versions of file APIs */ /* for O_TMPFILE */ #define _GNU_SOURCE #include #include #include #include #include #include #include #include #include #include "os.h" #include "file.h" #include "out.h" #define MAX_SIZE_LENGTH 64 #define DAX_REGION_ID_LEN 6 /* 5 digits + \0 */ /* * util_tmpfile_mkstemp -- (internal) create temporary file * if O_TMPFILE not supported */ static int util_tmpfile_mkstemp(const char *dir, const char *templ) { /* the templ must start with a path separator */ ASSERTeq(templ[0], '/'); int oerrno; int fd = -1; char *fullname = alloca(strlen(dir) + strlen(templ) + 1); (void) strcpy(fullname, dir); (void) strcat(fullname, templ); sigset_t set, oldset; sigfillset(&set); (void) sigprocmask(SIG_BLOCK, &set, &oldset); mode_t prev_umask = umask(S_IRWXG | S_IRWXO); fd = os_mkstemp(fullname); umask(prev_umask); if (fd < 0) { ERR("!mkstemp"); goto err; } (void) os_unlink(fullname); (void) sigprocmask(SIG_SETMASK, &oldset, NULL); LOG(3, "unlinked file is \"%s\"", fullname); return fd; err: oerrno = errno; (void) sigprocmask(SIG_SETMASK, &oldset, NULL); if (fd != -1) (void) os_close(fd); errno = oerrno; return -1; } /* * util_tmpfile -- create temporary file */ int util_tmpfile(const char *dir, const char *templ, int flags) { LOG(3, "dir \"%s\" template \"%s\" flags %x", dir, templ, flags); /* only O_EXCL is allowed here */ ASSERT(flags == 0 || flags == O_EXCL); #ifdef O_TMPFILE int fd = open(dir, O_TMPFILE | O_RDWR | flags, S_IRUSR | S_IWUSR); /* * Open can fail if underlying file system does not support O_TMPFILE * flag. */ if (fd >= 0) return fd; if (errno != EOPNOTSUPP) { ERR("!open"); return -1; } #endif return util_tmpfile_mkstemp(dir, templ); } /* * util_is_absolute_path -- check if the path is an absolute one */ int util_is_absolute_path(const char *path) { LOG(3, "path: %s", path); if (path[0] == OS_DIR_SEPARATOR) return 1; else return 0; } /* * util_create_mkdir -- creates new dir */ int util_file_mkdir(const char *path, mode_t mode) { LOG(3, "path: %s mode: %o", path, mode); return mkdir(path, mode); } /* * util_file_dir_open -- open a directory */ int util_file_dir_open(struct dir_handle *handle, const char *path) { LOG(3, "handle: %p path: %s", handle, path); handle->dirp = opendir(path); return handle->dirp == NULL; } /* * util_file_dir_next -- read next file in directory */ int util_file_dir_next(struct dir_handle *handle, struct file_info *info) { LOG(3, "handle: %p info: %p", handle, info); struct dirent *d = readdir(handle->dirp); if (d == NULL) return 1; /* break */ info->filename[NAME_MAX] = '\0'; strncpy(info->filename, d->d_name, NAME_MAX + 1); if (info->filename[NAME_MAX] != '\0') return -1; /* filename truncated */ info->is_dir = d->d_type == DT_DIR; return 0; /* continue */ } /* * util_file_dir_close -- close a directory */ int util_file_dir_close(struct dir_handle *handle) { LOG(3, "path: %p", handle); return closedir(handle->dirp); } /* * util_file_dir_remove -- remove directory */ int util_file_dir_remove(const char *path) { LOG(3, "path: %s", path); return rmdir(path); } /* * device_dax_alignment -- (internal) checks the alignment of given Device DAX */ static size_t device_dax_alignment(const char *path) { char spath[PATH_MAX]; size_t size = 0; char *daxpath; os_stat_t st; int olderrno; LOG(3, "path \"%s\"", path); if (os_stat(path, &st) < 0) { ERR("!stat \"%s\"", path); return 0; } snprintf(spath, PATH_MAX, "/sys/dev/char/%u:%u", os_major(st.st_rdev), os_minor(st.st_rdev)); daxpath = realpath(spath, NULL); if (!daxpath) { ERR("!realpath \"%s\"", spath); return 0; } if (util_safe_strcpy(spath, daxpath, sizeof(spath))) { ERR("util_safe_strcpy failed"); free(daxpath); return 0; } free(daxpath); while (spath[0] != '\0') { char sizebuf[MAX_SIZE_LENGTH + 1]; char *pos = strrchr(spath, '/'); char *endptr; size_t len; ssize_t rc; int fd; if (strcmp(spath, "/sys/devices") == 0) break; if (!pos) break; *pos = '\0'; len = strlen(spath); snprintf(&spath[len], sizeof(spath) - len, "/dax_region/align"); fd = os_open(spath, O_RDONLY); *pos = '\0'; if (fd < 0) continue; LOG(4, "device align path \"%s\"", spath); rc = read(fd, sizebuf, MAX_SIZE_LENGTH); os_close(fd); if (rc < 0) { ERR("!read"); return 0; } sizebuf[rc] = 0; /* null termination */ olderrno = errno; errno = 0; /* 'align' is in decimal format */ size = strtoull(sizebuf, &endptr, 10); if (endptr == sizebuf || *endptr != '\n' || (size == ULLONG_MAX && errno == ERANGE)) { ERR("invalid device alignment %s", sizebuf); size = 0; errno = olderrno; break; } /* * If the alignment value is not a power of two, try with * hex format, as this is how it was printed in older kernels. * Just in case someone is using kernel <4.9. */ if ((size & (size - 1)) != 0) { size = strtoull(sizebuf, &endptr, 16); if (endptr == sizebuf || *endptr != '\n' || (size == ULLONG_MAX && errno == ERANGE)) { ERR("invalid device alignment %s", sizebuf); size = 0; } } errno = olderrno; break; } LOG(4, "device alignment %zu", size); return size; } /* * util_file_device_dax_alignment -- returns internal Device DAX alignment */ size_t util_file_device_dax_alignment(const char *path) { LOG(3, "path \"%s\"", path); return device_dax_alignment(path); } /* * util_ddax_region_find -- returns Device DAX region id */ int util_ddax_region_find(const char *path) { LOG(3, "path \"%s\"", path); int dax_reg_id_fd; char dax_region_path[PATH_MAX]; char reg_id[DAX_REGION_ID_LEN]; char *end_addr; os_stat_t st; ASSERTne(path, NULL); if (os_stat(path, &st) < 0) { ERR("!stat \"%s\"", path); return -1; } dev_t dev_id = st.st_rdev; unsigned major = os_major(dev_id); unsigned minor = os_minor(dev_id); int ret = snprintf(dax_region_path, PATH_MAX, "/sys/dev/char/%u:%u/device/dax_region/id", major, minor); if (ret < 0) { ERR("snprintf(%p, %d, /sys/dev/char/%u:%u/device/" "dax_region/id, %u, %u): %d", dax_region_path, PATH_MAX, major, minor, major, minor, ret); return -1; } if ((dax_reg_id_fd = os_open(dax_region_path, O_RDONLY)) < 0) { LOG(1, "!open(\"%s\", O_RDONLY)", dax_region_path); return -1; } ssize_t len = read(dax_reg_id_fd, reg_id, DAX_REGION_ID_LEN); if (len == -1) { ERR("!read(%d, %p, %d)", dax_reg_id_fd, reg_id, DAX_REGION_ID_LEN); goto err; } else if (len < 2 || reg_id[len - 1] != '\n') { errno = EINVAL; ERR("!read(%d, %p, %d) invalid format", dax_reg_id_fd, reg_id, DAX_REGION_ID_LEN); goto err; } int olderrno = errno; errno = 0; long reg_num = strtol(reg_id, &end_addr, 10); if ((errno == ERANGE && (reg_num == LONG_MAX || reg_num == LONG_MIN)) || (errno != 0 && reg_num == 0)) { ERR("!strtol(%p, %p, 10)", reg_id, end_addr); goto err; } errno = olderrno; if (end_addr == reg_id) { ERR("!strtol(%p, %p, 10) no digits were found", reg_id, end_addr); goto err; } if (*end_addr != '\n') { ERR("!strtol(%s, %s, 10) invalid format", reg_id, end_addr); goto err; } os_close(dax_reg_id_fd); return (int)reg_num; err: os_close(dax_reg_id_fd); return -1; } pmdk-1.8/src/common/os_dimm.h0000664000000000000000000000420413615011243014670 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_dimm.h -- DIMMs API based on the ndctl library */ #ifndef PMDK_OS_DIMM_H #define PMDK_OS_DIMM_H 1 #include #include #include "os_badblock.h" #ifdef __cplusplus extern "C" { #endif int os_dimm_uid(const char *path, char *uid, size_t *len); int os_dimm_usc(const char *path, uint64_t *usc); int os_dimm_files_namespace_badblocks(const char *path, struct badblocks *bbs); int os_dimm_devdax_clear_badblocks_all(const char *path); int os_dimm_devdax_clear_badblocks(const char *path, struct badblocks *bbs); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/os_windows.c0000664000000000000000000003462713615011243015443 0ustar rootroot/* * Copyright 2017-2019, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_windows.c -- windows abstraction layer */ #include #include #include #include #include "alloc.h" #include "util.h" #include "os.h" #include "out.h" #define UTF8_BOM "\xEF\xBB\xBF" /* * os_open -- open abstraction layer */ int os_open(const char *pathname, int flags, ...) { wchar_t *path = util_toUTF16(pathname); if (path == NULL) return -1; int ret; if (flags & O_CREAT) { va_list arg; va_start(arg, flags); mode_t mode = va_arg(arg, mode_t); va_end(arg); ret = _wopen(path, flags, mode); } else { ret = _wopen(path, flags); } util_free_UTF16(path); /* BOM skipping should not modify errno */ int orig_errno = errno; /* * text files on windows can contain BOM. As we open files * in binary mode we have to detect bom and skip it */ if (ret != -1) { char bom[3]; if (_read(ret, bom, sizeof(bom)) != 3 || memcmp(bom, UTF8_BOM, 3) != 0) { /* UTF-8 bom not found - reset file to the beginning */ _lseek(ret, 0, SEEK_SET); } } errno = orig_errno; return ret; } /* * os_fsync -- fsync abstraction layer */ int os_fsync(int fd) { HANDLE handle = (HANDLE) _get_osfhandle(fd); if (handle == INVALID_HANDLE_VALUE) { errno = EBADF; return -1; } if (!FlushFileBuffers(handle)) { errno = EINVAL; return -1; } return 0; } /* * os_fsync_dir -- fsync the directory */ int os_fsync_dir(const char *dir_name) { /* XXX not used and not implemented */ ASSERT(0); return -1; } /* * os_stat -- stat abstraction layer */ int os_stat(const char *pathname, os_stat_t *buf) { wchar_t *path = util_toUTF16(pathname); if (path == NULL) return -1; int ret = _wstat64(path, buf); util_free_UTF16(path); return ret; } /* * os_unlink -- unlink abstraction layer */ int os_unlink(const char *pathname) { wchar_t *path = util_toUTF16(pathname); if (path == NULL) return -1; int ret = _wunlink(path); util_free_UTF16(path); return ret; } /* * os_access -- access abstraction layer */ int os_access(const char *pathname, int mode) { wchar_t *path = util_toUTF16(pathname); if (path == NULL) return -1; int ret = _waccess(path, mode); util_free_UTF16(path); return ret; } /* * os_skipBOM -- (internal) Skip BOM in file stream * * text files on windows can contain BOM. We have to detect bom and skip it. */ static void os_skipBOM(FILE *file) { if (file == NULL) return; /* BOM skipping should not modify errno */ int orig_errno = errno; /* UTF-8 BOM */ uint8_t bom[3]; size_t read_num = fread(bom, sizeof(bom[0]), sizeof(bom), file); if (read_num != ARRAY_SIZE(bom)) goto out; if (memcmp(bom, UTF8_BOM, ARRAY_SIZE(bom)) != 0) { /* UTF-8 bom not found - reset file to the beginning */ fseek(file, 0, SEEK_SET); } out: errno = orig_errno; } /* * os_fopen -- fopen abstraction layer */ FILE * os_fopen(const char *pathname, const char *mode) { wchar_t *path = util_toUTF16(pathname); if (path == NULL) return NULL; wchar_t *wmode = util_toUTF16(mode); if (wmode == NULL) { util_free_UTF16(path); return NULL; } FILE *ret = _wfopen(path, wmode); util_free_UTF16(path); util_free_UTF16(wmode); os_skipBOM(ret); return ret; } /* * os_fdopen -- fdopen abstraction layer */ FILE * os_fdopen(int fd, const char *mode) { FILE *ret = fdopen(fd, mode); os_skipBOM(ret); return ret; } /* * os_chmod -- chmod abstraction layer */ int os_chmod(const char *pathname, mode_t mode) { wchar_t *path = util_toUTF16(pathname); if (path == NULL) return -1; int ret = _wchmod(path, mode); util_free_UTF16(path); return ret; } /* * os_mkstemp -- generate a unique temporary filename from template */ int os_mkstemp(char *temp) { unsigned rnd; wchar_t *utemp = util_toUTF16(temp); if (utemp == NULL) return -1; wchar_t *path = _wmktemp(utemp); if (path == NULL) { util_free_UTF16(utemp); return -1; } wchar_t *npath = Malloc(sizeof(*npath) * wcslen(path) + _MAX_FNAME); if (npath == NULL) { util_free_UTF16(utemp); return -1; } wcscpy(npath, path); util_free_UTF16(utemp); /* * Use rand_s to generate more unique tmp file name than _mktemp do. * In case with multiple threads and multiple files even after close() * file name conflicts occurred. * It resolved issue with synchronous removing * multiples files by system. */ rand_s(&rnd); int ret = _snwprintf(npath + wcslen(npath), _MAX_FNAME, L"%u", rnd); if (ret < 0) goto out; /* * Use O_TEMPORARY flag to make sure the file is deleted when * the last file descriptor is closed. Also, it prevents opening * this file from another process. */ ret = _wopen(npath, O_RDWR | O_CREAT | O_EXCL | O_TEMPORARY, S_IWRITE | S_IREAD); out: Free(npath); return ret; } /* * os_posix_fallocate -- allocate file space */ int os_posix_fallocate(int fd, os_off_t offset, os_off_t len) { /* * From POSIX: * "EINVAL -- The len argument was zero or the offset argument was * less than zero." * * From Linux man-page: * "EINVAL -- offset was less than 0, or len was less than or * equal to 0" */ if (offset < 0 || len <= 0) return EINVAL; /* * From POSIX: * "EFBIG -- The value of offset+len is greater than the maximum * file size." * * Overflow can't be checked for by _chsize_s, since it only gets * the sum. */ if (offset + len < offset) return EFBIG; /* * posix_fallocate should not clobber errno, but * _filelengthi64 might set errno. */ int orig_errno = errno; __int64 current_size = _filelengthi64(fd); int file_length_errno = errno; errno = orig_errno; if (current_size < 0) return file_length_errno; __int64 requested_size = offset + len; if (requested_size <= current_size) return 0; return _chsize_s(fd, requested_size); } /* * os_ftruncate -- truncate a file to a specified length */ int os_ftruncate(int fd, os_off_t length) { return _chsize_s(fd, length); } /* * os_flock -- apply or remove an advisory lock on an open file */ int os_flock(int fd, int operation) { int flags = 0; SYSTEM_INFO systemInfo; GetSystemInfo(&systemInfo); switch (operation & (OS_LOCK_EX | OS_LOCK_SH | OS_LOCK_UN)) { case OS_LOCK_EX: case OS_LOCK_SH: if (operation & OS_LOCK_NB) flags = _LK_NBLCK; else flags = _LK_LOCK; break; case OS_LOCK_UN: flags = _LK_UNLCK; break; default: errno = EINVAL; return -1; } os_off_t filelen = _filelengthi64(fd); if (filelen < 0) return -1; /* for our purpose it's enough to lock the first page of the file */ long len = (filelen > systemInfo.dwPageSize) ? systemInfo.dwPageSize : (long)filelen; int res = _locking(fd, flags, len); if (res != 0 && errno == EACCES) errno = EWOULDBLOCK; /* for consistency with flock() */ return res; } /* * os_writev -- windows version of writev function * * XXX: _write and other similar functions are 32 bit on windows * if size of data is bigger then 2^32, this function * will be not atomic. */ ssize_t os_writev(int fd, const struct iovec *iov, int iovcnt) { size_t size = 0; /* XXX: _write is 32 bit on windows */ for (int i = 0; i < iovcnt; i++) size += iov[i].iov_len; void *buf = malloc(size); if (buf == NULL) return ENOMEM; char *it_buf = buf; for (int i = 0; i < iovcnt; i++) { memcpy(it_buf, iov[i].iov_base, iov[i].iov_len); it_buf += iov[i].iov_len; } ssize_t written = 0; while (size > 0) { int ret = _write(fd, buf, size >= MAXUINT ? MAXUINT : (unsigned)size); if (ret == -1) { written = -1; break; } written += ret; size -= ret; } free(buf); return written; } #define NSEC_IN_SEC 1000000000ull /* number of useconds between 1970-01-01T00:00:00Z and 1601-01-01T00:00:00Z */ #define DELTA_WIN2UNIX (11644473600000000ull) /* * clock_gettime -- returns elapsed time since the system was restarted * or since Epoch, depending on the mode id */ int os_clock_gettime(int id, struct timespec *ts) { switch (id) { case CLOCK_MONOTONIC: { LARGE_INTEGER time; LARGE_INTEGER frequency; QueryPerformanceFrequency(&frequency); QueryPerformanceCounter(&time); ts->tv_sec = time.QuadPart / frequency.QuadPart; ts->tv_nsec = (long)( (time.QuadPart % frequency.QuadPart) * NSEC_IN_SEC / frequency.QuadPart); } break; case CLOCK_REALTIME: { FILETIME ctime_ft; GetSystemTimeAsFileTime(&ctime_ft); ULARGE_INTEGER ctime = { .HighPart = ctime_ft.dwHighDateTime, .LowPart = ctime_ft.dwLowDateTime, }; ts->tv_sec = (ctime.QuadPart - DELTA_WIN2UNIX * 10) / 10000000; ts->tv_nsec = ((ctime.QuadPart - DELTA_WIN2UNIX * 10) % 10000000) * 100; } break; default: SetLastError(EINVAL); return -1; } return 0; } /* * os_setenv -- change or add an environment variable */ int os_setenv(const char *name, const char *value, int overwrite) { errno_t err; /* * If caller doesn't want to overwrite make sure that a environment * variable with the same name doesn't exist. */ if (!overwrite && getenv(name)) return 0; /* * _putenv_s returns a non-zero error code on failure but setenv * needs to return -1 on failure, let's translate the error code. */ if ((err = _putenv_s(name, value)) != 0) { errno = err; return -1; } return 0; } /* * os_unsetenv -- remove an environment variable */ int os_unsetenv(const char *name) { errno_t err; if ((err = _putenv_s(name, "")) != 0) { errno = err; return -1; } return 0; } /* * os_getenv -- getenv abstraction layer */ char * os_getenv(const char *name) { return getenv(name); } /* * rand_r -- rand_r for windows * * XXX: RAND_MAX is equal 0x7fff on Windows, so to get 32 bit random number * we need to merge two numbers returned by rand_s(). * It is not to the best solution as subsequences returned by rand_s are * not guaranteed to be independent. * * XXX: Windows doesn't implement deterministic thread-safe pseudorandom * generator (generator which can be initialized by seed ). * We have to chose between a deterministic nonthread-safe generator * (rand(), srand()) or a non-deterministic thread-safe generator(rand_s()) * as thread-safety is more important, a seed parameter is ignored in this * implementation. */ unsigned os_rand_r(unsigned *seedp) { UNREFERENCED_PARAMETER(seedp); unsigned part1, part2; rand_s(&part1); rand_s(&part2); return part1 << 16 | part2; } /* * sys_siglist -- map of signal to human readable messages like sys_siglist */ const char * const sys_siglist[] = { "Unknown signal 0", /* 0 */ "Hangup", /* 1 */ "Interrupt", /* 2 */ "Quit", /* 3 */ "Illegal instruction", /* 4 */ "Trace/breakpoint trap", /* 5 */ "Aborted", /* 6 */ "Bus error", /* 7 */ "Floating point exception", /* 8 */ "Killed", /* 9 */ "User defined signal 1", /* 10 */ "Segmentation fault", /* 11 */ "User defined signal 2", /* 12 */ "Broken pipe", /* 13 */ "Alarm clock", /* 14 */ "Terminated", /* 15 */ "Stack fault", /* 16 */ "Child exited", /* 17 */ "Continued", /* 18 */ "Stopped (signal)", /* 19 */ "Stopped", /* 20 */ "Stopped (tty input)", /* 21 */ "Stopped (tty output)", /* 22 */ "Urgent I/O condition", /* 23 */ "CPU time limit exceeded", /* 24 */ "File size limit exceeded", /* 25 */ "Virtual timer expired", /* 26 */ "Profiling timer expired", /* 27 */ "Window changed", /* 28 */ "I/O possible", /* 29 */ "Power failure", /* 30 */ "Bad system call", /* 31 */ "Unknown signal 32" /* 32 */ }; int sys_siglist_size = ARRAYSIZE(sys_siglist); /* * string constants for strsignal * XXX: ideally this should have the signal number as the suffix but then we * should use a buffer from thread local storage, so deferring the same till * we need it * NOTE: In Linux strsignal uses TLS for the same reason but if it fails to get * a thread local buffer it falls back to using a static buffer trading the * thread safety. */ #define STR_REALTIME_SIGNAL "Real-time signal" #define STR_UNKNOWN_SIGNAL "Unknown signal" /* * strsignal -- returns a string describing the signal number 'sig' * * XXX: According to POSIX, this one is of type 'char *', but in our * implementation it returns 'const char *'. */ const char * os_strsignal(int sig) { if (sig >= 0 && sig < ARRAYSIZE(sys_siglist)) return sys_siglist[sig]; else if (sig >= 34 && sig <= 64) return STR_REALTIME_SIGNAL; else return STR_UNKNOWN_SIGNAL; } int os_execv(const char *path, char *const argv[]) { wchar_t *wpath = util_toUTF16(path); if (wpath == NULL) return -1; int argc = 0; while (argv[argc]) argc++; int ret; wchar_t **wargv = Zalloc((argc + 1) * sizeof(wargv[0])); if (!wargv) { ret = -1; goto wargv_alloc_failed; } for (int i = 0; i < argc; ++i) { wargv[i] = util_toUTF16(argv[i]); if (!wargv[i]) { ret = -1; goto end; } } intptr_t iret = _wexecv(wpath, wargv); if (iret == 0) ret = 0; else ret = -1; end: for (int i = 0; i < argc; ++i) util_free_UTF16(wargv[i]); Free(wargv); wargv_alloc_failed: util_free_UTF16(wpath); return ret; } pmdk-1.8/src/common/shutdown_state.h0000664000000000000000000000465313615011243016324 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * shutdown_state.h -- unsafe shudown detection */ #ifndef PMDK_SHUTDOWN_STATE_H #define PMDK_SHUTDOWN_STATE_H 1 #include #ifdef __cplusplus extern "C" { #endif struct pool_replica; struct shutdown_state { uint64_t usc; uint64_t uuid; /* UID checksum */ uint8_t dirty; uint8_t reserved[39]; uint64_t checksum; }; int shutdown_state_init(struct shutdown_state *sds, struct pool_replica *rep); int shutdown_state_add_part(struct shutdown_state *sds, const char *path, struct pool_replica *rep); void shutdown_state_set_dirty(struct shutdown_state *sds, struct pool_replica *rep); void shutdown_state_clear_dirty(struct shutdown_state *sds, struct pool_replica *rep); int shutdown_state_check(struct shutdown_state *curr_sds, struct shutdown_state *pool_sds, struct pool_replica *rep); #ifdef __cplusplus } #endif #endif /* shutdown_state.h */ pmdk-1.8/src/common/os_deep.h0000664000000000000000000000377213615011243014670 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_deep.h -- abstraction layer for common usage of deep_* functions */ #ifndef PMDK_OS_DEEP_PERSIST_H #define PMDK_OS_DEEP_PERSIST_H 1 #include #include #include "set.h" #ifdef __cplusplus extern "C" { #endif int os_range_deep_common(uintptr_t addr, size_t len); int os_part_deep_common(struct pool_replica *rep, unsigned partidx, void *addr, size_t len, int flush); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/os_thread_windows.c0000664000000000000000000003602513615011243016764 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_thread_windows.c -- (imperfect) POSIX-like threads for Windows * * Loosely inspired by: * http://locklessinc.com/articles/pthreads_on_windows/ */ #include #include #include #include #include "os_thread.h" #include "util.h" #include "out.h" typedef struct { unsigned attr; CRITICAL_SECTION lock; } internal_os_mutex_t; typedef struct { unsigned attr; char is_write; SRWLOCK lock; } internal_os_rwlock_t; typedef struct { unsigned attr; CONDITION_VARIABLE cond; } internal_os_cond_t; typedef long long internal_os_once_t; typedef struct { HANDLE handle; } internal_semaphore_t; typedef struct { GROUP_AFFINITY affinity; } internal_os_cpu_set_t; typedef struct { HANDLE thread_handle; void *arg; void *(*start_routine)(void *); void *result; } internal_os_thread_t; /* number of useconds between 1970-01-01T00:00:00Z and 1601-01-01T00:00:00Z */ #define DELTA_WIN2UNIX (11644473600000000ull) #define TIMED_LOCK(action, ts) {\ if ((action) == TRUE)\ return 0;\ unsigned long long et = (ts)->tv_sec * 1000000000 + (ts)->tv_nsec;\ while (1) {\ FILETIME _t;\ GetSystemTimeAsFileTime(&_t);\ ULARGE_INTEGER _UI = {\ .HighPart = _t.dwHighDateTime,\ .LowPart = _t.dwLowDateTime,\ };\ if (100 * _UI.QuadPart - 1000 * DELTA_WIN2UNIX >= et)\ return ETIMEDOUT;\ if ((action) == TRUE)\ return 0;\ Sleep(1);\ }\ return ETIMEDOUT;\ } /* * os_mutex_init -- initializes mutex */ int os_mutex_init(os_mutex_t *__restrict mutex) { COMPILE_ERROR_ON(sizeof(os_mutex_t) < sizeof(internal_os_mutex_t)); internal_os_mutex_t *mutex_internal = (internal_os_mutex_t *)mutex; InitializeCriticalSection(&mutex_internal->lock); return 0; } /* * os_mutex_destroy -- destroys mutex */ int os_mutex_destroy(os_mutex_t *__restrict mutex) { internal_os_mutex_t *mutex_internal = (internal_os_mutex_t *)mutex; DeleteCriticalSection(&mutex_internal->lock); return 0; } /* * os_mutex_lock -- locks mutex */ _Use_decl_annotations_ int os_mutex_lock(os_mutex_t *__restrict mutex) { internal_os_mutex_t *mutex_internal = (internal_os_mutex_t *)mutex; EnterCriticalSection(&mutex_internal->lock); if (mutex_internal->lock.RecursionCount > 1) { LeaveCriticalSection(&mutex_internal->lock); FATAL("deadlock detected"); } return 0; } /* * os_mutex_trylock -- tries lock mutex */ _Use_decl_annotations_ int os_mutex_trylock(os_mutex_t *__restrict mutex) { internal_os_mutex_t *mutex_internal = (internal_os_mutex_t *)mutex; if (TryEnterCriticalSection(&mutex_internal->lock) == FALSE) return EBUSY; if (mutex_internal->lock.RecursionCount > 1) { LeaveCriticalSection(&mutex_internal->lock); return EBUSY; } return 0; } /* * os_mutex_timedlock -- tries lock mutex with timeout */ int os_mutex_timedlock(os_mutex_t *__restrict mutex, const struct timespec *abstime) { TIMED_LOCK((os_mutex_trylock(mutex) == 0), abstime); } /* * os_mutex_unlock -- unlocks mutex */ int os_mutex_unlock(os_mutex_t *__restrict mutex) { internal_os_mutex_t *mutex_internal = (internal_os_mutex_t *)mutex; LeaveCriticalSection(&mutex_internal->lock); return 0; } /* * os_rwlock_init -- initializes rwlock */ int os_rwlock_init(os_rwlock_t *__restrict rwlock) { COMPILE_ERROR_ON(sizeof(os_rwlock_t) < sizeof(internal_os_rwlock_t)); internal_os_rwlock_t *rwlock_internal = (internal_os_rwlock_t *)rwlock; InitializeSRWLock(&rwlock_internal->lock); return 0; } /* * os_rwlock_destroy -- destroys rwlock */ int os_rwlock_destroy(os_rwlock_t *__restrict rwlock) { /* do nothing */ UNREFERENCED_PARAMETER(rwlock); return 0; } /* * os_rwlock_rdlock -- get shared lock */ int os_rwlock_rdlock(os_rwlock_t *__restrict rwlock) { internal_os_rwlock_t *rwlock_internal = (internal_os_rwlock_t *)rwlock; AcquireSRWLockShared(&rwlock_internal->lock); rwlock_internal->is_write = 0; return 0; } /* * os_rwlock_wrlock -- get exclusive lock */ int os_rwlock_wrlock(os_rwlock_t *__restrict rwlock) { internal_os_rwlock_t *rwlock_internal = (internal_os_rwlock_t *)rwlock; AcquireSRWLockExclusive(&rwlock_internal->lock); rwlock_internal->is_write = 1; return 0; } /* * os_rwlock_tryrdlock -- tries get shared lock */ int os_rwlock_tryrdlock(os_rwlock_t *__restrict rwlock) { internal_os_rwlock_t *rwlock_internal = (internal_os_rwlock_t *)rwlock; if (TryAcquireSRWLockShared(&rwlock_internal->lock) == FALSE) { return EBUSY; } else { rwlock_internal->is_write = 0; return 0; } } /* * os_rwlock_trywrlock -- tries get exclusive lock */ _Use_decl_annotations_ int os_rwlock_trywrlock(os_rwlock_t *__restrict rwlock) { internal_os_rwlock_t *rwlock_internal = (internal_os_rwlock_t *)rwlock; if (TryAcquireSRWLockExclusive(&rwlock_internal->lock) == FALSE) { return EBUSY; } else { rwlock_internal->is_write = 1; return 0; } } /* * os_rwlock_timedrdlock -- gets shared lock with timeout */ int os_rwlock_timedrdlock(os_rwlock_t *__restrict rwlock, const struct timespec *abstime) { TIMED_LOCK((os_rwlock_tryrdlock(rwlock) == 0), abstime); } /* * os_rwlock_timedwrlock -- gets exclusive lock with timeout */ int os_rwlock_timedwrlock(os_rwlock_t *__restrict rwlock, const struct timespec *abstime) { TIMED_LOCK((os_rwlock_trywrlock(rwlock) == 0), abstime); } /* * os_rwlock_unlock -- unlocks rwlock */ _Use_decl_annotations_ int os_rwlock_unlock(os_rwlock_t *__restrict rwlock) { internal_os_rwlock_t *rwlock_internal = (internal_os_rwlock_t *)rwlock; if (rwlock_internal->is_write) ReleaseSRWLockExclusive(&rwlock_internal->lock); else ReleaseSRWLockShared(&rwlock_internal->lock); return 0; } /* * os_cond_init -- initializes condition variable */ int os_cond_init(os_cond_t *__restrict cond) { COMPILE_ERROR_ON(sizeof(os_cond_t) < sizeof(internal_os_cond_t)); internal_os_cond_t *cond_internal = (internal_os_cond_t *)cond; InitializeConditionVariable(&cond_internal->cond); return 0; } /* * os_cond_destroy -- destroys condition variable */ int os_cond_destroy(os_cond_t *__restrict cond) { /* do nothing */ UNREFERENCED_PARAMETER(cond); return 0; } /* * os_cond_broadcast -- broadcast condition variable */ int os_cond_broadcast(os_cond_t *__restrict cond) { internal_os_cond_t *cond_internal = (internal_os_cond_t *)cond; WakeAllConditionVariable(&cond_internal->cond); return 0; } /* * os_cond_wait -- signal condition variable */ int os_cond_signal(os_cond_t *__restrict cond) { internal_os_cond_t *cond_internal = (internal_os_cond_t *)cond; WakeConditionVariable(&cond_internal->cond); return 0; } /* * get_rel_wait -- (internal) convert timespec to windows timeout */ static DWORD get_rel_wait(const struct timespec *abstime) { struct __timeb64 t; _ftime64_s(&t); time_t now_ms = t.time * 1000 + t.millitm; time_t ms = (time_t)(abstime->tv_sec * 1000 + abstime->tv_nsec / 1000000); DWORD rel_wait = (DWORD)(ms - now_ms); return rel_wait < 0 ? 0 : rel_wait; } /* * os_cond_timedwait -- waits on condition variable with timeout */ int os_cond_timedwait(os_cond_t *__restrict cond, os_mutex_t *__restrict mutex, const struct timespec *abstime) { internal_os_cond_t *cond_internal = (internal_os_cond_t *)cond; internal_os_mutex_t *mutex_internal = (internal_os_mutex_t *)mutex; BOOL ret; SetLastError(0); ret = SleepConditionVariableCS(&cond_internal->cond, &mutex_internal->lock, get_rel_wait(abstime)); if (ret == FALSE) return (GetLastError() == ERROR_TIMEOUT) ? ETIMEDOUT : EINVAL; return 0; } /* * os_cond_wait -- waits on condition variable */ int os_cond_wait(os_cond_t *__restrict cond, os_mutex_t *__restrict mutex) { internal_os_cond_t *cond_internal = (internal_os_cond_t *)cond; internal_os_mutex_t *mutex_internal = (internal_os_mutex_t *)mutex; /* XXX - return error code based on GetLastError() */ BOOL ret; ret = SleepConditionVariableCS(&cond_internal->cond, &mutex_internal->lock, INFINITE); return (ret == FALSE) ? EINVAL : 0; } /* * os_once -- once-only function call */ int os_once(os_once_t *once, void (*func)(void)) { internal_os_once_t *once_internal = (internal_os_once_t *)once; internal_os_once_t tmp; while ((tmp = *once_internal) != 2) { if (tmp == 1) continue; /* another thread is already calling func() */ /* try to be the first one... */ if (!util_bool_compare_and_swap64(once_internal, tmp, 1)) continue; /* sorry, another thread was faster */ func(); if (!util_bool_compare_and_swap64(once_internal, 1, 2)) { ERR("error setting once"); return -1; } } return 0; } /* * os_tls_key_create -- creates a new tls key */ int os_tls_key_create(os_tls_key_t *key, void (*destructor)(void *)) { *key = FlsAlloc(destructor); if (*key == TLS_OUT_OF_INDEXES) return EAGAIN; return 0; } /* * os_tls_key_delete -- deletes key from tls */ int os_tls_key_delete(os_tls_key_t key) { if (!FlsFree(key)) return EINVAL; return 0; } /* * os_tls_set -- sets a value in tls */ int os_tls_set(os_tls_key_t key, const void *value) { if (!FlsSetValue(key, (LPVOID)value)) return ENOENT; return 0; } /* * os_tls_get -- gets a value from tls */ void * os_tls_get(os_tls_key_t key) { return FlsGetValue(key); } /* threading */ /* * os_thread_start_routine_wrapper is a start routine for _beginthreadex() and * it helps: * * - wrap the os_thread_create's start function */ static unsigned __stdcall os_thread_start_routine_wrapper(void *arg) { internal_os_thread_t *thread_info = (internal_os_thread_t *)arg; thread_info->result = thread_info->start_routine(thread_info->arg); return 0; } /* * os_thread_create -- starts a new thread */ int os_thread_create(os_thread_t *thread, const os_thread_attr_t *attr, void *(*start_routine)(void *), void *arg) { COMPILE_ERROR_ON(sizeof(os_thread_t) < sizeof(internal_os_thread_t)); internal_os_thread_t *thread_info = (internal_os_thread_t *)thread; thread_info->start_routine = start_routine; thread_info->arg = arg; thread_info->thread_handle = (HANDLE)_beginthreadex(NULL, 0, os_thread_start_routine_wrapper, thread_info, CREATE_SUSPENDED, NULL); if (thread_info->thread_handle == 0) { free(thread_info); return errno; } if (ResumeThread(thread_info->thread_handle) == -1) { free(thread_info); return EAGAIN; } return 0; } /* * os_thread_join -- joins a thread */ int os_thread_join(os_thread_t *thread, void **result) { internal_os_thread_t *internal_thread = (internal_os_thread_t *)thread; WaitForSingleObject(internal_thread->thread_handle, INFINITE); CloseHandle(internal_thread->thread_handle); if (result != NULL) *result = internal_thread->result; return 0; } /* * os_thread_self -- returns handle to calling thread */ void os_thread_self(os_thread_t *thread) { internal_os_thread_t *internal_thread = (internal_os_thread_t *)thread; internal_thread->thread_handle = GetCurrentThread(); } /* * os_cpu_zero -- clears cpu set */ void os_cpu_zero(os_cpu_set_t *set) { internal_os_cpu_set_t *internal_set = (internal_os_cpu_set_t *)set; memset(&internal_set->affinity, 0, sizeof(internal_set->affinity)); } /* * os_cpu_set -- adds cpu to set */ void os_cpu_set(size_t cpu, os_cpu_set_t *set) { internal_os_cpu_set_t *internal_set = (internal_os_cpu_set_t *)set; int sum = 0; int group_max = GetActiveProcessorGroupCount(); int group = 0; while (group < group_max) { sum += GetActiveProcessorCount(group); if (sum > cpu) { /* * XXX: can't set affinity to two different cpu groups */ if (internal_set->affinity.Group != group) { internal_set->affinity.Mask = 0; internal_set->affinity.Group = group; } cpu -= sum - GetActiveProcessorCount(group); internal_set->affinity.Mask |= 1LL << cpu; return; } group++; } FATAL("os_cpu_set cpu out of bounds"); } /* * os_thread_setaffinity_np -- sets affinity of the thread */ int os_thread_setaffinity_np(os_thread_t *thread, size_t set_size, const os_cpu_set_t *set) { internal_os_cpu_set_t *internal_set = (internal_os_cpu_set_t *)set; internal_os_thread_t *internal_thread = (internal_os_thread_t *)thread; int ret = SetThreadGroupAffinity(internal_thread->thread_handle, &internal_set->affinity, NULL); return ret != 0 ? 0 : EINVAL; } /* * os_semaphore_init -- initializes a new semaphore instance */ int os_semaphore_init(os_semaphore_t *sem, unsigned value) { internal_semaphore_t *internal_sem = (internal_semaphore_t *)sem; internal_sem->handle = CreateSemaphore(NULL, value, LONG_MAX, NULL); return internal_sem->handle != 0 ? 0 : -1; } /* * os_semaphore_destroy -- destroys a semaphore instance */ int os_semaphore_destroy(os_semaphore_t *sem) { internal_semaphore_t *internal_sem = (internal_semaphore_t *)sem; BOOL ret = CloseHandle(internal_sem->handle); return ret ? 0 : -1; } /* * os_semaphore_wait -- decreases the value of the semaphore */ int os_semaphore_wait(os_semaphore_t *sem) { internal_semaphore_t *internal_sem = (internal_semaphore_t *)sem; DWORD ret = WaitForSingleObject(internal_sem->handle, INFINITE); return ret == WAIT_OBJECT_0 ? 0 : -1; } /* * os_semaphore_trywait -- tries to decrease the value of the semaphore */ int os_semaphore_trywait(os_semaphore_t *sem) { internal_semaphore_t *internal_sem = (internal_semaphore_t *)sem; DWORD ret = WaitForSingleObject(internal_sem->handle, 0); if (ret == WAIT_TIMEOUT) errno = EAGAIN; return ret == WAIT_OBJECT_0 ? 0 : -1; } /* * os_semaphore_post -- increases the value of the semaphore */ int os_semaphore_post(os_semaphore_t *sem) { internal_semaphore_t *internal_sem = (internal_semaphore_t *)sem; BOOL ret = ReleaseSemaphore(internal_sem->handle, 1, NULL); return ret ? 0 : -1; } pmdk-1.8/src/common/shutdown_state.c0000664000000000000000000001445413615011243016317 0ustar rootroot/* * Copyright 2017-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * shutdown_state.c -- unsafe shudown detection */ #include #include #include #include "shutdown_state.h" #include "os_dimm.h" #include "out.h" #include "util.h" #include "os_deep.h" #include "set.h" #define FLUSH_SDS(sds, rep) \ if ((rep) != NULL) os_part_deep_common(rep, 0, sds, sizeof(*(sds)), 1) /* * shutdown_state_checksum -- (internal) counts SDS checksum and flush it */ static void shutdown_state_checksum(struct shutdown_state *sds, struct pool_replica *rep) { LOG(3, "sds %p", sds); util_checksum(sds, sizeof(*sds), &sds->checksum, 1, 0); FLUSH_SDS(sds, rep); } /* * shutdown_state_init -- initializes shutdown_state struct */ int shutdown_state_init(struct shutdown_state *sds, struct pool_replica *rep) { /* check if we didn't change size of shutdown_state accidentally */ COMPILE_ERROR_ON(sizeof(struct shutdown_state) != 64); LOG(3, "sds %p", sds); memset(sds, 0, sizeof(*sds)); shutdown_state_checksum(sds, rep); return 0; } /* * shutdown_state_add_part -- adds file uuid and usc to shutdown_state struct * * if path does not exist it will fail which does NOT mean shutdown failure */ int shutdown_state_add_part(struct shutdown_state *sds, const char *path, struct pool_replica *rep) { LOG(3, "sds %p, path %s", sds, path); size_t len = 0; char *uid; uint64_t usc; if (os_dimm_usc(path, &usc)) { LOG(2, "cannot read unsafe shutdown count for %s", path); return 1; } if (os_dimm_uid(path, NULL, &len)) { ERR("cannot read uuid of %s", path); return 1; } len += 4 - len % 4; uid = Zalloc(len); if (uid == NULL) { ERR("!Zalloc"); return 1; } if (os_dimm_uid(path, uid, &len)) { ERR("cannot read uuid of %s", path); Free(uid); return 1; } sds->usc = htole64(le64toh(sds->usc) + usc); uint64_t tmp; util_checksum(uid, len, &tmp, 1, 0); sds->uuid = htole64(le64toh(sds->uuid) + tmp); FLUSH_SDS(sds, rep); Free(uid); shutdown_state_checksum(sds, rep); return 0; } /* * shutdown_state_set_dirty -- sets dirty pool flag */ void shutdown_state_set_dirty(struct shutdown_state *sds, struct pool_replica *rep) { LOG(3, "sds %p", sds); sds->dirty = 1; rep->part[0].sds_dirty_modified = 1; FLUSH_SDS(sds, rep); shutdown_state_checksum(sds, rep); } /* * shutdown_state_clear_dirty -- clears dirty pool flag */ void shutdown_state_clear_dirty(struct shutdown_state *sds, struct pool_replica *rep) { LOG(3, "sds %p", sds); struct pool_set_part part = rep->part[0]; /* * If a dirty flag was set in previous program execution it should be * preserved as it stores information about potential ADR failure. */ if (part.sds_dirty_modified != 1) return; sds->dirty = 0; part.sds_dirty_modified = 0; FLUSH_SDS(sds, rep); shutdown_state_checksum(sds, rep); } /* * shutdown_state_reinit -- (internal) reinitializes shutdown_state struct */ static void shutdown_state_reinit(struct shutdown_state *curr_sds, struct shutdown_state *pool_sds, struct pool_replica *rep) { LOG(3, "curr_sds %p, pool_sds %p", curr_sds, pool_sds); shutdown_state_init(pool_sds, rep); pool_sds->uuid = htole64(curr_sds->uuid); pool_sds->usc = htole64(curr_sds->usc); pool_sds->dirty = 0; FLUSH_SDS(pool_sds, rep); shutdown_state_checksum(pool_sds, rep); } /* * shutdown_state_check -- compares and fixes shutdown state */ int shutdown_state_check(struct shutdown_state *curr_sds, struct shutdown_state *pool_sds, struct pool_replica *rep) { LOG(3, "curr_sds %p, pool_sds %p", curr_sds, pool_sds); if (util_is_zeroed(pool_sds, sizeof(*pool_sds)) && !util_is_zeroed(curr_sds, sizeof(*curr_sds))) { shutdown_state_reinit(curr_sds, pool_sds, rep); return 0; } bool is_uuid_usc_correct = le64toh(pool_sds->usc) == le64toh(curr_sds->usc) && le64toh(pool_sds->uuid) == le64toh(curr_sds->uuid); bool is_checksum_correct = util_checksum(pool_sds, sizeof(*pool_sds), &pool_sds->checksum, 0, 0); int dirty = pool_sds->dirty; if (!is_checksum_correct) { /* the program was killed during opening or closing the pool */ LOG(2, "incorrect checksum - SDS will be reinitialized"); shutdown_state_reinit(curr_sds, pool_sds, rep); return 0; } if (is_uuid_usc_correct) { if (dirty == 0) return 0; /* * the program was killed when the pool was opened * but there wasn't an ADR failure */ LOG(2, "the pool was not closed - SDS will be reinitialized"); shutdown_state_reinit(curr_sds, pool_sds, rep); return 0; } if (dirty == 0) { /* an ADR failure but the pool was closed */ LOG(2, "an ADR failure was detected but the pool was closed - SDS will be reinitialized"); shutdown_state_reinit(curr_sds, pool_sds, rep); return 0; } /* an ADR failure - the pool might be corrupted */ ERR("an ADR failure was detected, the pool might be corrupted"); return 1; } pmdk-1.8/src/common/os_auto_flush_windows.c0000664000000000000000000001437213615011243017667 0ustar rootroot/* * Copyright 2018-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_auto_flush_windows.c -- Windows abstraction layer for auto flush detection */ #include #include #include "alloc.h" #include "out.h" #include "os.h" #include "endian.h" #include "os_auto_flush_windows.h" /* * is_nfit_available -- (internal) check if platform supports NFIT table. */ static int is_nfit_available() { LOG(3, "is_nfit_available()"); DWORD signatures_size; char *signatures = NULL; int is_nfit = 0; DWORD offset = 0; signatures_size = EnumSystemFirmwareTables(ACPI_SIGNATURE, NULL, 0); if (signatures_size == 0) { ERR("!EnumSystemFirmwareTables"); return -1; } signatures = (char *)Malloc(signatures_size + 1); if (signatures == NULL) { ERR("!malloc"); return -1; } int ret = EnumSystemFirmwareTables(ACPI_SIGNATURE, signatures, signatures_size); signatures[signatures_size] = '\0'; if (ret != signatures_size) { ERR("!EnumSystemFirmwareTables"); goto err; } while (offset <= signatures_size) { int nfit_sig = strncmp(signatures + offset, NFIT_STR_SIGNATURE, NFIT_SIGNATURE_LEN); if (nfit_sig == 0) { is_nfit = 1; break; } offset += NFIT_SIGNATURE_LEN; } Free(signatures); return is_nfit; err: Free(signatures); return -1; } /* * is_auto_flush_cap_set -- (internal) check if specific * capabilities bits are set. * * ACPI 6.2A Specification: * Bit[0] - CPU Cache Flush to NVDIMM Durability on * Power Loss Capable. If set to 1, indicates that platform * ensures the entire CPU store data path is flushed to * persistent memory on system power loss. * Bit[1] - Memory Controller Flush to NVDIMM Durability on Power Loss Capable. * If set to 1, indicates that platform provides mechanisms to automatically * flush outstanding write data from the memory controller to persistent memory * in the event of platform power loss. Note: If bit 0 is set to 1 then this bit * shall be set to 1 as well. */ static int is_auto_flush_cap_set(uint32_t capabilities) { LOG(3, "is_auto_flush_cap_set capabilities 0x%" PRIx32, capabilities); int CPU_cache_flush = CHECK_BIT(capabilities, 0); int memory_controller_flush = CHECK_BIT(capabilities, 1); LOG(15, "CPU_cache_flush %d, memory_controller_flush %d", CPU_cache_flush, memory_controller_flush); if (memory_controller_flush == 1 && CPU_cache_flush == 1) return 1; return 0; } /* * parse_nfit_buffer -- (internal) parse nfit buffer * if platform_capabilities struct is available return pcs structure. */ static struct platform_capabilities parse_nfit_buffer(const unsigned char *nfit_buffer, unsigned long buffer_size) { LOG(3, "parse_nfit_buffer nfit_buffer %s, buffer_size %lu", nfit_buffer, buffer_size); uint16_t type; uint16_t length; size_t offset = sizeof(struct nfit_header); struct platform_capabilities pcs = {0}; while (offset < buffer_size) { type = *(nfit_buffer + offset); length = *(nfit_buffer + offset + 2); if (type == PCS_TYPE_NUMBER) { if (length == sizeof(struct platform_capabilities)) { memmove(&pcs, nfit_buffer + offset, length); return pcs; } } offset += length; } return pcs; } /* * os_auto_flush -- check if platform supports auto flush. */ int os_auto_flush(void) { LOG(3, NULL); DWORD nfit_buffer_size = 0; DWORD nfit_written = 0; PVOID nfit_buffer = NULL; struct nfit_header *nfit_data; struct platform_capabilities *pc = NULL; int eADR = 0; int is_nfit = is_nfit_available(); if (is_nfit == 0) { LOG(15, "ACPI NFIT table not available"); return 0; } if (is_nfit < 0 || is_nfit != 1) { LOG(1, "!is_nfit_available"); return -1; } /* get the entire nfit size */ nfit_buffer_size = GetSystemFirmwareTable( (DWORD)ACPI_SIGNATURE, (DWORD)NFIT_REV_SIGNATURE, NULL, 0); if (nfit_buffer_size == 0) { ERR("!GetSystemFirmwareTable"); return -1; } /* reserve buffer */ nfit_buffer = (unsigned char *)Malloc(nfit_buffer_size); if (nfit_buffer == NULL) { ERR("!malloc"); goto err; } /* write actual nfit to buffer */ nfit_written = GetSystemFirmwareTable( (DWORD)ACPI_SIGNATURE, (DWORD)NFIT_REV_SIGNATURE, nfit_buffer, nfit_buffer_size); if (nfit_written == 0) { ERR("!GetSystemFirmwareTable"); goto err; } if (nfit_buffer_size != nfit_written) { errno = ERROR_INVALID_DATA; ERR("!GetSystemFirmwareTable invalid data"); goto err; } nfit_data = (struct nfit_header *)nfit_buffer; int nfit_sig = strncmp(nfit_data->signature, NFIT_STR_SIGNATURE, NFIT_SIGNATURE_LEN); if (nfit_sig != 0) { ERR("!NFIT buffer has invalid data"); goto err; } struct platform_capabilities pcs = parse_nfit_buffer( nfit_buffer, nfit_buffer_size); eADR = is_auto_flush_cap_set(pcs.capabilities); Free(nfit_buffer); return eADR; err: Free(nfit_buffer); return -1; } pmdk-1.8/src/common/.cstyleignore0000664000000000000000000000001013615011243015567 0ustar rootrootqueue.h pmdk-1.8/src/common/uuid.h0000664000000000000000000000514413615011243014213 0ustar rootroot/* * Copyright 2014-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * uuid.h -- internal definitions for uuid module */ #ifndef PMDK_UUID_H #define PMDK_UUID_H 1 #include #include #ifdef __cplusplus extern "C" { #endif /* * Structure for binary version of uuid. From RFC4122, * https://tools.ietf.org/html/rfc4122 */ struct uuid { uint32_t time_low; uint16_t time_mid; uint16_t time_hi_and_ver; uint8_t clock_seq_hi; uint8_t clock_seq_low; uint8_t node[6]; }; #define POOL_HDR_UUID_LEN 16 /* uuid byte length */ #define POOL_HDR_UUID_STR_LEN 37 /* uuid string length */ #define POOL_HDR_UUID_GEN_FILE "/proc/sys/kernel/random/uuid" typedef unsigned char uuid_t[POOL_HDR_UUID_LEN]; /* 16 byte binary uuid value */ int util_uuid_generate(uuid_t uuid); int util_uuid_to_string(const uuid_t u, char *buf); int util_uuid_from_string(const char uuid[POOL_HDR_UUID_STR_LEN], struct uuid *ud); /* * uuidcmp -- compare two uuids */ static inline int uuidcmp(const uuid_t uuid1, const uuid_t uuid2) { return memcmp(uuid1, uuid2, POOL_HDR_UUID_LEN); } #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/pool_hdr.h0000664000000000000000000002242713615011243015056 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pool_hdr.h -- internal definitions for pool header module */ #ifndef PMDK_POOL_HDR_H #define PMDK_POOL_HDR_H 1 #include #include #include #include "uuid.h" #include "shutdown_state.h" #include "util.h" #include "page_size.h" #ifdef __cplusplus extern "C" { #endif /* * Number of bits per type in alignment descriptor */ #define ALIGNMENT_DESC_BITS 4 /* * architecture identification flags * * These flags allow to unambiguously determine the architecture * on which the pool was created. * * The alignment_desc field contains information about alignment * of the following basic types: * - char * - short * - int * - long * - long long * - size_t * - os_off_t * - float * - double * - long double * - void * * * The alignment of each type is computed as an offset of field * of specific type in the following structure: * struct { * char byte; * type field; * }; * * The value is decremented by 1 and masked by 4 bits. * Multiple alignments are stored on consecutive 4 bits of each * type in the order specified above. * * The values used in the machine, and machine_class fields are in * principle independent of operating systems, and object formats. * In practice they happen to match constants used in ELF object headers. */ struct arch_flags { uint64_t alignment_desc; /* alignment descriptor */ uint8_t machine_class; /* address size -- 64 bit or 32 bit */ uint8_t data; /* data encoding -- LE or BE */ uint8_t reserved[4]; uint16_t machine; /* required architecture */ }; #define POOL_HDR_ARCH_LEN sizeof(struct arch_flags) /* possible values of the machine class field in the above struct */ #define PMDK_MACHINE_CLASS_64 2 /* 64 bit pointers, 64 bit size_t */ /* possible values of the machine field in the above struct */ #define PMDK_MACHINE_X86_64 62 #define PMDK_MACHINE_AARCH64 183 #define PMDK_MACHINE_PPC64 21 /* possible values of the data field in the above struct */ #define PMDK_DATA_LE 1 /* 2's complement, little endian */ #define PMDK_DATA_BE 2 /* 2's complement, big endian */ /* * features flags */ typedef struct { uint32_t compat; /* mask: compatible "may" features */ uint32_t incompat; /* mask: "must support" features */ uint32_t ro_compat; /* mask: force RO if unsupported */ } features_t; /* * header used at the beginning of all types of memory pools * * for pools build on persistent memory, the integer types * below are stored in little-endian byte order. */ #define POOL_HDR_SIG_LEN 8 #define POOL_HDR_UNUSED_SIZE 1904 #define POOL_HDR_UNUSED2_SIZE 1976 #define POOL_HDR_ALIGN_PAD (PMEM_PAGESIZE - 4096) struct pool_hdr { char signature[POOL_HDR_SIG_LEN]; uint32_t major; /* format major version number */ features_t features; /* features flags */ uuid_t poolset_uuid; /* pool set UUID */ uuid_t uuid; /* UUID of this file */ uuid_t prev_part_uuid; /* prev part */ uuid_t next_part_uuid; /* next part */ uuid_t prev_repl_uuid; /* prev replica */ uuid_t next_repl_uuid; /* next replica */ uint64_t crtime; /* when created (seconds since epoch) */ struct arch_flags arch_flags; /* architecture identification flags */ unsigned char unused[POOL_HDR_UNUSED_SIZE]; /* must be zero */ /* not checksumed */ unsigned char unused2[POOL_HDR_UNUSED2_SIZE]; /* must be zero */ struct shutdown_state sds; /* shutdown status */ uint64_t checksum; /* checksum of above fields */ #if PMEM_PAGESIZE > 4096 /* prevent zero size array */ unsigned char align_pad[POOL_HDR_ALIGN_PAD]; /* alignment pad */ #endif }; #define POOL_HDR_SIZE (sizeof(struct pool_hdr)) #define POOL_DESC_SIZE PMEM_PAGESIZE void util_convert2le_hdr(struct pool_hdr *hdrp); void util_convert2h_hdr_nocheck(struct pool_hdr *hdrp); void util_get_arch_flags(struct arch_flags *arch_flags); int util_check_arch_flags(const struct arch_flags *arch_flags); features_t util_get_unknown_features(features_t features, features_t known); int util_feature_check(struct pool_hdr *hdrp, features_t features); int util_feature_cmp(features_t features, features_t ref); int util_feature_is_zero(features_t features); int util_feature_is_set(features_t features, features_t flag); void util_feature_enable(features_t *features, features_t new_feature); void util_feature_disable(features_t *features, features_t new_feature); const char *util_feature2str(features_t feature, features_t *found); features_t util_str2feature(const char *str); uint32_t util_str2pmempool_feature(const char *str); uint32_t util_feature2pmempool_feature(features_t feat); /* * set of macros for determining the alignment descriptor */ #define DESC_MASK ((1 << ALIGNMENT_DESC_BITS) - 1) #define alignment_of(t) offsetof(struct { char c; t x; }, x) #define alignment_desc_of(t) (((uint64_t)alignment_of(t) - 1) & DESC_MASK) #define alignment_desc()\ (alignment_desc_of(char) << 0 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(short) << 1 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(int) << 2 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(long) << 3 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(long long) << 4 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(size_t) << 5 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(off_t) << 6 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(float) << 7 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(double) << 8 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(long double) << 9 * ALIGNMENT_DESC_BITS) |\ (alignment_desc_of(void *) << 10 * ALIGNMENT_DESC_BITS) #define POOL_FEAT_ZERO 0x0000U static const features_t features_zero = {POOL_FEAT_ZERO, POOL_FEAT_ZERO, POOL_FEAT_ZERO}; /* * compat features */ #define POOL_FEAT_CHECK_BAD_BLOCKS 0x0001U /* check bad blocks in a pool */ #define POOL_FEAT_COMPAT_ALL \ (POOL_FEAT_CHECK_BAD_BLOCKS) #define FEAT_COMPAT(X) \ {POOL_FEAT_##X, POOL_FEAT_ZERO, POOL_FEAT_ZERO} /* * incompat features */ #define POOL_FEAT_SINGLEHDR 0x0001U /* pool header only in the first part */ #define POOL_FEAT_CKSUM_2K 0x0002U /* only first 2K of hdr checksummed */ #define POOL_FEAT_SDS 0x0004U /* check shutdown state */ #define POOL_FEAT_INCOMPAT_ALL \ (POOL_FEAT_SINGLEHDR | POOL_FEAT_CKSUM_2K | POOL_FEAT_SDS) /* * incompat features effective values (if applicable) */ #ifdef SDS_ENABLED #define POOL_E_FEAT_SDS POOL_FEAT_SDS #else #define POOL_E_FEAT_SDS 0x0000U /* empty */ #endif #define POOL_FEAT_COMPAT_VALID \ (POOL_FEAT_CHECK_BAD_BLOCKS) #define POOL_FEAT_INCOMPAT_VALID \ (POOL_FEAT_SINGLEHDR | POOL_FEAT_CKSUM_2K | POOL_E_FEAT_SDS) #if defined(_WIN32) || NDCTL_ENABLED #define POOL_FEAT_INCOMPAT_DEFAULT \ (POOL_FEAT_CKSUM_2K | POOL_E_FEAT_SDS) #else /* * shutdown state support on Linux requires root access on kernel < 4.20 with * ndctl < 63 so it is disabled by default */ #define POOL_FEAT_INCOMPAT_DEFAULT \ (POOL_FEAT_CKSUM_2K) #endif #if NDCTL_ENABLED #define POOL_FEAT_COMPAT_DEFAULT \ (POOL_FEAT_CHECK_BAD_BLOCKS) #else #define POOL_FEAT_COMPAT_DEFAULT \ (POOL_FEAT_ZERO) #endif #define FEAT_INCOMPAT(X) \ {POOL_FEAT_ZERO, POOL_FEAT_##X, POOL_FEAT_ZERO} #define POOL_FEAT_VALID \ {POOL_FEAT_COMPAT_VALID, POOL_FEAT_INCOMPAT_VALID, POOL_FEAT_ZERO} /* * defines the first not checksummed field - all fields after this will be * ignored during checksum calculations. */ #define POOL_HDR_CSUM_2K_END_OFF offsetof(struct pool_hdr, unused2) #define POOL_HDR_CSUM_4K_END_OFF offsetof(struct pool_hdr, checksum) /* * pick the first not checksummed field. 2K variant is used if * POOL_FEAT_CKSUM_2K incompat feature is set. */ #define POOL_HDR_CSUM_END_OFF(hdrp) \ ((hdrp)->features.incompat & POOL_FEAT_CKSUM_2K) \ ? POOL_HDR_CSUM_2K_END_OFF : POOL_HDR_CSUM_4K_END_OFF /* ignore shutdown state if incompat feature is disabled */ #define IGNORE_SDS(hdrp) \ (((hdrp) != NULL) && (((hdrp)->features.incompat & POOL_FEAT_SDS) == 0)) #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/os_deep_windows.c0000664000000000000000000000605113615011243016426 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_deep_windows.c -- Windows abstraction layer for deep_* functions */ #include #include "out.h" #include "os.h" #include "set.h" #include "libpmem.h" /* * os_range_deep_common -- call msnyc for non DEV dax */ int os_range_deep_common(uintptr_t addr, size_t len) { LOG(3, "os_range_deep_common addr %p len %lu", addr, len); if (len == 0) return 0; return pmem_msync((void *)addr, len); } /* * os_part_deep_common -- common function to handle both * deep_persist and deep_drain part flush cases. */ int os_part_deep_common(struct pool_replica *rep, unsigned partidx, void *addr, size_t len, int flush) { LOG(3, "part %p part %d addr %p len %lu flush %d", rep, partidx, addr, len, flush); if (!rep->is_pmem) { /* * In case of part on non-pmem call msync on the range * to deep flush the data. Deep drain is empty as all * data is msynced to persistence. */ if (!flush) return 0; if (pmem_msync(addr, len)) { LOG(1, "pmem_msync(%p, %lu)", addr, len); return -1; } return 0; } /* Call deep flush if it was requested */ if (flush) { LOG(15, "pmem_deep_flush addr %p, len %lu", addr, len); pmem_deep_flush(addr, len); } /* * Before deep drain call normal drain to ensure that data * is at least in WPQ. */ pmem_drain(); /* * For deep_drain on normal pmem it is enough to * call msync on one page. */ if (pmem_msync(addr, MIN(Pagesize, len))) { LOG(1, "pmem_msync(%p, %lu)", addr, len); return -1; } return 0; } pmdk-1.8/src/common/ctl_fallocate.c0000664000000000000000000000454313615011243016036 0ustar rootroot/* * Copyright 2018-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctl_fallocate.c -- implementation of the fallocate CTL namespace */ #include "ctl.h" #include "set.h" #include "out.h" #include "ctl_global.h" #include "file.h" static int CTL_READ_HANDLER(at_create)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int *arg_out = arg; *arg_out = Fallocate_at_create; return 0; } static int CTL_WRITE_HANDLER(at_create)(void *ctx, enum ctl_query_source source, void *arg, struct ctl_indexes *indexes) { int arg_in = *(int *)arg; Fallocate_at_create = arg_in; return 0; } static struct ctl_argument CTL_ARG(at_create) = CTL_ARG_BOOLEAN; static const struct ctl_node CTL_NODE(fallocate)[] = { CTL_LEAF_RW(at_create), CTL_NODE_END }; void ctl_fallocate_register(void) { CTL_REGISTER_MODULE(NULL, fallocate); } pmdk-1.8/src/common/set.h0000664000000000000000000003356313615011243014046 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * Copyright (c) 2016, Microsoft Corporation. All rights reserved. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * set.h -- internal definitions for set module */ #ifndef PMDK_SET_H #define PMDK_SET_H 1 #include #include #include #include #include "out.h" #include "vec.h" #include "pool_hdr.h" #include "librpmem.h" #ifdef __cplusplus extern "C" { #endif /* * pool sets & replicas */ #define POOLSET_HDR_SIG "PMEMPOOLSET" #define POOLSET_HDR_SIG_LEN 11 /* does NOT include '\0' */ #define POOLSET_REPLICA_SIG "REPLICA" #define POOLSET_REPLICA_SIG_LEN 7 /* does NOT include '\0' */ #define POOLSET_OPTION_SIG "OPTION" #define POOLSET_OPTION_SIG_LEN 6 /* does NOT include '\0' */ /* pool set option flags */ enum pool_set_option_flag { OPTION_UNKNOWN = 0x0, OPTION_SINGLEHDR = 0x1, /* pool headers only in the first part */ OPTION_NOHDRS = 0x2, /* no pool headers, remote replicas only */ }; struct pool_set_option { const char *name; enum pool_set_option_flag flag; }; #define POOL_LOCAL 0 #define POOL_REMOTE 1 #define REPLICAS_DISABLED 0 #define REPLICAS_ENABLED 1 /* util_pool_open flags */ #define POOL_OPEN_COW 1 /* copy-on-write mode */ #define POOL_OPEN_IGNORE_SDS 2 /* ignore shutdown state */ #define POOL_OPEN_IGNORE_BAD_BLOCKS 4 /* ignore bad blocks */ #define POOL_OPEN_CHECK_BAD_BLOCKS 8 /* check bad blocks */ enum del_parts_mode { DO_NOT_DELETE_PARTS, /* do not delete part files */ DELETE_CREATED_PARTS, /* delete only newly created parts files */ DELETE_ALL_PARTS /* force delete all parts files */ }; struct pool_set_part { /* populated by a pool set file parser */ const char *path; size_t filesize; /* aligned to page size */ int fd; int flags; /* stores flags used when opening the file */ /* valid only if fd >= 0 */ int is_dev_dax; /* indicates if the part is on device dax */ size_t alignment; /* internal alignment (Device DAX only) */ int created; /* indicates newly created (zeroed) file */ /* util_poolset_open/create */ void *remote_hdr; /* allocated header for remote replica */ void *hdr; /* base address of header */ size_t hdrsize; /* size of the header mapping */ int hdr_map_sync; /* header mapped with MAP_SYNC */ void *addr; /* base address of the mapping */ size_t size; /* size of the mapping - page aligned */ int map_sync; /* part has been mapped with MAP_SYNC flag */ int rdonly; /* is set based on compat features, affects */ /* the whole poolset */ uuid_t uuid; int has_bad_blocks; /* part file contains bad blocks */ int sds_dirty_modified; /* sds dirty flag was set */ }; struct pool_set_directory { const char *path; size_t resvsize; /* size of the address space reservation */ }; struct remote_replica { void *rpp; /* RPMEMpool opaque handle */ char *node_addr; /* address of a remote node */ /* poolset descriptor is a pool set file name on a remote node */ char *pool_desc; /* descriptor of a poolset */ }; struct pool_replica { unsigned nparts; unsigned nallocated; unsigned nhdrs; /* should be 0, 1 or nparts */ size_t repsize; /* total size of all the parts (mappings) */ size_t resvsize; /* min size of the address space reservation */ int is_pmem; /* true if all the parts are in PMEM */ struct remote_replica *remote; /* not NULL if the replica */ /* is a remote one */ VEC(, struct pool_set_directory) directory; struct pool_set_part part[]; }; struct pool_set { char *path; /* path of the poolset file */ unsigned nreplicas; uuid_t uuid; int rdonly; int zeroed; /* true if all the parts are new files */ size_t poolsize; /* the smallest replica size */ int has_bad_blocks; /* pool set contains bad blocks */ int remote; /* true if contains a remote replica */ unsigned options; /* enabled pool set options */ int directory_based; size_t resvsize; unsigned next_id; unsigned next_directory_id; int ignore_sds; /* don't use shutdown state */ struct pool_replica *replica[]; }; struct part_file { int is_remote; /* * Pointer to the part file structure - * - not-NULL only for a local part file */ struct pool_set_part *part; /* * Pointer to the replica structure - * - not-NULL only for a remote replica */ struct remote_replica *remote; }; struct pool_attr { char signature[POOL_HDR_SIG_LEN]; /* pool signature */ uint32_t major; /* format major version number */ features_t features; /* features flags */ unsigned char poolset_uuid[POOL_HDR_UUID_LEN]; /* pool uuid */ unsigned char first_part_uuid[POOL_HDR_UUID_LEN]; /* first part uuid */ unsigned char prev_repl_uuid[POOL_HDR_UUID_LEN]; /* prev replica uuid */ unsigned char next_repl_uuid[POOL_HDR_UUID_LEN]; /* next replica uuid */ unsigned char arch_flags[POOL_HDR_ARCH_LEN]; /* arch flags */ }; /* get index of the (r)th replica */ static inline unsigned REPidx(const struct pool_set *set, unsigned r) { ASSERTne(set->nreplicas, 0); return (set->nreplicas + r) % set->nreplicas; } /* get index of the (r + 1)th replica */ static inline unsigned REPNidx(const struct pool_set *set, unsigned r) { ASSERTne(set->nreplicas, 0); return (set->nreplicas + r + 1) % set->nreplicas; } /* get index of the (r - 1)th replica */ static inline unsigned REPPidx(const struct pool_set *set, unsigned r) { ASSERTne(set->nreplicas, 0); return (set->nreplicas + r - 1) % set->nreplicas; } /* get index of the (r)th part */ static inline unsigned PARTidx(const struct pool_replica *rep, unsigned p) { ASSERTne(rep->nparts, 0); return (rep->nparts + p) % rep->nparts; } /* get index of the (r + 1)th part */ static inline unsigned PARTNidx(const struct pool_replica *rep, unsigned p) { ASSERTne(rep->nparts, 0); return (rep->nparts + p + 1) % rep->nparts; } /* get index of the (r - 1)th part */ static inline unsigned PARTPidx(const struct pool_replica *rep, unsigned p) { ASSERTne(rep->nparts, 0); return (rep->nparts + p - 1) % rep->nparts; } /* get index of the (r)th part */ static inline unsigned HDRidx(const struct pool_replica *rep, unsigned p) { ASSERTne(rep->nhdrs, 0); return (rep->nhdrs + p) % rep->nhdrs; } /* get index of the (r + 1)th part */ static inline unsigned HDRNidx(const struct pool_replica *rep, unsigned p) { ASSERTne(rep->nhdrs, 0); return (rep->nhdrs + p + 1) % rep->nhdrs; } /* get index of the (r - 1)th part */ static inline unsigned HDRPidx(const struct pool_replica *rep, unsigned p) { ASSERTne(rep->nhdrs, 0); return (rep->nhdrs + p - 1) % rep->nhdrs; } /* get (r)th replica */ static inline struct pool_replica * REP(const struct pool_set *set, unsigned r) { return set->replica[REPidx(set, r)]; } /* get (r + 1)th replica */ static inline struct pool_replica * REPN(const struct pool_set *set, unsigned r) { return set->replica[REPNidx(set, r)]; } /* get (r - 1)th replica */ static inline struct pool_replica * REPP(const struct pool_set *set, unsigned r) { return set->replica[REPPidx(set, r)]; } /* get (p)th part */ static inline struct pool_set_part * PART(struct pool_replica *rep, unsigned p) { return &rep->part[PARTidx(rep, p)]; } /* get (p + 1)th part */ static inline struct pool_set_part * PARTN(struct pool_replica *rep, unsigned p) { return &rep->part[PARTNidx(rep, p)]; } /* get (p - 1)th part */ static inline struct pool_set_part * PARTP(struct pool_replica *rep, unsigned p) { return &rep->part[PARTPidx(rep, p)]; } /* get (p)th header */ static inline struct pool_hdr * HDR(struct pool_replica *rep, unsigned p) { return (struct pool_hdr *)(rep->part[HDRidx(rep, p)].hdr); } /* get (p + 1)th header */ static inline struct pool_hdr * HDRN(struct pool_replica *rep, unsigned p) { return (struct pool_hdr *)(rep->part[HDRNidx(rep, p)].hdr); } /* get (p - 1)th header */ static inline struct pool_hdr * HDRP(struct pool_replica *rep, unsigned p) { return (struct pool_hdr *)(rep->part[HDRPidx(rep, p)].hdr); } extern int Prefault_at_open; extern int Prefault_at_create; extern int SDS_at_create; extern int Fallocate_at_create; extern int COW_at_open; int util_poolset_parse(struct pool_set **setp, const char *path, int fd); int util_poolset_read(struct pool_set **setp, const char *path); int util_poolset_create_set(struct pool_set **setp, const char *path, size_t poolsize, size_t minsize, int ignore_sds); int util_poolset_open(struct pool_set *set); void util_poolset_close(struct pool_set *set, enum del_parts_mode del); void util_poolset_free(struct pool_set *set); int util_poolset_chmod(struct pool_set *set, mode_t mode); void util_poolset_fdclose(struct pool_set *set); void util_poolset_fdclose_always(struct pool_set *set); int util_is_poolset_file(const char *path); int util_poolset_foreach_part_struct(struct pool_set *set, int (*cb)(struct part_file *pf, void *arg), void *arg); int util_poolset_foreach_part(const char *path, int (*cb)(struct part_file *pf, void *arg), void *arg); size_t util_poolset_size(const char *path); int util_replica_deep_common(const void *addr, size_t len, struct pool_set *set, unsigned replica_id, int flush); int util_replica_deep_persist(const void *addr, size_t len, struct pool_set *set, unsigned replica_id); int util_replica_deep_drain(const void *addr, size_t len, struct pool_set *set, unsigned replica_id); int util_pool_create(struct pool_set **setp, const char *path, size_t poolsize, size_t minsize, size_t minpartsize, const struct pool_attr *attr, unsigned *nlanes, int can_have_rep); int util_pool_create_uuids(struct pool_set **setp, const char *path, size_t poolsize, size_t minsize, size_t minpartsize, const struct pool_attr *attr, unsigned *nlanes, int can_have_rep, int remote); int util_part_open(struct pool_set_part *part, size_t minsize, int create_part); void util_part_fdclose(struct pool_set_part *part); int util_replica_open(struct pool_set *set, unsigned repidx, int flags); int util_replica_set_attr(struct pool_replica *rep, const struct rpmem_pool_attr *rattr); void util_pool_hdr2attr(struct pool_attr *attr, struct pool_hdr *hdr); void util_pool_attr2hdr(struct pool_hdr *hdr, const struct pool_attr *attr); int util_replica_close(struct pool_set *set, unsigned repidx); int util_map_part(struct pool_set_part *part, void *addr, size_t size, size_t offset, int flags, int rdonly); int util_unmap_part(struct pool_set_part *part); int util_unmap_parts(struct pool_replica *rep, unsigned start_index, unsigned end_index); int util_header_create(struct pool_set *set, unsigned repidx, unsigned partidx, const struct pool_attr *attr, int overwrite); int util_map_hdr(struct pool_set_part *part, int flags, int rdonly); void util_unmap_hdr(struct pool_set_part *part); int util_pool_has_device_dax(struct pool_set *set); int util_pool_open_nocheck(struct pool_set *set, unsigned flags); int util_pool_open(struct pool_set **setp, const char *path, size_t minpartsize, const struct pool_attr *attr, unsigned *nlanes, void *addr, unsigned flags); int util_pool_open_remote(struct pool_set **setp, const char *path, int cow, size_t minpartsize, struct rpmem_pool_attr *rattr); void *util_pool_extend(struct pool_set *set, size_t *size, size_t minpartsize); void util_remote_init(void); void util_remote_fini(void); int util_update_remote_header(struct pool_set *set, unsigned repn); void util_remote_init_lock(void); void util_remote_destroy_lock(void); int util_pool_close_remote(RPMEMpool *rpp); void util_remote_unload(void); void util_replica_fdclose(struct pool_replica *rep); int util_poolset_remote_open(struct pool_replica *rep, unsigned repidx, size_t minsize, int create, void *pool_addr, size_t pool_size, unsigned *nlanes); int util_remote_load(void); int util_replica_open_remote(struct pool_set *set, unsigned repidx, int flags); int util_poolset_remote_replica_open(struct pool_set *set, unsigned repidx, size_t minsize, int create, unsigned *nlanes); int util_replica_close_local(struct pool_replica *rep, unsigned repn, enum del_parts_mode del); int util_replica_close_remote(struct pool_replica *rep, unsigned repn, enum del_parts_mode del); extern int (*Rpmem_persist)(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane, unsigned flags); extern int (*Rpmem_deep_persist)(RPMEMpool *rpp, size_t offset, size_t length, unsigned lane); extern int (*Rpmem_read)(RPMEMpool *rpp, void *buff, size_t offset, size_t length, unsigned lane); extern int (*Rpmem_close)(RPMEMpool *rpp); extern int (*Rpmem_remove)(const char *target, const char *pool_set_name, int flags); extern int (*Rpmem_set_attr)(RPMEMpool *rpp, const struct rpmem_pool_attr *rattr); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/Makefile0000664000000000000000000000331713615011243014534 0ustar rootroot# Copyright 2014-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/common/Makefile -- Makefile for common # LIBRARY_NAME = pmemcommon include pmemcommon.inc include ../Makefile.inc CFLAGS += $(LIBNDCTL_CFLAGS) CFLAGS += -DUSE_LIBDL pmdk-1.8/src/common/alloc.c0000664000000000000000000000765613615011243014344 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include #include "alloc.h" #include "fault_injection.h" #include "out.h" Malloc_func fn_malloc = malloc; Realloc_func fn_realloc = realloc; #if FAULT_INJECTION static __thread int malloc_num; static __thread int fail_malloc_num; static __thread const char *fail_malloc_from; void * _flt_Malloc(size_t size, const char *func) { if (fail_malloc_from && strcmp(func, fail_malloc_from) == 0) { if (++malloc_num == fail_malloc_num) { errno = ENOMEM; return NULL; } } return fn_malloc(size); } static __thread int realloc_num; static __thread int fail_realloc_num; static __thread const char *fail_realloc_from; void * _flt_Realloc(void *ptr, size_t size, const char *func) { if (fail_realloc_from && strcmp(func, fail_realloc_from) == 0) { if (++realloc_num == fail_realloc_num) { errno = ENOMEM; return NULL; } } return fn_realloc(ptr, size); } void common_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { switch (type) { case PMEM_MALLOC: malloc_num = 0; fail_malloc_num = nth; fail_malloc_from = at; break; case PMEM_REALLOC: realloc_num = 0; fail_realloc_num = nth; fail_realloc_from = at; break; default: FATAL("unknown allocation type"); } } int common_fault_injection_enabled(void) { return 1; } #else void *_Malloc(size_t size) { return fn_malloc(size); } void *_Realloc(void *ptr, size_t size) { return fn_realloc(ptr, size); } #endif void set_func_malloc(void *(*malloc_func)(size_t size)) { fn_malloc = (malloc_func == NULL) ? malloc : malloc_func; } void set_func_realloc(void *(*realloc_func)(void *ptr, size_t size)) { fn_realloc = (realloc_func == NULL) ? realloc : realloc_func; } /* * our versions of malloc & friends start off pointing to the libc versions */ Free_func Free = free; Strdup_func Strdup = strdup; /* * Zalloc -- allocate zeroed memory */ void * Zalloc(size_t sz) { void *ret = Malloc(sz); if (!ret) return NULL; return memset(ret, 0, sz); } /* * util_set_alloc_funcs -- allow one to override malloc, etc. */ void util_set_alloc_funcs(void *(*malloc_func)(size_t size), void (*free_func)(void *ptr), void *(*realloc_func)(void *ptr, size_t size), char *(*strdup_func)(const char *s)) { set_func_malloc(malloc_func); Free = (free_func == NULL) ? free : free_func; set_func_realloc(realloc_func); Strdup = (strdup_func == NULL) ? strdup : strdup_func; } pmdk-1.8/src/common/pmemcommon.h0000664000000000000000000000420613615011243015412 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmemcommon.h -- definitions for "common" module */ #ifndef PMEMCOMMON_H #define PMEMCOMMON_H 1 #include "util.h" #include "out.h" #include "mmap.h" #ifdef __cplusplus extern "C" { #endif static inline void common_init(const char *log_prefix, const char *log_level_var, const char *log_file_var, int major_version, int minor_version) { util_init(); out_init(log_prefix, log_level_var, log_file_var, major_version, minor_version); util_mmap_init(); } static inline void common_fini(void) { util_mmap_fini(); out_fini(); } #ifdef __cplusplus } #endif #endif pmdk-1.8/src/common/file_windows.c0000664000000000000000000001305713615011243015733 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * file_windows.c -- Windows emulation of Linux-specific system calls */ /* * XXX - The initial approach to PMDK for Windows port was to minimize the * amount of changes required in the core part of the library, and to avoid * preprocessor conditionals, if possible. For that reason, some of the * Linux system calls that have no equivalents on Windows have been emulated * using Windows API. * Note that it was not a goal to fully emulate POSIX-compliant behavior * of mentioned functions. They are used only internally, so current * implementation is just good enough to satisfy PMDK needs and to make it * work on Windows. */ #include #include #include #include "alloc.h" #include "file.h" #include "out.h" #include "os.h" /* * util_tmpfile -- create a temporary file */ int util_tmpfile(const char *dir, const char *templ, int flags) { LOG(3, "dir \"%s\" template \"%s\" flags %x", dir, templ, flags); /* only O_EXCL is allowed here */ ASSERT(flags == 0 || flags == O_EXCL); int oerrno; int fd = -1; size_t len = strlen(dir) + strlen(templ) + 1; char *fullname = Malloc(sizeof(*fullname) * len); if (fullname == NULL) { ERR("!Malloc"); return -1; } int ret = _snprintf(fullname, len, "%s%s", dir, templ); if (ret < 0 || ret >= len) { ERR("snprintf: %d", ret); goto err; } LOG(4, "fullname \"%s\"", fullname); /* * XXX - block signals and modify file creation mask for the time * of mkstmep() execution. Restore previous settings once the file * is created. */ fd = os_mkstemp(fullname); if (fd < 0) { ERR("!os_mkstemp"); goto err; } /* * There is no point to use unlink() here. First, because it does not * work on open files. Second, because the file is created with * O_TEMPORARY flag, and it looks like such temp files cannot be open * from another process, even though they are visible on * the filesystem. */ Free(fullname); return fd; err: Free(fullname); oerrno = errno; if (fd != -1) (void) os_close(fd); errno = oerrno; return -1; } /* * util_is_absolute_path -- check if the path is absolute */ int util_is_absolute_path(const char *path) { LOG(3, "path \"%s\"", path); if (path == NULL || path[0] == '\0') return 0; if (path[0] == '\\' || path[1] == ':') return 1; return 0; } /* * util_file_mkdir -- creates new dir */ int util_file_mkdir(const char *path, mode_t mode) { /* * On windows we cannot create read only dir so mode * parameter is useless. */ UNREFERENCED_PARAMETER(mode); LOG(3, "path: %s mode: %d", path, mode); return _mkdir(path); } /* * util_file_dir_open -- open a directory */ int util_file_dir_open(struct dir_handle *handle, const char *path) { /* init handle */ handle->handle = NULL; handle->path = path; return 0; } /* * util_file_dir_next - read next file in directory */ int util_file_dir_next(struct dir_handle *handle, struct file_info *info) { WIN32_FIND_DATAA data; if (handle->handle == NULL) { handle->handle = FindFirstFileA(handle->path, &data); if (handle->handle == NULL) return 1; } else { if (FindNextFileA(handle->handle, &data) == 0) return 1; } info->filename[NAME_MAX] = '\0'; strncpy(info->filename, data.cFileName, NAME_MAX + 1); if (info->filename[NAME_MAX] != '\0') return -1; /* filename truncated */ info->is_dir = data.dwFileAttributes == FILE_ATTRIBUTE_DIRECTORY; return 0; } /* * util_file_dir_close -- close a directory */ int util_file_dir_close(struct dir_handle *handle) { return FindClose(handle->handle); } /* * util_file_dir_close -- remove directory */ int util_file_dir_remove(const char *path) { return RemoveDirectoryA(path) == 0 ? -1 : 0; } /* * util_file_device_dax_alignment -- returns internal Device DAX alignment */ size_t util_file_device_dax_alignment(const char *path) { LOG(3, "path \"%s\"", path); return 0; } /* * util_ddax_region_find -- returns DEV dax region id that contains file */ int util_ddax_region_find(const char *path) { LOG(3, "path \"%s\"", path); return -1; } pmdk-1.8/src/common/rand.h0000664000000000000000000000352613615011243014173 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rand.h -- random utils */ #ifndef RAND_H #define RAND_H 1 #include typedef uint64_t rng_t[4]; uint64_t hash64(uint64_t x); uint64_t rnd64_r(rng_t *rng); void randomize_r(rng_t *rng, uint64_t seed); uint64_t rnd64(void); void randomize(uint64_t seed); #endif pmdk-1.8/src/common/errno_freebsd.h0000664000000000000000000000357713615011243016074 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * errno_freebsd.h -- map Linux errno's to something close on FreeBSD */ #ifndef PMDK_ERRNO_FREEBSD_H #define PMDK_ERRNO_FREEBSD_H 1 #ifdef __FreeBSD__ #define EBADFD EBADF #define ELIBACC EINVAL #define EMEDIUMTYPE EOPNOTSUPP #define ENOMEDIUM ENODEV #define EREMOTEIO EIO #endif #endif /* PMDK_ERRNO_FREEBSD_H */ pmdk-1.8/src/common/os_posix.c0000664000000000000000000002045513615011243015105 0ustar rootroot/* * Copyright 2017-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * os_posix.c -- abstraction layer for basic Posix functions */ #define _GNU_SOURCE #include #include #include #ifdef __FreeBSD__ #include #endif #include #include #include #include #include #include #include #include #include "util.h" #include "out.h" #include "os.h" /* * os_open -- open abstraction layer */ int os_open(const char *pathname, int flags, ...) { int mode_required = (flags & O_CREAT) == O_CREAT; #ifdef O_TMPFILE mode_required |= (flags & O_TMPFILE) == O_TMPFILE; #endif if (mode_required) { va_list arg; va_start(arg, flags); /* Clang requires int due to auto-promotion */ int mode = va_arg(arg, int); va_end(arg); return open(pathname, flags, (mode_t)mode); } else { return open(pathname, flags); } } /* * os_fsync -- fsync abstraction layer */ int os_fsync(int fd) { return fsync(fd); } /* * os_fsync_dir -- fsync the directory */ int os_fsync_dir(const char *dir_name) { int fd = os_open(dir_name, O_RDONLY | O_DIRECTORY); if (fd < 0) return -1; int ret = os_fsync(fd); os_close(fd); return ret; } /* * os_stat -- stat abstraction layer */ int os_stat(const char *pathname, os_stat_t *buf) { return stat(pathname, buf); } /* * os_unlink -- unlink abstraction layer */ int os_unlink(const char *pathname) { return unlink(pathname); } /* * os_access -- access abstraction layer */ int os_access(const char *pathname, int mode) { return access(pathname, mode); } /* * os_fopen -- fopen abstraction layer */ FILE * os_fopen(const char *pathname, const char *mode) { return fopen(pathname, mode); } /* * os_fdopen -- fdopen abstraction layer */ FILE * os_fdopen(int fd, const char *mode) { return fdopen(fd, mode); } /* * os_chmod -- chmod abstraction layer */ int os_chmod(const char *pathname, mode_t mode) { return chmod(pathname, mode); } /* * os_mkstemp -- mkstemp abstraction layer */ int os_mkstemp(char *temp) { return mkstemp(temp); } /* * os_posix_fallocate -- posix_fallocate abstraction layer */ int os_posix_fallocate(int fd, os_off_t offset, off_t len) { #ifdef __FreeBSD__ struct stat fbuf; struct statfs fsbuf; /* * XXX Workaround for https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=223287 * * FreeBSD implements posix_fallocate with a simple block allocation/zero * loop. If the requested size is unreasonably large, this can result in * an uninterruptable system call that will suck up all the space in the * file system and could take hours to fail. To avoid this, make a crude * check to see if the requested allocation is larger than the available * space in the file system (minus any blocks already allocated to the * file), and if so, immediately return ENOSPC. We do the check only if * the offset is 0; otherwise, trying to figure out how many additional * blocks are required is too complicated. * * This workaround is here mostly to fail "absurdly" large requests for * testing purposes; however, it is coded to allow normal (albeit slow) * operation if the space can actually be allocated. Because of the way * PMDK uses posix_fallocate, supporting Linux-style fallocate in * FreeBSD should be considered. */ if (offset == 0) { if (fstatfs(fd, &fsbuf) == -1 || fstat(fd, &fbuf) == -1) return errno; size_t reqd_blocks = ((size_t)len + (fsbuf.f_bsize - 1)) / fsbuf.f_bsize; if (fbuf.st_blocks > 0) { if (reqd_blocks >= (size_t)fbuf.st_blocks) reqd_blocks -= (size_t)fbuf.st_blocks; else reqd_blocks = 0; } if (reqd_blocks > (size_t)fsbuf.f_bavail) return ENOSPC; } #endif /* * First, try to alloc the whole thing in one go. This allows ENOSPC to * fail immediately -- allocating piece by piece would fill the storage * just to abort halfway. */ int err = posix_fallocate(fd, offset, len); if (err != ENOMEM && err != EINTR) return err; /* * Workaround for a bug in tmpfs where it fails large but reasonable * requests that exceed available DRAM but fit within swap space. And * even if a request fits within DRAM, tmpfs will evict other tasks * just to reserve space. * * We also want to survive random unrelated signals. Profilers spam * the program with SIGVTALRM/SIGPROF, anything run from a terminal can * receive SIGNWINCH, etc. As fallocate is a long-running syscall, * let's restart it, but in a way that avoids infinite loops. * * Thus: * * limit a single syscall to 1GB * * ignore sporadic signals * * on repeated failures, start reducing syscall size * * ... but not below 1MB */ os_off_t chunk = 1LL << 30; /* 1GB */ int tries = 0; while (len) { if (chunk > len) chunk = len; int err = posix_fallocate(fd, offset, chunk); if (!err) { offset += chunk; len -= chunk; tries = 0; } else if (err != ENOMEM && err != EINTR) { return err; } else if (++tries == 5) { tries = 0; chunk /= 2; /* * Within memory pressure or a signal storm, small * allocs are more likely to get through, but once we * get this small, something is badly wrong. */ if (chunk < 1LL << 20) /* 1MB */ return err; } } return 0; } /* * os_ftruncate -- ftruncate abstraction layer */ int os_ftruncate(int fd, os_off_t length) { return ftruncate(fd, length); } /* * os_flock -- flock abstraction layer */ int os_flock(int fd, int operation) { int opt = 0; if (operation & OS_LOCK_EX) opt |= LOCK_EX; if (operation & OS_LOCK_SH) opt |= LOCK_SH; if (operation & OS_LOCK_UN) opt |= LOCK_UN; if (operation & OS_LOCK_NB) opt |= LOCK_NB; return flock(fd, opt); } /* * os_writev -- writev abstraction layer */ ssize_t os_writev(int fd, const struct iovec *iov, int iovcnt) { return writev(fd, iov, iovcnt); } /* * os_clock_gettime -- clock_gettime abstraction layer */ int os_clock_gettime(int id, struct timespec *ts) { return clock_gettime(id, ts); } /* * os_rand_r -- rand_r abstraction layer */ unsigned os_rand_r(unsigned *seedp) { return (unsigned)rand_r(seedp); } /* * os_unsetenv -- unsetenv abstraction layer */ int os_unsetenv(const char *name) { return unsetenv(name); } /* * os_setenv -- setenv abstraction layer */ int os_setenv(const char *name, const char *value, int overwrite) { return setenv(name, value, overwrite); } /* * secure_getenv -- provide GNU secure_getenv for FreeBSD */ #if defined(__FreeBSD__) static char * secure_getenv(const char *name) { if (issetugid() != 0) return NULL; return getenv(name); } #endif /* * os_getenv -- getenv abstraction layer */ char * os_getenv(const char *name) { return secure_getenv(name); } /* * os_strsignal -- strsignal abstraction layer */ const char * os_strsignal(int sig) { return strsignal(sig); } int os_execv(const char *path, char *const argv[]) { return execv(path, argv); } pmdk-1.8/src/common/valgrind_internal.h0000664000000000000000000002672513615011243016757 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * valgrind_internal.h -- internal definitions for valgrind macros */ #ifndef PMDK_VALGRIND_INTERNAL_H #define PMDK_VALGRIND_INTERNAL_H 1 #if !defined(_WIN32) && !defined(__FreeBSD__) #ifndef VALGRIND_ENABLED #define VALGRIND_ENABLED 1 #endif #endif #if VALGRIND_ENABLED #define VG_PMEMCHECK_ENABLED 1 #define VG_HELGRIND_ENABLED 1 #define VG_MEMCHECK_ENABLED 1 #define VG_DRD_ENABLED 1 #endif #if VG_PMEMCHECK_ENABLED || VG_HELGRIND_ENABLED || VG_MEMCHECK_ENABLED || \ VG_DRD_ENABLED #define ANY_VG_TOOL_ENABLED 1 #else #define ANY_VG_TOOL_ENABLED 0 #endif #if ANY_VG_TOOL_ENABLED extern unsigned _On_valgrind; #define On_valgrind __builtin_expect(_On_valgrind, 0) #include "valgrind/valgrind.h" #else #define On_valgrind (0) #endif #if VG_HELGRIND_ENABLED #include "valgrind/helgrind.h" #endif #if VG_DRD_ENABLED #include "valgrind/drd.h" #endif #if VG_HELGRIND_ENABLED || VG_DRD_ENABLED #define VALGRIND_ANNOTATE_HAPPENS_BEFORE(obj) do {\ if (On_valgrind) \ ANNOTATE_HAPPENS_BEFORE((obj));\ } while (0) #define VALGRIND_ANNOTATE_HAPPENS_AFTER(obj) do {\ if (On_valgrind) \ ANNOTATE_HAPPENS_AFTER((obj));\ } while (0) #define VALGRIND_ANNOTATE_NEW_MEMORY(addr, size) do {\ if (On_valgrind) \ ANNOTATE_NEW_MEMORY((addr), (size));\ } while (0) #define VALGRIND_ANNOTATE_IGNORE_READS_BEGIN() do {\ if (On_valgrind) \ ANNOTATE_IGNORE_READS_BEGIN();\ } while (0) #define VALGRIND_ANNOTATE_IGNORE_READS_END() do {\ if (On_valgrind) \ ANNOTATE_IGNORE_READS_END();\ } while (0) #define VALGRIND_ANNOTATE_IGNORE_WRITES_BEGIN() do {\ if (On_valgrind) \ ANNOTATE_IGNORE_WRITES_BEGIN();\ } while (0) #define VALGRIND_ANNOTATE_IGNORE_WRITES_END() do {\ if (On_valgrind) \ ANNOTATE_IGNORE_WRITES_END();\ } while (0) /* Supported by both helgrind and drd. */ #define VALGRIND_HG_DRD_DISABLE_CHECKING(addr, size) do {\ if (On_valgrind) \ VALGRIND_HG_DISABLE_CHECKING((addr), (size));\ } while (0) #else #define VALGRIND_ANNOTATE_HAPPENS_BEFORE(obj) do { (void)(obj); } while (0) #define VALGRIND_ANNOTATE_HAPPENS_AFTER(obj) do { (void)(obj); } while (0) #define VALGRIND_ANNOTATE_NEW_MEMORY(addr, size) do {\ (void) (addr);\ (void) (size);\ } while (0) #define VALGRIND_ANNOTATE_IGNORE_READS_BEGIN() do {} while (0) #define VALGRIND_ANNOTATE_IGNORE_READS_END() do {} while (0) #define VALGRIND_ANNOTATE_IGNORE_WRITES_BEGIN() do {} while (0) #define VALGRIND_ANNOTATE_IGNORE_WRITES_END() do {} while (0) #define VALGRIND_HG_DRD_DISABLE_CHECKING(addr, size) do {\ (void) (addr);\ (void) (size);\ } while (0) #endif #if VG_PMEMCHECK_ENABLED #include "valgrind/pmemcheck.h" void pobj_emit_log(const char *func, int order); void pmem_emit_log(const char *func, int order); extern int _Pmreorder_emit; #define Pmreorder_emit __builtin_expect(_Pmreorder_emit, 0) #define VALGRIND_REGISTER_PMEM_MAPPING(addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_REGISTER_PMEM_MAPPING((addr), (len));\ } while (0) #define VALGRIND_REGISTER_PMEM_FILE(desc, base_addr, size, offset) do {\ if (On_valgrind)\ VALGRIND_PMC_REGISTER_PMEM_FILE((desc), (base_addr), (size), \ (offset));\ } while (0) #define VALGRIND_REMOVE_PMEM_MAPPING(addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_REMOVE_PMEM_MAPPING((addr), (len));\ } while (0) #define VALGRIND_CHECK_IS_PMEM_MAPPING(addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_CHECK_IS_PMEM_MAPPING((addr), (len));\ } while (0) #define VALGRIND_PRINT_PMEM_MAPPINGS do {\ if (On_valgrind)\ VALGRIND_PMC_PRINT_PMEM_MAPPINGS;\ } while (0) #define VALGRIND_DO_FLUSH(addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_DO_FLUSH((addr), (len));\ } while (0) #define VALGRIND_DO_FENCE do {\ if (On_valgrind)\ VALGRIND_PMC_DO_FENCE;\ } while (0) #define VALGRIND_DO_PERSIST(addr, len) do {\ if (On_valgrind) {\ VALGRIND_PMC_DO_FLUSH((addr), (len));\ VALGRIND_PMC_DO_FENCE;\ }\ } while (0) #define VALGRIND_SET_CLEAN(addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_SET_CLEAN(addr, len);\ } while (0) #define VALGRIND_WRITE_STATS do {\ if (On_valgrind)\ VALGRIND_PMC_WRITE_STATS;\ } while (0) #define VALGRIND_EMIT_LOG(emit_log) do {\ if (On_valgrind)\ VALGRIND_PMC_EMIT_LOG((emit_log));\ } while (0) #define VALGRIND_START_TX do {\ if (On_valgrind)\ VALGRIND_PMC_START_TX;\ } while (0) #define VALGRIND_START_TX_N(txn) do {\ if (On_valgrind)\ VALGRIND_PMC_START_TX_N(txn);\ } while (0) #define VALGRIND_END_TX do {\ if (On_valgrind)\ VALGRIND_PMC_END_TX;\ } while (0) #define VALGRIND_END_TX_N(txn) do {\ if (On_valgrind)\ VALGRIND_PMC_END_TX_N(txn);\ } while (0) #define VALGRIND_ADD_TO_TX(addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_ADD_TO_TX(addr, len);\ } while (0) #define VALGRIND_ADD_TO_TX_N(txn, addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_ADD_TO_TX_N(txn, addr, len);\ } while (0) #define VALGRIND_REMOVE_FROM_TX(addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_REMOVE_FROM_TX(addr, len);\ } while (0) #define VALGRIND_REMOVE_FROM_TX_N(txn, addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_REMOVE_FROM_TX_N(txn, addr, len);\ } while (0) #define VALGRIND_ADD_TO_GLOBAL_TX_IGNORE(addr, len) do {\ if (On_valgrind)\ VALGRIND_PMC_ADD_TO_GLOBAL_TX_IGNORE(addr, len);\ } while (0) /* * Logs library and function name with proper suffix * to pmemcheck store log file. */ #define PMEMOBJ_API_START()\ if (Pmreorder_emit)\ pobj_emit_log(__func__, 0); #define PMEMOBJ_API_END()\ if (Pmreorder_emit)\ pobj_emit_log(__func__, 1); #define PMEM_API_START()\ if (Pmreorder_emit)\ pmem_emit_log(__func__, 0); #define PMEM_API_END()\ if (Pmreorder_emit)\ pmem_emit_log(__func__, 1); #else #define Pmreorder_emit (0) #define VALGRIND_REGISTER_PMEM_MAPPING(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_REGISTER_PMEM_FILE(desc, base_addr, size, offset) do {\ (void) (desc);\ (void) (base_addr);\ (void) (size);\ (void) (offset);\ } while (0) #define VALGRIND_REMOVE_PMEM_MAPPING(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_CHECK_IS_PMEM_MAPPING(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_PRINT_PMEM_MAPPINGS do {} while (0) #define VALGRIND_DO_FLUSH(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_DO_FENCE do {} while (0) #define VALGRIND_DO_PERSIST(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_SET_CLEAN(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_WRITE_STATS do {} while (0) #define VALGRIND_EMIT_LOG(emit_log) do {\ (void) (emit_log);\ } while (0) #define VALGRIND_START_TX do {} while (0) #define VALGRIND_START_TX_N(txn) do { (void) (txn); } while (0) #define VALGRIND_END_TX do {} while (0) #define VALGRIND_END_TX_N(txn) do {\ (void) (txn);\ } while (0) #define VALGRIND_ADD_TO_TX(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_ADD_TO_TX_N(txn, addr, len) do {\ (void) (txn);\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_REMOVE_FROM_TX(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_REMOVE_FROM_TX_N(txn, addr, len) do {\ (void) (txn);\ (void) (addr);\ (void) (len);\ } while (0) #define VALGRIND_ADD_TO_GLOBAL_TX_IGNORE(addr, len) do {\ (void) (addr);\ (void) (len);\ } while (0) #define PMEMOBJ_API_START() do {} while (0) #define PMEMOBJ_API_END() do {} while (0) #define PMEM_API_START() do {} while (0) #define PMEM_API_END() do {} while (0) #endif #if VG_MEMCHECK_ENABLED #include "valgrind/memcheck.h" #define VALGRIND_DO_DISABLE_ERROR_REPORTING do {\ if (On_valgrind)\ VALGRIND_DISABLE_ERROR_REPORTING;\ } while (0) #define VALGRIND_DO_ENABLE_ERROR_REPORTING do {\ if (On_valgrind)\ VALGRIND_ENABLE_ERROR_REPORTING;\ } while (0) #define VALGRIND_DO_CREATE_MEMPOOL(heap, rzB, is_zeroed) do {\ if (On_valgrind)\ VALGRIND_CREATE_MEMPOOL(heap, rzB, is_zeroed);\ } while (0) #define VALGRIND_DO_DESTROY_MEMPOOL(heap) do {\ if (On_valgrind)\ VALGRIND_DESTROY_MEMPOOL(heap);\ } while (0) #define VALGRIND_DO_MEMPOOL_ALLOC(heap, addr, size) do {\ if (On_valgrind)\ VALGRIND_MEMPOOL_ALLOC(heap, addr, size);\ } while (0) #define VALGRIND_DO_MEMPOOL_FREE(heap, addr) do {\ if (On_valgrind)\ VALGRIND_MEMPOOL_FREE(heap, addr);\ } while (0) #define VALGRIND_DO_MEMPOOL_CHANGE(heap, addrA, addrB, size) do {\ if (On_valgrind)\ VALGRIND_MEMPOOL_CHANGE(heap, addrA, addrB, size);\ } while (0) #define VALGRIND_DO_MAKE_MEM_DEFINED(addr, len) do {\ if (On_valgrind)\ VALGRIND_MAKE_MEM_DEFINED(addr, len);\ } while (0) #define VALGRIND_DO_MAKE_MEM_UNDEFINED(addr, len) do {\ if (On_valgrind)\ VALGRIND_MAKE_MEM_UNDEFINED(addr, len);\ } while (0) #define VALGRIND_DO_MAKE_MEM_NOACCESS(addr, len) do {\ if (On_valgrind)\ VALGRIND_MAKE_MEM_NOACCESS(addr, len);\ } while (0) #define VALGRIND_DO_CHECK_MEM_IS_ADDRESSABLE(addr, len) do {\ if (On_valgrind)\ VALGRIND_CHECK_MEM_IS_ADDRESSABLE(addr, len);\ } while (0) #else #define VALGRIND_DO_DISABLE_ERROR_REPORTING do {} while (0) #define VALGRIND_DO_ENABLE_ERROR_REPORTING do {} while (0) #define VALGRIND_DO_CREATE_MEMPOOL(heap, rzB, is_zeroed)\ do { (void) (heap); (void) (rzB); (void) (is_zeroed); } while (0) #define VALGRIND_DO_DESTROY_MEMPOOL(heap)\ do { (void) (heap); } while (0) #define VALGRIND_DO_MEMPOOL_ALLOC(heap, addr, size)\ do { (void) (heap); (void) (addr); (void) (size); } while (0) #define VALGRIND_DO_MEMPOOL_FREE(heap, addr)\ do { (void) (heap); (void) (addr); } while (0) #define VALGRIND_DO_MEMPOOL_CHANGE(heap, addrA, addrB, size)\ do {\ (void) (heap); (void) (addrA); (void) (addrB); (void) (size);\ } while (0) #define VALGRIND_DO_MAKE_MEM_DEFINED(addr, len)\ do { (void) (addr); (void) (len); } while (0) #define VALGRIND_DO_MAKE_MEM_UNDEFINED(addr, len)\ do { (void) (addr); (void) (len); } while (0) #define VALGRIND_DO_MAKE_MEM_NOACCESS(addr, len)\ do { (void) (addr); (void) (len); } while (0) #define VALGRIND_DO_CHECK_MEM_IS_ADDRESSABLE(addr, len)\ do { (void) (addr); (void) (len); } while (0) #endif #endif pmdk-1.8/src/libpmempool/0000775000000000000000000000000013615011243014117 5ustar rootrootpmdk-1.8/src/libpmempool/pool.h0000664000000000000000000001215313615011243015243 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pool.h -- internal definitions for pool processing functions */ #ifndef POOL_H #define POOL_H #include #include #include "libpmemobj.h" #include "queue.h" #include "set.h" #include "log.h" #include "blk.h" #include "btt_layout.h" #ifdef __cplusplus extern "C" { #endif #include "alloc.h" #include "fault_injection.h" enum pool_type { POOL_TYPE_UNKNOWN = (1 << 0), POOL_TYPE_LOG = (1 << 1), POOL_TYPE_BLK = (1 << 2), POOL_TYPE_OBJ = (1 << 3), POOL_TYPE_BTT = (1 << 4), POOL_TYPE_ANY = POOL_TYPE_UNKNOWN | POOL_TYPE_LOG | POOL_TYPE_BLK | POOL_TYPE_OBJ | POOL_TYPE_BTT, }; struct pool_params { enum pool_type type; char signature[POOL_HDR_SIG_LEN]; features_t features; size_t size; mode_t mode; int is_poolset; int is_part; int is_dev_dax; int is_pmem; union { struct { uint64_t bsize; } blk; struct { char layout[PMEMOBJ_MAX_LAYOUT]; } obj; }; }; struct pool_set_file { int fd; char *fname; void *addr; size_t size; struct pool_set *poolset; time_t mtime; mode_t mode; }; struct arena { PMDK_TAILQ_ENTRY(arena) next; struct btt_info btt_info; uint32_t id; bool valid; bool zeroed; uint64_t offset; uint8_t *flog; size_t flogsize; uint32_t *map; size_t mapsize; }; struct pool_data { struct pool_params params; struct pool_set_file *set_file; int blk_no_layout; union { struct pool_hdr pool; struct pmemlog log; struct pmemblk blk; } hdr; enum { UUID_NOP = 0, UUID_FROM_BTT, UUID_NOT_FROM_BTT, } uuid_op; struct arena bttc; PMDK_TAILQ_HEAD(arenashead, arena) arenas; uint32_t narenas; }; struct pool_data *pool_data_alloc(PMEMpoolcheck *ppc); void pool_data_free(struct pool_data *pool); void pool_params_from_header(struct pool_params *params, const struct pool_hdr *hdr); int pool_set_parse(struct pool_set **setp, const char *path); void *pool_set_file_map(struct pool_set_file *file, uint64_t offset); int pool_read(struct pool_data *pool, void *buff, size_t nbytes, uint64_t off); int pool_write(struct pool_data *pool, const void *buff, size_t nbytes, uint64_t off); int pool_copy(struct pool_data *pool, const char *dst_path, int overwrite); int pool_set_part_copy(struct pool_set_part *dpart, struct pool_set_part *spart, int overwrite); int pool_memset(struct pool_data *pool, uint64_t off, int c, size_t count); unsigned pool_set_files_count(struct pool_set_file *file); int pool_set_file_map_headers(struct pool_set_file *file, int rdonly, int prv); void pool_set_file_unmap_headers(struct pool_set_file *file); void pool_hdr_default(enum pool_type type, struct pool_hdr *hdrp); enum pool_type pool_hdr_get_type(const struct pool_hdr *hdrp); enum pool_type pool_set_type(struct pool_set *set); const char *pool_get_pool_type_str(enum pool_type type); int pool_btt_info_valid(struct btt_info *infop); int pool_blk_get_first_valid_arena(struct pool_data *pool, struct arena *arenap); int pool_blk_bsize_valid(uint32_t bsize, uint64_t fsize); uint64_t pool_next_arena_offset(struct pool_data *pool, uint64_t header_offset); uint64_t pool_get_first_valid_btt(struct pool_data *pool, struct btt_info *infop, uint64_t offset, bool *zeroed); size_t pool_get_min_size(enum pool_type); #if FAULT_INJECTION void pmempool_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at); int pmempool_fault_injection_enabled(void); #else static inline void pmempool_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { abort(); } static inline int pmempool_fault_injection_enabled(void) { return 0; } #endif #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmempool/check_pool_hdr.c0000664000000000000000000006737113615011243017244 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_pool_hdr.c -- pool header check */ #include #include #include #include #include "out.h" #include "util_pmem.h" #include "libpmempool.h" #include "libpmem.h" #include "pmempool.h" #include "pool.h" #include "set.h" #include "check_util.h" #define NO_COMMON_POOLSET_UUID "%sno common pool_hdr.poolset_uuid" #define INVALID_UUID "%sinvalid pool_hdr.uuid" #define INVALID_CHECKSUM "%sinvalid pool_hdr.checksum" enum question { Q_DEFAULT_SIGNATURE, Q_DEFAULT_MAJOR, Q_DEFAULT_COMPAT_FEATURES, Q_DEFAULT_INCOMPAT_FEATURES, Q_DEFAULT_RO_COMPAT_FEATURES, Q_ZERO_UNUSED_AREA, Q_ARCH_FLAGS, Q_CRTIME, Q_CHECKSUM, Q_POOLSET_UUID_SET, Q_POOLSET_UUID_FROM_BTT_INFO, Q_POOLSET_UUID_REGENERATE, Q_UUID_SET, Q_UUID_REGENERATE, Q_NEXT_PART_UUID_SET, Q_PREV_PART_UUID_SET, Q_NEXT_REPL_UUID_SET, Q_PREV_REPL_UUID_SET }; /* * pool_hdr_possible_type -- (internal) return possible type of pool */ static enum pool_type pool_hdr_possible_type(PMEMpoolcheck *ppc) { if (pool_blk_get_first_valid_arena(ppc->pool, &ppc->pool->bttc)) return POOL_TYPE_BLK; return POOL_TYPE_UNKNOWN; } /* * pool_hdr_valid -- (internal) return true if pool header is valid */ static int pool_hdr_valid(struct pool_hdr *hdrp) { return !util_is_zeroed((void *)hdrp, sizeof(*hdrp)) && util_checksum(hdrp, sizeof(*hdrp), &hdrp->checksum, 0, POOL_HDR_CSUM_END_OFF(hdrp)); } /* * pool_supported -- (internal) check if pool type is supported */ static int pool_supported(enum pool_type type) { switch (type) { case POOL_TYPE_LOG: return 1; case POOL_TYPE_BLK: return 1; case POOL_TYPE_OBJ: default: return 0; } } /* * pool_hdr_preliminary_check -- (internal) check pool header checksum and pool * parameters */ static int pool_hdr_preliminary_check(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); CHECK_INFO(ppc, "%schecking pool header", loc->prefix); if (util_is_zeroed((void *)&loc->hdr, sizeof(loc->hdr))) { if (CHECK_IS_NOT(ppc, REPAIR)) { check_end(ppc->data); ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, "%sempty pool hdr", loc->prefix); } } else if (loc->hdr_valid) { enum pool_type type = pool_hdr_get_type(&loc->hdr); if (type == POOL_TYPE_UNKNOWN) { if (CHECK_IS_NOT(ppc, REPAIR)) { check_end(ppc->data); ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, "%sinvalid signature", loc->prefix); } CHECK_INFO(ppc, "%sinvalid signature", loc->prefix); } else { /* valid check sum */ CHECK_INFO(ppc, "%spool header correct", loc->prefix); loc->step = CHECK_STEP_COMPLETE; return 0; } } else if (CHECK_IS_NOT(ppc, REPAIR)) { check_end(ppc->data); ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, "%sincorrect pool header", loc->prefix); } else { CHECK_INFO(ppc, "%sincorrect pool header", loc->prefix); } ASSERT(CHECK_IS(ppc, REPAIR)); if (ppc->pool->params.type == POOL_TYPE_UNKNOWN) { ppc->pool->params.type = pool_hdr_possible_type(ppc); if (ppc->pool->params.type == POOL_TYPE_UNKNOWN) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; return CHECK_ERR(ppc, "cannot determine pool type"); } } if (!pool_supported(ppc->pool->params.type)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; return CHECK_ERR(ppc, "the repair of %s pools is not supported", pool_get_pool_type_str(ppc->pool->params.type)); } return 0; } /* * pool_hdr_default_check -- (internal) check some default values in pool header */ static int pool_hdr_default_check(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); ASSERT(CHECK_IS(ppc, REPAIR)); struct pool_hdr def_hdr; pool_hdr_default(ppc->pool->params.type, &def_hdr); if (memcmp(loc->hdr.signature, def_hdr.signature, POOL_HDR_SIG_LEN)) { CHECK_ASK(ppc, Q_DEFAULT_SIGNATURE, "%spool_hdr.signature is not valid.|Do you want to set " "it to %.8s?", loc->prefix, def_hdr.signature); } if (loc->hdr.major != def_hdr.major) { CHECK_ASK(ppc, Q_DEFAULT_MAJOR, "%spool_hdr.major is not valid.|Do you want to set it " "to default value 0x%x?", loc->prefix, def_hdr.major); } features_t unknown = util_get_unknown_features( loc->hdr.features, def_hdr.features); if (unknown.compat) { CHECK_ASK(ppc, Q_DEFAULT_COMPAT_FEATURES, "%spool_hdr.features.compat is not valid.|Do you want " "to set it to default value 0x%x?", loc->prefix, def_hdr.features.compat); } if (unknown.incompat) { CHECK_ASK(ppc, Q_DEFAULT_INCOMPAT_FEATURES, "%spool_hdr.features.incompat is not valid.|Do you " "want to set it to default value 0x%x?", loc->prefix, def_hdr.features.incompat); } if (unknown.ro_compat) { CHECK_ASK(ppc, Q_DEFAULT_RO_COMPAT_FEATURES, "%spool_hdr.features.ro_compat is not valid.|Do you " "want to set it to default value 0x%x?", loc->prefix, def_hdr.features.ro_compat); } if (!util_is_zeroed(loc->hdr.unused, sizeof(loc->hdr.unused))) { CHECK_ASK(ppc, Q_ZERO_UNUSED_AREA, "%sunused area is not filled by zeros.|Do you want to " "fill it up?", loc->prefix); } return check_questions_sequence_validate(ppc); } /* * pool_hdr_default_fix -- (internal) fix some default values in pool header */ static int pool_hdr_default_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); ASSERTne(loc, NULL); struct pool_hdr def_hdr; pool_hdr_default(ppc->pool->params.type, &def_hdr); switch (question) { case Q_DEFAULT_SIGNATURE: CHECK_INFO(ppc, "%ssetting pool_hdr.signature to %.8s", loc->prefix, def_hdr.signature); memcpy(&loc->hdr.signature, &def_hdr.signature, POOL_HDR_SIG_LEN); break; case Q_DEFAULT_MAJOR: CHECK_INFO(ppc, "%ssetting pool_hdr.major to 0x%x", loc->prefix, def_hdr.major); loc->hdr.major = def_hdr.major; break; case Q_DEFAULT_COMPAT_FEATURES: CHECK_INFO(ppc, "%ssetting pool_hdr.features.compat to 0x%x", loc->prefix, def_hdr.features.compat); loc->hdr.features.compat = def_hdr.features.compat; break; case Q_DEFAULT_INCOMPAT_FEATURES: CHECK_INFO(ppc, "%ssetting pool_hdr.features.incompat to 0x%x", loc->prefix, def_hdr.features.incompat); loc->hdr.features.incompat = def_hdr.features.incompat; break; case Q_DEFAULT_RO_COMPAT_FEATURES: CHECK_INFO(ppc, "%ssetting pool_hdr.features.ro_compat to 0x%x", loc->prefix, def_hdr.features.ro_compat); loc->hdr.features.ro_compat = def_hdr.features.ro_compat; break; case Q_ZERO_UNUSED_AREA: CHECK_INFO(ppc, "%ssetting pool_hdr.unused to zeros", loc->prefix); memset(loc->hdr.unused, 0, sizeof(loc->hdr.unused)); break; default: ERR("not implemented question id: %u", question); } return 0; } /* * pool_hdr_quick_check -- (internal) end check if pool header is valid */ static int pool_hdr_quick_check(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); if (pool_hdr_valid(loc->hdrp)) loc->step = CHECK_STEP_COMPLETE; return 0; } /* * pool_hdr_nondefault -- (internal) validate custom value fields */ static int pool_hdr_nondefault(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); if (loc->hdr.crtime > (uint64_t)ppc->pool->set_file->mtime) { const char * const error = "%spool_hdr.crtime is not valid"; if (CHECK_IS_NOT(ppc, REPAIR)) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, error, loc->prefix); } else if (CHECK_IS_NOT(ppc, ADVANCED)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_INFO(ppc, "%s" REQUIRE_ADVANCED, loc->prefix); return CHECK_ERR(ppc, error, loc->prefix); } CHECK_ASK(ppc, Q_CRTIME, "%spool_hdr.crtime is not valid.|Do you want to set it " "to file's modtime [%s]?", loc->prefix, check_get_time_str(ppc->pool->set_file->mtime)); } if (loc->valid_part_hdrp && memcmp(&loc->valid_part_hdrp->arch_flags, &loc->hdr.arch_flags, sizeof(struct arch_flags)) != 0) { const char * const error = "%spool_hdr.arch_flags is not valid"; if (CHECK_IS_NOT(ppc, REPAIR)) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, error, loc->prefix); } CHECK_ASK(ppc, Q_ARCH_FLAGS, "%spool_hdr.arch_flags is not valid.|Do you want to " "copy it from a valid part?", loc->prefix); } return check_questions_sequence_validate(ppc); } /* * pool_hdr_nondefault_fix -- (internal) fix custom value fields */ static int pool_hdr_nondefault_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); ASSERTne(loc, NULL); uint64_t *flags = NULL; switch (question) { case Q_CRTIME: CHECK_INFO(ppc, "%ssetting pool_hdr.crtime to file's modtime: " "%s", loc->prefix, check_get_time_str(ppc->pool->set_file->mtime)); util_convert2h_hdr_nocheck(&loc->hdr); loc->hdr.crtime = (uint64_t)ppc->pool->set_file->mtime; util_convert2le_hdr(&loc->hdr); break; case Q_ARCH_FLAGS: flags = (uint64_t *)&loc->valid_part_hdrp->arch_flags; CHECK_INFO(ppc, "%ssetting pool_hdr.arch_flags to 0x%08" PRIx64 "%08" PRIx64, loc->prefix, flags[0], flags[1]); util_convert2h_hdr_nocheck(&loc->hdr); memcpy(&loc->hdr.arch_flags, &loc->valid_part_hdrp->arch_flags, sizeof(struct arch_flags)); util_convert2le_hdr(&loc->hdr); break; default: ERR("not implemented question id: %u", question); } return 0; } /* * pool_hdr_poolset_uuid -- (internal) check poolset_uuid field */ static int pool_hdr_poolset_uuid_find(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); /* * If the pool header is valid and there is not other parts or replicas * in the poolset its poolset_uuid is also valid. */ if (loc->hdr_valid && loc->single_repl && loc->single_part) return 0; if (loc->replica != 0 || loc->part != 0) goto after_lookup; /* for blk pool we can take the UUID from BTT Info header */ if (ppc->pool->params.type == POOL_TYPE_BLK && ppc->pool->bttc.valid) { loc->valid_puuid = &ppc->pool->bttc.btt_info.parent_uuid; if (uuidcmp(loc->hdr.poolset_uuid, *loc->valid_puuid) != 0) { CHECK_ASK(ppc, Q_POOLSET_UUID_FROM_BTT_INFO, "%sinvalid pool_hdr.poolset_uuid.|Do you want " "to set it to %s from BTT Info?", loc->prefix, check_get_uuid_str(*loc->valid_puuid)); goto exit_question; } } if (loc->single_part && loc->single_repl) { /* * If the pool is not blk pool or BTT Info header is invalid * there is no other way to validate poolset uuid. */ return 0; } /* * if all valid poolset part files have the same poolset uuid it is * the valid poolset uuid * if all part files have the same poolset uuid it is valid poolset uuid */ struct pool_set *poolset = ppc->pool->set_file->poolset; unsigned nreplicas = poolset->nreplicas; uuid_t *common_puuid = loc->valid_puuid; for (unsigned r = 0; r < nreplicas; r++) { struct pool_replica *rep = REP(poolset, r); for (unsigned p = 0; p < rep->nhdrs; p++) { struct pool_hdr *hdr = HDR(rep, p); /* * find poolset uuid if it is the same for all part * files */ if (common_puuid == NULL) { if (r == 0 && p == 0) { common_puuid = &hdr->poolset_uuid; } } else if (uuidcmp(*common_puuid, hdr->poolset_uuid) != 0) { common_puuid = NULL; } if (!pool_hdr_valid(hdr)) continue; /* * find poolset uuid if it is the same for all valid * part files */ if (loc->valid_puuid == NULL) { loc->valid_puuid = &hdr->poolset_uuid; } else if (uuidcmp(*loc->valid_puuid, hdr->poolset_uuid) != 0) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, "the poolset contains " "part files from various poolsets"); } } } if (!loc->valid_puuid && common_puuid) loc->valid_puuid = common_puuid; if (loc->valid_puuid) goto after_lookup; if (CHECK_IS_NOT(ppc, REPAIR)) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, NO_COMMON_POOLSET_UUID, loc->prefix); } else if (CHECK_IS_NOT(ppc, ADVANCED)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_INFO(ppc, "%s" REQUIRE_ADVANCED, loc->prefix); return CHECK_ERR(ppc, NO_COMMON_POOLSET_UUID, loc->prefix); } else { CHECK_ASK(ppc, Q_POOLSET_UUID_REGENERATE, NO_COMMON_POOLSET_UUID ".|Do you want to regenerate pool_hdr.poolset_uuid?", loc->prefix); goto exit_question; } after_lookup: if (loc->valid_puuid) { if (uuidcmp(*loc->valid_puuid, loc->hdr.poolset_uuid) != 0) { if (CHECK_IS_NOT(ppc, REPAIR)) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, "%sinvalid " "pool_hdr.poolset_uuid", loc->prefix); } CHECK_ASK(ppc, Q_POOLSET_UUID_SET, "%sinvalid " "pool_hdr.poolset_uuid.|Do you want to set " "it to %s from a valid part file?", loc->prefix, check_get_uuid_str(*loc->valid_puuid)); } } exit_question: return check_questions_sequence_validate(ppc); } /* * pool_hdr_poolset_uuid_fix -- (internal) fix poolset_uuid field */ static int pool_hdr_poolset_uuid_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); ASSERTne(loc, NULL); switch (question) { case Q_POOLSET_UUID_SET: case Q_POOLSET_UUID_FROM_BTT_INFO: CHECK_INFO(ppc, "%ssetting pool_hdr.poolset_uuid to %s", loc->prefix, check_get_uuid_str(*loc->valid_puuid)); memcpy(loc->hdr.poolset_uuid, loc->valid_puuid, POOL_HDR_UUID_LEN); if (question == Q_POOLSET_UUID_SET) ppc->pool->uuid_op = UUID_NOT_FROM_BTT; else ppc->pool->uuid_op = UUID_FROM_BTT; break; case Q_POOLSET_UUID_REGENERATE: if (util_uuid_generate(loc->hdr.poolset_uuid) != 0) { ppc->result = CHECK_RESULT_INTERNAL_ERROR; return CHECK_ERR(ppc, "%suuid generation failed", loc->prefix); } CHECK_INFO(ppc, "%ssetting pool_hdr.pooset_uuid to %s", loc->prefix, check_get_uuid_str(loc->hdr.poolset_uuid)); ppc->pool->uuid_op = UUID_NOT_FROM_BTT; break; default: ERR("not implemented question id: %u", question); } return 0; } #define COMPARE_TO_FIRST_PART_ONLY 2 /* * pool_hdr_uuid_find -- (internal) check UUID value */ static int pool_hdr_uuid_find(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); /* * If the pool header is valid and there is not other parts or replicas * in the poolset its uuid is also valid. */ if (loc->hdr_valid && loc->single_repl && loc->single_part) return 0; int hdrs_valid[] = { loc->next_part_hdr_valid, loc->prev_part_hdr_valid, loc->next_repl_hdr_valid, loc->prev_repl_hdr_valid}; uuid_t *uuids[] = { &loc->next_part_hdrp->prev_part_uuid, &loc->prev_part_hdrp->next_part_uuid, &loc->next_repl_hdrp->prev_repl_uuid, &loc->prev_repl_hdrp->next_repl_uuid }; /* * if all valid poolset part files have the same uuid links to this part * file it is valid uuid * if all links have the same uuid and it is single file pool it is also * the valid uuid */ loc->valid_uuid = NULL; if (loc->hdr_valid) loc->valid_uuid = &loc->hdr.uuid; uuid_t *common_uuid = uuids[0]; COMPILE_ERROR_ON(ARRAY_SIZE(uuids) != ARRAY_SIZE(hdrs_valid)); COMPILE_ERROR_ON(COMPARE_TO_FIRST_PART_ONLY >= ARRAY_SIZE(uuids)); for (unsigned i = 0; i < ARRAY_SIZE(uuids); ++i) { if (i > 0 && common_uuid != NULL) { if (uuidcmp(*common_uuid, *uuids[i]) != 0) { common_uuid = NULL; } } if (i >= COMPARE_TO_FIRST_PART_ONLY && loc->part != 0) continue; if (!hdrs_valid[i]) continue; if (!loc->valid_uuid) { loc->valid_uuid = uuids[i]; } else if (uuidcmp(*loc->valid_uuid, *uuids[i]) != 0) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, "%sambiguous pool_hdr.uuid", loc->prefix); } } if (!loc->valid_uuid && common_uuid) loc->valid_uuid = common_uuid; if (loc->valid_uuid != NULL) { if (uuidcmp(*loc->valid_uuid, loc->hdr.uuid) != 0) { CHECK_ASK(ppc, Q_UUID_SET, INVALID_UUID ".|Do you want " "to set it to %s from a valid part file?", loc->prefix, check_get_uuid_str(*loc->valid_uuid)); } } else if (CHECK_IS(ppc, ADVANCED)) { CHECK_ASK(ppc, Q_UUID_REGENERATE, INVALID_UUID ".|Do you want " "to regenerate it?", loc->prefix); } else if (CHECK_IS(ppc, REPAIR)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_INFO(ppc, "%s" REQUIRE_ADVANCED, loc->prefix); return CHECK_ERR(ppc, INVALID_UUID, loc->prefix); } else { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, INVALID_UUID, loc->prefix); } return check_questions_sequence_validate(ppc); } /* * pool_hdr_uuid_fix -- (internal) fix UUID value */ static int pool_hdr_uuid_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); ASSERTne(loc, NULL); switch (question) { case Q_UUID_SET: CHECK_INFO(ppc, "%ssetting pool_hdr.uuid to %s", loc->prefix, check_get_uuid_str(*loc->valid_uuid)); memcpy(loc->hdr.uuid, loc->valid_uuid, POOL_HDR_UUID_LEN); break; case Q_UUID_REGENERATE: if (util_uuid_generate(loc->hdr.uuid) != 0) { ppc->result = CHECK_RESULT_INTERNAL_ERROR; return CHECK_ERR(ppc, "%suuid generation failed", loc->prefix); } CHECK_INFO(ppc, "%ssetting pool_hdr.uuid to %s", loc->prefix, check_get_uuid_str(loc->hdr.uuid)); break; default: ERR("not implemented question id: %u", question); } return 0; } /* * pool_hdr_uuid_links -- (internal) check UUID links values */ static int pool_hdr_uuid_links(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); /* * If the pool header is valid and there is not other parts or replicas * in the poolset its uuid links are also valid. */ if (loc->hdr_valid && loc->single_repl && loc->single_part) return 0; uuid_t *links[] = { &loc->hdr.next_part_uuid, &loc->hdr.prev_part_uuid, &loc->hdr.next_repl_uuid, &loc->hdr.prev_repl_uuid}; uuid_t *uuids[] = { &loc->next_part_hdrp->uuid, &loc->prev_part_hdrp->uuid, &loc->next_repl_hdrp->uuid, &loc->prev_repl_hdrp->uuid }; uint32_t questions[] = { Q_NEXT_PART_UUID_SET, Q_PREV_PART_UUID_SET, Q_NEXT_REPL_UUID_SET, Q_PREV_REPL_UUID_SET }; const char *fields[] = { "pool_hdr.next_part_uuid", "pool_hdr.prev_part_uuid", "pool_hdr.next_repl_uuid", "pool_hdr.prev_repl_uuid" }; COMPILE_ERROR_ON(ARRAY_SIZE(links) != ARRAY_SIZE(uuids)); COMPILE_ERROR_ON(ARRAY_SIZE(links) != ARRAY_SIZE(questions)); COMPILE_ERROR_ON(ARRAY_SIZE(links) != ARRAY_SIZE(fields)); for (uint64_t i = 0; i < ARRAY_SIZE(links); ++i) { if (uuidcmp(*links[i], *uuids[i]) == 0) continue; if (CHECK_IS(ppc, REPAIR)) { CHECK_ASK(ppc, questions[i], "%sinvalid %s.|Do you want to set it to a " "valid value?", loc->prefix, fields[i]); } else { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, "%sinvalid %s", loc->prefix, fields[i]); } } return check_questions_sequence_validate(ppc); } /* * pool_hdr_uuid_links_fix -- (internal) fix UUID links values */ static int pool_hdr_uuid_links_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); ASSERTne(loc, NULL); switch (question) { case Q_NEXT_PART_UUID_SET: CHECK_INFO(ppc, "%ssetting pool_hdr.next_part_uuid to %s", loc->prefix, check_get_uuid_str(loc->next_part_hdrp->uuid)); memcpy(loc->hdr.next_part_uuid, loc->next_part_hdrp->uuid, POOL_HDR_UUID_LEN); break; case Q_PREV_PART_UUID_SET: CHECK_INFO(ppc, "%ssetting pool_hdr.prev_part_uuid to %s", loc->prefix, check_get_uuid_str(loc->prev_part_hdrp->uuid)); memcpy(loc->hdr.prev_part_uuid, loc->prev_part_hdrp->uuid, POOL_HDR_UUID_LEN); break; case Q_NEXT_REPL_UUID_SET: CHECK_INFO(ppc, "%ssetting pool_hdr.next_repl_uuid to %s", loc->prefix, check_get_uuid_str(loc->next_repl_hdrp->uuid)); memcpy(loc->hdr.next_repl_uuid, loc->next_repl_hdrp->uuid, POOL_HDR_UUID_LEN); break; case Q_PREV_REPL_UUID_SET: CHECK_INFO(ppc, "%ssetting pool_hdr.prev_repl_uuid to %s", loc->prefix, check_get_uuid_str(loc->prev_repl_hdrp->uuid)); memcpy(loc->hdr.prev_repl_uuid, loc->prev_repl_hdrp->uuid, POOL_HDR_UUID_LEN); break; default: ERR("not implemented question id: %u", question); } return 0; } /* * pool_hdr_checksum -- (internal) validate checksum */ static int pool_hdr_checksum(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); if (loc->hdr_valid) return 0; if (CHECK_IS_NOT(ppc, REPAIR)) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, INVALID_CHECKSUM, loc->prefix); } else if (CHECK_IS_NOT(ppc, ADVANCED)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_INFO(ppc, "%s" REQUIRE_ADVANCED, loc->prefix); return CHECK_ERR(ppc, INVALID_CHECKSUM, loc->prefix); } CHECK_ASK(ppc, Q_CHECKSUM, INVALID_CHECKSUM ".|Do you want to " "regenerate checksum?", loc->prefix); return check_questions_sequence_validate(ppc); } /* * pool_hdr_checksum_fix -- (internal) fix checksum */ static int pool_hdr_checksum_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); ASSERTne(loc, NULL); switch (question) { case Q_CHECKSUM: util_checksum(&loc->hdr, sizeof(loc->hdr), &loc->hdr.checksum, 1, POOL_HDR_CSUM_END_OFF(&loc->hdr)); CHECK_INFO(ppc, "%ssetting pool_hdr.checksum to 0x%jx", loc->prefix, le64toh(loc->hdr.checksum)); break; default: ERR("not implemented question id: %u", question); } return 0; } struct step { int (*check)(PMEMpoolcheck *, location *); int (*fix)(PMEMpoolcheck *, location *, uint32_t, void *); }; static const struct step steps_initial[] = { { .check = pool_hdr_preliminary_check, }, { .check = pool_hdr_default_check, }, { .fix = pool_hdr_default_fix, .check = pool_hdr_quick_check, }, { .check = pool_hdr_nondefault, }, { .fix = pool_hdr_nondefault_fix, }, { .check = NULL, .fix = NULL, }, }; static const struct step steps_uuids[] = { { .check = pool_hdr_poolset_uuid_find, }, { .fix = pool_hdr_poolset_uuid_fix, }, { .check = pool_hdr_uuid_find, }, { .fix = pool_hdr_uuid_fix, }, { .check = pool_hdr_uuid_links, }, { .fix = pool_hdr_uuid_links_fix, }, { .check = pool_hdr_checksum, }, { .fix = pool_hdr_checksum_fix, }, { .check = NULL, .fix = NULL, }, }; /* * step_exe -- (internal) perform single step according to its parameters */ static int step_exe(PMEMpoolcheck *ppc, const struct step *steps, location *loc, struct pool_replica *rep, unsigned nreplicas) { const struct step *step = &steps[loc->step++]; if (!step->fix) return step->check(ppc, loc); if (!check_has_answer(ppc->data)) return 0; if (check_answer_loop(ppc, loc, NULL, 1, step->fix)) return -1; util_convert2le_hdr(&loc->hdr); memcpy(loc->hdrp, &loc->hdr, sizeof(loc->hdr)); loc->hdr_valid = pool_hdr_valid(loc->hdrp); util_persist_auto(rep->part[0].is_dev_dax, loc->hdrp, sizeof(*loc->hdrp)); util_convert2h_hdr_nocheck(&loc->hdr); loc->pool_hdr_modified = 1; /* execute check after fix if available */ if (step->check) return step->check(ppc, loc); return 0; } /* * init_location_data -- (internal) prepare location information */ static void init_location_data(PMEMpoolcheck *ppc, location *loc) { /* prepare prefix for messages */ unsigned nfiles = pool_set_files_count(ppc->pool->set_file); if (ppc->result != CHECK_RESULT_PROCESS_ANSWERS) { if (nfiles > 1) { int ret = snprintf(loc->prefix, PREFIX_MAX_SIZE, "replica %u part %u: ", loc->replica, loc->part); if (ret < 0 || ret >= PREFIX_MAX_SIZE) FATAL("snprintf: %d", ret); } else loc->prefix[0] = '\0'; loc->step = 0; } /* get neighboring parts and replicas and briefly validate them */ const struct pool_set *poolset = ppc->pool->set_file->poolset; loc->single_repl = poolset->nreplicas == 1; loc->single_part = poolset->replica[loc->replica]->nparts == 1; struct pool_replica *rep = REP(poolset, loc->replica); struct pool_replica *next_rep = REPN(poolset, loc->replica); struct pool_replica *prev_rep = REPP(poolset, loc->replica); loc->hdrp = HDR(rep, loc->part); memcpy(&loc->hdr, loc->hdrp, sizeof(loc->hdr)); util_convert2h_hdr_nocheck(&loc->hdr); loc->hdr_valid = pool_hdr_valid(loc->hdrp); loc->next_part_hdrp = HDRN(rep, loc->part); loc->prev_part_hdrp = HDRP(rep, loc->part); loc->next_repl_hdrp = HDR(next_rep, 0); loc->prev_repl_hdrp = HDR(prev_rep, 0); loc->next_part_hdr_valid = pool_hdr_valid(loc->next_part_hdrp); loc->prev_part_hdr_valid = pool_hdr_valid(loc->prev_part_hdrp); loc->next_repl_hdr_valid = pool_hdr_valid(loc->next_repl_hdrp); loc->prev_repl_hdr_valid = pool_hdr_valid(loc->prev_repl_hdrp); if (!loc->valid_part_done || loc->valid_part_replica != loc->replica) { loc->valid_part_hdrp = NULL; for (unsigned p = 0; p < rep->nhdrs; ++p) { if (pool_hdr_valid(HDR(rep, p))) { loc->valid_part_hdrp = HDR(rep, p); break; } } loc->valid_part_done = true; } } /* * check_pool_hdr -- entry point for pool header checks */ void check_pool_hdr(PMEMpoolcheck *ppc) { LOG(3, NULL); location *loc = check_get_step_data(ppc->data); unsigned nreplicas = ppc->pool->set_file->poolset->nreplicas; struct pool_set *poolset = ppc->pool->set_file->poolset; for (; loc->replica < nreplicas; loc->replica++) { struct pool_replica *rep = poolset->replica[loc->replica]; for (; loc->part < rep->nhdrs; loc->part++) { init_location_data(ppc, loc); /* do all checks */ while (CHECK_NOT_COMPLETE(loc, steps_initial)) { ASSERT(loc->step < ARRAY_SIZE(steps_initial)); if (step_exe(ppc, steps_initial, loc, rep, nreplicas)) return; } } loc->part = 0; } memcpy(&ppc->pool->hdr.pool, poolset->replica[0]->part[0].hdr, sizeof(struct pool_hdr)); if (loc->pool_hdr_modified) { struct pool_hdr hdr; memcpy(&hdr, &ppc->pool->hdr.pool, sizeof(struct pool_hdr)); util_convert2h_hdr_nocheck(&hdr); pool_params_from_header(&ppc->pool->params, &hdr); } } /* * check_pool_hdr_uuids -- entry point for pool header links checks */ void check_pool_hdr_uuids(PMEMpoolcheck *ppc) { LOG(3, NULL); location *loc = check_get_step_data(ppc->data); unsigned nreplicas = ppc->pool->set_file->poolset->nreplicas; struct pool_set *poolset = ppc->pool->set_file->poolset; for (; loc->replica < nreplicas; loc->replica++) { struct pool_replica *rep = poolset->replica[loc->replica]; for (; loc->part < rep->nparts; loc->part++) { init_location_data(ppc, loc); /* do all checks */ while (CHECK_NOT_COMPLETE(loc, steps_uuids)) { ASSERT(loc->step < ARRAY_SIZE(steps_uuids)); if (step_exe(ppc, steps_uuids, loc, rep, nreplicas)) return; } } loc->part = 0; } memcpy(&ppc->pool->hdr.pool, poolset->replica[0]->part[0].hdr, sizeof(struct pool_hdr)); if (loc->pool_hdr_modified) { struct pool_hdr hdr; memcpy(&hdr, &ppc->pool->hdr.pool, sizeof(struct pool_hdr)); util_convert2h_hdr_nocheck(&hdr); pool_params_from_header(&ppc->pool->params, &hdr); } } pmdk-1.8/src/libpmempool/check_backup.c0000664000000000000000000002241313615011243016667 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_backup.c -- pre-check backup */ #include #include #include #include "out.h" #include "file.h" #include "os.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check_util.h" enum question { Q_OVERWRITE_EXISTING_FILE, Q_OVERWRITE_EXISTING_PARTS }; /* * location_release -- (internal) release poolset structure */ static void location_release(location *loc) { if (loc->set) { util_poolset_free(loc->set); loc->set = NULL; } } /* * backup_nonpoolset_requirements -- (internal) check backup requirements */ static int backup_nonpoolset_requirements(PMEMpoolcheck *ppc, location *loc) { LOG(3, "backup_path %s", ppc->backup_path); int exists = util_file_exists(ppc->backup_path); if (exists < 0) { return CHECK_ERR(ppc, "unable to access the backup destination: %s", ppc->backup_path); } if (!exists) { errno = 0; return 0; } if ((size_t)util_file_get_size(ppc->backup_path) != ppc->pool->set_file->size) { ppc->result = CHECK_RESULT_ERROR; return CHECK_ERR(ppc, "destination of the backup does not match the size of the source pool file: %s", ppc->backup_path); } if (CHECK_WITHOUT_FIXING(ppc)) { location_release(loc); loc->step = CHECK_STEP_COMPLETE; return 0; } CHECK_ASK(ppc, Q_OVERWRITE_EXISTING_FILE, "destination of the backup already exists.|Do you want to overwrite it?"); return check_questions_sequence_validate(ppc); } /* * backup_nonpoolset_overwrite -- (internal) overwrite pool */ static int backup_nonpoolset_overwrite(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); ASSERTne(loc, NULL); switch (question) { case Q_OVERWRITE_EXISTING_FILE: if (pool_copy(ppc->pool, ppc->backup_path, 1 /* overwrite */)) { location_release(loc); ppc->result = CHECK_RESULT_ERROR; return CHECK_ERR(ppc, "cannot perform backup"); } location_release(loc); loc->step = CHECK_STEP_COMPLETE; return 0; default: ERR("not implemented question id: %u", question); } return 0; } /* * backup_nonpoolset_create -- (internal) create backup */ static int backup_nonpoolset_create(PMEMpoolcheck *ppc, location *loc) { CHECK_INFO(ppc, "creating backup file: %s", ppc->backup_path); if (pool_copy(ppc->pool, ppc->backup_path, 0)) { location_release(loc); ppc->result = CHECK_RESULT_ERROR; return CHECK_ERR(ppc, "cannot perform backup"); } location_release(loc); loc->step = CHECK_STEP_COMPLETE; return 0; } /* * backup_poolset_requirements -- (internal) check backup requirements */ static int backup_poolset_requirements(PMEMpoolcheck *ppc, location *loc) { LOG(3, "backup_path %s", ppc->backup_path); if (ppc->pool->set_file->poolset->nreplicas > 1) { CHECK_INFO(ppc, "backup of a poolset with multiple replicas is not supported"); goto err; } if (pool_set_parse(&loc->set, ppc->backup_path)) { CHECK_INFO_ERRNO(ppc, "invalid poolset backup file: %s", ppc->backup_path); goto err; } if (loc->set->nreplicas > 1) { CHECK_INFO(ppc, "backup to a poolset with multiple replicas is not supported"); goto err_poolset; } ASSERTeq(loc->set->nreplicas, 1); struct pool_replica *srep = ppc->pool->set_file->poolset->replica[0]; struct pool_replica *drep = loc->set->replica[0]; if (srep->nparts != drep->nparts) { CHECK_INFO(ppc, "number of part files in the backup poolset must match number of part files in the source poolset"); goto err_poolset; } int overwrite_required = 0; for (unsigned p = 0; p < srep->nparts; p++) { int exists = util_file_exists(drep->part[p].path); if (exists < 0) { CHECK_INFO(ppc, "unable to access the part of the destination poolset: %s", ppc->backup_path); goto err_poolset; } if (srep->part[p].filesize != drep->part[p].filesize) { CHECK_INFO(ppc, "size of the part %u of the backup poolset does not match source poolset", p); goto err_poolset; } if (!exists) { errno = 0; continue; } overwrite_required = true; if ((size_t)util_file_get_size(drep->part[p].path) != srep->part[p].filesize) { CHECK_INFO(ppc, "destination of the backup part does not match size of the source part file: %s", drep->part[p].path); goto err_poolset; } } if (CHECK_WITHOUT_FIXING(ppc)) { location_release(loc); loc->step = CHECK_STEP_COMPLETE; return 0; } if (overwrite_required) { CHECK_ASK(ppc, Q_OVERWRITE_EXISTING_PARTS, "part files of the destination poolset of the backup already exist.|" "Do you want to overwrite them?"); } return check_questions_sequence_validate(ppc); err_poolset: location_release(loc); err: ppc->result = CHECK_RESULT_ERROR; return CHECK_ERR(ppc, "unable to backup poolset"); } /* * backup_poolset -- (internal) backup the poolset */ static int backup_poolset(PMEMpoolcheck *ppc, location *loc, int overwrite) { struct pool_replica *srep = ppc->pool->set_file->poolset->replica[0]; struct pool_replica *drep = loc->set->replica[0]; for (unsigned p = 0; p < srep->nparts; p++) { if (overwrite == 0) { CHECK_INFO(ppc, "creating backup file: %s", drep->part[p].path); } if (pool_set_part_copy(&drep->part[p], &srep->part[p], overwrite)) { location_release(loc); ppc->result = CHECK_RESULT_ERROR; CHECK_INFO(ppc, "unable to create backup file"); return CHECK_ERR(ppc, "unable to backup poolset"); } } return 0; } /* * backup_poolset_overwrite -- (internal) backup poolset with overwrite */ static int backup_poolset_overwrite(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); ASSERTne(loc, NULL); switch (question) { case Q_OVERWRITE_EXISTING_PARTS: if (backup_poolset(ppc, loc, 1 /* overwrite */)) { location_release(loc); ppc->result = CHECK_RESULT_ERROR; return CHECK_ERR(ppc, "cannot perform backup"); } location_release(loc); loc->step = CHECK_STEP_COMPLETE; return 0; default: ERR("not implemented question id: %u", question); } return 0; } /* * backup_poolset_create -- (internal) backup poolset */ static int backup_poolset_create(PMEMpoolcheck *ppc, location *loc) { if (backup_poolset(ppc, loc, 0)) { location_release(loc); ppc->result = CHECK_RESULT_ERROR; return CHECK_ERR(ppc, "cannot perform backup"); } location_release(loc); loc->step = CHECK_STEP_COMPLETE; return 0; } struct step { int (*check)(PMEMpoolcheck *, location *); int (*fix)(PMEMpoolcheck *, location *, uint32_t, void *); int poolset; }; static const struct step steps[] = { { .check = backup_nonpoolset_requirements, .poolset = false, }, { .fix = backup_nonpoolset_overwrite, .poolset = false, }, { .check = backup_nonpoolset_create, .poolset = false }, { .check = backup_poolset_requirements, .poolset = true, }, { .fix = backup_poolset_overwrite, .poolset = true, }, { .check = backup_poolset_create, .poolset = true }, { .check = NULL, .fix = NULL, }, }; /* * step_exe -- (internal) perform single step according to its parameters */ static int step_exe(PMEMpoolcheck *ppc, location *loc) { ASSERT(loc->step < ARRAY_SIZE(steps)); const struct step *step = &steps[loc->step++]; if (step->poolset == 0 && ppc->pool->params.is_poolset == 1) return 0; if (!step->fix) return step->check(ppc, loc); if (!check_has_answer(ppc->data)) return 0; if (check_answer_loop(ppc, loc, NULL, 1, step->fix)) return -1; ppc->result = CHECK_RESULT_CONSISTENT; return 0; } /* * check_backup -- perform backup if requested and needed */ void check_backup(PMEMpoolcheck *ppc) { LOG(3, "backup_path %s", ppc->backup_path); if (ppc->backup_path == NULL) return; location *loc = check_get_step_data(ppc->data); /* do all checks */ while (CHECK_NOT_COMPLETE(loc, steps)) { if (step_exe(ppc, loc)) break; } } pmdk-1.8/src/libpmempool/replica.c0000664000000000000000000017442113615011243015713 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * replica.c -- groups all commands for replica manipulation */ #include "replica.h" #include #include #include #include #include #include #include #include #include #include "obj.h" #include "palloc.h" #include "file.h" #include "os.h" #include "out.h" #include "pool_hdr.h" #include "set.h" #include "util.h" #include "uuid.h" #include "shutdown_state.h" #include "os_dimm.h" #include "badblock.h" /* * check_flags_sync -- (internal) check if flags are supported for sync */ static int check_flags_sync(unsigned flags) { flags &= ~(PMEMPOOL_SYNC_DRY_RUN | PMEMPOOL_SYNC_FIX_BAD_BLOCKS); return flags > 0; } /* * check_flags_transform -- (internal) check if flags are supported for * transform */ static int check_flags_transform(unsigned flags) { flags &= ~PMEMPOOL_TRANSFORM_DRY_RUN; return flags > 0; } /* * replica_align_badblock_offset_length -- align offset and length * of the bad block for the given part */ void replica_align_badblock_offset_length(size_t *offset, size_t *length, struct pool_set *set_in, unsigned repn, unsigned partn) { LOG(3, "offset %zu, length %zu, pool_set %p, replica %u, part %u", *offset, *length, set_in, repn, partn); size_t alignment = set_in->replica[repn]->part[partn].alignment; size_t off = ALIGN_DOWN(*offset, alignment); size_t len = ALIGN_UP(*length + (*offset - off), alignment); *offset = off; *length = len; } /* * replica_get_part_data_len -- get data length for given part */ size_t replica_get_part_data_len(struct pool_set *set_in, unsigned repn, unsigned partn) { size_t alignment = set_in->replica[repn]->part[partn].alignment; size_t hdrsize = (set_in->options & OPTION_SINGLEHDR) ? 0 : alignment; return ALIGN_DOWN(set_in->replica[repn]->part[partn].filesize, alignment) - ((partn == 0) ? POOL_HDR_SIZE : hdrsize); } /* * replica_get_part_offset -- get part's offset from the beginning of replica */ uint64_t replica_get_part_offset(struct pool_set *set, unsigned repn, unsigned partn) { return (uint64_t)set->replica[repn]->part[partn].addr - (uint64_t)set->replica[repn]->part[0].addr; } /* * replica_get_part_data_offset -- get data length before given part */ uint64_t replica_get_part_data_offset(struct pool_set *set, unsigned repn, unsigned partn) { if (partn == 0) return POOL_HDR_SIZE; return (uint64_t)set->replica[repn]->part[partn].addr - (uint64_t)set->replica[repn]->part[0].addr; } /* * replica_remove_part -- unlink part from replica */ int replica_remove_part(struct pool_set *set, unsigned repn, unsigned partn, int fix_bad_blocks) { LOG(3, "set %p repn %u partn %u fix_bad_blocks %i", set, repn, partn, fix_bad_blocks); struct pool_set_part *part = PART(REP(set, repn), partn); if (part->fd != -1) { os_close(part->fd); part->fd = -1; } int olderrno = errno; enum file_type type = util_file_get_type(part->path); if (type == OTHER_ERROR) return -1; /* if the part is a device dax, clear its bad blocks */ if (type == TYPE_DEVDAX && fix_bad_blocks && os_dimm_devdax_clear_badblocks_all(part->path)) { ERR("clearing bad blocks in device dax failed -- '%s'", part->path); errno = EIO; return -1; } if (type == TYPE_NORMAL && util_unlink(part->path)) { ERR("!removing part %u from replica %u failed", partn, repn); return -1; } errno = olderrno; LOG(4, "Removed part %s number %u from replica %u", part->path, partn, repn); return 0; } /* * create_replica_health_status -- (internal) create helping structure for * storing replica's health status */ static struct replica_health_status * create_replica_health_status(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); unsigned nparts = set->replica[repn]->nparts; struct replica_health_status *replica_hs; replica_hs = Zalloc(sizeof(struct replica_health_status) + nparts * sizeof(struct part_health_status)); if (replica_hs == NULL) { ERR("!Zalloc for replica health status"); return NULL; } replica_hs->nparts = nparts; replica_hs->nhdrs = set->replica[repn]->nhdrs; return replica_hs; } /* * replica_part_remove_recovery_file -- remove bad blocks' recovery file */ static int replica_part_remove_recovery_file(struct part_health_status *phs) { LOG(3, "phs %p", phs); if (phs->recovery_file_name == NULL || phs->recovery_file_exists == 0) return 0; if (os_unlink(phs->recovery_file_name) < 0) { ERR("!removing the bad block recovery file failed -- '%s'", phs->recovery_file_name); return -1; } LOG(3, "bad block recovery file removed -- '%s'", phs->recovery_file_name); phs->recovery_file_exists = 0; return 0; } /* * replica_remove_all_recovery_files -- remove all recovery files */ int replica_remove_all_recovery_files(struct poolset_health_status *set_hs) { LOG(3, "set_hs %p", set_hs); int ret = 0; for (unsigned r = 0; r < set_hs->nreplicas; ++r) { struct replica_health_status *rhs = set_hs->replica[r]; for (unsigned p = 0; p < rhs->nparts; ++p) ret |= replica_part_remove_recovery_file(&rhs->part[p]); } return ret; } /* * replica_free_poolset_health_status -- free memory allocated for helping * structure */ void replica_free_poolset_health_status(struct poolset_health_status *set_hs) { LOG(3, "set_hs %p", set_hs); for (unsigned r = 0; r < set_hs->nreplicas; ++r) { struct replica_health_status *rep_hs = set_hs->replica[r]; for (unsigned p = 0; p < rep_hs->nparts; ++p) { Free(rep_hs->part[p].recovery_file_name); Free(rep_hs->part[p].bbs.bbv); } Free(set_hs->replica[r]); } Free(set_hs); } /* * replica_create_poolset_health_status -- create helping structure for storing * poolset's health status */ int replica_create_poolset_health_status(struct pool_set *set, struct poolset_health_status **set_hsp) { LOG(3, "set %p, set_hsp %p", set, set_hsp); unsigned nreplicas = set->nreplicas; struct poolset_health_status *set_hs; set_hs = Zalloc(sizeof(struct poolset_health_status) + nreplicas * sizeof(struct replica_health_status *)); if (set_hs == NULL) { ERR("!Zalloc for poolset health state"); return -1; } set_hs->nreplicas = nreplicas; for (unsigned i = 0; i < nreplicas; ++i) { struct replica_health_status *replica_hs = create_replica_health_status(set, i); if (replica_hs == NULL) { replica_free_poolset_health_status(set_hs); return -1; } set_hs->replica[i] = replica_hs; } *set_hsp = set_hs; return 0; } /* * replica_is_part_broken -- check if part is marked as broken in the helping * structure */ int replica_is_part_broken(unsigned repn, unsigned partn, struct poolset_health_status *set_hs) { struct replica_health_status *rhs = REP_HEALTH(set_hs, repn); return (rhs->flags & IS_BROKEN) || (PART_HEALTH(rhs, partn) & IS_BROKEN); } /* * is_replica_broken -- check if any part in the replica is marked as broken */ int replica_is_replica_broken(unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "repn %u, set_hs %p", repn, set_hs); struct replica_health_status *r_hs = REP_HEALTH(set_hs, repn); if (r_hs->flags & IS_BROKEN) return 1; for (unsigned p = 0; p < r_hs->nparts; ++p) { if (replica_is_part_broken(repn, p, set_hs)) return 1; } return 0; } /* * replica_is_replica_consistent -- check if replica is not marked as * inconsistent */ int replica_is_replica_consistent(unsigned repn, struct poolset_health_status *set_hs) { return !(REP_HEALTH(set_hs, repn)->flags & IS_INCONSISTENT); } /* * replica_has_bad_blocks -- check if replica has bad blocks */ int replica_has_bad_blocks(unsigned repn, struct poolset_health_status *set_hs) { return REP_HEALTH(set_hs, repn)->flags & HAS_BAD_BLOCKS; } /* * replica_part_has_bad_blocks -- check if replica's part has bad blocks */ int replica_part_has_bad_blocks(struct part_health_status *phs) { return phs->flags & HAS_BAD_BLOCKS; } /* * replica_part_has_corrupted_header -- (internal) check if replica's part * has bad blocks in the header (corrupted header) */ int replica_part_has_corrupted_header(unsigned repn, unsigned partn, struct poolset_health_status *set_hs) { struct replica_health_status *rhs = REP_HEALTH(set_hs, repn); return PART_HEALTH(rhs, partn) & HAS_CORRUPTED_HEADER; } /* * replica_has_corrupted_header -- (internal) check if replica has bad blocks * in the header (corrupted header) */ static int replica_has_corrupted_header(unsigned repn, struct poolset_health_status *set_hs) { return REP_HEALTH(set_hs, repn)->flags & HAS_CORRUPTED_HEADER; } /* * replica_is_replica_healthy -- check if replica is unbroken and consistent */ int replica_is_replica_healthy(unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "repn %u, set_hs %p", repn, set_hs); int ret = !replica_is_replica_broken(repn, set_hs) && replica_is_replica_consistent(repn, set_hs) && !replica_has_bad_blocks(repn, set_hs); LOG(4, "return %i", ret); return ret; } /* * replica_has_healthy_header -- (internal) check if replica has healthy headers */ static int replica_has_healthy_header(unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "repn %u, set_hs %p", repn, set_hs); int ret = !replica_is_replica_broken(repn, set_hs) && replica_is_replica_consistent(repn, set_hs) && !replica_has_corrupted_header(repn, set_hs); LOG(4, "return %i", ret); return ret; } /* * replica_is_poolset_healthy -- check if all replicas in a poolset are not * marked as broken nor inconsistent in the * helping structure */ int replica_is_poolset_healthy(struct poolset_health_status *set_hs) { LOG(3, "set_hs %p", set_hs); for (unsigned r = 0; r < set_hs->nreplicas; ++r) { if (!replica_is_replica_healthy(r, set_hs)) return 0; } return 1; } /* * replica_is_poolset_transformed -- check if the flag indicating a call from * pmempool_transform is on */ int replica_is_poolset_transformed(unsigned flags) { return flags & IS_TRANSFORMED; } /* * replica_find_unbroken_part_with_header -- find a part number in a given * replica, which is not marked as broken in the helping structure and contains * a pool header */ unsigned replica_find_unbroken_part(unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "repn %u, set_hs %p", repn, set_hs); for (unsigned p = 0; p < REP_HEALTH(set_hs, repn)->nhdrs; ++p) { if (!replica_is_part_broken(repn, p, set_hs)) return p; } return UNDEF_PART; } /* * replica_find_healthy_replica -- find a replica which is a good source of data */ unsigned replica_find_healthy_replica(struct poolset_health_status *set_hs) { LOG(3, "set_hs %p", set_hs); for (unsigned r = 0; r < set_hs->nreplicas; ++r) { if (replica_is_replica_healthy(r, set_hs)) { LOG(4, "return %i", r); return r; } } LOG(4, "return %i", UNDEF_REPLICA); return UNDEF_REPLICA; } /* * replica_find_replica_healthy_header -- find a replica with a healthy header */ unsigned replica_find_replica_healthy_header(struct poolset_health_status *set_hs) { LOG(3, "set_hs %p", set_hs); for (unsigned r = 0; r < set_hs->nreplicas; ++r) { if (replica_has_healthy_header(r, set_hs)) { LOG(4, "return %i", r); return r; } } LOG(4, "return %i", UNDEF_REPLICA); return UNDEF_REPLICA; } /* * replica_check_store_size -- (internal) store size from pool descriptor for * replica */ static int replica_check_store_size(struct pool_set *set, struct poolset_health_status *set_hs, unsigned repn) { LOG(3, "set %p, set_hs %p, repn %u", set, set_hs, repn); struct pool_replica *rep = set->replica[repn]; struct pmemobjpool pop; if (rep->remote) { memcpy(&pop.hdr, rep->part[0].hdr, sizeof(pop.hdr)); void *descr = (void *)((uintptr_t)&pop + POOL_HDR_SIZE); if (Rpmem_read(rep->remote->rpp, descr, POOL_HDR_SIZE, sizeof(pop) - POOL_HDR_SIZE, 0)) { return -1; } } else { /* round up map size to Mmap align size */ if (util_map_part(&rep->part[0], NULL, ALIGN_UP(sizeof(pop), rep->part[0].alignment), 0, MAP_SHARED, 1)) { return -1; } memcpy(&pop, rep->part[0].addr, sizeof(pop)); util_unmap_part(&rep->part[0]); } void *dscp = (void *)((uintptr_t)&pop + sizeof(pop.hdr)); if (!util_checksum(dscp, OBJ_DSC_P_SIZE, &pop.checksum, 0, 0)) { set_hs->replica[repn]->flags |= IS_BROKEN; return 0; } set_hs->replica[repn]->pool_size = pop.heap_offset + pop.heap_size; return 0; } /* * check_store_all_sizes -- (internal) store sizes from pool descriptor for all * healthy replicas */ static int check_store_all_sizes(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) { if (!replica_has_healthy_header(r, set_hs)) continue; if (replica_check_store_size(set, set_hs, r)) return -1; } return 0; } /* * check_and_open_poolset_part_files -- (internal) for each part in a poolset * check if the part files are accessible, and if not, mark it as broken * in a helping structure; then open the part file */ static int check_and_open_poolset_part_files(struct pool_set *set, struct poolset_health_status *set_hs, unsigned flags) { LOG(3, "set %p, set_hs %p, flags %u", set, set_hs, flags); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; if (rep->remote) { if (util_replica_open_remote(set, r, 0)) { LOG(1, "cannot open remote replica no %u", r); return -1; } unsigned nlanes = REMOTE_NLANES; int ret = util_poolset_remote_open(rep, r, rep->repsize, 0, rep->part[0].addr, rep->resvsize, &nlanes); if (ret) { rep_hs->flags |= IS_BROKEN; LOG(1, "remote replica #%u marked as BROKEN", r); } continue; } for (unsigned p = 0; p < rep->nparts; ++p) { const char *path = rep->part[p].path; enum file_type type = util_file_get_type(path); if (type < 0 || os_access(path, R_OK|W_OK) != 0) { LOG(1, "part file %s is not accessible", path); errno = 0; rep_hs->part[p].flags |= IS_BROKEN; if (is_dry_run(flags)) continue; } if (util_part_open(&rep->part[p], 0, 0)) { if (type == TYPE_DEVDAX) { LOG(1, "opening part on Device DAX %s failed", path); return -1; } LOG(1, "opening part %s failed", path); errno = 0; rep_hs->part[p].flags |= IS_BROKEN; } } } return 0; } /* * map_all_unbroken_headers -- (internal) map all headers in a poolset, * skipping those marked as broken in a helping * structure */ static int map_all_unbroken_headers(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; if (rep->remote) continue; for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip broken parts */ if (replica_is_part_broken(r, p, set_hs)) continue; LOG(4, "mapping header for part %u, replica %u", p, r); if (util_map_hdr(&rep->part[p], MAP_SHARED, 0) != 0) { LOG(1, "header mapping failed - part #%d", p); rep_hs->part[p].flags |= IS_BROKEN; } } } return 0; } /* * unmap_all_headers -- (internal) unmap all headers in a poolset */ static int unmap_all_headers(struct pool_set *set) { LOG(3, "set %p", set); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; util_replica_close(set, r); if (rep->remote && rep->remote->rpp) { Rpmem_close(rep->remote->rpp); rep->remote->rpp = NULL; } } return 0; } /* * check_checksums_and_signatures -- (internal) check if checksums * and signatures are correct for parts * in a given replica */ static int check_checksums_and_signatures(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); struct replica_health_status *rep_hs = REP_HEALTH(set_hs, r); /* * Checksums and signatures of remote replicas are checked * during opening them on the remote side by the rpmem daemon. * The local version of remote headers does not contain * such data. */ if (rep->remote) continue; for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip broken parts */ if (replica_is_part_broken(r, p, set_hs)) continue; /* check part's checksum */ LOG(4, "checking checksum for part %u, replica %u", p, r); struct pool_hdr *hdr = HDR(rep, p); if (!util_checksum(hdr, sizeof(*hdr), &hdr->checksum, 0, POOL_HDR_CSUM_END_OFF(hdr))) { ERR("invalid checksum of pool header"); rep_hs->part[p].flags |= IS_BROKEN; } else if (util_is_zeroed(hdr, sizeof(*hdr))) { rep_hs->part[p].flags |= IS_BROKEN; } enum pool_type type = pool_hdr_get_type(hdr); if (type == POOL_TYPE_UNKNOWN) { ERR("invalid signature"); rep_hs->part[p].flags |= IS_BROKEN; } } } return 0; } /* * replica_badblocks_recovery_file_save -- save bad blocks in the bad blocks * recovery file before clearing them */ static int replica_badblocks_recovery_file_save(struct part_health_status *part_hs) { LOG(3, "part_health_status %p", part_hs); ASSERTeq(part_hs->recovery_file_exists, 1); ASSERTne(part_hs->recovery_file_name, NULL); struct badblocks *bbs = &part_hs->bbs; char *path = part_hs->recovery_file_name; int ret = -1; int fd = os_open(path, O_WRONLY | O_TRUNC); if (fd < 0) { ERR("!opening bad block recovery file failed -- '%s'", path); return -1; } FILE *recovery_file_name = os_fdopen(fd, "w"); if (recovery_file_name == NULL) { ERR( "!opening a file stream for bad block recovery file failed -- '%s'", path); os_close(fd); return -1; } /* save bad blocks */ for (unsigned i = 0; i < bbs->bb_cnt; i++) { ASSERT(bbs->bbv[i].length != 0); fprintf(recovery_file_name, "%llu %u\n", bbs->bbv[i].offset, bbs->bbv[i].length); } if (fflush(recovery_file_name) == EOF) { ERR("!flushing bad block recovery file failed -- '%s'", path); goto exit_error; } if (os_fsync(fd) < 0) { ERR("!syncing bad block recovery file failed -- '%s'", path); goto exit_error; } /* save the finish flag */ fprintf(recovery_file_name, "0 0\n"); if (fflush(recovery_file_name) == EOF) { ERR("!flushing bad block recovery file failed -- '%s'", path); goto exit_error; } if (os_fsync(fd) < 0) { ERR("!syncing bad block recovery file failed -- '%s'", path); goto exit_error; } LOG(3, "bad blocks saved in the recovery file -- '%s'", path); ret = 0; exit_error: os_fclose(recovery_file_name); return ret; } /* * replica_part_badblocks_recovery_file_read -- read bad blocks * from the bad block recovery file * for the current part */ static int replica_part_badblocks_recovery_file_read(struct part_health_status *part_hs) { LOG(3, "part_health_status %p", part_hs); ASSERT(part_hs->recovery_file_exists); ASSERTne(part_hs->recovery_file_name, NULL); VEC(bbsvec, struct bad_block) bbv = VEC_INITIALIZER; char *path = part_hs->recovery_file_name; struct bad_block bb; int ret = -1; FILE *recovery_file = os_fopen(path, "r"); if (!recovery_file) { ERR("!opening the recovery file for reading failed -- '%s'", path); return -1; } unsigned long long min_offset = 0; /* minimum possible offset */ do { if (fscanf(recovery_file, "%llu %u\n", &bb.offset, &bb.length) < 2) { LOG(1, "incomplete bad block recovery file -- '%s'", path); ret = 1; goto error_exit; } if (bb.offset == 0 && bb.length == 0) { /* finish_flag */ break; } /* check if bad blocks build an increasing sequence */ if (bb.offset < min_offset) { ERR( "wrong format of bad block recovery file (bad blocks are not sorted by the offset in ascending order) -- '%s'", path); errno = EINVAL; ret = -1; goto error_exit; } /* update the minimum possible offset */ min_offset = bb.offset + bb.length; bb.nhealthy = NO_HEALTHY_REPLICA; /* unknown healthy replica */ /* add the new bad block to the vector */ if (VEC_PUSH_BACK(&bbv, bb)) goto error_exit; } while (1); part_hs->bbs.bbv = VEC_ARR(&bbv); part_hs->bbs.bb_cnt = (unsigned)VEC_SIZE(&bbv); os_fclose(recovery_file); LOG(1, "bad blocks read from the recovery file -- '%s'", path); return 0; error_exit: VEC_DELETE(&bbv); os_fclose(recovery_file); return ret; } /* status returned by the replica_badblocks_recovery_files_check() function */ enum badblocks_recovery_files_status { RECOVERY_FILES_ERROR = -1, RECOVERY_FILES_DO_NOT_EXIST = 0, RECOVERY_FILES_EXIST_ALL = 1, RECOVERY_FILES_NOT_ALL_EXIST = 2 }; /* * replica_badblocks_recovery_files_check -- (internal) check if bad blocks * recovery files exist */ static enum badblocks_recovery_files_status replica_badblocks_recovery_files_check(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); int recovery_file_exists = 0; int recovery_file_does_not_exist = 0; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; if (rep->remote) { /* * Bad blocks in remote replicas currently are fixed * during opening by removing and recreating * the whole remote replica. */ continue; } for (unsigned p = 0; p < rep->nparts; ++p) { const char *path = PART(rep, p)->path; struct part_health_status *part_hs = &rep_hs->part[p]; int exists = util_file_exists(path); if (exists < 0) return -1; if (!exists) { /* part file does not exist - skip it */ continue; } part_hs->recovery_file_name = badblocks_recovery_file_alloc(set->path, r, p); if (part_hs->recovery_file_name == NULL) { LOG(1, "allocating name of bad block recovery file failed"); return RECOVERY_FILES_ERROR; } exists = util_file_exists(part_hs->recovery_file_name); if (exists < 0) return -1; part_hs->recovery_file_exists = exists; if (part_hs->recovery_file_exists) { LOG(3, "bad block recovery file exists: %s", part_hs->recovery_file_name); recovery_file_exists = 1; } else { LOG(3, "bad block recovery file does not exist: %s", part_hs->recovery_file_name); recovery_file_does_not_exist = 1; } } } if (recovery_file_exists) { if (recovery_file_does_not_exist) { LOG(4, "return RECOVERY_FILES_NOT_ALL_EXIST"); return RECOVERY_FILES_NOT_ALL_EXIST; } else { LOG(4, "return RECOVERY_FILES_EXIST_ALL"); return RECOVERY_FILES_EXIST_ALL; } } LOG(4, "return RECOVERY_FILES_DO_NOT_EXIST"); return RECOVERY_FILES_DO_NOT_EXIST; } /* * replica_badblocks_recovery_files_read -- (internal) read bad blocks from all * bad block recovery files for all parts */ static int replica_badblocks_recovery_files_read(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); int ret; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; /* XXX: not supported yet */ if (rep->remote) continue; for (unsigned p = 0; p < rep->nparts; ++p) { const char *path = PART(rep, p)->path; struct part_health_status *part_hs = &rep_hs->part[p]; int exists = util_file_exists(path); if (exists < 0) return -1; if (!exists) { /* the part does not exist */ continue; } LOG(1, "reading bad blocks from the recovery file -- '%s'", part_hs->recovery_file_name); ret = replica_part_badblocks_recovery_file_read( part_hs); if (ret < 0) { LOG(1, "reading bad blocks from the recovery file failed -- '%s'", part_hs->recovery_file_name); return -1; } if (ret > 0) { LOG(1, "incomplete bad block recovery file detected -- '%s'", part_hs->recovery_file_name); return 1; } if (part_hs->bbs.bb_cnt) { LOG(3, "part %u contains %u bad blocks -- '%s'", p, part_hs->bbs.bb_cnt, path); } } } return 0; } /* * replica_badblocks_recovery_files_create_empty -- (internal) create one empty * bad block recovery file * for each part file */ static int replica_badblocks_recovery_files_create_empty(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); struct part_health_status *part_hs; const char *path; int fd; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; /* XXX: not supported yet */ if (rep->remote) continue; for (unsigned p = 0; p < rep->nparts; ++p) { part_hs = &rep_hs->part[p]; path = PART(rep, p)->path; if (!part_hs->recovery_file_name) continue; fd = os_open(part_hs->recovery_file_name, O_RDWR | O_CREAT | O_EXCL, 0600); if (fd < 0) { ERR( "!creating an empty bad block recovery file failed -- '%s' (part file '%s')", part_hs->recovery_file_name, path); return -1; } os_close(fd); char *file_name = Strdup(part_hs->recovery_file_name); if (file_name == NULL) { ERR("!Strdup"); return -1; } char *dir_name = dirname(file_name); /* fsync the file's directory */ if (os_fsync_dir(dir_name) < 0) { ERR( "!syncing the directory of the bad block recovery file failed -- '%s' (part file '%s')", dir_name, path); Free(file_name); return -1; } Free(file_name); part_hs->recovery_file_exists = 1; } } return 0; } /* * replica_badblocks_recovery_files_save -- (internal) save bad blocks * in the bad block recovery files */ static int replica_badblocks_recovery_files_save(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; /* XXX: not supported yet */ if (rep->remote) continue; for (unsigned p = 0; p < rep->nparts; ++p) { struct part_health_status *part_hs = &rep_hs->part[p]; if (!part_hs->recovery_file_name) continue; int ret = replica_badblocks_recovery_file_save(part_hs); if (ret < 0) { LOG(1, "opening bad block recovery file failed -- '%s'", part_hs->recovery_file_name); return -1; } } } return 0; } /* * replica_badblocks_get -- (internal) get all bad blocks and save them * in part_hs->bbs structures. * Returns 1 if any bad block was found, 0 otherwise. */ static int replica_badblocks_get(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); int bad_blocks_found = 0; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; /* XXX: not supported yet */ if (rep->remote) continue; for (unsigned p = 0; p < rep->nparts; ++p) { const char *path = PART(rep, p)->path; struct part_health_status *part_hs = &rep_hs->part[p]; int exists = util_file_exists(path); if (exists < 0) return -1; if (!exists) continue; int ret = os_badblocks_get(path, &part_hs->bbs); if (ret < 0) { ERR( "!checking the pool part for bad blocks failed -- '%s'", path); return -1; } if (part_hs->bbs.bb_cnt) { LOG(3, "part %u contains %u bad blocks -- '%s'", p, part_hs->bbs.bb_cnt, path); bad_blocks_found = 1; } } } return bad_blocks_found; } /* * check_badblocks_in_header -- (internal) check if bad blocks corrupted * the header */ static int check_badblocks_in_header(struct badblocks *bbs) { for (unsigned b = 0; b < bbs->bb_cnt; b++) if (bbs->bbv[b].offset < POOL_HDR_SIZE) return 1; return 0; } /* * replica_badblocks_clear -- (internal) clear all bad blocks */ static int replica_badblocks_clear(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); int ret; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; /* XXX: not supported yet */ if (rep->remote) continue; for (unsigned p = 0; p < rep->nparts; ++p) { const char *path = PART(rep, p)->path; struct part_health_status *part_hs = &rep_hs->part[p]; int exists = util_file_exists(path); if (exists < 0) return -1; if (!exists) { /* the part does not exist */ continue; } if (part_hs->bbs.bb_cnt == 0) { /* no bad blocks found */ continue; } /* bad blocks were found */ part_hs->flags |= HAS_BAD_BLOCKS; rep_hs->flags |= HAS_BAD_BLOCKS; if (check_badblocks_in_header(&part_hs->bbs)) { part_hs->flags |= HAS_CORRUPTED_HEADER; if (p == 0) rep_hs->flags |= HAS_CORRUPTED_HEADER; } ret = os_badblocks_clear(path, &part_hs->bbs); if (ret < 0) { LOG(1, "clearing bad blocks in replica failed -- '%s'", path); return -1; } } } return 0; } /* * replica_badblocks_check_or_clear -- (internal) check if replica contains * bad blocks when in dry run * or clear them otherwise */ static int replica_badblocks_check_or_clear(struct pool_set *set, struct poolset_health_status *set_hs, int dry_run, int called_from_sync, int check_bad_blocks, int fix_bad_blocks) { LOG(3, "set %p, set_hs %p, dry_run %i, called_from_sync %i, " "check_bad_blocks %i, fix_bad_blocks %i", set, set_hs, dry_run, called_from_sync, check_bad_blocks, fix_bad_blocks); #define ERR_MSG_BB \ " please read the manual first and use this option\n"\ " ONLY IF you are sure that you know what you are doing" enum badblocks_recovery_files_status status; int ret; /* check all bad block recovery files */ status = replica_badblocks_recovery_files_check(set, set_hs); /* phase #1 - error handling */ switch (status) { case RECOVERY_FILES_ERROR: LOG(1, "checking bad block recovery files failed"); return -1; case RECOVERY_FILES_EXIST_ALL: case RECOVERY_FILES_NOT_ALL_EXIST: if (!called_from_sync) { ERR( "error: a bad block recovery file exists, run 'pmempool sync --bad-blocks' to fix bad blocks first"); return -1; } if (!fix_bad_blocks) { ERR( "error: a bad block recovery file exists, but the '--bad-blocks' option is not set\n" ERR_MSG_BB); return -1; } break; default: break; }; /* * The pool is checked for bad blocks only if: * 1) compat feature POOL_FEAT_CHECK_BAD_BLOCKS is set * OR: * 2) the '--bad-blocks' option is set * * Bad blocks are cleared and fixed only if: * - the '--bad-blocks' option is set */ if (!fix_bad_blocks && !check_bad_blocks) { LOG(3, "skipping bad blocks checking"); return 0; } /* phase #2 - reading recovery files */ switch (status) { case RECOVERY_FILES_EXIST_ALL: /* read all bad block recovery files */ ret = replica_badblocks_recovery_files_read(set, set_hs); if (ret < 0) { LOG(1, "checking bad block recovery files failed"); return -1; } if (ret > 0) { /* incomplete bad block recovery file was detected */ LOG(1, "warning: incomplete bad block recovery file detected\n" " - all recovery files will be removed"); /* changing status to RECOVERY_FILES_NOT_ALL_EXIST */ status = RECOVERY_FILES_NOT_ALL_EXIST; } break; case RECOVERY_FILES_NOT_ALL_EXIST: LOG(1, "warning: one of bad block recovery files does not exist\n" " - all recovery files will be removed"); break; default: break; }; if (status == RECOVERY_FILES_NOT_ALL_EXIST) { /* * At least one of bad block recovery files does not exist, * or an incomplete bad block recovery file was detected, * so all recovery files have to be removed. */ if (!dry_run) { LOG(1, "removing all bad block recovery files..."); ret = replica_remove_all_recovery_files(set_hs); if (ret < 0) { LOG(1, "removing bad block recovery files failed"); return -1; } } else { LOG(1, "all bad block recovery files would be removed"); } /* changing status to RECOVERY_FILES_DO_NOT_EXIST */ status = RECOVERY_FILES_DO_NOT_EXIST; } if (status == RECOVERY_FILES_DO_NOT_EXIST) { /* * There are no bad block recovery files, * so let's check bad blocks. */ int bad_blocks_found = replica_badblocks_get(set, set_hs); if (bad_blocks_found < 0) { if (errno == ENOTSUP) { LOG(1, BB_NOT_SUPP); return -1; } LOG(1, "checking bad blocks failed"); return -1; } if (!bad_blocks_found) { LOG(4, "no bad blocks found"); return 0; } /* bad blocks were found */ if (!called_from_sync) { ERR( "error: bad blocks found, run 'pmempool sync --bad-blocks' to fix bad blocks first"); return -1; } if (!fix_bad_blocks) { ERR( "error: bad blocks found, but the '--bad-blocks' option is not set\n" ERR_MSG_BB); return -1; } if (dry_run) { /* dry-run - do nothing */ LOG(1, "warning: bad blocks were found"); return 0; } /* create one empty recovery file for each part file */ ret = replica_badblocks_recovery_files_create_empty(set, set_hs); if (ret < 0) { LOG(1, "creating empty bad block recovery files failed"); return -1; } /* save bad blocks in recovery files */ ret = replica_badblocks_recovery_files_save(set, set_hs); if (ret < 0) { LOG(1, "saving bad block recovery files failed"); return -1; } } if (dry_run) { /* dry-run - do nothing */ LOG(1, "bad blocks would be cleared"); return 0; } ret = replica_badblocks_clear(set, set_hs); if (ret < 0) { ERR("clearing bad blocks failed"); return -1; } return 0; } /* * check_shutdown_state -- (internal) check if replica has * healthy shutdown_state */ static int check_shutdown_state(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) {\ struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; struct pool_hdr *hdrp = HDR(rep, 0); if (rep->remote) continue; if (hdrp == NULL) { /* cannot verify shutdown state */ rep_hs->flags |= IS_BROKEN; continue; } struct shutdown_state curr_sds; shutdown_state_init(&curr_sds, NULL); for (unsigned p = 0; p < rep->nparts; ++p) { const char *path = PART(rep, p)->path; const int exists = util_file_exists(path); if (exists < 0) return -1; /* * skip missing parts to avoid false positive shutdown * state failure detection */ if (!exists) continue; if (shutdown_state_add_part(&curr_sds, path, NULL)) { rep_hs->flags |= IS_BROKEN; break; } } if (rep_hs->flags & IS_BROKEN) continue; /* make a copy of sds as we shouldn't modify a pool */ struct shutdown_state pool_sds = hdrp->sds; if (shutdown_state_check(&curr_sds, &pool_sds, NULL)) rep_hs->flags |= IS_BROKEN; } return 0; } /* * check_uuids_between_parts -- (internal) check if uuids between adjacent * parts are consistent for a given replica */ static int check_uuids_between_parts(struct pool_set *set, unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "set %p, repn %u, set_hs %p", set, repn, set_hs); struct pool_replica *rep = REP(set, repn); /* check poolset_uuid consistency between replica's parts */ LOG(4, "checking consistency of poolset uuid in replica %u", repn); uuid_t poolset_uuid; int uuid_stored = 0; unsigned part_stored = UNDEF_PART; for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip broken parts */ if (replica_is_part_broken(repn, p, set_hs)) continue; if (!uuid_stored) { memcpy(poolset_uuid, HDR(rep, p)->poolset_uuid, POOL_HDR_UUID_LEN); uuid_stored = 1; part_stored = p; continue; } if (uuidcmp(HDR(rep, p)->poolset_uuid, poolset_uuid)) { ERR( "different poolset uuids in parts from the same replica (repn %u, parts %u and %u) - cannot synchronize", repn, part_stored, p); errno = EINVAL; return -1; } } /* check if all uuids for adjacent replicas are the same across parts */ LOG(4, "checking consistency of adjacent replicas' uuids in replica %u", repn); unsigned unbroken_p = UNDEF_PART; for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip broken parts */ if (replica_is_part_broken(repn, p, set_hs)) continue; if (unbroken_p == UNDEF_PART) { unbroken_p = p; continue; } struct pool_hdr *hdrp = HDR(rep, p); int prev_differ = uuidcmp(HDR(rep, unbroken_p)->prev_repl_uuid, hdrp->prev_repl_uuid); int next_differ = uuidcmp(HDR(rep, unbroken_p)->next_repl_uuid, hdrp->next_repl_uuid); if (prev_differ || next_differ) { ERR( "different adjacent replica UUID between parts (repn %u, parts %u and %u) - cannot synchronize", repn, unbroken_p, p); errno = EINVAL; return -1; } } /* check parts linkage */ LOG(4, "checking parts linkage in replica %u", repn); for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip broken parts */ if (replica_is_part_broken(repn, p, set_hs)) continue; struct pool_hdr *hdrp = HDR(rep, p); struct pool_hdr *next_hdrp = HDRN(rep, p); int next_is_broken = replica_is_part_broken(repn, p + 1, set_hs); if (!next_is_broken) { int next_decoupled = uuidcmp(next_hdrp->prev_part_uuid, hdrp->uuid) || uuidcmp(hdrp->next_part_uuid, next_hdrp->uuid); if (next_decoupled) { ERR( "two consecutive unbroken parts are not linked to each other (repn %u, parts %u and %u) - cannot synchronize", repn, p, p + 1); errno = EINVAL; return -1; } } } return 0; } /* * check_replicas_consistency -- (internal) check if all uuids within each * replica are consistent */ static int check_replicas_consistency(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) { if (check_uuids_between_parts(set, r, set_hs)) return -1; } return 0; } /* * check_replica_options -- (internal) check if options are consistent in the * replica */ static int check_replica_options(struct pool_set *set, unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "set %p, repn %u, set_hs %p", set, repn, set_hs); struct pool_replica *rep = REP(set, repn); struct replica_health_status *rep_hs = REP_HEALTH(set_hs, repn); for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip broken parts */ if (replica_is_part_broken(repn, p, set_hs)) continue; struct pool_hdr *hdr = HDR(rep, p); if (((hdr->features.incompat & POOL_FEAT_SINGLEHDR) == 0) != ((set->options & OPTION_SINGLEHDR) == 0)) { LOG(1, "improper options are set in part %u's header in replica %u", p, repn); rep_hs->part[p].flags |= IS_BROKEN; } } return 0; } /* * check_options -- (internal) check if options are consistent in all replicas */ static int check_options(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) { if (check_replica_options(set, r, set_hs)) return -1; } return 0; } /* * check_replica_poolset_uuids - (internal) check if poolset_uuid fields are * consistent among all parts of a replica; * the replica is initially considered as * consistent */ static int check_replica_poolset_uuids(struct pool_set *set, unsigned repn, uuid_t poolset_uuid, struct poolset_health_status *set_hs) { LOG(3, "set %p, repn %u, poolset_uuid %p, set_hs %p", set, repn, poolset_uuid, set_hs); struct pool_replica *rep = REP(set, repn); for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip broken parts */ if (replica_is_part_broken(repn, p, set_hs)) continue; if (uuidcmp(HDR(rep, p)->poolset_uuid, poolset_uuid)) { /* * two internally consistent replicas have * different poolset_uuid */ return -1; } else { /* * it is sufficient to check only one part * from internally consistent replica */ break; } } return 0; } /* * check_poolset_uuids -- (internal) check if poolset_uuid fields are consistent * among all internally consistent replicas */ static int check_poolset_uuids(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); /* find a replica with healthy header */ unsigned r_h = replica_find_replica_healthy_header(set_hs); if (r_h == UNDEF_REPLICA) { ERR("no healthy replica found"); return -1; } uuid_t poolset_uuid; memcpy(poolset_uuid, HDR(REP(set, r_h), 0)->poolset_uuid, POOL_HDR_UUID_LEN); for (unsigned r = 0; r < set->nreplicas; ++r) { /* skip inconsistent replicas */ if (!replica_is_replica_consistent(r, set_hs) || r == r_h) continue; if (check_replica_poolset_uuids(set, r, poolset_uuid, set_hs)) { ERR( "inconsistent poolset uuids between replicas %u and %u - cannot synchronize", r_h, r); return -1; } } return 0; } /* * get_replica_uuid -- (internal) get replica uuid */ static int get_replica_uuid(struct pool_replica *rep, unsigned repn, struct poolset_health_status *set_hs, uuid_t **uuidpp) { unsigned nhdrs = rep->nhdrs; if (!replica_is_part_broken(repn, 0, set_hs)) { /* the first part is not broken */ *uuidpp = &HDR(rep, 0)->uuid; return 0; } else if (nhdrs > 1 && !replica_is_part_broken(repn, 1, set_hs)) { /* the second part is not broken */ *uuidpp = &HDR(rep, 1)->prev_part_uuid; return 0; } else if (nhdrs > 1 && !replica_is_part_broken(repn, nhdrs - 1, set_hs)) { /* the last part is not broken */ *uuidpp = &HDR(rep, nhdrs - 1)->next_part_uuid; return 0; } else { /* cannot get replica uuid */ return -1; } } /* * check_uuids_between_replicas -- (internal) check if uuids between internally * consistent adjacent replicas are consistent */ static int check_uuids_between_replicas(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) { /* skip comparing inconsistent pairs of replicas */ if (!replica_is_replica_consistent(r, set_hs) || !replica_is_replica_consistent(r + 1, set_hs)) continue; struct pool_replica *rep = REP(set, r); struct pool_replica *rep_n = REPN(set, r); /* get uuids of the two adjacent replicas */ uuid_t *rep_uuidp = NULL; uuid_t *rep_n_uuidp = NULL; unsigned r_n = REPN_HEALTHidx(set_hs, r); if (get_replica_uuid(rep, r, set_hs, &rep_uuidp)) LOG(2, "cannot get replica uuid, replica %u", r); if (get_replica_uuid(rep_n, r_n, set_hs, &rep_n_uuidp)) LOG(2, "cannot get replica uuid, replica %u", r_n); /* * check if replica uuids are consistent between two adjacent * replicas */ unsigned p = replica_find_unbroken_part(r, set_hs); unsigned p_n = replica_find_unbroken_part(r_n, set_hs); if (p_n != UNDEF_PART && rep_uuidp != NULL && uuidcmp(*rep_uuidp, HDR(rep_n, p_n)->prev_repl_uuid)) { ERR( "inconsistent replica uuids between replicas %u and %u", r, r_n); return -1; } if (p != UNDEF_PART && rep_n_uuidp != NULL && uuidcmp(*rep_n_uuidp, HDR(rep, p)->next_repl_uuid)) { ERR( "inconsistent replica uuids between replicas %u and %u", r, r_n); return -1; } /* * check if replica uuids on borders of a broken replica are * consistent */ unsigned r_nn = REPN_HEALTHidx(set_hs, r_n); if (set->nreplicas > 1 && p != UNDEF_PART && replica_is_replica_broken(r_n, set_hs) && replica_is_replica_consistent(r_nn, set_hs)) { unsigned p_nn = replica_find_unbroken_part(r_nn, set_hs); if (p_nn == UNDEF_PART) { LOG(2, "cannot compare uuids on borders of replica %u", r); continue; } struct pool_replica *rep_nn = REP(set, r_nn); if (uuidcmp(HDR(rep, p)->next_repl_uuid, HDR(rep_nn, p_nn)->prev_repl_uuid)) { ERR( "inconsistent replica uuids on borders of replica %u", r); return -1; } } } return 0; } /* * check_replica_cycles -- (internal) check if healthy replicas form cycles * shorter than the number of all replicas */ static int check_replica_cycles(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); unsigned first_healthy; unsigned count_healthy = 0; for (unsigned r = 0; r < set->nreplicas; ++r) { if (!replica_is_replica_healthy(r, set_hs)) { count_healthy = 0; continue; } if (count_healthy == 0) first_healthy = r; ++count_healthy; struct pool_hdr *hdrh = PART(REP(set, first_healthy), 0)->hdr; struct pool_hdr *hdr = PART(REP(set, r), 0)->hdr; if (uuidcmp(hdrh->uuid, hdr->next_repl_uuid) == 0 && count_healthy < set->nreplicas) { /* * Healthy replicas form a cycle shorter than * the number of all replicas; for the user it * means that: */ ERR( "alien replica found (probably coming from a different poolset)"); return -1; } } return 0; } /* * check_replica_sizes -- (internal) check if all replicas are large * enough to hold data from a healthy replica */ static int check_replica_sizes(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); ssize_t pool_size = -1; for (unsigned r = 0; r < set->nreplicas; ++r) { /* skip broken replicas */ if (!replica_is_replica_healthy(r, set_hs)) continue; /* get the size of a pool in the replica */ ssize_t replica_pool_size; if (REP(set, r)->remote) /* XXX: no way to get the size of a remote pool yet */ replica_pool_size = (ssize_t)set->poolsize; else replica_pool_size = replica_get_pool_size(set, r); if (replica_pool_size < 0) { LOG(1, "getting pool size from replica %u failed", r); set_hs->replica[r]->flags |= IS_BROKEN; continue; } /* check if the pool is bigger than minimum size */ enum pool_type type = pool_hdr_get_type(HDR(REP(set, r), 0)); if ((size_t)replica_pool_size < pool_get_min_size(type)) { LOG(1, "pool size from replica %u is smaller than the minimum size allowed for the pool", r); set_hs->replica[r]->flags |= IS_BROKEN; continue; } /* check if each replica is big enough to hold the pool data */ if (set->poolsize < (size_t)replica_pool_size) { ERR( "some replicas are too small to hold synchronized data"); return -1; } if (pool_size < 0) { pool_size = replica_pool_size; continue; } /* check if pools in all healthy replicas are of equal size */ if (pool_size != replica_pool_size) { ERR("pool sizes from different replicas differ"); return -1; } } return 0; } /* * replica_read_features -- (internal) read features from the header */ static int replica_read_features(struct pool_set *set, struct poolset_health_status *set_hs, features_t *features) { LOG(3, "set %p set_hs %p features %p", set, set_hs, features); ASSERTne(features, NULL); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; struct replica_health_status *rep_hs = set_hs->replica[r]; if (rep->remote) { if (rep_hs->flags & IS_BROKEN) continue; struct pool_hdr *hdrp = rep->part[0].hdr; memcpy(features, &hdrp->features, sizeof(*features)); return 0; } for (unsigned p = 0; p < rep->nparts; p++) { struct pool_set_part *part = &rep->part[p]; if (part->fd == -1) continue; if (util_map_hdr(part, MAP_SHARED, 0) != 0) { LOG(1, "header mapping failed"); return -1; } struct pool_hdr *hdrp = part->hdr; memcpy(features, &hdrp->features, sizeof(*features)); util_unmap_hdr(part); return 0; } } /* no healthy replica/part found */ return -1; } /* * replica_check_poolset_health -- check if a given poolset can be considered as * healthy, and store the status in a helping structure */ int replica_check_poolset_health(struct pool_set *set, struct poolset_health_status **set_hsp, int called_from_sync, unsigned flags) { LOG(3, "set %p, set_hsp %p, called_from_sync %i, flags %u", set, set_hsp, called_from_sync, flags); if (replica_create_poolset_health_status(set, set_hsp)) { LOG(1, "creating poolset health status failed"); return -1; } struct poolset_health_status *set_hs = *set_hsp; /* check if part files exist and are accessible */ if (check_and_open_poolset_part_files(set, set_hs, flags)) { LOG(1, "poolset part files check failed"); goto err; } features_t features; int check_bad_blks; int fix_bad_blks = called_from_sync && fix_bad_blocks(flags); if (fix_bad_blks) { /* * We will fix bad blocks, so we cannot read features here, * because reading could fail, because of bad blocks. * We will read features after having bad blocks fixed. * * Fixing bad blocks implies checking bad blocks. */ check_bad_blks = 1; } else { /* * We will not fix bad blocks, so we have to read features here. */ if (replica_read_features(set, set_hs, &features)) { LOG(1, "reading features failed"); goto err; } check_bad_blks = features.compat & POOL_FEAT_CHECK_BAD_BLOCKS; } /* check for bad blocks when in dry run or clear them otherwise */ if (replica_badblocks_check_or_clear(set, set_hs, is_dry_run(flags), called_from_sync, check_bad_blks, fix_bad_blks)) { LOG(1, "replica bad_blocks check failed"); goto err; } /* read features after fixing bad blocks */ if (fix_bad_blks && replica_read_features(set, set_hs, &features)) { LOG(1, "reading features failed"); goto err; } /* set ignore_sds flag basing on features read from the header */ set->ignore_sds = !(features.incompat & POOL_FEAT_SDS); /* map all headers */ map_all_unbroken_headers(set, set_hs); /* * Check if checksums and signatures are correct for all parts * in all replicas. */ check_checksums_and_signatures(set, set_hs); /* check if option flags are consistent */ if (check_options(set, set_hs)) { LOG(1, "flags check failed"); goto err; } if (!set->ignore_sds && check_shutdown_state(set, set_hs)) { LOG(1, "replica shutdown_state check failed"); goto err; } /* check if uuids in parts across each replica are consistent */ if (check_replicas_consistency(set, set_hs)) { LOG(1, "replica consistency check failed"); goto err; } /* check poolset_uuid values between replicas */ if (check_poolset_uuids(set, set_hs)) { LOG(1, "poolset uuids check failed"); goto err; } /* check if uuids for adjacent replicas are consistent */ if (check_uuids_between_replicas(set, set_hs)) { LOG(1, "replica uuids check failed"); goto err; } /* check if healthy replicas make up another poolset */ if (check_replica_cycles(set, set_hs)) { LOG(1, "replica cycles check failed"); goto err; } /* check if replicas are large enough */ if (check_replica_sizes(set, set_hs)) { LOG(1, "replica sizes check failed"); goto err; } if (check_store_all_sizes(set, set_hs)) { LOG(1, "reading pool sizes failed"); goto err; } unmap_all_headers(set); util_poolset_fdclose_always(set); return 0; err: errno = EINVAL; unmap_all_headers(set); util_poolset_fdclose_always(set); replica_free_poolset_health_status(set_hs); return -1; } /* * replica_get_pool_size -- find the effective size (mapped) of a pool based * on metadata from given replica */ ssize_t replica_get_pool_size(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); struct pool_set_part *part = PART(REP(set, repn), 0); int should_close_part = 0; int should_unmap_part = 0; if (part->fd == -1) { if (util_part_open(part, 0, 0)) return -1; should_close_part = 1; } if (part->addr == NULL) { if (util_map_part(part, NULL, ALIGN_UP(sizeof(PMEMobjpool), part->alignment), 0, MAP_SHARED, 1)) { util_part_fdclose(part); return -1; } should_unmap_part = 1; } PMEMobjpool *pop = (PMEMobjpool *)part->addr; ssize_t ret = (ssize_t)(pop->heap_offset + pop->heap_size); if (should_unmap_part) util_unmap_part(part); if (should_close_part) util_part_fdclose(part); return ret; } /* * replica_check_part_sizes -- check if all parts are large enough */ int replica_check_part_sizes(struct pool_set *set, size_t min_size) { LOG(3, "set %p, min_size %zu", set, min_size); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; if (rep->remote != NULL) /* skip remote replicas */ continue; for (unsigned p = 0; p < rep->nparts; ++p) { if (PART(rep, p)->filesize < min_size) { ERR("replica %u, part %u: file is too small", r, p); errno = EINVAL; return -1; } } } return 0; } /* * replica_check_local_part_dir -- check if directory for the part file * exists */ int replica_check_local_part_dir(struct pool_set *set, unsigned repn, unsigned partn) { LOG(3, "set %p, repn %u, partn %u", set, repn, partn); char *path = Strdup(PART(REP(set, repn), partn)->path); const char *dir = dirname(path); os_stat_t sb; if (os_stat(dir, &sb) != 0 || !(sb.st_mode & S_IFDIR)) { ERR( "directory %s for part %u in replica %u does not exist or is not accessible", path, partn, repn); Free(path); return -1; } Free(path); return 0; } /* * replica_check_part_dirs -- (internal) check if directories for part files * exist */ int replica_check_part_dirs(struct pool_set *set) { LOG(3, "set %p", set); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; if (rep->remote != NULL) /* skip remote replicas */ continue; for (unsigned p = 0; p < rep->nparts; ++p) { if (replica_check_local_part_dir(set, r, p)) return -1; } } return 0; } /* * replica_open_replica_part_files -- open all part files for a replica */ int replica_open_replica_part_files(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); struct pool_replica *rep = set->replica[repn]; for (unsigned p = 0; p < rep->nparts; ++p) { /* skip already opened files */ if (rep->part[p].fd != -1) continue; if (util_part_open(&rep->part[p], 0, 0)) { LOG(1, "part files open failed for replica %u, part %u", repn, p); errno = EINVAL; goto err; } } return 0; err: util_replica_fdclose(set->replica[repn]); return -1; } /* * replica_open_poolset_part_files -- open all part files for a poolset */ int replica_open_poolset_part_files(struct pool_set *set) { LOG(3, "set %p", set); for (unsigned r = 0; r < set->nreplicas; ++r) { if (set->replica[r]->remote) continue; if (replica_open_replica_part_files(set, r)) { LOG(1, "opening replica %u, part files failed", r); goto err; } } return 0; err: util_poolset_fdclose_always(set); return -1; } /* * pmempool_syncU -- synchronize replicas within a poolset */ #ifndef _WIN32 static inline #endif int pmempool_syncU(const char *poolset, unsigned flags) { LOG(3, "poolset %s, flags %u", poolset, flags); ASSERTne(poolset, NULL); /* check if poolset has correct signature */ if (util_is_poolset_file(poolset) != 1) { ERR("file is not a poolset file"); goto err; } /* check if flags are supported */ if (check_flags_sync(flags)) { ERR("unsupported flags"); errno = EINVAL; goto err; } /* open poolset file */ int fd = util_file_open(poolset, NULL, 0, O_RDONLY); if (fd < 0) { ERR("cannot open a poolset file"); goto err; } /* fill up pool_set structure */ struct pool_set *set = NULL; if (util_poolset_parse(&set, poolset, fd)) { ERR("parsing input poolset failed"); goto err_close_file; } if (set->nreplicas == 1) { ERR("no replica(s) found in the pool set"); errno = EINVAL; goto err_close_file; } if (set->remote && util_remote_load()) { ERR("remote replication not available"); errno = ENOTSUP; goto err_close_file; } /* sync all replicas */ if (replica_sync(set, NULL, flags)) { LOG(1, "synchronization failed"); goto err_close_all; } util_poolset_close(set, DO_NOT_DELETE_PARTS); os_close(fd); return 0; err_close_all: util_poolset_close(set, DO_NOT_DELETE_PARTS); err_close_file: os_close(fd); err: if (errno == 0) errno = EINVAL; return -1; } #ifndef _WIN32 /* * pmempool_sync -- synchronize replicas within a poolset */ int pmempool_sync(const char *poolset, unsigned flags) { return pmempool_syncU(poolset, flags); } #else /* * pmempool_syncW -- synchronize replicas within a poolset in widechar */ int pmempool_syncW(const wchar_t *poolset, unsigned flags) { char *path = util_toUTF8(poolset); if (path == NULL) { ERR("Invalid poolest file path."); return -1; } int ret = pmempool_syncU(path, flags); util_free_UTF8(path); return ret; } #endif /* * pmempool_transformU -- alter poolset structure */ #ifndef _WIN32 static inline #endif int pmempool_transformU(const char *poolset_src, const char *poolset_dst, unsigned flags) { LOG(3, "poolset_src %s, poolset_dst %s, flags %u", poolset_src, poolset_dst, flags); ASSERTne(poolset_src, NULL); ASSERTne(poolset_dst, NULL); /* check if the source poolset has correct signature */ if (util_is_poolset_file(poolset_src) != 1) { ERR("source file is not a poolset file"); goto err; } /* check if the destination poolset has correct signature */ if (util_is_poolset_file(poolset_dst) != 1) { ERR("destination file is not a poolset file"); goto err; } /* check if flags are supported */ if (check_flags_transform(flags)) { ERR("unsupported flags"); errno = EINVAL; goto err; } /* open the source poolset file */ int fd_in = util_file_open(poolset_src, NULL, 0, O_RDONLY); if (fd_in < 0) { ERR("cannot open source poolset file"); goto err; } /* parse the source poolset file */ struct pool_set *set_in = NULL; if (util_poolset_parse(&set_in, poolset_src, fd_in)) { ERR("parsing source poolset failed"); os_close(fd_in); goto err; } os_close(fd_in); /* open the destination poolset file */ int fd_out = util_file_open(poolset_dst, NULL, 0, O_RDONLY); if (fd_out < 0) { ERR("cannot open destination poolset file"); goto err; } enum del_parts_mode del = DO_NOT_DELETE_PARTS; /* parse the destination poolset file */ struct pool_set *set_out = NULL; if (util_poolset_parse(&set_out, poolset_dst, fd_out)) { ERR("parsing destination poolset failed"); os_close(fd_out); goto err_free_poolin; } os_close(fd_out); /* check if the source poolset is of a correct type */ enum pool_type ptype = pool_set_type(set_in); if (ptype != POOL_TYPE_OBJ) { errno = EINVAL; ERR("transform is not supported for given pool type: %s", pool_get_pool_type_str(ptype)); goto err_free_poolout; } /* load remote library if needed */ if (set_in->remote && util_remote_load()) { ERR("remote replication not available"); goto err_free_poolout; } if (set_out->remote && util_remote_load()) { ERR("remote replication not available"); goto err_free_poolout; } del = is_dry_run(flags) ? DO_NOT_DELETE_PARTS : DELETE_CREATED_PARTS; /* transform poolset */ if (replica_transform(set_in, set_out, flags)) { LOG(1, "transformation failed"); goto err_free_poolout; } util_poolset_close(set_in, DO_NOT_DELETE_PARTS); util_poolset_close(set_out, DO_NOT_DELETE_PARTS); return 0; err_free_poolout: util_poolset_close(set_out, del); err_free_poolin: util_poolset_close(set_in, DO_NOT_DELETE_PARTS); err: if (errno == 0) errno = EINVAL; return -1; } #ifndef _WIN32 /* * pmempool_transform -- alter poolset structure */ int pmempool_transform(const char *poolset_src, const char *poolset_dst, unsigned flags) { return pmempool_transformU(poolset_src, poolset_dst, flags); } #else /* * pmempool_transformW -- alter poolset structure in widechar */ int pmempool_transformW(const wchar_t *poolset_src, const wchar_t *poolset_dst, unsigned flags) { char *path_src = util_toUTF8(poolset_src); if (path_src == NULL) { ERR("Invalid source poolest file path."); return -1; } char *path_dst = util_toUTF8(poolset_dst); if (path_dst == NULL) { ERR("Invalid destination poolest file path."); Free(path_src); return -1; } int ret = pmempool_transformU(path_src, path_dst, flags); util_free_UTF8(path_src); util_free_UTF8(path_dst); return ret; } #endif pmdk-1.8/src/libpmempool/feature.c0000664000000000000000000004465313615011243015732 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * feature.c -- implementation of pmempool_feature_(enable|disable|query)() */ #include #include #include #include #include #include "libpmempool.h" #include "util_pmem.h" #include "pool_hdr.h" #include "pool.h" #define RW 0 #define RDONLY 1 #define FEATURE_INCOMPAT(X) \ (features_t)FEAT_INCOMPAT(X) static const features_t f_singlehdr = FEAT_INCOMPAT(SINGLEHDR); static const features_t f_cksum_2k = FEAT_INCOMPAT(CKSUM_2K); static const features_t f_sds = FEAT_INCOMPAT(SDS); static const features_t f_chkbb = FEAT_COMPAT(CHECK_BAD_BLOCKS); #define FEAT_INVALID \ {UINT32_MAX, UINT32_MAX, UINT32_MAX}; static const features_t f_invalid = FEAT_INVALID; #define FEATURE_MAXPRINT ((size_t)1024) /* * buff_concat -- (internal) concat formatted string to string buffer */ static int buff_concat(char *buff, size_t *pos, const char *fmt, ...) { va_list ap; va_start(ap, fmt); const size_t size = FEATURE_MAXPRINT - *pos - 1; int ret = vsnprintf(buff + *pos, size, fmt, ap); va_end(ap); if (ret < 0) { ERR("vsprintf"); return ret; } if ((size_t)ret >= size) { ERR("buffer truncated %d >= %zu", ret, size); return -1; } *pos += (size_t)ret; return 0; } /* * buff_concat_features -- (internal) concat features string to string buffer */ static int buff_concat_features(char *buff, size_t *pos, features_t f) { return buff_concat(buff, pos, "{compat 0x%x, incompat 0x%x, ro_compat 0x%x}", f.compat, f.incompat, f.ro_compat); } /* * poolset_close -- (internal) close pool set */ static void poolset_close(struct pool_set *set) { for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); ASSERT(!rep->remote); for (unsigned p = 0; p < rep->nparts; ++p) { util_unmap_hdr(PART(rep, p)); } } util_poolset_close(set, DO_NOT_DELETE_PARTS); } /* * features_check -- (internal) check if features are correct */ static int features_check(features_t *features, struct pool_hdr *hdrp) { static char msg[FEATURE_MAXPRINT]; struct pool_hdr hdr; memcpy(&hdr, hdrp, sizeof(hdr)); util_convert2h_hdr_nocheck(&hdr); /* (features != f_invlaid) <=> features is set */ if (!util_feature_cmp(*features, f_invalid)) { /* features from current and previous headers have to match */ if (!util_feature_cmp(*features, hdr.features)) { size_t pos = 0; if (buff_concat_features(msg, &pos, hdr.features)) goto err; if (buff_concat(msg, &pos, "%s", " != ")) goto err; if (buff_concat_features(msg, &pos, *features)) goto err; ERR("features mismatch detected: %s", msg); return -1; } else { return 0; } } features_t unknown = util_get_unknown_features( hdr.features, (features_t)POOL_FEAT_VALID); /* all features are known */ if (util_feature_is_zero(unknown)) { memcpy(features, &hdr.features, sizeof(*features)); return 0; } /* unknown features detected - print error message */ size_t pos = 0; if (buff_concat_features(msg, &pos, unknown)) goto err; ERR("invalid features detected: %s", msg); err: return -1; } /* * get_pool_open_flags -- (internal) generate pool open flags */ static inline unsigned get_pool_open_flags(struct pool_set *set, int rdonly) { unsigned flags = 0; if (rdonly == RDONLY && !util_pool_has_device_dax(set)) flags = POOL_OPEN_COW; flags |= POOL_OPEN_IGNORE_BAD_BLOCKS; return flags; } /* * get_mmap_flags -- (internal) generate mmap flags */ static inline int get_mmap_flags(struct pool_set_part *part, int rdonly) { if (part->is_dev_dax) return MAP_SHARED; else return rdonly ? MAP_PRIVATE : MAP_SHARED; } /* * poolset_open -- (internal) open pool set */ static struct pool_set * poolset_open(const char *path, int rdonly) { struct pool_set *set; features_t features = FEAT_INVALID; /* read poolset */ int ret = util_poolset_create_set(&set, path, 0, 0, true); if (ret < 0) { ERR("cannot open pool set -- '%s'", path); goto err_poolset; } if (set->remote) { ERR("poolsets with remote replicas are not supported"); errno = EINVAL; goto err_open; } /* open a memory pool */ unsigned flags = get_pool_open_flags(set, rdonly); if (util_pool_open_nocheck(set, flags)) goto err_open; /* map all headers and check features */ for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); ASSERT(!rep->remote); for (unsigned p = 0; p < rep->nparts; ++p) { struct pool_set_part *part = PART(rep, p); int mmap_flags = get_mmap_flags(part, rdonly); if (util_map_hdr(part, mmap_flags, rdonly)) { part->hdr = NULL; goto err_map_hdr; } if (features_check(&features, HDR(rep, p))) { ERR( "invalid features - replica #%d part #%d", r, p); goto err_open; } } } return set; err_map_hdr: /* unmap all headers */ for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); ASSERT(!rep->remote); for (unsigned p = 0; p < rep->nparts; ++p) { util_unmap_hdr(PART(rep, p)); } } err_open: /* close the memory pool and release pool set structure */ util_poolset_close(set, DO_NOT_DELETE_PARTS); err_poolset: return NULL; } /* * get_hdr -- (internal) read header in host byte order */ static struct pool_hdr * get_hdr(struct pool_set *set, unsigned rep, unsigned part) { static struct pool_hdr hdr; /* copy header */ struct pool_hdr *hdrp = HDR(REP(set, rep), part); memcpy(&hdr, hdrp, sizeof(hdr)); /* convert to host byte order and return */ util_convert2h_hdr_nocheck(&hdr); return &hdr; } /* * set_hdr -- (internal) convert header to little-endian, checksum and write */ static void set_hdr(struct pool_set *set, unsigned rep, unsigned part, struct pool_hdr *src) { /* convert to little-endian and set new checksum */ const size_t skip_off = POOL_HDR_CSUM_END_OFF(src); util_convert2le_hdr(src); util_checksum(src, sizeof(*src), &src->checksum, 1, skip_off); /* write header */ struct pool_replica *replica = REP(set, rep); struct pool_hdr *dst = HDR(replica, part); memcpy(dst, src, sizeof(*src)); util_persist_auto(PART(replica, part)->is_dev_dax, dst, sizeof(*src)); } typedef enum { DISABLED, ENABLED } fstate_t; #define FEATURE_IS_ENABLED_STR "feature already enabled: %s" #define FEATURE_IS_DISABLED_STR "feature already disabled: %s" /* * require_feature_is -- (internal) check if required feature is enabled * (or disabled) */ static int require_feature_is(struct pool_set *set, features_t feature, fstate_t req_state) { struct pool_hdr *hdrp = get_hdr((set), 0, 0); fstate_t state = util_feature_is_set(hdrp->features, feature) ? ENABLED : DISABLED; if (state == req_state) return 1; const char *msg = (state == ENABLED) ? FEATURE_IS_ENABLED_STR : FEATURE_IS_DISABLED_STR; LOG(3, msg, util_feature2str(feature, NULL)); return 0; } #define FEATURE_IS_NOT_ENABLED_PRIOR_STR "enable %s prior to %s %s" #define FEATURE_IS_NOT_DISABLED_PRIOR_STR "disable %s prior to %s %s" /* * require_other_feature_is -- (internal) check if other feature is enabled * (or disabled) in case the other feature has to be enabled (or disabled) * prior to the main one */ static int require_other_feature_is(struct pool_set *set, features_t other, fstate_t req_state, features_t feature, const char *cause) { struct pool_hdr *hdrp = get_hdr((set), 0, 0); fstate_t state = util_feature_is_set(hdrp->features, other) ? ENABLED : DISABLED; if (state == req_state) return 1; const char *msg = (req_state == ENABLED) ? FEATURE_IS_NOT_ENABLED_PRIOR_STR : FEATURE_IS_NOT_DISABLED_PRIOR_STR; ERR(msg, util_feature2str(other, NULL), cause, util_feature2str(feature, NULL)); return 0; } /* * feature_set -- (internal) enable (or disable) feature */ static void feature_set(struct pool_set *set, features_t feature, int value) { for (unsigned r = 0; r < set->nreplicas; ++r) { for (unsigned p = 0; p < REP(set, r)->nparts; ++p) { struct pool_hdr *hdrp = get_hdr(set, r, p); if (value == ENABLED) util_feature_enable(&hdrp->features, feature); else util_feature_disable(&hdrp->features, feature); set_hdr(set, r, p, hdrp); } } } /* * query_feature -- (internal) query feature value */ static int query_feature(const char *path, features_t feature) { struct pool_set *set = poolset_open(path, RDONLY); if (!set) goto err_open; struct pool_hdr *hdrp = get_hdr(set, 0, 0); const int query = util_feature_is_set(hdrp->features, feature); poolset_close(set); return query; err_open: return -1; } /* * unsupported_feature -- (internal) report unsupported feature */ static inline int unsupported_feature(features_t feature) { ERR("unsupported feature: %s", util_feature2str(feature, NULL)); errno = EINVAL; return -1; } /* * enable_singlehdr -- (internal) enable POOL_FEAT_SINGLEHDR */ static int enable_singlehdr(const char *path) { return unsupported_feature(f_singlehdr); } /* * disable_singlehdr -- (internal) disable POOL_FEAT_SINGLEHDR */ static int disable_singlehdr(const char *path) { return unsupported_feature(f_singlehdr); } /* * query_singlehdr -- (internal) query POOL_FEAT_SINGLEHDR */ static int query_singlehdr(const char *path) { return query_feature(path, f_singlehdr); } /* * enable_checksum_2k -- (internal) enable POOL_FEAT_CKSUM_2K */ static int enable_checksum_2k(const char *path) { struct pool_set *set = poolset_open(path, RW); if (!set) return -1; if (require_feature_is(set, f_cksum_2k, DISABLED)) feature_set(set, f_cksum_2k, ENABLED); poolset_close(set); return 0; } /* * disable_checksum_2k -- (internal) disable POOL_FEAT_CKSUM_2K */ static int disable_checksum_2k(const char *path) { struct pool_set *set = poolset_open(path, RW); if (!set) return -1; int ret = 0; if (!require_feature_is(set, f_cksum_2k, ENABLED)) goto exit; /* check if POOL_FEAT_SDS is disabled */ if (!require_other_feature_is(set, f_sds, DISABLED, f_cksum_2k, "disabling")) { ret = -1; goto exit; } feature_set(set, f_cksum_2k, DISABLED); exit: poolset_close(set); return ret; } /* * query_checksum_2k -- (internal) query POOL_FEAT_CKSUM_2K */ static int query_checksum_2k(const char *path) { return query_feature(path, f_cksum_2k); } /* * enable_shutdown_state -- (internal) enable POOL_FEAT_SDS */ static int enable_shutdown_state(const char *path) { struct pool_set *set = poolset_open(path, RW); if (!set) return -1; int ret = 0; if (!require_feature_is(set, f_sds, DISABLED)) goto exit; /* check if POOL_FEAT_CKSUM_2K is enabled */ if (!require_other_feature_is(set, f_cksum_2k, ENABLED, f_sds, "enabling")) { ret = -1; goto exit; } feature_set(set, f_sds, ENABLED); exit: poolset_close(set); return ret; } /* * reset_shutdown_state -- zero all shutdown structures */ static void reset_shutdown_state(struct pool_set *set) { for (unsigned rep = 0; rep < set->nreplicas; ++rep) { for (unsigned part = 0; part < REP(set, rep)->nparts; ++part) { struct pool_hdr *hdrp = HDR(REP(set, rep), part); shutdown_state_init(&hdrp->sds, REP(set, rep)); } } } /* * disable_shutdown_state -- (internal) disable POOL_FEAT_SDS */ static int disable_shutdown_state(const char *path) { struct pool_set *set = poolset_open(path, RW); if (!set) return -1; if (require_feature_is(set, f_sds, ENABLED)) { feature_set(set, f_sds, DISABLED); reset_shutdown_state(set); } poolset_close(set); return 0; } /* * query_shutdown_state -- (internal) query POOL_FEAT_SDS */ static int query_shutdown_state(const char *path) { return query_feature(path, f_sds); } /* * enable_badblocks_checking -- (internal) enable POOL_FEAT_CHECK_BAD_BLOCKS */ static int enable_badblocks_checking(const char *path) { #ifdef _WIN32 ERR("bad blocks checking is not supported on Windows"); return -1; #else struct pool_set *set = poolset_open(path, RW); if (!set) return -1; if (require_feature_is(set, f_chkbb, DISABLED)) feature_set(set, f_chkbb, ENABLED); poolset_close(set); return 0; #endif } /* * disable_badblocks_checking -- (internal) disable POOL_FEAT_CHECK_BAD_BLOCKS */ static int disable_badblocks_checking(const char *path) { struct pool_set *set = poolset_open(path, RW); if (!set) return -1; int ret = 0; if (!require_feature_is(set, f_chkbb, ENABLED)) goto exit; feature_set(set, f_chkbb, DISABLED); exit: poolset_close(set); return ret; } /* * query_badblocks_checking -- (internal) query POOL_FEAT_CHECK_BAD_BLOCKS */ static int query_badblocks_checking(const char *path) { return query_feature(path, f_chkbb); } struct feature_funcs { int (*enable)(const char *); int (*disable)(const char *); int (*query)(const char *); }; static struct feature_funcs features[] = { { .enable = enable_singlehdr, .disable = disable_singlehdr, .query = query_singlehdr }, { .enable = enable_checksum_2k, .disable = disable_checksum_2k, .query = query_checksum_2k }, { .enable = enable_shutdown_state, .disable = disable_shutdown_state, .query = query_shutdown_state }, { .enable = enable_badblocks_checking, .disable = disable_badblocks_checking, .query = query_badblocks_checking }, }; #define FEATURE_FUNCS_MAX ARRAY_SIZE(features) /* * are_flags_valid -- (internal) check if flags are valid */ static inline int are_flags_valid(unsigned flags) { if (flags != 0) { ERR("invalid flags: 0x%x", flags); errno = EINVAL; return 0; } return 1; } /* * is_feature_valid -- (internal) check if feature is valid */ static inline int is_feature_valid(uint32_t feature) { if (feature >= FEATURE_FUNCS_MAX) { ERR("invalid feature: 0x%x", feature); errno = EINVAL; return 0; } return 1; } /* * pmempool_feature_enableU -- enable pool set feature */ #ifndef _WIN32 static inline #endif int pmempool_feature_enableU(const char *path, enum pmempool_feature feature, unsigned flags) { LOG(3, "path %s feature %x flags %x", path, feature, flags); if (!is_feature_valid(feature)) return -1; if (!are_flags_valid(flags)) return -1; return features[feature].enable(path); } /* * pmempool_feature_disableU -- disable pool set feature */ #ifndef _WIN32 static inline #endif int pmempool_feature_disableU(const char *path, enum pmempool_feature feature, unsigned flags) { LOG(3, "path %s feature %x flags %x", path, feature, flags); if (!is_feature_valid(feature)) return -1; if (!are_flags_valid(flags)) return -1; return features[feature].disable(path); } /* * pmempool_feature_queryU -- query pool set feature */ #ifndef _WIN32 static inline #endif int pmempool_feature_queryU(const char *path, enum pmempool_feature feature, unsigned flags) { LOG(3, "path %s feature %x flags %x", path, feature, flags); /* * XXX: Windows does not allow function call in a constant expressions */ #ifndef _WIN32 #define CHECK_INCOMPAT_MAPPING(FEAT, ENUM) \ COMPILE_ERROR_ON( \ util_feature2pmempool_feature(FEATURE_INCOMPAT(FEAT)) != ENUM) CHECK_INCOMPAT_MAPPING(SINGLEHDR, PMEMPOOL_FEAT_SINGLEHDR); CHECK_INCOMPAT_MAPPING(CKSUM_2K, PMEMPOOL_FEAT_CKSUM_2K); CHECK_INCOMPAT_MAPPING(SDS, PMEMPOOL_FEAT_SHUTDOWN_STATE); #undef CHECK_INCOMPAT_MAPPING #endif if (!is_feature_valid(feature)) return -1; if (!are_flags_valid(flags)) return -1; return features[feature].query(path); } #ifndef _WIN32 /* * pmempool_feature_enable -- enable pool set feature */ int pmempool_feature_enable(const char *path, enum pmempool_feature feature, unsigned flags) { return pmempool_feature_enableU(path, feature, flags); } #else /* * pmempool_feature_enableW -- enable pool set feature as widechar */ int pmempool_feature_enableW(const wchar_t *path, enum pmempool_feature feature, unsigned flags) { char *upath = util_toUTF8(path); if (upath == NULL) { ERR("Invalid poolest/pool file path."); return -1; } int ret = pmempool_feature_enableU(upath, feature, flags); util_free_UTF8(upath); return ret; } #endif #ifndef _WIN32 /* * pmempool_feature_disable -- disable pool set feature */ int pmempool_feature_disable(const char *path, enum pmempool_feature feature, unsigned flags) { return pmempool_feature_disableU(path, feature, flags); } #else /* * pmempool_feature_disableW -- disable pool set feature as widechar */ int pmempool_feature_disableW(const wchar_t *path, enum pmempool_feature feature, unsigned flags) { char *upath = util_toUTF8(path); if (upath == NULL) { ERR("Invalid poolest/pool file path."); return -1; } int ret = pmempool_feature_disableU(upath, feature, flags); util_free_UTF8(upath); return ret; } #endif #ifndef _WIN32 /* * pmempool_feature_query -- query pool set feature */ int pmempool_feature_query(const char *path, enum pmempool_feature feature, unsigned flags) { return pmempool_feature_queryU(path, feature, flags); } #else /* * pmempool_feature_queryW -- query pool set feature as widechar */ int pmempool_feature_queryW(const wchar_t *path, enum pmempool_feature feature, unsigned flags) { char *upath = util_toUTF8(path); if (upath == NULL) { ERR("Invalid poolest/pool file path."); return -1; } int ret = pmempool_feature_queryU(upath, feature, flags); util_free_UTF8(upath); return ret; } #endif pmdk-1.8/src/libpmempool/libpmempool.vcxproj0000664000000000000000000001631313615011243020057 0ustar rootroot Debug x64 Release x64 {f7c6c6b6-4142-4c82-8699-4a9d8183181b} {0b1818eb-bdc8-4865-964f-db8bf05cfd86} {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} {901f04db-e1a5-4a41-8b81-9d31c19acd59} {CF9A0883-6334-44C7-AC29-349468C78E27} DynamicLibrary libpmempool libpmempool en-US 14.0 10.0.16299.0 10.0.10240.0 DynamicLibrary true v140 DynamicLibrary false false v140 $(SolutionDir)\libpmemobj;$(SolutionDir)\libpmemblk;$(SolutionDir)\libpmemlog;%(AdditionalIncludeDirectories) $(SolutionDir)\libpmemobj;$(SolutionDir)\libpmemblk;$(SolutionDir)\libpmemlog;%(AdditionalIncludeDirectories) pmdk-1.8/src/libpmempool/check_bad_blocks.c0000664000000000000000000000543413615011243017511 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_bad_blocks.c -- pre-check bad_blocks */ #include #include #include #include "out.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check_util.h" #include "os_badblock.h" #include "badblock.h" /* * check_bad_blocks -- check poolset for bad_blocks */ void check_bad_blocks(PMEMpoolcheck *ppc) { LOG(3, "ppc %p", ppc); int ret; if (!(ppc->pool->params.features.compat & POOL_FEAT_CHECK_BAD_BLOCKS)) { /* skipping checking poolset for bad blocks */ ppc->result = CHECK_RESULT_CONSISTENT; return; } if (ppc->pool->set_file->poolset) { ret = badblocks_check_poolset(ppc->pool->set_file->poolset, 0); } else { ret = os_badblocks_check_file(ppc->pool->set_file->fname); } if (ret < 0) { if (errno == ENOTSUP) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_ERR(ppc, BB_NOT_SUPP); return; } ppc->result = CHECK_RESULT_ERROR; CHECK_ERR(ppc, "checking poolset for bad blocks failed -- '%s'", ppc->path); return; } if (ret > 0) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_ERR(ppc, "poolset contains bad blocks, use 'pmempool info --bad-blocks=yes' to print or 'pmempool sync --bad-blocks' to clear them"); } } pmdk-1.8/src/libpmempool/check_btt_map_flog.c0000664000000000000000000004154113615011243020062 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_btt_map_flog.c -- check BTT Map and Flog */ #include #include #include #include "out.h" #include "btt.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check_util.h" enum questions { Q_REPAIR_MAP, Q_REPAIR_FLOG, }; /* * flog_read -- (internal) read and convert flog from file */ static int flog_read(PMEMpoolcheck *ppc, struct arena *arenap) { uint64_t flogoff = arenap->offset + arenap->btt_info.flogoff; arenap->flogsize = btt_flog_size(arenap->btt_info.nfree); arenap->flog = malloc(arenap->flogsize); if (!arenap->flog) { ERR("!malloc"); goto error_malloc; } if (pool_read(ppc->pool, arenap->flog, arenap->flogsize, flogoff)) goto error_read; uint8_t *ptr = arenap->flog; uint32_t i; for (i = 0; i < arenap->btt_info.nfree; i++) { struct btt_flog *flog = (struct btt_flog *)ptr; btt_flog_convert2h(&flog[0]); btt_flog_convert2h(&flog[1]); ptr += BTT_FLOG_PAIR_ALIGN; } return 0; error_read: free(arenap->flog); arenap->flog = NULL; error_malloc: return -1; } /* * map_read -- (internal) read and convert map from file */ static int map_read(PMEMpoolcheck *ppc, struct arena *arenap) { uint64_t mapoff = arenap->offset + arenap->btt_info.mapoff; arenap->mapsize = btt_map_size(arenap->btt_info.external_nlba); ASSERT(arenap->mapsize != 0); arenap->map = malloc(arenap->mapsize); if (!arenap->map) { ERR("!malloc"); goto error_malloc; } if (pool_read(ppc->pool, arenap->map, arenap->mapsize, mapoff)) { goto error_read; } uint32_t i; for (i = 0; i < arenap->btt_info.external_nlba; i++) arenap->map[i] = le32toh(arenap->map[i]); return 0; error_read: free(arenap->map); arenap->map = NULL; error_malloc: return -1; } /* * list_item -- item for simple list */ struct list_item { PMDK_LIST_ENTRY(list_item) next; uint32_t val; }; /* * list -- simple list for storing numbers */ struct list { PMDK_LIST_HEAD(listhead, list_item) head; uint32_t count; }; /* * list_alloc -- (internal) allocate an empty list */ static struct list * list_alloc(void) { struct list *list = malloc(sizeof(struct list)); if (!list) { ERR("!malloc"); return NULL; } PMDK_LIST_INIT(&list->head); list->count = 0; return list; } /* * list_push -- (internal) insert new element to the list */ static struct list_item * list_push(struct list *list, uint32_t val) { struct list_item *item = malloc(sizeof(*item)); if (!item) { ERR("!malloc"); return NULL; } item->val = val; list->count++; PMDK_LIST_INSERT_HEAD(&list->head, item, next); return item; } /* * list_pop -- (internal) pop element from list head */ static int list_pop(struct list *list, uint32_t *valp) { if (!PMDK_LIST_EMPTY(&list->head)) { struct list_item *i = PMDK_LIST_FIRST(&list->head); PMDK_LIST_REMOVE(i, next); if (valp) *valp = i->val; free(i); list->count--; return 1; } return 0; } /* * list_free -- (internal) free the list */ static void list_free(struct list *list) { while (list_pop(list, NULL)) ; free(list); } /* * cleanup -- (internal) prepare resources for map and flog check */ static int cleanup(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); if (loc->list_unmap) list_free(loc->list_unmap); if (loc->list_flog_inval) list_free(loc->list_flog_inval); if (loc->list_inval) list_free(loc->list_inval); if (loc->fbitmap) free(loc->fbitmap); if (loc->bitmap) free(loc->bitmap); if (loc->dup_bitmap) free(loc->dup_bitmap); return 0; } /* * init -- (internal) initialize map and flog check */ static int init(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); struct arena *arenap = loc->arenap; /* read flog and map entries */ if (flog_read(ppc, arenap)) { CHECK_ERR(ppc, "arena %u: cannot read BTT Flog", arenap->id); goto error; } if (map_read(ppc, arenap)) { CHECK_ERR(ppc, "arena %u: cannot read BTT Map", arenap->id); goto error; } /* create bitmaps for checking duplicated blocks */ uint32_t bitmapsize = howmany(arenap->btt_info.internal_nlba, 8); loc->bitmap = calloc(bitmapsize, 1); if (!loc->bitmap) { ERR("!calloc"); CHECK_ERR(ppc, "arena %u: cannot allocate memory for blocks " "bitmap", arenap->id); goto error; } loc->dup_bitmap = calloc(bitmapsize, 1); if (!loc->dup_bitmap) { ERR("!calloc"); CHECK_ERR(ppc, "arena %u: cannot allocate memory for " "duplicated blocks bitmap", arenap->id); goto error; } loc->fbitmap = calloc(bitmapsize, 1); if (!loc->fbitmap) { ERR("!calloc"); CHECK_ERR(ppc, "arena %u: cannot allocate memory for BTT Flog " "bitmap", arenap->id); goto error; } /* list of invalid map entries */ loc->list_inval = list_alloc(); if (!loc->list_inval) { CHECK_ERR(ppc, "arena %u: cannot allocate memory for invalid BTT map " "entries list", arenap->id); goto error; } /* list of invalid flog entries */ loc->list_flog_inval = list_alloc(); if (!loc->list_flog_inval) { CHECK_ERR(ppc, "arena %u: cannot allocate memory for invalid BTT Flog " "entries list", arenap->id); goto error; } /* list of unmapped blocks */ loc->list_unmap = list_alloc(); if (!loc->list_unmap) { CHECK_ERR(ppc, "arena %u: cannot allocate memory for unmaped blocks " "list", arenap->id); goto error; } return 0; error: ppc->result = CHECK_RESULT_ERROR; cleanup(ppc, loc); return -1; } /* * map_get_postmap_lba -- extract postmap LBA from map entry */ static inline uint32_t map_get_postmap_lba(struct arena *arenap, uint32_t i) { uint32_t entry = arenap->map[i]; /* if map record is in initial state (flags == 0b00) */ if (map_entry_is_initial(entry)) return i; /* read postmap LBA otherwise */ return entry & BTT_MAP_ENTRY_LBA_MASK; } /* * map_entry_check -- (internal) check single map entry */ static int map_entry_check(PMEMpoolcheck *ppc, location *loc, uint32_t i) { struct arena *arenap = loc->arenap; uint32_t lba = map_get_postmap_lba(arenap, i); /* add duplicated and invalid entries to list */ if (lba < arenap->btt_info.internal_nlba) { if (util_isset(loc->bitmap, lba)) { CHECK_INFO(ppc, "arena %u: BTT Map entry %u duplicated " "at %u", arenap->id, lba, i); util_setbit(loc->dup_bitmap, lba); if (!list_push(loc->list_inval, i)) return -1; } else util_setbit(loc->bitmap, lba); } else { CHECK_INFO(ppc, "arena %u: invalid BTT Map entry at %u", arenap->id, i); if (!list_push(loc->list_inval, i)) return -1; } return 0; } /* * flog_entry_check -- (internal) check single flog entry */ static int flog_entry_check(PMEMpoolcheck *ppc, location *loc, uint32_t i, uint8_t **ptr) { struct arena *arenap = loc->arenap; /* flog entry consists of two btt_flog structures */ struct btt_flog *flog = (struct btt_flog *)*ptr; int next; struct btt_flog *flog_cur = btt_flog_get_valid(flog, &next); /* insert invalid and duplicated indexes to list */ if (!flog_cur) { CHECK_INFO(ppc, "arena %u: invalid BTT Flog entry at %u", arenap->id, i); if (!list_push(loc->list_flog_inval, i)) return -1; goto next; } uint32_t entry = flog_cur->old_map & BTT_MAP_ENTRY_LBA_MASK; uint32_t new_entry = flog_cur->new_map & BTT_MAP_ENTRY_LBA_MASK; /* * Check if lba is in extranal_nlba range, and check if both old_map and * new_map are in internal_nlba range. */ if (flog_cur->lba >= arenap->btt_info.external_nlba || entry >= arenap->btt_info.internal_nlba || new_entry >= arenap->btt_info.internal_nlba) { CHECK_INFO(ppc, "arena %u: invalid BTT Flog entry at %u", arenap->id, i); if (!list_push(loc->list_flog_inval, i)) return -1; goto next; } if (util_isset(loc->fbitmap, entry)) { /* * here we have two flog entries which holds the same free block */ CHECK_INFO(ppc, "arena %u: duplicated BTT Flog entry at %u\n", arenap->id, i); if (!list_push(loc->list_flog_inval, i)) return -1; } else if (util_isset(loc->bitmap, entry)) { /* here we have probably an unfinished write */ if (util_isset(loc->bitmap, new_entry)) { /* Both old_map and new_map are already used in map. */ CHECK_INFO(ppc, "arena %u: duplicated BTT Flog entry " "at %u", arenap->id, i); util_setbit(loc->dup_bitmap, new_entry); if (!list_push(loc->list_flog_inval, i)) return -1; } else { /* * Unfinished write. Next time pool is opened, the map * will be updated to new_map. */ util_setbit(loc->bitmap, new_entry); util_setbit(loc->fbitmap, entry); } } else { int flog_valid = 1; /* * Either flog entry is in its initial state: * - current_btt_flog entry is first one in pair and * - current_btt_flog.old_map == current_btt_flog.new_map and * - current_btt_flog.seq == 0b01 and * - second flog entry in pair is zeroed * or * current_btt_flog.old_map != current_btt_flog.new_map */ if (entry == new_entry) flog_valid = (next == 1) && (flog_cur->seq == 1) && util_is_zeroed((const void *)&flog[1], sizeof(flog[1])); if (flog_valid) { /* totally fine case */ util_setbit(loc->bitmap, entry); util_setbit(loc->fbitmap, entry); } else { CHECK_INFO(ppc, "arena %u: invalid BTT Flog entry at " "%u", arenap->id, i); if (!list_push(loc->list_flog_inval, i)) return -1; } } next: *ptr += BTT_FLOG_PAIR_ALIGN; return 0; } /* * arena_map_flog_check -- (internal) check map and flog */ static int arena_map_flog_check(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); struct arena *arenap = loc->arenap; /* check map entries */ uint32_t i; for (i = 0; i < arenap->btt_info.external_nlba; i++) { if (map_entry_check(ppc, loc, i)) goto error_push; } /* check flog entries */ uint8_t *ptr = arenap->flog; for (i = 0; i < arenap->btt_info.nfree; i++) { if (flog_entry_check(ppc, loc, i, &ptr)) goto error_push; } /* check unmapped blocks and insert to list */ for (i = 0; i < arenap->btt_info.internal_nlba; i++) { if (!util_isset(loc->bitmap, i)) { CHECK_INFO(ppc, "arena %u: unmapped block %u", arenap->id, i); if (!list_push(loc->list_unmap, i)) goto error_push; } } if (loc->list_unmap->count) CHECK_INFO(ppc, "arena %u: number of unmapped blocks: %u", arenap->id, loc->list_unmap->count); if (loc->list_inval->count) CHECK_INFO(ppc, "arena %u: number of invalid BTT Map entries: " "%u", arenap->id, loc->list_inval->count); if (loc->list_flog_inval->count) CHECK_INFO(ppc, "arena %u: number of invalid BTT Flog entries: " "%u", arenap->id, loc->list_flog_inval->count); if (CHECK_IS_NOT(ppc, REPAIR) && loc->list_unmap->count > 0) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; check_end(ppc->data); goto cleanup; } /* * We are able to repair if and only if number of unmapped blocks is * equal to sum of invalid map and flog entries. */ if (loc->list_unmap->count != (loc->list_inval->count + loc->list_flog_inval->count)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_ERR(ppc, "arena %u: cannot repair BTT Map and Flog", arenap->id); goto cleanup; } if (CHECK_IS_NOT(ppc, ADVANCED) && loc->list_inval->count + loc->list_flog_inval->count > 0) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_INFO(ppc, REQUIRE_ADVANCED); CHECK_ERR(ppc, "BTT Map and / or BTT Flog contain invalid " "entries"); check_end(ppc->data); goto cleanup; } if (loc->list_inval->count > 0) { CHECK_ASK(ppc, Q_REPAIR_MAP, "Do you want to repair invalid " "BTT Map entries?"); } if (loc->list_flog_inval->count > 0) { CHECK_ASK(ppc, Q_REPAIR_FLOG, "Do you want to repair invalid " "BTT Flog entries?"); } return check_questions_sequence_validate(ppc); error_push: CHECK_ERR(ppc, "arena %u: cannot allocate momory for list item", arenap->id); ppc->result = CHECK_RESULT_ERROR; cleanup: cleanup(ppc, loc); return -1; } /* * arena_map_flog_fix -- (internal) fix map and flog */ static int arena_map_flog_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *ctx) { LOG(3, NULL); ASSERTeq(ctx, NULL); ASSERTne(loc, NULL); struct arena *arenap = loc->arenap; uint32_t inval; uint32_t unmap; switch (question) { case Q_REPAIR_MAP: /* * Cause first of duplicated map entries seems valid till we * find second of them we must find all first map entries * pointing to the postmap LBA's we know are duplicated to mark * them with error flag. */ for (uint32_t i = 0; i < arenap->btt_info.external_nlba; i++) { uint32_t lba = map_get_postmap_lba(arenap, i); if (lba >= arenap->btt_info.internal_nlba) continue; if (!util_isset(loc->dup_bitmap, lba)) continue; arenap->map[i] = BTT_MAP_ENTRY_ERROR | lba; util_clrbit(loc->dup_bitmap, lba); CHECK_INFO(ppc, "arena %u: storing 0x%x at %u BTT Map entry", arenap->id, arenap->map[i], i); } /* * repair invalid or duplicated map entries by using unmapped * blocks */ while (list_pop(loc->list_inval, &inval)) { if (!list_pop(loc->list_unmap, &unmap)) { ppc->result = CHECK_RESULT_ERROR; return -1; } arenap->map[inval] = unmap | BTT_MAP_ENTRY_ERROR; CHECK_INFO(ppc, "arena %u: storing 0x%x at %u BTT Map " "entry", arenap->id, arenap->map[inval], inval); } break; case Q_REPAIR_FLOG: /* repair invalid flog entries using unmapped blocks */ while (list_pop(loc->list_flog_inval, &inval)) { if (!list_pop(loc->list_unmap, &unmap)) { ppc->result = CHECK_RESULT_ERROR; return -1; } struct btt_flog *flog = (struct btt_flog *) (arenap->flog + inval * BTT_FLOG_PAIR_ALIGN); memset(&flog[1], 0, sizeof(flog[1])); uint32_t entry = unmap | BTT_MAP_ENTRY_ERROR; flog[0].lba = inval; flog[0].new_map = entry; flog[0].old_map = entry; flog[0].seq = 1; CHECK_INFO(ppc, "arena %u: repairing BTT Flog at %u " "with free block entry 0x%x", loc->arenap->id, inval, entry); } break; default: ERR("not implemented question id: %u", question); } return 0; } struct step { int (*check)(PMEMpoolcheck *, location *); int (*fix)(PMEMpoolcheck *, location *, uint32_t, void *); }; static const struct step steps[] = { { .check = init, }, { .check = arena_map_flog_check, }, { .fix = arena_map_flog_fix, }, { .check = cleanup, }, { .check = NULL, .fix = NULL, }, }; /* * step_exe -- (internal) perform single step according to its parameters */ static inline int step_exe(PMEMpoolcheck *ppc, location *loc) { ASSERT(loc->step < ARRAY_SIZE(steps)); const struct step *step = &steps[loc->step++]; if (!step->fix) return step->check(ppc, loc); if (!check_answer_loop(ppc, loc, NULL, 1, step->fix)) return 0; cleanup(ppc, loc); return -1; } /* * check_btt_map_flog -- perform check and fixing of map and flog */ void check_btt_map_flog(PMEMpoolcheck *ppc) { LOG(3, NULL); location *loc = check_get_step_data(ppc->data); if (ppc->pool->blk_no_layout) return; /* initialize check */ if (!loc->arenap && loc->narena == 0 && ppc->result != CHECK_RESULT_PROCESS_ANSWERS) { CHECK_INFO(ppc, "checking BTT Map and Flog"); loc->arenap = PMDK_TAILQ_FIRST(&ppc->pool->arenas); loc->narena = 0; } while (loc->arenap != NULL) { /* add info about checking next arena */ if (ppc->result != CHECK_RESULT_PROCESS_ANSWERS && loc->step == 0) { CHECK_INFO(ppc, "arena %u: checking BTT Map and Flog", loc->narena); } /* do all checks */ while (CHECK_NOT_COMPLETE(loc, steps)) { if (step_exe(ppc, loc)) return; } /* jump to next arena */ loc->arenap = PMDK_TAILQ_NEXT(loc->arenap, next); loc->narena++; loc->step = 0; } } pmdk-1.8/src/libpmempool/replica.h0000664000000000000000000001706513615011243015720 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * replica.h -- module for synchronizing and transforming poolset */ #ifndef REPLICA_H #define REPLICA_H #include "libpmempool.h" #include "pool.h" #include "os_badblock.h" #ifdef __cplusplus extern "C" { #endif #define UNDEF_REPLICA UINT_MAX #define UNDEF_PART UINT_MAX /* * A part marked as broken does not exist or is damaged so that * it cannot be opened and has to be recreated. */ #define IS_BROKEN (1U << 0) /* * A replica marked as inconsistent exists but has inconsistent metadata * (e.g. inconsistent parts or replicas linkage) */ #define IS_INCONSISTENT (1U << 1) /* * A part or replica marked in this way has bad blocks inside. */ #define HAS_BAD_BLOCKS (1U << 2) /* * A part marked in this way has bad blocks in the header */ #define HAS_CORRUPTED_HEADER (1U << 3) /* * A flag which can be passed to sync_replica() to indicate that the function is * called by pmempool_transform */ #define IS_TRANSFORMED (1U << 10) /* * Number of lanes utilized when working with remote replicas */ #define REMOTE_NLANES 1 /* * Helping structures for storing part's health status */ struct part_health_status { unsigned flags; struct badblocks bbs; /* structure with bad blocks */ char *recovery_file_name; /* name of bad block recovery file */ int recovery_file_exists; /* bad block recovery file exists */ }; /* * Helping structures for storing replica and poolset's health status */ struct replica_health_status { unsigned nparts; unsigned nhdrs; /* a flag for the replica */ unsigned flags; /* effective size of a pool, valid only for healthy replica */ size_t pool_size; /* flags for each part */ struct part_health_status part[]; }; struct poolset_health_status { unsigned nreplicas; /* a flag for the poolset */ unsigned flags; /* health statuses for each replica */ struct replica_health_status *replica[]; }; /* get index of the (r)th replica health status */ static inline unsigned REP_HEALTHidx(struct poolset_health_status *set, unsigned r) { ASSERTne(set->nreplicas, 0); return (set->nreplicas + r) % set->nreplicas; } /* get index of the (r + 1)th replica health status */ static inline unsigned REPN_HEALTHidx(struct poolset_health_status *set, unsigned r) { ASSERTne(set->nreplicas, 0); return (set->nreplicas + r + 1) % set->nreplicas; } /* get (p)th part health status */ static inline unsigned PART_HEALTHidx(struct replica_health_status *rep, unsigned p) { ASSERTne(rep->nparts, 0); return (rep->nparts + p) % rep->nparts; } /* get (r)th replica health status */ static inline struct replica_health_status * REP_HEALTH(struct poolset_health_status *set, unsigned r) { return set->replica[REP_HEALTHidx(set, r)]; } /* get (p)th part health status */ static inline unsigned PART_HEALTH(struct replica_health_status *rep, unsigned p) { return rep->part[PART_HEALTHidx(rep, p)].flags; } uint64_t replica_get_part_offset(struct pool_set *set, unsigned repn, unsigned partn); void replica_align_badblock_offset_length(size_t *offset, size_t *length, struct pool_set *set_in, unsigned repn, unsigned partn); size_t replica_get_part_data_len(struct pool_set *set_in, unsigned repn, unsigned partn); uint64_t replica_get_part_data_offset(struct pool_set *set_in, unsigned repn, unsigned part); /* * is_dry_run -- (internal) check whether only verification mode is enabled */ static inline bool is_dry_run(unsigned flags) { /* * PMEMPOOL_SYNC_DRY_RUN and PMEMPOOL_TRANSFORM_DRY_RUN * have to have the same value in order to use this common function. */ ASSERT_COMPILE_ERROR_ON(PMEMPOOL_SYNC_DRY_RUN != PMEMPOOL_TRANSFORM_DRY_RUN); return flags & PMEMPOOL_SYNC_DRY_RUN; } /* * fix_bad_blocks -- (internal) fix bad blocks - it causes reading or creating * bad blocks recovery files * (depending on if they exist or not) */ static inline bool fix_bad_blocks(unsigned flags) { return flags & PMEMPOOL_SYNC_FIX_BAD_BLOCKS; } int replica_remove_all_recovery_files(struct poolset_health_status *set_hs); int replica_remove_part(struct pool_set *set, unsigned repn, unsigned partn, int fix_bad_blocks); int replica_create_poolset_health_status(struct pool_set *set, struct poolset_health_status **set_hsp); void replica_free_poolset_health_status(struct poolset_health_status *set_s); int replica_check_poolset_health(struct pool_set *set, struct poolset_health_status **set_hs, int called_from_sync, unsigned flags); int replica_is_part_broken(unsigned repn, unsigned partn, struct poolset_health_status *set_hs); int replica_has_bad_blocks(unsigned repn, struct poolset_health_status *set_hs); int replica_part_has_bad_blocks(struct part_health_status *phs); int replica_part_has_corrupted_header(unsigned repn, unsigned partn, struct poolset_health_status *set_hs); unsigned replica_find_unbroken_part(unsigned repn, struct poolset_health_status *set_hs); int replica_is_replica_broken(unsigned repn, struct poolset_health_status *set_hs); int replica_is_replica_consistent(unsigned repn, struct poolset_health_status *set_hs); int replica_is_replica_healthy(unsigned repn, struct poolset_health_status *set_hs); unsigned replica_find_healthy_replica( struct poolset_health_status *set_hs); unsigned replica_find_replica_healthy_header( struct poolset_health_status *set_hs); int replica_is_poolset_healthy(struct poolset_health_status *set_hs); int replica_is_poolset_transformed(unsigned flags); ssize_t replica_get_pool_size(struct pool_set *set, unsigned repn); int replica_check_part_sizes(struct pool_set *set, size_t min_size); int replica_check_part_dirs(struct pool_set *set); int replica_check_local_part_dir(struct pool_set *set, unsigned repn, unsigned partn); int replica_open_replica_part_files(struct pool_set *set, unsigned repn); int replica_open_poolset_part_files(struct pool_set *set); int replica_sync(struct pool_set *set_in, struct poolset_health_status *set_hs, unsigned flags); int replica_transform(struct pool_set *set_in, struct pool_set *set_out, unsigned flags); #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmempool/libpmempool.def0000664000000000000000000000410513615011243017116 0ustar rootroot;;;; Begin Copyright Notice ; ; Copyright 2016, Intel Corporation ; ; Redistribution and use in source and binary forms, with or without ; modification, are permitted provided that the following conditions ; are met: ; ; * Redistributions of source code must retain the above copyright ; notice, this list of conditions and the following disclaimer. ; ; * Redistributions in binary form must reproduce the above copyright ; notice, this list of conditions and the following disclaimer in ; the documentation and/or other materials provided with the ; distribution. ; ; * Neither the name of the copyright holder nor the names of its ; contributors may be used to endorse or promote products derived ; from this software without specific prior written permission. ; ; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS ; "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT ; LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ; A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT ; OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, ; SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT ; LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, ; DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY ; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT ; (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE ; OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ; ;;;; End Copyright Notice LIBRARY libpmempool VERSION 1.0 EXPORTS pmempool_check_versionU pmempool_check_versionW pmempool_errormsgU pmempool_errormsgW pmempool_check_initU pmempool_check_initW pmempool_checkU pmempool_checkW pmempool_check_end pmempool_syncU pmempool_syncW pmempool_transformU pmempool_transformW pmempool_rmU pmempool_rmW pmempool_feature_enableU pmempool_feature_enableW pmempool_feature_disableU pmempool_feature_disableW pmempool_feature_queryU pmempool_feature_queryW DllMain pmdk-1.8/src/libpmempool/libpmempool.vcxproj.filters0000664000000000000000000001561013615011243021525 0ustar rootroot Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files {cd079c79-5441-413e-b2ba-99fed2b0b779} {9ad93d4f-a9d1-4e38-94a1-77e36acc268f} Source Files Source Files pmdk-1.8/src/libpmempool/sync.c0000664000000000000000000012555713615011243015256 0ustar rootroot/* * Copyright 2016-2020, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * sync.c -- a module for poolset synchronizing */ #include #include #include #include #include #include #include #include "libpmem.h" #include "replica.h" #include "out.h" #include "os.h" #include "util_pmem.h" #include "util.h" #ifdef USE_RPMEM #include "rpmem_common.h" #include "rpmem_ssh.h" #endif #define BB_DATA_STR "offset 0x%llx, length 0x%x, nhealthy %i" /* defines 'struct bb_vec' - the vector of the 'struct bad_block' structures */ VEC(bb_vec, struct bad_block); /* * validate_args -- (internal) check whether passed arguments are valid */ static int validate_args(struct pool_set *set) { LOG(3, "set %p", set); ASSERTne(set, NULL); /* the checks below help detect use of incorrect poolset file */ /* * check if all parts in the poolset are large enough * (now replication works only for pmemobj pools) */ if (replica_check_part_sizes(set, PMEMOBJ_MIN_POOL)) { LOG(2, "part sizes check failed"); goto err; } /* * check if all directories for part files exist */ if (replica_check_part_dirs(set)) { LOG(2, "part directories check failed"); goto err; } return 0; err: if (errno == 0) errno = EINVAL; return -1; } /* * sync_copy_data -- (internal) copy data from the healthy replica * to the broken one */ static int sync_copy_data(void *src_addr, void *dst_addr, size_t off, size_t len, struct pool_replica *rep_h, struct pool_replica *rep, const struct pool_set_part *part) { LOG(3, "src_addr %p dst_addr %p off %zu len %zu " "rep_h %p rep %p part %p", src_addr, dst_addr, off, len, rep_h, rep, part); int ret; if (rep->remote) { LOG(10, "copying data (offset 0x%zx length 0x%zx) to remote node -- '%s' on '%s'", off, len, rep->remote->pool_desc, rep->remote->node_addr); ret = Rpmem_persist(rep->remote->rpp, off, len, 0, 0); if (ret) { LOG(1, "copying data to remote node failed -- '%s' on '%s'", rep->remote->pool_desc, rep->remote->node_addr); return -1; } } else if (rep_h->remote) { LOG(10, "reading data (offset 0x%zx length 0x%zx) from remote node -- '%s' on '%s'", off, len, rep_h->remote->pool_desc, rep_h->remote->node_addr); ret = Rpmem_read(rep_h->remote->rpp, dst_addr, off, len, 0); if (ret) { LOG(1, "reading data from remote node failed -- '%s' on '%s'", rep_h->remote->pool_desc, rep_h->remote->node_addr); return -1; } } else { LOG(10, "copying data (offset 0x%zx length 0x%zx) from local replica -- '%s'", off, len, rep_h->part[0].path); /* copy all data */ memcpy(dst_addr, src_addr, len); util_persist(part->is_dev_dax, dst_addr, len); } return 0; } /* * sync_recreate_header -- (internal) recreate the header */ static int sync_recreate_header(struct pool_set *set, unsigned r, unsigned p, struct pool_hdr *src_hdr) { LOG(3, "set %p replica %u part %u src_hdr %p", set, r, p, src_hdr); struct pool_attr attr; util_pool_hdr2attr(&attr, src_hdr); if (util_header_create(set, r, p, &attr, 1) != 0) { LOG(1, "part headers create failed for replica %u part %u", r, p); errno = EINVAL; return -1; } return 0; } /* * sync_mark_replica_no_badblocks -- (internal) mark replica as not having * bad blocks */ static void sync_mark_replica_no_badblocks(unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "repn %u set_hs %p", repn, set_hs); struct replica_health_status *rhs = REP_HEALTH(set_hs, repn); if (rhs->flags & HAS_BAD_BLOCKS) { rhs->flags &= ~HAS_BAD_BLOCKS; LOG(4, "replica %u has no bad blocks now", repn); } } /* * sync_mark_part_no_badblocks -- (internal) mark part as not having bad blocks */ static void sync_mark_part_no_badblocks(unsigned repn, unsigned partn, struct poolset_health_status *set_hs) { LOG(3, "repn %u partn %u set_hs %p", repn, partn, set_hs); struct replica_health_status *rhs = REP_HEALTH(set_hs, repn); if (rhs->part[PART_HEALTHidx(rhs, partn)].flags & HAS_BAD_BLOCKS) { rhs->part[PART_HEALTHidx(rhs, partn)].flags &= ~HAS_BAD_BLOCKS; LOG(4, "replica %u part %u has no bad blocks now", repn, partn); } } /* * sync_recalc_badblocks -- (internal) recalculate offset and length * of bad blocks to absolute ones * (relative to the beginning of the pool) */ static int sync_recalc_badblocks(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p set_hs %p", set, set_hs); /* header size for all headers but the first one */ size_t hdrsize = (set->options & (OPTION_SINGLEHDR | OPTION_NOHDRS)) ? 0 : Mmap_align; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); struct replica_health_status *rep_hs = set_hs->replica[r]; for (unsigned p = 0; p < rep->nparts; ++p) { struct part_health_status *phs = &rep_hs->part[p]; if (!replica_part_has_bad_blocks(phs)) { /* skip parts with no bad blocks */ continue; } ASSERTne(phs->bbs.bb_cnt, 0); ASSERTne(phs->bbs.bbv, NULL); LOG(10, "Replica %u part %u HAS %u bad blocks", r, p, phs->bbs.bb_cnt); size_t part_off = replica_get_part_offset(set, r, p); for (unsigned i = 0; i < phs->bbs.bb_cnt; i++) { LOG(10, "relative bad block #%i: offset %llu, length %u", i, phs->bbs.bbv[i].offset, phs->bbs.bbv[i].length); size_t off = phs->bbs.bbv[i].offset; size_t len = phs->bbs.bbv[i].length; if (len + off <= hdrsize) continue; /* parts #>0 are mapped without the header */ if (p > 0 && hdrsize > 0) { if (off >= hdrsize) { /* * Bad block does not overlap * with the header, so only * adjust the offset. */ off -= hdrsize; } else { /* * Bad block overlaps * with the header, * so adjust the length * and zero the offset. */ len -= hdrsize - off; off = 0; } } replica_align_badblock_offset_length(&off, &len, set, r, p); phs->bbs.bbv[i].offset = part_off + off; phs->bbs.bbv[i].length = (unsigned)len; LOG(10, "absolute bad block #%i: offset 0x%llx, length 0x%x", i, phs->bbs.bbv[i].offset, phs->bbs.bbv[i].length); } } } return 0; } /* * sync_badblocks_find_healthy_replica -- (internal) look for a healthy replica * for each bad block * * This function looks for a healthy replica for each bad block. Bad blocks * can overlap across replicas, so each bad block may have to be divided * into smaller parts which can be fixed using different healthy replica. * * Key variables: * - bbv_all[] - array containing all (possibly divided) bad blocks * from all previous replicas. * - bbv_aux[] - array containing all (possibly divided) bad blocks * from all previous parts of the current replica merged with * these bad blocks from bbv_all[] that have offsets less or equal * the greatest bad block's offset in the previous part. * * This function merges bad blocks from bbv_all[] with bad blocks * from the current part and writes the outcome bad blocks to bbv_aux[]. * Only bad blocks with offsets less or equal the greatest bad block's offset * in the current part will be moved from bbv_all[] to bbv_aux[]. * The rest of them has to be moved at the end by sync_badblocks_move_vec(). * * bbv_aux[] becomes new bbv_all[] and bbv_aux[] is zeroed * before checking the next replica (bbv_all = bbv_aux; bbv_aux = 0). * * For example (all replicas have only one part): * - bbv_all with rep#0: |__----___________----__| * - merged with rep#1: |____----_______----____| * - gives such bbv_aux: |__11--00_______00--11__| * - merged with rep#2: |__________---__________| * - gives such bbv_aux: |__112200__000__002211__| (all bad blocks can be fixed) * * where: * '_' stands for a healthy block (no bad block) * '-' stands for a bad block with nhealthy == NO_HEALTHY_REPLICA * 'N' stands for a bad block with nhealthy == N (can be fixed using rep#N) */ static int sync_badblocks_find_healthy_replica(struct part_health_status *phs, int rep, struct bb_vec *pbbv_all, struct bb_vec *pbbv_aux, unsigned *i_all) { LOG(3, "phs %p rep %i pbbv_all %p pbbv_aux %p i_all %i", phs, rep, pbbv_all, pbbv_aux, *i_all); struct bad_block bb_add; /* the element which is being added */ struct bad_block bb_new; /* a new element */ struct bad_block *pbb_all; /* current element of bbv_all[] */ unsigned long long beg_prev; unsigned long long end_prev; unsigned long long beg_new; unsigned long long end_new; unsigned len_prev; unsigned len_new; size_t size_all = VEC_SIZE(pbbv_all); if (size_all == 0) { /* there were no bad blocks so far, so fill up bbv_aux[] */ for (unsigned i = 0; i < phs->bbs.bb_cnt; i++) { bb_add = phs->bbs.bbv[i]; if (rep > 0) /* bad block can be fixed with replica #0 */ bb_add.nhealthy = 0; if (VEC_PUSH_BACK(pbbv_aux, bb_add)) return -1; LOG(10, "added bad block (prev-empty): " BB_DATA_STR, bb_add.offset, bb_add.length, bb_add.nhealthy); } } else { if (*i_all < size_all) { pbb_all = VEC_GET(pbbv_all, (*i_all)++); } else { pbb_all = NULL; } for (unsigned i = 0; i < phs->bbs.bb_cnt; i++) { bb_new = phs->bbs.bbv[i]; LOG(10, " * (%u) inserting new bad block: " BB_DATA_STR, i + 1, bb_new.offset, bb_new.length, bb_new.nhealthy); if (pbb_all == NULL || pbb_all->length == 0) { if (*i_all < size_all) pbb_all = VEC_GET(pbbv_all, (*i_all)++); else pbb_all = NULL; } /* all from bbv_all before the bb_new */ while (pbb_all != NULL && pbb_all->offset + pbb_all->length - 1 < bb_new.offset) { if (pbb_all->nhealthy == NO_HEALTHY_REPLICA) /* can be fixed with this replica */ pbb_all->nhealthy = rep; if (VEC_PUSH_BACK(pbbv_aux, *pbb_all)) return -1; LOG(10, "added bad block (prev-before): " BB_DATA_STR, pbb_all->offset, pbb_all->length, pbb_all->nhealthy); if (*i_all < size_all) { pbb_all = VEC_GET(pbbv_all, (*i_all)++); } else { pbb_all = NULL; break; } } beg_new = bb_new.offset; len_new = bb_new.length; end_new = beg_new + len_new - 1; /* all pbb_all overlapping with the bb_new */ while (len_new > 0 && pbb_all != NULL) { beg_prev = pbb_all->offset; len_prev = pbb_all->length; end_prev = beg_prev + len_prev - 1; /* check if new overlaps with prev */ if (end_prev < beg_new || end_new < beg_prev) break; /* * 1st part: non-overlapping part * of pbb_all or bb_new */ if (beg_prev < beg_new) { /* non-overlapping part of pbb_all */ bb_add.offset = beg_prev; bb_add.length = (unsigned) (beg_new - beg_prev); if (pbb_all->nhealthy != NO_HEALTHY_REPLICA) { bb_add.nhealthy = pbb_all->nhealthy; } else { /* * It can be fixed with * this replica. */ bb_add.nhealthy = rep; } if (VEC_PUSH_BACK(pbbv_aux, bb_add)) return -1; LOG(10, "added bad block (prev-only): " BB_DATA_STR, bb_add.offset, bb_add.length, bb_add.nhealthy); beg_prev += bb_add.length; len_prev -= bb_add.length; } else if (beg_new < beg_prev) { /* non-overlapping part of bb_new */ bb_add.offset = beg_new; bb_add.length = (unsigned) (beg_prev - beg_new); if (rep == 0) { bb_add.nhealthy = NO_HEALTHY_REPLICA; } else { /* * It can be fixed with any * previous replica, so let's * choose replia #0. */ bb_add.nhealthy = 0; } if (VEC_PUSH_BACK(pbbv_aux, bb_add)) return -1; LOG(10, "added bad block (new-only): " BB_DATA_STR, bb_add.offset, bb_add.length, bb_add.nhealthy); beg_new += bb_add.length; len_new -= bb_add.length; } /* * 2nd part: overlapping part * of pbb_all and bb_new */ if (len_prev <= len_new) { bb_add.offset = beg_prev; bb_add.length = len_prev; beg_new += len_prev; len_new -= len_prev; /* whole pbb_all was added */ len_prev = 0; } else { bb_add.offset = beg_new; bb_add.length = len_new; beg_prev += len_new; len_prev -= len_new; /* whole bb_new was added */ len_new = 0; } bb_add.nhealthy = pbb_all->nhealthy; if (VEC_PUSH_BACK(pbbv_aux, bb_add)) return -1; LOG(10, "added bad block (common): " BB_DATA_STR, bb_add.offset, bb_add.length, bb_add.nhealthy); /* update pbb_all */ pbb_all->offset = beg_prev; pbb_all->length = len_prev; if (len_prev == 0) { if (*i_all < size_all) pbb_all = VEC_GET(pbbv_all, (*i_all)++); else pbb_all = NULL; } } /* the rest of the bb_new */ if (len_new > 0) { bb_add.offset = beg_new; bb_add.length = len_new; if (rep > 0) /* it can be fixed with replica #0 */ bb_add.nhealthy = 0; else bb_add.nhealthy = NO_HEALTHY_REPLICA; if (VEC_PUSH_BACK(pbbv_aux, bb_add)) return -1; LOG(10, "added bad block (new-rest): " BB_DATA_STR, bb_add.offset, bb_add.length, bb_add.nhealthy); } } if (pbb_all != NULL && pbb_all->length > 0 && *i_all > 0) /* this pbb_all will be used again in the next part */ (*i_all)--; } return 0; } /* * sync_badblocks_assign_healthy_replica -- (internal) assign healthy replica * for each bad block */ static int sync_badblocks_assign_healthy_replica(struct part_health_status *phs, int rep, struct bb_vec *pbbv_all, unsigned *i_all) { LOG(3, "phs %p rep %i pbbv_all %p i_all %i", phs, rep, pbbv_all, *i_all); struct bad_block bb_new; /* a new element */ struct bad_block bb_old; /* an old element */ struct bad_block *pbb_all; /* current element of bbv_all[] */ unsigned length_left; struct bb_vec bbv_new = VEC_INITIALIZER; size_t size_all = VEC_SIZE(pbbv_all); pbb_all = VEC_GET(pbbv_all, *i_all); for (unsigned i = 0; i < phs->bbs.bb_cnt; i++) { bb_old = phs->bbs.bbv[i]; LOG(10, "assigning old bad block: " BB_DATA_STR, bb_old.offset, bb_old.length, bb_old.nhealthy); /* * Skip all bad blocks from bbv_all with offsets * less than the offset of the current bb_old. */ while (pbb_all->offset < bb_old.offset) { /* (*i_all) has to be less than (size_all - 1) */ ASSERT(*i_all < size_all - 1); pbb_all = VEC_GET(pbbv_all, ++(*i_all)); } bb_new.offset = bb_old.offset; length_left = bb_old.length; while (length_left > 0) { LOG(10, "checking saved bad block: " BB_DATA_STR, pbb_all->offset, pbb_all->length, pbb_all->nhealthy); ASSERTeq(pbb_all->offset, bb_new.offset); ASSERT(pbb_all->length <= length_left); bb_new.length = pbb_all->length; bb_new.nhealthy = pbb_all->nhealthy; if (VEC_PUSH_BACK(&bbv_new, bb_new)) goto error_exit; LOG(10, "added new bad block: " BB_DATA_STR, bb_new.offset, bb_new.length, bb_new.nhealthy); bb_new.offset += bb_new.length; length_left -= bb_new.length; if (length_left == 0) continue; /* (*i_all) has to be less than (size_all - 1) */ ASSERT(*i_all < size_all - 1); pbb_all = VEC_GET(pbbv_all, ++(*i_all)); } } Free(phs->bbs.bbv); phs->bbs.bbv = VEC_ARR(&bbv_new); phs->bbs.bb_cnt = (unsigned)VEC_SIZE(&bbv_new); LOG(10, "added %u new bad blocks", phs->bbs.bb_cnt); return 0; error_exit: VEC_DELETE(&bbv_new); return -1; } /* * sync_badblocks_move_vec -- (internal) move bad blocks from vector pbbv_all * to vector pbbv_aux */ static int sync_badblocks_move_vec(struct bb_vec *pbbv_all, struct bb_vec *pbbv_aux, unsigned i_all, unsigned rep) { LOG(3, "pbbv_all %p pbbv_aux %p i_all %u rep %u", pbbv_all, pbbv_aux, i_all, rep); size_t size_all = VEC_SIZE(pbbv_all); struct bad_block *pbb_all; while (i_all < size_all) { pbb_all = VEC_GET(pbbv_all, i_all++); if (pbb_all->length == 0) continue; if (pbb_all->nhealthy == NO_HEALTHY_REPLICA && rep > 0) /* it can be fixed using the last replica */ pbb_all->nhealthy = (int)rep; if (VEC_PUSH_BACK(pbbv_aux, *pbb_all)) return -1; LOG(10, "added bad block (prev-after): " BB_DATA_STR, pbb_all->offset, pbb_all->length, pbb_all->nhealthy); } return 0; } /* * sync_check_bad_blocks_overlap -- (internal) check if there are uncorrectable * bad blocks (bad blocks overlapping * in all replicas) */ static int sync_check_bad_blocks_overlap(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p set_hs %p", set, set_hs); struct bb_vec bbv_all = VEC_INITIALIZER; struct bb_vec bbv_aux = VEC_INITIALIZER; int ret = -1; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); struct replica_health_status *rep_hs = set_hs->replica[r]; unsigned i_all = 0; /* index in bbv_all */ for (unsigned p = 0; p < rep->nparts; ++p) { struct part_health_status *phs = &rep_hs->part[p]; if (!replica_part_has_bad_blocks(phs)) { /* skip parts with no bad blocks */ continue; } ASSERTne(phs->bbs.bb_cnt, 0); ASSERTne(phs->bbs.bbv, NULL); LOG(10, "Replica %u part %u HAS %u bad blocks", r, p, phs->bbs.bb_cnt); /* * This function merges bad blocks from bbv_all * with bad blocks from the current part * and writes the outcome bad blocks to bbv_aux. * Only bad blocks with offsets less or equal * the greatest bad block's offset in the current part * will be moved from bbv_all to bbv_aux. * The rest of them has to be moved at the end * by sync_badblocks_move_vec() below. */ if (sync_badblocks_find_healthy_replica(phs, (int)r, &bbv_all, &bbv_aux, &i_all)) goto exit; } /* * Move the rest of bad blocks from bbv_all to bbv_aux * (for more details see the comment above). * All these bad blocks can be fixed using the last replica 'r'. */ if (sync_badblocks_move_vec(&bbv_all, &bbv_aux, i_all, r)) return -1; /* bbv_aux becomes a new bbv_all */ VEC_MOVE(&bbv_all, &bbv_aux); i_all = 0; } ret = 0; /* check if there is an uncorrectable bad block */ size_t size_all = VEC_SIZE(&bbv_all); for (unsigned i = 0; i < size_all; i++) { struct bad_block *pbb_all = VEC_GET(&bbv_all, i); if (pbb_all->nhealthy == NO_HEALTHY_REPLICA) { ret = 1; /* this bad block cannot be fixed */ LOG(1, "uncorrectable bad block found: offset 0x%llx, length 0x%x", pbb_all->offset, pbb_all->length); goto exit; } } /* * All bad blocks can be fixed, * so assign healthy replica for each of them. */ for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); struct replica_health_status *rep_hs = set_hs->replica[r]; if (!replica_has_bad_blocks(r, set_hs)) { /* skip replicas with no bad blocks */ continue; } unsigned i_all = 0; /* index in bbv_all */ for (unsigned p = 0; p < rep->nparts; ++p) { struct part_health_status *phs = &rep_hs->part[p]; if (!replica_part_has_bad_blocks(phs)) { /* skip parts with no bad blocks */ continue; } if (sync_badblocks_assign_healthy_replica(phs, (int)r, &bbv_all, &i_all)) goto exit; } } exit: VEC_DELETE(&bbv_aux); VEC_DELETE(&bbv_all); return ret; } /* * sync_badblocks_data -- (internal) clear bad blocks in replica */ static int sync_badblocks_data(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); struct pool_replica *rep_h; for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); struct replica_health_status *rep_hs = set_hs->replica[r]; for (unsigned p = 0; p < rep->nparts; ++p) { struct part_health_status *phs = &rep_hs->part[p]; if (!replica_part_has_bad_blocks(phs)) { /* skip parts with no bad blocks */ continue; } ASSERTne(phs->bbs.bb_cnt, 0); ASSERTne(phs->bbs.bbv, NULL); const struct pool_set_part *part = &rep->part[p]; size_t part_off = replica_get_part_offset(set, r, p); for (unsigned i = 0; i < phs->bbs.bb_cnt; i++) { size_t off = phs->bbs.bbv[i].offset - part_off; size_t len = phs->bbs.bbv[i].length; ASSERT(phs->bbs.bbv[i].nhealthy >= 0); rep_h = REP(set, (unsigned)phs->bbs.bbv[i].nhealthy); void *src_addr = ADDR_SUM(rep_h->part[0].addr, part_off + off); void *dst_addr = ADDR_SUM(part->addr, off); if (sync_copy_data(src_addr, dst_addr, part_off + off, len, rep_h, rep, part)) return -1; } /* free array of bad blocks */ Free(phs->bbs.bbv); phs->bbs.bbv = NULL; /* mark part as having no bad blocks */ sync_mark_part_no_badblocks(r, p, set_hs); } /* mark replica as having no bad blocks */ sync_mark_replica_no_badblocks(r, set_hs); } LOG(1, "all bad blocks have been fixed"); if (replica_remove_all_recovery_files(set_hs)) { LOG(1, "removing bad block recovery files failed"); return -1; } return 0; } /* * recreate_broken_parts -- (internal) create parts in place of the broken ones */ static int recreate_broken_parts(struct pool_set *set, struct poolset_health_status *set_hs, int fix_bad_blocks) { LOG(3, "set %p set_hs %p fix_bad_blocks %i", set, set_hs, fix_bad_blocks); for (unsigned r = 0; r < set_hs->nreplicas; ++r) { if (set->replica[r]->remote) continue; struct pool_replica *broken_r = set->replica[r]; for (unsigned p = 0; p < set_hs->replica[r]->nparts; ++p) { /* skip unbroken parts */ if (!replica_is_part_broken(r, p, set_hs)) continue; /* remove parts from broken replica */ if (replica_remove_part(set, r, p, fix_bad_blocks)) { LOG(2, "cannot remove part"); return -1; } /* create removed part and open it */ if (util_part_open(&broken_r->part[p], 0, 1 /* create */)) { LOG(2, "cannot open/create parts"); return -1; } sync_mark_part_no_badblocks(r, p, set_hs); } } return 0; } /* * fill_struct_part_uuids -- (internal) set part uuids in pool_set structure */ static void fill_struct_part_uuids(struct pool_set *set, unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "set %p, repn %u, set_hs %p", set, repn, set_hs); struct pool_replica *rep = REP(set, repn); struct pool_hdr *hdrp; for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip broken parts */ if (replica_is_part_broken(repn, p, set_hs)) continue; hdrp = HDR(rep, p); memcpy(rep->part[p].uuid, hdrp->uuid, POOL_HDR_UUID_LEN); } } /* * is_uuid_already_used -- (internal) check if given uuid is assigned to * any of the earlier replicas */ static int is_uuid_already_used(uuid_t uuid, struct pool_set *set, unsigned repn) { for (unsigned r = 0; r < repn; ++r) { if (uuidcmp(uuid, PART(REP(set, r), 0)->uuid) == 0) return 1; } return 0; } /* * fill_struct_broken_part_uuids -- (internal) set part uuids in pool_set * structure */ static int fill_struct_broken_part_uuids(struct pool_set *set, unsigned repn, struct poolset_health_status *set_hs, unsigned flags) { LOG(3, "set %p, repn %u, set_hs %p, flags %u", set, repn, set_hs, flags); struct pool_replica *rep = REP(set, repn); struct pool_hdr *hdrp; for (unsigned p = 0; p < rep->nhdrs; ++p) { /* skip unbroken parts */ if (!replica_is_part_broken(repn, p, set_hs)) continue; /* check if part was damaged or was added by transform */ if (replica_is_poolset_transformed(flags)) { /* generate new uuid for this part */ if (util_uuid_generate(rep->part[p].uuid) < 0) { ERR("cannot generate pool set part UUID"); errno = EINVAL; return -1; } continue; } if (!replica_is_part_broken(repn, p - 1, set_hs) && !(set->options & OPTION_SINGLEHDR)) { /* try to get part uuid from the previous part */ hdrp = HDRP(rep, p); memcpy(rep->part[p].uuid, hdrp->next_part_uuid, POOL_HDR_UUID_LEN); } else if (!replica_is_part_broken(repn, p + 1, set_hs) && !(set->options & OPTION_SINGLEHDR)) { /* try to get part uuid from the next part */ hdrp = HDRN(rep, p); memcpy(rep->part[p].uuid, hdrp->prev_part_uuid, POOL_HDR_UUID_LEN); } else if (p == 0 && !replica_is_part_broken(repn - 1, 0, set_hs)) { /* try to get part uuid from the previous replica */ hdrp = HDR(REPP(set, repn), 0); if (is_uuid_already_used(hdrp->next_repl_uuid, set, repn)) { ERR( "repeated uuid - some replicas were created with a different poolset file"); errno = EINVAL; return -1; } memcpy(rep->part[p].uuid, hdrp->next_repl_uuid, POOL_HDR_UUID_LEN); } else if (p == 0 && !replica_is_part_broken(repn + 1, 0, set_hs)) { /* try to get part uuid from the next replica */ hdrp = HDR(REPN(set, repn), 0); if (is_uuid_already_used(hdrp->prev_repl_uuid, set, repn)) { ERR( "repeated uuid - some replicas were created with a different poolset file"); errno = EINVAL; return -1; } memcpy(rep->part[p].uuid, hdrp->prev_repl_uuid, POOL_HDR_UUID_LEN); } else { /* generate new uuid for this part */ if (util_uuid_generate(rep->part[p].uuid) < 0) { ERR("cannot generate pool set part UUID"); errno = EINVAL; return -1; } } } return 0; } /* * fill_struct_uuids -- (internal) fill fields in pool_set needed for further * altering of uuids */ static int fill_struct_uuids(struct pool_set *set, unsigned src_replica, struct poolset_health_status *set_hs, unsigned flags) { LOG(3, "set %p, src_replica %u, set_hs %p, flags %u", set, src_replica, set_hs, flags); /* set poolset uuid */ struct pool_hdr *src_hdr0 = HDR(REP(set, src_replica), 0); memcpy(set->uuid, src_hdr0->poolset_uuid, POOL_HDR_UUID_LEN); /* set unbroken parts' uuids */ for (unsigned r = 0; r < set->nreplicas; ++r) { fill_struct_part_uuids(set, r, set_hs); } /* set broken parts' uuids */ for (unsigned r = 0; r < set->nreplicas; ++r) { if (fill_struct_broken_part_uuids(set, r, set_hs, flags)) return -1; } return 0; } /* * create_headers_for_broken_parts -- (internal) create headers for all new * parts created in place of the broken ones */ static int create_headers_for_broken_parts(struct pool_set *set, unsigned src_replica, struct poolset_health_status *set_hs) { LOG(3, "set %p, src_replica %u, set_hs %p", set, src_replica, set_hs); struct pool_hdr *src_hdr = HDR(REP(set, src_replica), 0); for (unsigned r = 0; r < set_hs->nreplicas; ++r) { /* skip unbroken replicas */ if (!replica_is_replica_broken(r, set_hs) && !replica_has_bad_blocks(r, set_hs)) continue; for (unsigned p = 0; p < set_hs->replica[r]->nhdrs; p++) { /* skip unbroken parts */ if (!replica_is_part_broken(r, p, set_hs) && !replica_part_has_corrupted_header(r, p, set_hs)) continue; if (sync_recreate_header(set, r, p, src_hdr)) return -1; } } return 0; } /* * copy_data_to_broken_parts -- (internal) copy data to all parts created * in place of the broken ones */ static int copy_data_to_broken_parts(struct pool_set *set, unsigned healthy_replica, unsigned flags, struct poolset_health_status *set_hs) { LOG(3, "set %p, healthy_replica %u, flags %u, set_hs %p", set, healthy_replica, flags, set_hs); /* get pool size from healthy replica */ size_t poolsize = set->poolsize; for (unsigned r = 0; r < set_hs->nreplicas; ++r) { /* skip unbroken and consistent replicas */ if (replica_is_replica_healthy(r, set_hs)) continue; struct pool_replica *rep = REP(set, r); struct pool_replica *rep_h = REP(set, healthy_replica); for (unsigned p = 0; p < rep->nparts; ++p) { /* skip unbroken parts from consistent replicas */ if (!replica_is_part_broken(r, p, set_hs) && replica_is_replica_consistent(r, set_hs)) continue; const struct pool_set_part *part = &rep->part[p]; size_t off = replica_get_part_data_offset(set, r, p); size_t len = replica_get_part_data_len(set, r, p); /* do not allow copying too much data */ if (off >= poolsize) continue; if (off + len > poolsize || rep->remote) len = poolsize - off; /* * First part of replica is mapped * with header */ size_t fpoff = (p == 0) ? POOL_HDR_SIZE : 0; void *src_addr = ADDR_SUM(rep_h->part[0].addr, off); void *dst_addr = ADDR_SUM(part->addr, fpoff); if (sync_copy_data(src_addr, dst_addr, off, len, rep_h, rep, part)) return -1; } } return 0; } /* * grant_created_parts_perm -- (internal) set RW permission rights to all * the parts created in place of the broken ones */ static int grant_created_parts_perm(struct pool_set *set, unsigned src_repn, struct poolset_health_status *set_hs) { LOG(3, "set %p, src_repn %u, set_hs %p", set, src_repn, set_hs); /* choose the default permissions */ mode_t def_mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IWGRP; /* get permissions of the first part of the source replica */ mode_t src_mode; os_stat_t sb; if (REP(set, src_repn)->remote) { src_mode = def_mode; } else if (os_stat(PART(REP(set, src_repn), 0)->path, &sb) != 0) { ERR("cannot check file permissions of %s (replica %u, part %u)", PART(REP(set, src_repn), 0)->path, src_repn, 0); src_mode = def_mode; } else { src_mode = sb.st_mode; } /* set permissions to all recreated parts */ for (unsigned r = 0; r < set_hs->nreplicas; ++r) { /* skip unbroken replicas */ if (!replica_is_replica_broken(r, set_hs)) continue; if (set->replica[r]->remote) continue; for (unsigned p = 0; p < set_hs->replica[r]->nparts; p++) { /* skip parts which were not created */ if (!PART(REP(set, r), p)->created) continue; LOG(4, "setting permissions for part %u, replica %u", p, r); /* set rights to those of existing part files */ if (os_chmod(PART(REP(set, r), p)->path, src_mode)) { ERR( "cannot set permission rights for created parts: replica %u, part %u", r, p); errno = EPERM; return -1; } } } return 0; } /* * update_parts_linkage -- (internal) set uuids linking recreated parts within * a replica */ static int update_parts_linkage(struct pool_set *set, unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "set %p, repn %u, set_hs %p", set, repn, set_hs); struct pool_replica *rep = REP(set, repn); for (unsigned p = 0; p < rep->nhdrs; ++p) { struct pool_hdr *hdrp = HDR(rep, p); struct pool_hdr *prev_hdrp = HDRP(rep, p); struct pool_hdr *next_hdrp = HDRN(rep, p); /* set uuids in the current part */ memcpy(hdrp->prev_part_uuid, PARTP(rep, p)->uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->next_part_uuid, PARTN(rep, p)->uuid, POOL_HDR_UUID_LEN); util_checksum(hdrp, sizeof(*hdrp), &hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(hdrp)); /* set uuids in the previous part */ memcpy(prev_hdrp->next_part_uuid, PART(rep, p)->uuid, POOL_HDR_UUID_LEN); util_checksum(prev_hdrp, sizeof(*prev_hdrp), &prev_hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(prev_hdrp)); /* set uuids in the next part */ memcpy(next_hdrp->prev_part_uuid, PART(rep, p)->uuid, POOL_HDR_UUID_LEN); util_checksum(next_hdrp, sizeof(*next_hdrp), &next_hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(next_hdrp)); /* store pool's header */ util_persist(PART(rep, p)->is_dev_dax, hdrp, sizeof(*hdrp)); util_persist(PARTP(rep, p)->is_dev_dax, prev_hdrp, sizeof(*prev_hdrp)); util_persist(PARTN(rep, p)->is_dev_dax, next_hdrp, sizeof(*next_hdrp)); } return 0; } /* * update_replicas_linkage -- (internal) update uuids linking replicas */ static int update_replicas_linkage(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); struct pool_replica *rep = REP(set, repn); struct pool_replica *prev_r = REPP(set, repn); struct pool_replica *next_r = REPN(set, repn); ASSERT(rep->nparts > 0); ASSERT(prev_r->nparts > 0); ASSERT(next_r->nparts > 0); /* set uuids in the current replica */ for (unsigned p = 0; p < rep->nhdrs; ++p) { struct pool_hdr *hdrp = HDR(rep, p); memcpy(hdrp->prev_repl_uuid, PART(prev_r, 0)->uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->next_repl_uuid, PART(next_r, 0)->uuid, POOL_HDR_UUID_LEN); util_checksum(hdrp, sizeof(*hdrp), &hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(hdrp)); /* store pool's header */ util_persist(PART(rep, p)->is_dev_dax, hdrp, sizeof(*hdrp)); } /* set uuids in the previous replica */ for (unsigned p = 0; p < prev_r->nhdrs; ++p) { struct pool_hdr *prev_hdrp = HDR(prev_r, p); memcpy(prev_hdrp->next_repl_uuid, PART(rep, 0)->uuid, POOL_HDR_UUID_LEN); util_checksum(prev_hdrp, sizeof(*prev_hdrp), &prev_hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(prev_hdrp)); /* store pool's header */ util_persist(PART(prev_r, p)->is_dev_dax, prev_hdrp, sizeof(*prev_hdrp)); } /* set uuids in the next replica */ for (unsigned p = 0; p < next_r->nhdrs; ++p) { struct pool_hdr *next_hdrp = HDR(next_r, p); memcpy(next_hdrp->prev_repl_uuid, PART(rep, 0)->uuid, POOL_HDR_UUID_LEN); util_checksum(next_hdrp, sizeof(*next_hdrp), &next_hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(next_hdrp)); /* store pool's header */ util_persist(PART(next_r, p)->is_dev_dax, next_hdrp, sizeof(*next_hdrp)); } return 0; } /* * update_poolset_uuids -- (internal) update poolset uuid in recreated parts */ static int update_poolset_uuids(struct pool_set *set, unsigned repn, struct poolset_health_status *set_hs) { LOG(3, "set %p, repn %u, set_hs %p", set, repn, set_hs); struct pool_replica *rep = REP(set, repn); for (unsigned p = 0; p < rep->nhdrs; ++p) { struct pool_hdr *hdrp = HDR(rep, p); memcpy(hdrp->poolset_uuid, set->uuid, POOL_HDR_UUID_LEN); util_checksum(hdrp, sizeof(*hdrp), &hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(hdrp)); /* store pool's header */ util_persist(PART(rep, p)->is_dev_dax, hdrp, sizeof(*hdrp)); } return 0; } /* * update_remote_headers -- (internal) update headers of existing remote * replicas */ static int update_remote_headers(struct pool_set *set) { LOG(3, "set %p", set); for (unsigned r = 0; r < set->nreplicas; ++ r) { /* skip local or just created replicas */ if (REP(set, r)->remote == NULL || PART(REP(set, r), 0)->created == 1) continue; if (util_update_remote_header(set, r)) { LOG(1, "updating header of a remote replica no. %u failed", r); return -1; } } return 0; } /* * update_uuids -- (internal) set all uuids that might have changed or be unset * after recreating parts */ static int update_uuids(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; ++r) { if (!replica_is_replica_healthy(r, set_hs)) update_parts_linkage(set, r, set_hs); update_replicas_linkage(set, r); update_poolset_uuids(set, r, set_hs); } if (update_remote_headers(set)) return -1; return 0; } /* * remove_remote -- (internal) remove remote pool */ static int remove_remote(const char *target, const char *pool_set) { LOG(3, "target %s, pool_set %s", target, pool_set); #ifdef USE_RPMEM struct rpmem_target_info *info = rpmem_target_parse(target); if (!info) goto err_parse; struct rpmem_ssh *ssh = rpmem_ssh_exec(info, "--remove", pool_set, "--force", NULL); if (!ssh) { goto err_ssh_exec; } if (rpmem_ssh_monitor(ssh, 0)) goto err_ssh_monitor; int ret = rpmem_ssh_close(ssh); rpmem_target_free(info); return ret; err_ssh_monitor: rpmem_ssh_close(ssh); err_ssh_exec: rpmem_target_free(info); err_parse: return -1; #else FATAL("remote replication not supported"); return -1; #endif } /* * open_remote_replicas -- (internal) open all unbroken remote replicas */ static int open_remote_replicas(struct pool_set *set, struct poolset_health_status *set_hs) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; if (!rep->remote) continue; if (!replica_is_replica_healthy(r, set_hs)) continue; unsigned nlanes = REMOTE_NLANES; int ret = util_poolset_remote_replica_open(set, r, set->poolsize, 0, &nlanes); if (ret) { LOG(1, "Opening '%s' on '%s' failed", rep->remote->pool_desc, rep->remote->node_addr); return ret; } } return 0; } /* * create_remote_replicas -- (internal) recreate all broken replicas */ static int create_remote_replicas(struct pool_set *set, struct poolset_health_status *set_hs, unsigned flags) { LOG(3, "set %p, set_hs %p", set, set_hs); for (unsigned r = 0; r < set->nreplicas; r++) { struct pool_replica *rep = set->replica[r]; if (!rep->remote) continue; if (replica_is_replica_healthy(r, set_hs)) continue; if (!replica_is_poolset_transformed(flags)) { /* ignore errors from remove operation */ remove_remote(rep->remote->node_addr, rep->remote->pool_desc); } unsigned nlanes = REMOTE_NLANES; int ret = util_poolset_remote_replica_open(set, r, set->poolsize, 1, &nlanes); if (ret) { LOG(1, "Creating '%s' on '%s' failed", rep->remote->pool_desc, rep->remote->node_addr); return ret; } } return 0; } /* * sync_replica -- synchronize data across replicas within a poolset */ int replica_sync(struct pool_set *set, struct poolset_health_status *s_hs, unsigned flags) { LOG(3, "set %p, flags %u", set, flags); int ret = 0; struct poolset_health_status *set_hs = NULL; /* check if we already know the poolset health status */ if (s_hs == NULL) { /* validate poolset before checking its health */ if (validate_args(set)) return -1; /* examine poolset's health */ if (replica_check_poolset_health(set, &set_hs, 1 /* called from sync */, flags)) { LOG(1, "poolset health check failed"); return -1; } /* check if poolset is broken; if not, nothing to do */ if (replica_is_poolset_healthy(set_hs)) { LOG(1, "poolset is healthy"); goto out; } } else { set_hs = s_hs; } /* find a replica with healthy header; it will be the source of data */ unsigned healthy_replica = replica_find_healthy_replica(set_hs); unsigned healthy_header = healthy_replica; if (healthy_header == UNDEF_REPLICA) { healthy_header = replica_find_replica_healthy_header(set_hs); if (healthy_header == UNDEF_REPLICA) { ERR("no healthy replica found"); errno = EINVAL; ret = -1; goto out; } } /* in dry-run mode we can stop here */ if (is_dry_run(flags)) { LOG(1, "Sync in dry-run mode finished successfully"); goto out; } /* recreate broken parts */ if (recreate_broken_parts(set, set_hs, fix_bad_blocks(flags))) { ERR("recreating broken parts failed"); ret = -1; goto out; } /* open all part files */ if (replica_open_poolset_part_files(set)) { ERR("opening poolset part files failed"); ret = -1; goto out; } /* map all replicas */ if (util_poolset_open(set)) { ERR("opening poolset failed"); ret = -1; goto out; } /* this is required for opening remote pools */ set->poolsize = set_hs->replica[healthy_header]->pool_size; LOG(3, "setting the pool size (%zu) from replica #%u", set->poolsize, healthy_header); /* open all remote replicas */ if (open_remote_replicas(set, set_hs)) { ERR("opening remote replicas failed"); ret = -1; goto out; } /* recalculate offset and length of bad blocks */ if (sync_recalc_badblocks(set, set_hs)) { LOG(1, "syncing bad blocks data failed"); ret = -1; goto out; } /* * Check if there are uncorrectable bad blocks * (bad blocks overlapping in all replicas). */ int status = sync_check_bad_blocks_overlap(set, set_hs); if (status == -1) { LOG(1, "checking bad blocks failed"); ret = -1; goto out; } if (status == 1) { ERR( "a part of the pool has uncorrectable errors in all replicas"); errno = EINVAL; ret = -1; goto out; } LOG(3, "bad blocks do not overlap"); /* sync data in bad blocks */ if (sync_badblocks_data(set, set_hs)) { LOG(1, "syncing bad blocks data failed"); ret = -1; goto out; } /* find one good replica; it will be the source of data */ healthy_replica = replica_find_healthy_replica(set_hs); if (healthy_replica == UNDEF_REPLICA) { ERR("no healthy replica found"); errno = EINVAL; ret = -1; goto out; } /* update uuid fields in the set structure with part headers */ if (fill_struct_uuids(set, healthy_replica, set_hs, flags)) { ERR("gathering uuids failed"); ret = -1; goto out; } /* create headers for broken parts */ if (create_headers_for_broken_parts(set, healthy_replica, set_hs)) { ERR("creating headers for broken parts failed"); ret = -1; goto out; } /* create all remote replicas */ if (create_remote_replicas(set, set_hs, flags)) { ERR("creating remote replicas failed"); ret = -1; goto out; } /* check and copy data if possible */ if (copy_data_to_broken_parts(set, healthy_replica, flags, set_hs)) { ERR("copying data to broken parts failed"); ret = -1; goto out; } /* update uuids of replicas and parts */ if (update_uuids(set, set_hs)) { ERR("updating uuids failed"); ret = -1; goto out; } /* grant permissions to all created parts */ if (grant_created_parts_perm(set, healthy_replica, set_hs)) { ERR("granting permissions to created parts failed"); ret = -1; } out: if (s_hs == NULL) replica_free_poolset_health_status(set_hs); return ret; } pmdk-1.8/src/libpmempool/check_sds.c0000664000000000000000000001766013615011243016223 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_shutdown_state.c -- shutdown state check */ #include #include #include #include #include "out.h" #include "util_pmem.h" #include "libpmempool.h" #include "libpmem.h" #include "pmempool.h" #include "pool.h" #include "set.h" #include "check_util.h" enum question { Q_RESET_SDS, }; #define SDS_CHECK_STR "checking shutdown state" #define SDS_OK_STR "shutdown state correct" #define SDS_DIRTY_STR "shutdown state is dirty" #define ADR_FAILURE_STR \ "an ADR failure was detected - your pool might be corrupted" #define ZERO_SDS_STR \ "Do you want to zero shutdown state?" #define RESET_SDS_STR \ "Do you want to reset shutdown state at your own risk? " \ "If you have more then one replica you will have to " \ "synchronize your pool after this operation." #define SDS_FAIL_MSG(hdrp) \ IGNORE_SDS(hdrp) ? SDS_DIRTY_STR : ADR_FAILURE_STR #define SDS_REPAIR_MSG(hdrp) \ IGNORE_SDS(hdrp) \ ? SDS_DIRTY_STR ".|" ZERO_SDS_STR \ : ADR_FAILURE_STR ".|" RESET_SDS_STR /* * sds_check_replica -- (internal) check if replica is healthy */ static int sds_check_replica(location *loc) { LOG(3, NULL); struct pool_replica *rep = REP(loc->set, loc->replica); if (rep->remote) return 0; /* make a copy of sds as we shouldn't modify a pool */ struct shutdown_state old_sds = loc->hdr.sds; struct shutdown_state curr_sds; if (IGNORE_SDS(&loc->hdr)) return util_is_zeroed(&old_sds, sizeof(old_sds)) ? 0 : -1; shutdown_state_init(&curr_sds, NULL); /* get current shutdown state */ for (unsigned p = 0; p < rep->nparts; ++p) { if (shutdown_state_add_part(&curr_sds, PART(rep, p)->path, NULL)) return -1; } /* compare current and old shutdown state */ return shutdown_state_check(&curr_sds, &old_sds, NULL); } /* * sds_check -- (internal) check shutdown_state */ static int sds_check(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); CHECK_INFO(ppc, "%s" SDS_CHECK_STR, loc->prefix); /* shutdown state is valid */ if (!sds_check_replica(loc)) { CHECK_INFO(ppc, "%s" SDS_OK_STR, loc->prefix); loc->step = CHECK_STEP_COMPLETE; return 0; } /* shutdown state is NOT valid and can NOT be repaired */ if (CHECK_IS_NOT(ppc, REPAIR)) { check_end(ppc->data); ppc->result = CHECK_RESULT_NOT_CONSISTENT; return CHECK_ERR(ppc, "%s%s", loc->prefix, SDS_FAIL_MSG(&loc->hdr)); } /* shutdown state is NOT valid but can be repaired */ CHECK_ASK(ppc, Q_RESET_SDS, "%s%s", loc->prefix, SDS_REPAIR_MSG(&loc->hdr)); return check_questions_sequence_validate(ppc); } /* * sds_fix -- (internal) fix shutdown state */ static int sds_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *context) { LOG(3, NULL); switch (question) { case Q_RESET_SDS: CHECK_INFO(ppc, "%sresetting pool_hdr.sds", loc->prefix); memset(&loc->hdr.sds, 0, sizeof(loc->hdr.sds)); ++loc->healthy_replicas; break; default: ERR("not implemented question id: %u", question); } return 0; } struct step { int (*check)(PMEMpoolcheck *, location *); int (*fix)(PMEMpoolcheck *, location *, uint32_t, void *); }; static const struct step steps[] = { { .check = sds_check, }, { .fix = sds_fix, }, { .check = NULL, .fix = NULL, }, }; /* * step_exe -- (internal) perform single step according to its parameters */ static int step_exe(PMEMpoolcheck *ppc, const struct step *steps, location *loc) { const struct step *step = &steps[loc->step++]; if (!step->fix) return step->check(ppc, loc); if (!check_has_answer(ppc->data)) return 0; if (check_answer_loop(ppc, loc, NULL, 0 /* fail on no */, step->fix)) return -1; util_convert2le_hdr(&loc->hdr); memcpy(loc->hdrp, &loc->hdr, sizeof(loc->hdr)); util_persist_auto(loc->is_dev_dax, loc->hdrp, sizeof(*loc->hdrp)); util_convert2h_hdr_nocheck(&loc->hdr); loc->pool_hdr_modified = 1; return 0; } /* * init_prefix -- prepare prefix for messages */ static void init_prefix(location *loc) { if (loc->set->nreplicas > 1) { int ret = snprintf(loc->prefix, PREFIX_MAX_SIZE, "replica %u: ", loc->replica); if (ret < 0 || ret >= PREFIX_MAX_SIZE) FATAL("snprintf: %d", ret); } else loc->prefix[0] = '\0'; loc->step = 0; } /* * init_location_data -- (internal) prepare location information */ static void init_location_data(PMEMpoolcheck *ppc, location *loc) { ASSERTeq(loc->part, 0); loc->set = ppc->pool->set_file->poolset; if (ppc->result != CHECK_RESULT_PROCESS_ANSWERS) init_prefix(loc); struct pool_replica *rep = REP(loc->set, loc->replica); loc->hdrp = HDR(rep, loc->part); memcpy(&loc->hdr, loc->hdrp, sizeof(loc->hdr)); util_convert2h_hdr_nocheck(&loc->hdr); loc->is_dev_dax = PART(rep, 0)->is_dev_dax; } /* * sds_get_healthy_replicas_num -- (internal) get number of healthy replicas */ static void sds_get_healthy_replicas_num(PMEMpoolcheck *ppc, location *loc) { const unsigned nreplicas = ppc->pool->set_file->poolset->nreplicas; loc->healthy_replicas = 0; loc->part = 0; for (; loc->replica < nreplicas; loc->replica++) { init_location_data(ppc, loc); if (!sds_check_replica(loc)) { ++loc->healthy_replicas; /* healthy replica found */ } } loc->replica = 0; /* reset replica index */ } /* * check_sds -- entry point for shutdown state checks */ void check_sds(PMEMpoolcheck *ppc) { LOG(3, NULL); const unsigned nreplicas = ppc->pool->set_file->poolset->nreplicas; location *loc = check_get_step_data(ppc->data); if (!loc->init_done) { sds_get_healthy_replicas_num(ppc, loc); if (loc->healthy_replicas == nreplicas) { /* all replicas have healthy shutdown state */ /* print summary */ for (; loc->replica < nreplicas; loc->replica++) { init_prefix(loc); CHECK_INFO(ppc, "%s" SDS_CHECK_STR, loc->prefix); CHECK_INFO(ppc, "%s" SDS_OK_STR, loc->prefix); } return; } else if (loc->healthy_replicas > 0) { ppc->sync_required = true; return; } loc->init_done = true; } /* produce single healthy replica */ loc->part = 0; for (; loc->replica < nreplicas; loc->replica++) { init_location_data(ppc, loc); while (CHECK_NOT_COMPLETE(loc, steps)) { ASSERT(loc->step < ARRAY_SIZE(steps)); if (step_exe(ppc, steps, loc)) return; } if (loc->healthy_replicas) break; } if (loc->healthy_replicas == 0) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; CHECK_ERR(ppc, "cannot complete repair, reverting changes"); } else if (loc->healthy_replicas < nreplicas) { ppc->sync_required = true; } } pmdk-1.8/src/libpmempool/transform.c0000664000000000000000000007103713615011243016306 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * transform.c -- a module for poolset transforming */ #include #include #include #include #include #include #include #include #include #include "replica.h" #include "out.h" #include "file.h" #include "os.h" #include "libpmem.h" #include "util_pmem.h" /* * poolset_compare_status - a helping structure for gathering corresponding * replica numbers when comparing poolsets */ struct poolset_compare_status { unsigned nreplicas; unsigned flags; unsigned replica[]; }; /* * type of transform operation to be done */ enum transform_op { NOT_TRANSFORMABLE, ADD_REPLICAS, RM_REPLICAS, ADD_HDRS, RM_HDRS, }; /* * check_if_part_used_once -- (internal) check if the part is used only once in * the rest of the poolset */ static int check_if_part_used_once(struct pool_set *set, unsigned repn, unsigned partn) { LOG(3, "set %p, repn %u, partn %u", set, repn, partn); struct pool_replica *rep = REP(set, repn); char *path = util_part_realpath(PART(rep, partn)->path); if (path == NULL) { LOG(1, "cannot get absolute path for %s, replica %u, part %u", PART(rep, partn)->path, repn, partn); errno = 0; path = strdup(PART(rep, partn)->path); if (path == NULL) { ERR("!strdup"); return -1; } } int ret = 0; for (unsigned r = repn; r < set->nreplicas; ++r) { struct pool_replica *repr = set->replica[r]; /* skip remote replicas */ if (repr->remote != NULL) continue; /* avoid superfluous comparisons */ unsigned i = (r == repn) ? partn + 1 : 0; for (unsigned p = i; p < repr->nparts; ++p) { char *pathp = util_part_realpath(PART(repr, p)->path); if (pathp == NULL) { if (errno != ENOENT) { ERR("realpath failed for %s, errno %d", PART(repr, p)->path, errno); ret = -1; goto out; } LOG(1, "cannot get absolute path for %s," " replica %u, part %u", PART(rep, partn)->path, repn, partn); pathp = strdup(PART(repr, p)->path); errno = 0; } int result = util_compare_file_inodes(path, pathp); if (result == 0) { /* same file used multiple times */ ERR("some part file's path is" " used multiple times"); ret = -1; errno = EINVAL; free(pathp); goto out; } else if (result < 0) { ERR("comparing file inodes failed for %s and" " %s", path, pathp); ret = -1; free(pathp); goto out; } free(pathp); } } out: free(path); return ret; } /* * check_if_remote_replica_used_once -- (internal) check if remote replica is * used only once in the rest of the * poolset */ static int check_if_remote_replica_used_once(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); struct remote_replica *rep = REP(set, repn)->remote; ASSERTne(rep, NULL); for (unsigned r = repn + 1; r < set->nreplicas; ++r) { /* skip local replicas */ if (REP(set, r)->remote == NULL) continue; struct remote_replica *repr = REP(set, r)->remote; /* XXX: add comparing resolved addresses of the nodes */ if (strcmp(rep->node_addr, repr->node_addr) == 0 && strcmp(rep->pool_desc, repr->pool_desc) == 0) { ERR("remote replica %u is used multiple times", repn); errno = EINVAL; return -1; } } return 0; } /* * check_paths -- (internal) check if directories for part files exist * and if paths for part files do not repeat in the poolset */ static int check_paths(struct pool_set *set) { LOG(3, "set %p", set); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = set->replica[r]; if (rep->remote != NULL) { if (check_if_remote_replica_used_once(set, r)) return -1; } else { for (unsigned p = 0; p < rep->nparts; ++p) { if (replica_check_local_part_dir(set, r, p)) return -1; if (check_if_part_used_once(set, r, p)) return -1; } } } return 0; } /* * validate_args -- (internal) check whether passed arguments are valid */ static int validate_args(struct pool_set *set_in, struct pool_set *set_out) { LOG(3, "set_in %p, set_out %p", set_in, set_out); if (set_in->directory_based) { ERR("transform of directory poolsets is not supported"); errno = EINVAL; return -1; } /* * check if all parts in the target poolset are large enough * (now replication works only for pmemobj pools) */ if (replica_check_part_sizes(set_out, PMEMOBJ_MIN_POOL)) { ERR("part sizes check failed"); return -1; } /* * check if all directories for part files exist and if part files * do not reoccur in the poolset */ if (check_paths(set_out)) return -1; /* * check if set_out has enough size, i.e. if the target poolset * structure has enough capacity to accommodate the effective size of * the source poolset */ ssize_t master_pool_size = replica_get_pool_size(set_in, 0); if (master_pool_size < 0) { ERR("getting pool size from master replica failed"); return -1; } if (set_out->poolsize < (size_t)master_pool_size) { ERR("target poolset is too small"); errno = EINVAL; return -1; } return 0; } /* * create poolset_compare_status -- (internal) create structure for gathering * status of poolset comparison */ static int create_poolset_compare_status(struct pool_set *set, struct poolset_compare_status **set_sp) { LOG(3, "set %p, set_sp %p", set, set_sp); struct poolset_compare_status *set_s; set_s = Zalloc(sizeof(struct poolset_compare_status) + set->nreplicas * sizeof(unsigned)); if (set_s == NULL) { ERR("!Zalloc for poolset status"); return -1; } for (unsigned r = 0; r < set->nreplicas; ++r) set_s->replica[r] = UNDEF_REPLICA; set_s->nreplicas = set->nreplicas; *set_sp = set_s; return 0; } /* * compare_parts -- (internal) check if two parts can be considered the same */ static int compare_parts(struct pool_set_part *p1, struct pool_set_part *p2) { LOG(3, "p1 %p, p2 %p", p1, p2); LOG(4, "p1->path: %s, p1->filesize: %lu", p1->path, p1->filesize); LOG(4, "p2->path: %s, p2->filesize: %lu", p2->path, p2->filesize); return strcmp(p1->path, p2->path) || (p1->filesize != p2->filesize); } /* * compare_replicas -- (internal) check if two replicas are different */ static int compare_replicas(struct pool_replica *r1, struct pool_replica *r2) { LOG(3, "r1 %p, r2 %p", r1, r2); LOG(4, "r1->nparts: %u, r2->nparts: %u", r1->nparts, r2->nparts); /* both replicas are local */ if (r1->remote == NULL && r2->remote == NULL) { if (r1->nparts != r2->nparts) return 1; for (unsigned p = 0; p < r1->nparts; ++p) { if (compare_parts(&r1->part[p], &r2->part[p])) return 1; } return 0; } /* both replicas are remote */ if (r1->remote != NULL && r2->remote != NULL) { return strcmp(r1->remote->node_addr, r2->remote->node_addr) || strcmp(r1->remote->pool_desc, r2->remote->pool_desc); } /* a remote and a local replicas */ return 1; } /* * check_compare_poolsets_status -- (internal) find different replicas between * two poolsets; for each replica which has * a counterpart in the other poolset store * the other replica's number in a helping * structure */ static int check_compare_poolsets_status(struct pool_set *set_in, struct pool_set *set_out, struct poolset_compare_status *set_in_s, struct poolset_compare_status *set_out_s) { LOG(3, "set_in %p, set_out %p, set_in_s %p, set_out_s %p", set_in, set_out, set_in_s, set_out_s); for (unsigned ri = 0; ri < set_in->nreplicas; ++ri) { struct pool_replica *rep_in = REP(set_in, ri); for (unsigned ro = 0; ro < set_out->nreplicas; ++ro) { struct pool_replica *rep_out = REP(set_out, ro); LOG(1, "comparing rep_in %u with rep_out %u", ri, ro); /* skip different replicas */ if (compare_replicas(rep_in, rep_out)) continue; if (set_in_s->replica[ri] != UNDEF_REPLICA || set_out_s->replica[ro] != UNDEF_REPLICA) { /* there are more than one counterparts */ ERR("there are more then one corresponding" " replicas; cannot transform"); errno = EINVAL; return -1; } set_in_s->replica[ri] = ro; set_out_s->replica[ro] = ri; } } return 0; } /* * check_compare_poolset_options -- (internal) check poolset options */ static int check_compare_poolsets_options(struct pool_set *set_in, struct pool_set *set_out, struct poolset_compare_status *set_in_s, struct poolset_compare_status *set_out_s) { if (set_in->options & OPTION_SINGLEHDR) set_in_s->flags |= OPTION_SINGLEHDR; if (set_out->options & OPTION_SINGLEHDR) set_out_s->flags |= OPTION_SINGLEHDR; if ((set_in->options & OPTION_NOHDRS) || (set_out->options & OPTION_NOHDRS)) { errno = EINVAL; ERR( "the NOHDRS poolset option is not supported in local poolset files"); return -1; } return 0; } /* * compare_poolsets -- (internal) compare two poolsets; for each replica which * has a counterpart in the other poolset store the other * replica's number in a helping structure */ static int compare_poolsets(struct pool_set *set_in, struct pool_set *set_out, struct poolset_compare_status **set_in_s, struct poolset_compare_status **set_out_s) { LOG(3, "set_in %p, set_out %p, set_in_s %p, set_out_s %p", set_in, set_out, set_in_s, set_out_s); if (create_poolset_compare_status(set_in, set_in_s)) return -1; if (create_poolset_compare_status(set_out, set_out_s)) goto err_free_in; if (check_compare_poolsets_status(set_in, set_out, *set_in_s, *set_out_s)) goto err_free_out; if (check_compare_poolsets_options(set_in, set_out, *set_in_s, *set_out_s)) goto err_free_out; return 0; err_free_out: Free(*set_out_s); err_free_in: Free(*set_in_s); return -1; } /* * replica_counterpart -- (internal) returns index of a counterpart replica */ static unsigned replica_counterpart(unsigned repn, struct poolset_compare_status *set_s) { return set_s->replica[repn]; } /* * are_poolsets_transformable -- (internal) check if poolsets can be transformed * one into the other; also gather info about * replicas's health */ static enum transform_op identify_transform_operation(struct poolset_compare_status *set_in_s, struct poolset_compare_status *set_out_s, struct poolset_health_status *set_in_hs, struct poolset_health_status *set_out_hs) { LOG(3, "set_in_s %p, set_out_s %p", set_in_s, set_out_s); int has_replica_to_keep = 0; int is_removing_replicas = 0; int is_adding_replicas = 0; /* check if there are replicas to be removed */ for (unsigned r = 0; r < set_in_s->nreplicas; ++r) { unsigned c = replica_counterpart(r, set_in_s); if (c != UNDEF_REPLICA) { LOG(2, "replica %u has a counterpart %u", r, set_in_s->replica[r]); has_replica_to_keep = 1; REP_HEALTH(set_out_hs, c)->pool_size = REP_HEALTH(set_in_hs, r)->pool_size; } else { LOG(2, "replica %u has no counterpart", r); is_removing_replicas = 1; } } /* make sure we have at least one replica to keep */ if (!has_replica_to_keep) { ERR("there must be at least one replica left"); return NOT_TRANSFORMABLE; } /* check if there are replicas to be added */ for (unsigned r = 0; r < set_out_s->nreplicas; ++r) { if (replica_counterpart(r, set_out_s) == UNDEF_REPLICA) { LOG(2, "Replica %u from output set has no counterpart", r); if (is_removing_replicas) { ERR( "adding and removing replicas at the same time is not allowed"); return NOT_TRANSFORMABLE; } REP_HEALTH(set_out_hs, r)->flags |= IS_BROKEN; is_adding_replicas = 1; } } /* check if there is anything to do */ if (!is_removing_replicas && !is_adding_replicas && (set_in_s->flags & OPTION_SINGLEHDR) == (set_out_s->flags & OPTION_SINGLEHDR)) { ERR("both poolsets are equal"); return NOT_TRANSFORMABLE; } /* allow changing the SINGLEHDR option only as the sole operation */ if ((is_removing_replicas || is_adding_replicas) && (set_in_s->flags & OPTION_SINGLEHDR) != (set_out_s->flags & OPTION_SINGLEHDR)) { ERR( "cannot add/remove replicas and change the SINGLEHDR option at the same time"); return NOT_TRANSFORMABLE; } if (is_removing_replicas) return RM_REPLICAS; if (is_adding_replicas) return ADD_REPLICAS; if (set_out_s->flags & OPTION_SINGLEHDR) return RM_HDRS; if (set_in_s->flags & OPTION_SINGLEHDR) return ADD_HDRS; ASSERT(0); return NOT_TRANSFORMABLE; } /* * do_added_parts_exist -- (internal) check if any part of the replicas that are * to be added (marked as broken) already exists */ static int do_added_parts_exist(struct pool_set *set, struct poolset_health_status *set_hs) { for (unsigned r = 0; r < set->nreplicas; ++r) { /* skip unbroken (i.e. not being added) replicas */ if (!replica_is_replica_broken(r, set_hs)) continue; struct pool_replica *rep = REP(set, r); /* skip remote replicas */ if (rep->remote) continue; for (unsigned p = 0; p < rep->nparts; ++p) { /* check if part file exists */ int oerrno = errno; int exists = util_file_exists(rep->part[p].path); if (exists < 0) return -1; if (exists && !rep->part[p].is_dev_dax) { LOG(1, "part file %s exists", rep->part[p].path); return 1; } errno = oerrno; } } return 0; } /* * delete_replicas -- (internal) delete replicas which do not have their * counterpart set in the helping status structure */ static int delete_replicas(struct pool_set *set, struct poolset_compare_status *set_s) { LOG(3, "set %p, set_s %p", set, set_s); for (unsigned r = 0; r < set->nreplicas; ++r) { struct pool_replica *rep = REP(set, r); if (replica_counterpart(r, set_s) == UNDEF_REPLICA) { if (!rep->remote) { if (util_replica_close_local(rep, r, DELETE_ALL_PARTS)) return -1; } else { if (util_replica_close_remote(rep, r, DELETE_ALL_PARTS)) return -1; } } } return 0; } /* * copy_replica_data_fw -- (internal) copy data between replicas of two * poolsets, starting from the beginning of the * second part */ static void copy_replica_data_fw(struct pool_set *set_dst, struct pool_set *set_src, unsigned repn) { LOG(3, "set_in %p, set_out %p, repn %u", set_src, set_dst, repn); ssize_t pool_size = replica_get_pool_size(set_src, repn); if (pool_size < 0) { LOG(1, "getting pool size from replica %u failed", repn); pool_size = (ssize_t)set_src->poolsize; } size_t len = (size_t)pool_size - POOL_HDR_SIZE - replica_get_part_data_len(set_src, repn, 0); void *src = PART(REP(set_src, repn), 1)->addr; void *dst = PART(REP(set_dst, repn), 1)->addr; size_t count = len / POOL_HDR_SIZE; while (count-- > 0) { pmem_memcpy_persist(dst, src, POOL_HDR_SIZE); src = ADDR_SUM(src, POOL_HDR_SIZE); dst = ADDR_SUM(dst, POOL_HDR_SIZE); } } /* * copy_replica_data_bw -- (internal) copy data between replicas of two * poolsets, starting from the end of the pool */ static void copy_replica_data_bw(struct pool_set *set_dst, struct pool_set *set_src, unsigned repn) { LOG(3, "set_in %p, set_out %p, repn %u", set_src, set_dst, repn); ssize_t pool_size = replica_get_pool_size(set_src, repn); if (pool_size < 0) { LOG(1, "getting pool size from replica %u failed", repn); pool_size = (ssize_t)set_src->poolsize; } size_t len = (size_t)pool_size - POOL_HDR_SIZE - replica_get_part_data_len(set_src, repn, 0); size_t count = len / POOL_HDR_SIZE; void *src = ADDR_SUM(PART(REP(set_src, repn), 1)->addr, len); void *dst = ADDR_SUM(PART(REP(set_dst, repn), 1)->addr, len); while (count-- > 0) { src = ADDR_SUM(src, -(ssize_t)POOL_HDR_SIZE); dst = ADDR_SUM(dst, -(ssize_t)POOL_HDR_SIZE); pmem_memcpy_persist(dst, src, POOL_HDR_SIZE); } } /* * create_missing_headers -- (internal) create headers for all parts but the * first one */ static int create_missing_headers(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); struct pool_hdr *src_hdr = HDR(REP(set, repn), 0); for (unsigned p = 1; p < set->replica[repn]->nhdrs; ++p) { struct pool_attr attr; util_pool_hdr2attr(&attr, src_hdr); attr.features.incompat &= (uint32_t)(~POOL_FEAT_SINGLEHDR); if (util_header_create(set, repn, p, &attr, 1) != 0) { LOG(1, "part headers create failed for" " replica %u part %u", repn, p); errno = EINVAL; return -1; } } return 0; } /* * update_replica_header -- (internal) update field values in the first header * in the replica */ static void update_replica_header(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); struct pool_replica *rep = REP(set, repn); struct pool_set_part *part = PART(REP(set, repn), 0); struct pool_hdr *hdr = (struct pool_hdr *)part->hdr; if (set->options & OPTION_SINGLEHDR) { hdr->features.incompat |= POOL_FEAT_SINGLEHDR; memcpy(hdr->next_part_uuid, hdr->uuid, POOL_HDR_UUID_LEN); memcpy(hdr->prev_part_uuid, hdr->uuid, POOL_HDR_UUID_LEN); } else { hdr->features.incompat &= (uint32_t)(~POOL_FEAT_SINGLEHDR); } util_checksum(hdr, sizeof(*hdr), &hdr->checksum, 1, POOL_HDR_CSUM_END_OFF(hdr)); util_persist_auto(rep->is_pmem, hdr, sizeof(*hdr)); } /* * fill_replica_struct_uuids -- (internal) gather all uuids required for the * replica in the helper structure */ static int fill_replica_struct_uuids(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); struct pool_replica *rep = REP(set, repn); memcpy(PART(rep, 0)->uuid, HDR(rep, 0)->uuid, POOL_HDR_UUID_LEN); for (unsigned p = 1; p < rep->nhdrs; ++p) { if (util_uuid_generate(rep->part[p].uuid) < 0) { ERR("cannot generate part UUID"); errno = EINVAL; return -1; } } return 0; } /* * update_uuids -- (internal) update uuids in all headers in the replica */ static void update_uuids(struct pool_set *set, unsigned repn) { LOG(3, "set %p, repn %u", set, repn); struct pool_replica *rep = REP(set, repn); struct pool_hdr *hdr0 = HDR(rep, 0); for (unsigned p = 0; p < rep->nhdrs; ++p) { struct pool_hdr *hdrp = HDR(rep, p); memcpy(hdrp->next_part_uuid, PARTN(rep, p)->uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->prev_part_uuid, PARTP(rep, p)->uuid, POOL_HDR_UUID_LEN); /* Avoid calling memcpy() on identical regions */ if (p != 0) { memcpy(hdrp->next_repl_uuid, hdr0->next_repl_uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->prev_repl_uuid, hdr0->prev_repl_uuid, POOL_HDR_UUID_LEN); memcpy(hdrp->poolset_uuid, hdr0->poolset_uuid, POOL_HDR_UUID_LEN); } util_checksum(hdrp, sizeof(*hdrp), &hdrp->checksum, 1, POOL_HDR_CSUM_END_OFF(hdrp)); util_persist(PART(rep, p)->is_dev_dax, hdrp, sizeof(*hdrp)); } } /* * copy_part_fds -- (internal) copy poolset part file descriptors between * two poolsets */ static void copy_part_fds(struct pool_set *set_dst, struct pool_set *set_src) { ASSERTeq(set_src->nreplicas, set_dst->nreplicas); for (unsigned r = 0; r < set_dst->nreplicas; ++r) { ASSERTeq(REP(set_src, r)->nparts, REP(set_dst, r)->nparts); for (unsigned p = 0; p < REP(set_dst, r)->nparts; ++p) { PART(REP(set_dst, r), p)->fd = PART(REP(set_src, r), p)->fd; } } } /* * remove_hdrs_replica -- (internal) remove headers from the replica */ static int remove_hdrs_replica(struct pool_set *set_in, struct pool_set *set_out, unsigned repn) { LOG(3, "set %p, repn %u", set_in, repn); int ret = 0; /* open all part files of the input replica */ if (replica_open_replica_part_files(set_in, repn)) { LOG(1, "opening replica %u, part files failed", repn); ret = -1; goto out; } /* share part file descriptors between poolset structures */ copy_part_fds(set_out, set_in); /* map the whole input replica */ if (util_replica_open(set_in, repn, MAP_SHARED)) { LOG(1, "opening input replica failed: replica %u", repn); ret = -1; goto out_close; } /* map the whole output replica */ if (util_replica_open(set_out, repn, MAP_SHARED)) { LOG(1, "opening output replica failed: replica %u", repn); ret = -1; goto out_unmap_in; } /* move data between the two mappings of the replica */ if (REP(set_in, repn)->nparts > 1) copy_replica_data_fw(set_out, set_in, repn); /* make changes to the first part's header */ update_replica_header(set_out, repn); util_replica_close(set_out, repn); out_unmap_in: util_replica_close(set_in, repn); out_close: util_replica_fdclose(REP(set_in, repn)); out: return ret; } /* * add_hdrs_replica -- (internal) add lacking headers to the replica * * when the operation fails and returns -1, the replica remains untouched */ static int add_hdrs_replica(struct pool_set *set_in, struct pool_set *set_out, unsigned repn) { LOG(3, "set %p, repn %u", set_in, repn); int ret = 0; /* open all part files of the input replica */ if (replica_open_replica_part_files(set_in, repn)) { LOG(1, "opening replica %u, part files failed", repn); ret = -1; goto out; } /* share part file descriptors between poolset structures */ copy_part_fds(set_out, set_in); /* map the whole input replica */ if (util_replica_open(set_in, repn, MAP_SHARED)) { LOG(1, "opening input replica failed: replica %u", repn); ret = -1; goto out_close; } /* map the whole output replica */ if (util_replica_open(set_out, repn, MAP_SHARED)) { LOG(1, "opening output replica failed: replica %u", repn); ret = -1; goto out_unmap_in; } /* generate new uuids for lacking headers */ if (fill_replica_struct_uuids(set_out, repn)) { LOG(1, "generating lacking uuids for parts failed: replica %u", repn); ret = -1; goto out_unmap_out; } /* copy data between the two mappings of the replica */ if (REP(set_in, repn)->nparts > 1) copy_replica_data_bw(set_out, set_in, repn); /* create the missing headers */ if (create_missing_headers(set_out, repn)) { LOG(1, "creating lacking headers failed: replica %u", repn); /* * copy the data back, so we could fall back to the original * state */ if (REP(set_in, repn)->nparts > 1) copy_replica_data_fw(set_in, set_out, repn); ret = -1; goto out_unmap_out; } /* make changes to the first part's header */ update_replica_header(set_out, repn); /* store new uuids in all headers and update linkage in the replica */ update_uuids(set_out, repn); out_unmap_out: util_replica_close(set_out, repn); out_unmap_in: util_replica_close(set_in, repn); out_close: util_replica_fdclose(REP(set_in, repn)); out: return ret; } /* * remove_hdrs -- (internal) transform a poolset without the SINGLEHDR option * (with headers) into a poolset with the SINGLEHDR option * (without headers) */ static int remove_hdrs(struct pool_set *set_in, struct pool_set *set_out, struct poolset_health_status *set_in_hs, unsigned flags) { LOG(3, "set_in %p, set_out %p, set_in_hs %p, flags %u", set_in, set_out, set_in_hs, flags); for (unsigned r = 0; r < set_in->nreplicas; ++r) { if (remove_hdrs_replica(set_in, set_out, r)) { LOG(1, "removing headers from replica %u failed", r); /* mark all previous replicas as damaged */ while (--r < set_in->nreplicas) REP_HEALTH(set_in_hs, r)->flags |= IS_BROKEN; return -1; } } return 0; } /* * add_hdrs -- (internal) transform a poolset with the SINGLEHDR option (without * headers) into a poolset without the SINGLEHDR option (with * headers) */ static int add_hdrs(struct pool_set *set_in, struct pool_set *set_out, struct poolset_health_status *set_in_hs, unsigned flags) { LOG(3, "set_in %p, set_out %p, set_in_hs %p, flags %u", set_in, set_out, set_in_hs, flags); for (unsigned r = 0; r < set_in->nreplicas; ++r) { if (add_hdrs_replica(set_in, set_out, r)) { LOG(1, "adding headers to replica %u failed", r); /* mark all previous replicas as damaged */ while (--r < set_in->nreplicas) REP_HEALTH(set_in_hs, r)->flags |= IS_BROKEN; return -1; } } return 0; } /* * transform_replica -- transforming one poolset into another */ int replica_transform(struct pool_set *set_in, struct pool_set *set_out, unsigned flags) { LOG(3, "set_in %p, set_out %p", set_in, set_out); int ret = 0; /* validate user arguments */ if (validate_args(set_in, set_out)) return -1; /* check if the source poolset is healthy */ struct poolset_health_status *set_in_hs = NULL; if (replica_check_poolset_health(set_in, &set_in_hs, 0 /* called from transform */, flags)) { ERR("source poolset health check failed"); return -1; } if (!replica_is_poolset_healthy(set_in_hs)) { ERR("source poolset is broken"); ret = -1; errno = EINVAL; goto free_hs_in; } /* copy value of the ignore_sds flag from the input poolset */ set_out->ignore_sds = set_in->ignore_sds; struct poolset_health_status *set_out_hs = NULL; if (replica_create_poolset_health_status(set_out, &set_out_hs)) { ERR("creating poolset health status failed"); ret = -1; goto free_hs_in; } /* check if the poolsets are transformable */ struct poolset_compare_status *set_in_cs = NULL; struct poolset_compare_status *set_out_cs = NULL; if (compare_poolsets(set_in, set_out, &set_in_cs, &set_out_cs)) { ERR("comparing poolsets failed"); ret = -1; goto free_hs_out; } enum transform_op operation = identify_transform_operation(set_in_cs, set_out_cs, set_in_hs, set_out_hs); if (operation == NOT_TRANSFORMABLE) { LOG(1, "poolsets are not transformable"); ret = -1; errno = EINVAL; goto free_cs; } if (operation == RM_HDRS) { if (!is_dry_run(flags) && remove_hdrs(set_in, set_out, set_in_hs, flags)) { ERR("removing headers failed; falling back to the " "input poolset"); if (replica_sync(set_in, set_in_hs, flags | IS_TRANSFORMED)) { LOG(1, "falling back to the input poolset " "failed"); } else { LOG(1, "falling back to the input poolset " "succeeded"); } ret = -1; } goto free_cs; } if (operation == ADD_HDRS) { if (!is_dry_run(flags) && add_hdrs(set_in, set_out, set_in_hs, flags)) { ERR("adding headers failed; falling back to the " "input poolset"); if (replica_sync(set_in, set_in_hs, flags | IS_TRANSFORMED)) { LOG(1, "falling back to the input poolset " "failed"); } else { LOG(1, "falling back to the input poolset " "succeeded"); } ret = -1; } goto free_cs; } if (operation == ADD_REPLICAS) { /* * check if any of the parts that are to be added already exists */ if (do_added_parts_exist(set_out, set_out_hs)) { ERR("some parts being added already exist"); ret = -1; errno = EINVAL; goto free_cs; } } /* signal that sync is called by transform */ if (replica_sync(set_out, set_out_hs, flags | IS_TRANSFORMED)) { ret = -1; goto free_cs; } if (operation == RM_REPLICAS) { if (!is_dry_run(flags) && delete_replicas(set_in, set_in_cs)) ret = -1; } free_cs: Free(set_in_cs); Free(set_out_cs); free_hs_out: replica_free_poolset_health_status(set_out_hs); free_hs_in: replica_free_poolset_health_status(set_in_hs); return ret; } pmdk-1.8/src/libpmempool/check_util.c0000664000000000000000000004137213615011243016404 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_util.c -- check utility functions */ #include #include #include "out.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check_util.h" #define CHECK_END UINT_MAX /* separate info part of message from question part of message */ #define MSG_SEPARATOR '|' /* error part of message must have '.' at the end */ #define MSG_PLACE_OF_SEPARATION '.' #define MAX_MSG_STR_SIZE 8192 #define CHECK_ANSWER_YES "yes" #define CHECK_ANSWER_NO "no" #define STR_MAX 256 #define TIME_STR_FMT "%a %b %d %Y %H:%M:%S" #define UUID_STR_MAX 37 enum check_answer { PMEMPOOL_CHECK_ANSWER_EMPTY, PMEMPOOL_CHECK_ANSWER_YES, PMEMPOOL_CHECK_ANSWER_NO, PMEMPOOL_CHECK_ANSWER_DEFAULT, }; /* queue of check statuses */ struct check_status { PMDK_TAILQ_ENTRY(check_status) next; struct pmempool_check_status status; unsigned question; enum check_answer answer; char *msg; }; PMDK_TAILQ_HEAD(check_status_head, check_status); /* check control context */ struct check_data { unsigned step; location step_data; struct check_status *error; struct check_status_head infos; struct check_status_head questions; struct check_status_head answers; struct check_status *check_status_cache; }; /* * check_data_alloc -- allocate and initialize check_data structure */ struct check_data * check_data_alloc(void) { LOG(3, NULL); struct check_data *data = calloc(1, sizeof(*data)); if (data == NULL) { ERR("!calloc"); return NULL; } PMDK_TAILQ_INIT(&data->infos); PMDK_TAILQ_INIT(&data->questions); PMDK_TAILQ_INIT(&data->answers); return data; } /* * check_data_free -- clean and deallocate check_data */ void check_data_free(struct check_data *data) { LOG(3, NULL); if (data->error != NULL) { free(data->error); data->error = NULL; } if (data->check_status_cache != NULL) { free(data->check_status_cache); data->check_status_cache = NULL; } while (!PMDK_TAILQ_EMPTY(&data->infos)) { struct check_status *statp = PMDK_TAILQ_FIRST(&data->infos); PMDK_TAILQ_REMOVE(&data->infos, statp, next); free(statp); } while (!PMDK_TAILQ_EMPTY(&data->questions)) { struct check_status *statp = PMDK_TAILQ_FIRST(&data->questions); PMDK_TAILQ_REMOVE(&data->questions, statp, next); free(statp); } while (!PMDK_TAILQ_EMPTY(&data->answers)) { struct check_status *statp = PMDK_TAILQ_FIRST(&data->answers); PMDK_TAILQ_REMOVE(&data->answers, statp, next); free(statp); } free(data); } /* * check_step_get - return current check step number */ uint32_t check_step_get(struct check_data *data) { return data->step; } /* * check_step_inc -- move to next step number */ void check_step_inc(struct check_data *data) { if (check_is_end_util(data)) return; ++data->step; memset(&data->step_data, 0, sizeof(location)); } /* * check_get_step_data -- return pointer to check step data */ location * check_get_step_data(struct check_data *data) { return &data->step_data; } /* * check_end -- mark check as ended */ void check_end(struct check_data *data) { LOG(3, NULL); data->step = CHECK_END; } /* * check_is_end_util -- return if check has ended */ int check_is_end_util(struct check_data *data) { return data->step == CHECK_END; } /* * status_alloc -- (internal) allocate and initialize check_status */ static inline struct check_status * status_alloc(void) { struct check_status *status = malloc(sizeof(*status)); if (!status) FATAL("!malloc"); status->msg = malloc(sizeof(char) * MAX_MSG_STR_SIZE); if (!status->msg) { free(status); FATAL("!malloc"); } status->status.str.msg = status->msg; status->answer = PMEMPOOL_CHECK_ANSWER_EMPTY; status->question = CHECK_INVALID_QUESTION; return status; } /* * status_release -- (internal) release check_status */ static void status_release(struct check_status *status) { #ifdef _WIN32 /* dealloc duplicate string after conversion */ if (status->status.str.msg != status->msg) free((void *)status->status.str.msg); #endif free(status->msg); free(status); } /* * status_msg_info_only -- (internal) separate info part of the message * * If message is in form of "info.|question" it modifies it as follows * "info\0|question" */ static inline int status_msg_info_only(const char *msg) { char *sep = strchr(msg, MSG_SEPARATOR); if (sep) { ASSERTne(sep, msg); --sep; ASSERTeq(*sep, MSG_PLACE_OF_SEPARATION); *sep = '\0'; return 0; } return -1; } /* * status_msg_info_and_question -- (internal) join info and question * * If message is in form "info.|question" it will replace MSG_SEPARATOR '|' with * space to get "info. question" */ static inline int status_msg_info_and_question(const char *msg) { char *sep = strchr(msg, MSG_SEPARATOR); if (sep) { *sep = ' '; return 0; } return -1; } /* * status_push -- (internal) push single status object */ static int status_push(PMEMpoolcheck *ppc, struct check_status *st, uint32_t question) { if (st->status.type == PMEMPOOL_CHECK_MSG_TYPE_ERROR) { ASSERTeq(ppc->data->error, NULL); ppc->data->error = st; return -1; } else if (st->status.type == PMEMPOOL_CHECK_MSG_TYPE_INFO) { if (CHECK_IS(ppc, VERBOSE)) PMDK_TAILQ_INSERT_TAIL(&ppc->data->infos, st, next); else check_status_release(ppc, st); return 0; } /* st->status.type == PMEMPOOL_CHECK_MSG_TYPE_QUESTION */ if (CHECK_IS_NOT(ppc, REPAIR)) { /* error status */ if (status_msg_info_only(st->msg)) { ERR("no error message for the user"); st->msg[0] = '\0'; } st->status.type = PMEMPOOL_CHECK_MSG_TYPE_ERROR; return status_push(ppc, st, question); } if (CHECK_IS(ppc, ALWAYS_YES)) { if (!status_msg_info_only(st->msg)) { /* information status */ st->status.type = PMEMPOOL_CHECK_MSG_TYPE_INFO; status_push(ppc, st, question); st = status_alloc(); } /* answer status */ ppc->result = CHECK_RESULT_PROCESS_ANSWERS; st->question = question; st->answer = PMEMPOOL_CHECK_ANSWER_YES; st->status.type = PMEMPOOL_CHECK_MSG_TYPE_QUESTION; PMDK_TAILQ_INSERT_TAIL(&ppc->data->answers, st, next); } else { /* question message */ status_msg_info_and_question(st->msg); st->question = question; ppc->result = CHECK_RESULT_ASK_QUESTIONS; st->answer = PMEMPOOL_CHECK_ANSWER_EMPTY; PMDK_TAILQ_INSERT_TAIL(&ppc->data->questions, st, next); } return 0; } /* * check_status_create -- create single status, push it to proper queue * * MSG_SEPARATOR character in fmt is treated as message separator. If creating * question but check arguments do not allow to make any changes (asking any * question is pointless) it takes part of message before MSG_SEPARATOR * character and use it to create error message. Character just before separator * must be a MSG_PLACE_OF_SEPARATION character. Return non 0 value if error * status would be created. * * The arg is an additional argument for specified type of status. */ int check_status_create(PMEMpoolcheck *ppc, enum pmempool_check_msg_type type, uint32_t arg, const char *fmt, ...) { if (CHECK_IS_NOT(ppc, VERBOSE) && type == PMEMPOOL_CHECK_MSG_TYPE_INFO) return 0; struct check_status *st = status_alloc(); ASSERT(CHECK_IS(ppc, FORMAT_STR)); va_list ap; va_start(ap, fmt); int p = vsnprintf(st->msg, MAX_MSG_STR_SIZE, fmt, ap); va_end(ap); /* append possible strerror at the end of the message */ if (type != PMEMPOOL_CHECK_MSG_TYPE_QUESTION && arg && p > 0) { char buff[UTIL_MAX_ERR_MSG]; util_strerror((int)arg, buff, UTIL_MAX_ERR_MSG); int ret = snprintf(st->msg + p, MAX_MSG_STR_SIZE - (size_t)p, ": %s", buff); if (ret < 0 || ret >= (int)(MAX_MSG_STR_SIZE - (size_t)p)) { ERR("snprintf: %d", ret); status_release(st); return -1; } } st->status.type = type; return status_push(ppc, st, arg); } /* * check_status_release -- release single status object */ void check_status_release(PMEMpoolcheck *ppc, struct check_status *status) { if (status->status.type == PMEMPOOL_CHECK_MSG_TYPE_ERROR) ppc->data->error = NULL; status_release(status); } /* * pop_status -- (internal) pop single message from check_status queue */ static struct check_status * pop_status(struct check_data *data, struct check_status_head *queue) { if (!PMDK_TAILQ_EMPTY(queue)) { ASSERTeq(data->check_status_cache, NULL); data->check_status_cache = PMDK_TAILQ_FIRST(queue); PMDK_TAILQ_REMOVE(queue, data->check_status_cache, next); return data->check_status_cache; } return NULL; } /* * check_pop_question -- pop single question from questions queue */ struct check_status * check_pop_question(struct check_data *data) { return pop_status(data, &data->questions); } /* * check_pop_info -- pop single info from information queue */ struct check_status * check_pop_info(struct check_data *data) { return pop_status(data, &data->infos); } /* * check_pop_error -- pop error from state */ struct check_status * check_pop_error(struct check_data *data) { if (data->error) { ASSERTeq(data->check_status_cache, NULL); data->check_status_cache = data->error; data->error = NULL; return data->check_status_cache; } return NULL; } #ifdef _WIN32 void cache_to_utf8(struct check_data *data, char *buf, size_t size) { if (data->check_status_cache == NULL) return; struct check_status *status = data->check_status_cache; /* if it was a question, convert it and the answer to utf8 */ if (status->status.type == PMEMPOOL_CHECK_MSG_TYPE_QUESTION) { struct pmempool_check_statusW *wstatus = (struct pmempool_check_statusW *)&status->status; wchar_t *wstring = (wchar_t *)wstatus->str.msg; status->status.str.msg = util_toUTF8(wstring); if (status->status.str.msg == NULL) FATAL("!malloc"); util_free_UTF16(wstring); if (util_toUTF8_buff(wstatus->str.answer, buf, size) != 0) FATAL("Invalid answer conversion %s", out_get_errormsg()); status->status.str.answer = buf; } } #endif /* * check_clear_status_cache -- release check_status from cache */ void check_clear_status_cache(struct check_data *data) { if (data->check_status_cache) { switch (data->check_status_cache->status.type) { case PMEMPOOL_CHECK_MSG_TYPE_INFO: case PMEMPOOL_CHECK_MSG_TYPE_ERROR: /* * Info and error statuses are disposable. After showing * them to the user we have to release them. */ status_release(data->check_status_cache); data->check_status_cache = NULL; break; case PMEMPOOL_CHECK_MSG_TYPE_QUESTION: /* * Question status after being showed to the user carry * users answer. It must be kept till answer would be * processed so it can not be released from cache. It * has to be pushed to the answers queue, processed and * released after that. */ break; default: ASSERT(0); } } } /* * status_answer_push -- (internal) push single answer to answers queue */ static void status_answer_push(struct check_data *data, struct check_status *st) { ASSERTeq(st->status.type, PMEMPOOL_CHECK_MSG_TYPE_QUESTION); PMDK_TAILQ_INSERT_TAIL(&data->answers, st, next); } /* * check_push_answer -- process answer and push it to answers queue */ int check_push_answer(PMEMpoolcheck *ppc) { if (ppc->data->check_status_cache == NULL) return 0; /* check if answer is "yes" or "no" */ struct check_status *status = ppc->data->check_status_cache; if (status->status.str.answer != NULL) { if (strcmp(status->status.str.answer, CHECK_ANSWER_YES) == 0) status->answer = PMEMPOOL_CHECK_ANSWER_YES; else if (strcmp(status->status.str.answer, CHECK_ANSWER_NO) == 0) status->answer = PMEMPOOL_CHECK_ANSWER_NO; } if (status->answer == PMEMPOOL_CHECK_ANSWER_EMPTY) { /* invalid answer provided */ status_answer_push(ppc->data, ppc->data->check_status_cache); ppc->data->check_status_cache = NULL; CHECK_INFO(ppc, "Answer must be either %s or %s", CHECK_ANSWER_YES, CHECK_ANSWER_NO); return -1; } /* push answer */ PMDK_TAILQ_INSERT_TAIL(&ppc->data->answers, ppc->data->check_status_cache, next); ppc->data->check_status_cache = NULL; return 0; } /* * check_has_error - check if error exists */ bool check_has_error(struct check_data *data) { return data->error != NULL; } /* * check_has_answer - check if any answer exists */ bool check_has_answer(struct check_data *data) { return !PMDK_TAILQ_EMPTY(&data->answers); } /* * pop_answer -- (internal) pop single answer from answers queue */ static struct check_status * pop_answer(struct check_data *data) { struct check_status *ret = NULL; if (!PMDK_TAILQ_EMPTY(&data->answers)) { ret = PMDK_TAILQ_FIRST(&data->answers); PMDK_TAILQ_REMOVE(&data->answers, ret, next); } return ret; } /* * check_status_get_util -- extract pmempool_check_status from check_status */ struct pmempool_check_status * check_status_get_util(struct check_status *status) { return &status->status; } /* * check_answer_loop -- loop through all available answers and process them */ int check_answer_loop(PMEMpoolcheck *ppc, location *data, void *ctx, int fail_on_no, int (*callback)(PMEMpoolcheck *, location *, uint32_t, void *ctx)) { struct check_status *answer; while ((answer = pop_answer(ppc->data)) != NULL) { /* if answer is "no" we cannot fix an issue */ if (answer->answer != PMEMPOOL_CHECK_ANSWER_YES) { if (fail_on_no || answer->answer != PMEMPOOL_CHECK_ANSWER_NO) { CHECK_ERR(ppc, "cannot complete repair, reverting changes"); ppc->result = CHECK_RESULT_NOT_CONSISTENT; goto error; } ppc->result = CHECK_RESULT_REPAIRED; check_status_release(ppc, answer); continue; } /* perform fix */ if (callback(ppc, data, answer->question, ctx)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; goto error; } if (ppc->result == CHECK_RESULT_ERROR) goto error; /* fix succeeded */ ppc->result = CHECK_RESULT_REPAIRED; check_status_release(ppc, answer); } return 0; error: check_status_release(ppc, answer); return -1; } /* * check_questions_sequence_validate -- generate return value from result * * Sequence of questions can result in one of the following results: CONSISTENT, * REPAIRED, ASK_QUESTIONS of PROCESS_ANSWERS. If result == ASK_QUESTIONS it * returns -1 to indicate existence of unanswered questions. */ int check_questions_sequence_validate(PMEMpoolcheck *ppc) { ASSERT(ppc->result == CHECK_RESULT_CONSISTENT || ppc->result == CHECK_RESULT_ASK_QUESTIONS || ppc->result == CHECK_RESULT_PROCESS_ANSWERS || ppc->result == CHECK_RESULT_REPAIRED); if (ppc->result == CHECK_RESULT_ASK_QUESTIONS) { ASSERT(!PMDK_TAILQ_EMPTY(&ppc->data->questions)); return -1; } return 0; } /* * check_get_time_str -- returns time in human-readable format */ const char * check_get_time_str(time_t time) { static char str_buff[STR_MAX] = {0, }; struct tm *tm = util_localtime(&time); if (tm) strftime(str_buff, STR_MAX, TIME_STR_FMT, tm); else { int ret = snprintf(str_buff, STR_MAX, "unknown"); if (ret < 0) { ERR("failed to get time str"); return ""; } } return str_buff; } /* * check_get_uuid_str -- returns uuid in human readable format */ const char * check_get_uuid_str(uuid_t uuid) { static char uuid_str[UUID_STR_MAX] = {0, }; int ret = util_uuid_to_string(uuid, uuid_str); if (ret != 0) { ERR("failed to covert uuid to string"); return ""; } return uuid_str; } /* * pmempool_check_insert_arena -- insert arena to list */ void check_insert_arena(PMEMpoolcheck *ppc, struct arena *arenap) { PMDK_TAILQ_INSERT_TAIL(&ppc->pool->arenas, arenap, next); ppc->pool->narenas++; } pmdk-1.8/src/libpmempool/check_util.h0000664000000000000000000001500213615011243016400 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_util.h -- internal definitions check util */ #ifndef CHECK_UTIL_H #define CHECK_UTIL_H #include #include #include #ifdef __cplusplus extern "C" { #endif #define CHECK_STEP_COMPLETE UINT_MAX #define CHECK_INVALID_QUESTION UINT_MAX #define REQUIRE_ADVANCED "the following error can be fixed using " \ "PMEMPOOL_CHECK_ADVANCED flag" #ifndef min #define min(a, b) ((a) < (b) ? (a) : (b)) #endif /* check control context */ struct check_data; struct arena; /* queue of check statuses */ struct check_status; /* container storing state of all check steps */ #define PREFIX_MAX_SIZE 30 typedef struct { unsigned init_done; unsigned step; unsigned replica; unsigned part; int single_repl; int single_part; struct pool_set *set; int is_dev_dax; struct pool_hdr *hdrp; /* copy of the pool header in host byte order */ struct pool_hdr hdr; int hdr_valid; /* * If pool header has been modified this field indicates that * the pool parameters structure requires refresh. */ int pool_hdr_modified; unsigned healthy_replicas; struct pool_hdr *next_part_hdrp; struct pool_hdr *prev_part_hdrp; struct pool_hdr *next_repl_hdrp; struct pool_hdr *prev_repl_hdrp; int next_part_hdr_valid; int prev_part_hdr_valid; int next_repl_hdr_valid; int prev_repl_hdr_valid; /* valid poolset uuid */ uuid_t *valid_puuid; /* valid part uuid */ uuid_t *valid_uuid; /* valid part pool header */ struct pool_hdr *valid_part_hdrp; int valid_part_done; unsigned valid_part_replica; char prefix[PREFIX_MAX_SIZE]; struct arena *arenap; uint64_t offset; uint32_t narena; uint8_t *bitmap; uint8_t *dup_bitmap; uint8_t *fbitmap; struct list *list_inval; struct list *list_flog_inval; struct list *list_unmap; struct { int btti_header; int btti_backup; } valid; struct { struct btt_info btti; uint64_t btti_offset; } pool_valid; } location; /* check steps */ void check_bad_blocks(PMEMpoolcheck *ppc); void check_backup(PMEMpoolcheck *ppc); void check_pool_hdr(PMEMpoolcheck *ppc); void check_pool_hdr_uuids(PMEMpoolcheck *ppc); void check_sds(PMEMpoolcheck *ppc); void check_log(PMEMpoolcheck *ppc); void check_blk(PMEMpoolcheck *ppc); void check_btt_info(PMEMpoolcheck *ppc); void check_btt_map_flog(PMEMpoolcheck *ppc); void check_write(PMEMpoolcheck *ppc); struct check_data *check_data_alloc(void); void check_data_free(struct check_data *data); uint32_t check_step_get(struct check_data *data); void check_step_inc(struct check_data *data); location *check_get_step_data(struct check_data *data); void check_end(struct check_data *data); int check_is_end_util(struct check_data *data); int check_status_create(PMEMpoolcheck *ppc, enum pmempool_check_msg_type type, uint32_t arg, const char *fmt, ...) FORMAT_PRINTF(4, 5); void check_status_release(PMEMpoolcheck *ppc, struct check_status *status); void check_clear_status_cache(struct check_data *data); struct check_status *check_pop_question(struct check_data *data); struct check_status *check_pop_error(struct check_data *data); struct check_status *check_pop_info(struct check_data *data); bool check_has_error(struct check_data *data); bool check_has_answer(struct check_data *data); int check_push_answer(PMEMpoolcheck *ppc); struct pmempool_check_status *check_status_get_util( struct check_status *status); int check_status_is(struct check_status *status, enum pmempool_check_msg_type type); /* create info status */ #define CHECK_INFO(ppc, ...)\ check_status_create(ppc, PMEMPOOL_CHECK_MSG_TYPE_INFO, 0, __VA_ARGS__) /* create info status and append error message based on errno */ #define CHECK_INFO_ERRNO(ppc, ...)\ check_status_create(ppc, PMEMPOOL_CHECK_MSG_TYPE_INFO,\ (uint32_t)errno, __VA_ARGS__) /* create error status */ #define CHECK_ERR(ppc, ...)\ check_status_create(ppc, PMEMPOOL_CHECK_MSG_TYPE_ERROR, 0, __VA_ARGS__) /* create question status */ #define CHECK_ASK(ppc, question, ...)\ check_status_create(ppc, PMEMPOOL_CHECK_MSG_TYPE_QUESTION, question,\ __VA_ARGS__) #define CHECK_NOT_COMPLETE(loc, steps)\ ((loc)->step != CHECK_STEP_COMPLETE &&\ ((steps)[(loc)->step].check != NULL ||\ (steps)[(loc)->step].fix != NULL)) int check_answer_loop(PMEMpoolcheck *ppc, location *data, void *ctx, int fail_on_no, int (*callback)(PMEMpoolcheck *, location *, uint32_t, void *ctx)); int check_questions_sequence_validate(PMEMpoolcheck *ppc); const char *check_get_time_str(time_t time); const char *check_get_uuid_str(uuid_t uuid); const char *check_get_pool_type_str(enum pool_type type); void check_insert_arena(PMEMpoolcheck *ppc, struct arena *arenap); #ifdef _WIN32 void cache_to_utf8(struct check_data *data, char *buf, size_t size); #endif #define CHECK_IS(ppc, flag)\ util_flag_isset((ppc)->args.flags, PMEMPOOL_CHECK_ ## flag) #define CHECK_IS_NOT(ppc, flag)\ util_flag_isclr((ppc)->args.flags, PMEMPOOL_CHECK_ ## flag) #define CHECK_WITHOUT_FIXING(ppc)\ CHECK_IS_NOT(ppc, REPAIR) || CHECK_IS(ppc, DRY_RUN) #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmempool/libpmempool_main.c0000664000000000000000000000424213615011243017610 0ustar rootroot/* * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmempool_main.c -- entry point for libpmempool.dll * * XXX - This is a placeholder. All the library initialization/cleanup * that is done in library ctors/dtors, as well as TLS initialization * should be moved here. */ #include void libpmempool_init(void); void libpmempool_fini(void); int APIENTRY DllMain(HINSTANCE hInstance, DWORD dwReason, LPVOID lpReserved) { switch (dwReason) { case DLL_PROCESS_ATTACH: libpmempool_init(); break; case DLL_THREAD_ATTACH: case DLL_THREAD_DETACH: break; case DLL_PROCESS_DETACH: libpmempool_fini(); break; } return TRUE; } pmdk-1.8/src/libpmempool/check_btt_info.c0000664000000000000000000003170213615011243017227 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_btt_info.c -- check BTT Info */ #include #include #include #include "out.h" #include "util.h" #include "btt.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check_util.h" enum question { Q_RESTORE_FROM_BACKUP, Q_REGENERATE, Q_REGENERATE_CHECKSUM, Q_RESTORE_FROM_HEADER }; /* * location_release -- (internal) release check_btt_info_loc allocations */ static void location_release(location *loc) { free(loc->arenap); loc->arenap = NULL; } /* * btt_info_checksum -- (internal) check BTT Info checksum */ static int btt_info_checksum(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); loc->arenap = calloc(1, sizeof(struct arena)); if (!loc->arenap) { ERR("!calloc"); ppc->result = CHECK_RESULT_INTERNAL_ERROR; CHECK_ERR(ppc, "cannot allocate memory for arena"); goto error_cleanup; } /* read the BTT Info header at well known offset */ if (pool_read(ppc->pool, &loc->arenap->btt_info, sizeof(loc->arenap->btt_info), loc->offset)) { CHECK_ERR(ppc, "arena %u: cannot read BTT Info header", loc->arenap->id); ppc->result = CHECK_RESULT_ERROR; goto error_cleanup; } loc->arenap->id = ppc->pool->narenas; /* BLK is consistent even without BTT Layout */ if (ppc->pool->params.type == POOL_TYPE_BLK) { int is_zeroed = util_is_zeroed((const void *) &loc->arenap->btt_info, sizeof(loc->arenap->btt_info)); if (is_zeroed) { CHECK_INFO(ppc, "BTT Layout not written"); loc->step = CHECK_STEP_COMPLETE; ppc->pool->blk_no_layout = 1; location_release(loc); check_end(ppc->data); return 0; } } /* check consistency of BTT Info */ if (pool_btt_info_valid(&loc->arenap->btt_info)) { CHECK_INFO(ppc, "arena %u: BTT Info header checksum correct", loc->arenap->id); loc->valid.btti_header = 1; } else if (CHECK_IS_NOT(ppc, REPAIR)) { CHECK_ERR(ppc, "arena %u: BTT Info header checksum incorrect", loc->arenap->id); ppc->result = CHECK_RESULT_NOT_CONSISTENT; check_end(ppc->data); goto error_cleanup; } return 0; error_cleanup: location_release(loc); return -1; } /* * btt_info_backup -- (internal) check BTT Info backup */ static int btt_info_backup(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); /* check BTT Info backup consistency */ const size_t btt_info_size = sizeof(ppc->pool->bttc.btt_info); uint64_t btt_info_off = pool_next_arena_offset(ppc->pool, loc->offset) - btt_info_size; if (pool_read(ppc->pool, &ppc->pool->bttc.btt_info, btt_info_size, btt_info_off)) { CHECK_ERR(ppc, "arena %u: cannot read BTT Info backup", loc->arenap->id); goto error; } /* check whether this BTT Info backup is valid */ if (pool_btt_info_valid(&ppc->pool->bttc.btt_info)) { loc->valid.btti_backup = 1; /* restore BTT Info from backup */ if (!loc->valid.btti_header && CHECK_IS(ppc, REPAIR)) CHECK_ASK(ppc, Q_RESTORE_FROM_BACKUP, "arena %u: BTT " "Info header checksum incorrect.|Restore BTT " "Info from backup?", loc->arenap->id); } /* * if BTT Info backup require repairs it will be fixed in further steps */ return check_questions_sequence_validate(ppc); error: ppc->result = CHECK_RESULT_ERROR; location_release(loc); return -1; } /* * btt_info_from_backup_fix -- (internal) fix BTT Info using its backup */ static int btt_info_from_backup_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *ctx) { LOG(3, NULL); ASSERTeq(ctx, NULL); ASSERTne(loc, NULL); switch (question) { case Q_RESTORE_FROM_BACKUP: CHECK_INFO(ppc, "arena %u: restoring BTT Info header from backup", loc->arenap->id); memcpy(&loc->arenap->btt_info, &ppc->pool->bttc.btt_info, sizeof(loc->arenap->btt_info)); loc->valid.btti_header = 1; break; default: ERR("not implemented question id: %u", question); } return 0; } /* * btt_info_gen -- (internal) ask whether try to regenerate BTT Info */ static int btt_info_gen(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); if (loc->valid.btti_header) return 0; ASSERT(CHECK_IS(ppc, REPAIR)); if (!loc->pool_valid.btti_offset) { ppc->result = CHECK_RESULT_NOT_CONSISTENT; check_end(ppc->data); return CHECK_ERR(ppc, "can not find any valid BTT Info"); } CHECK_ASK(ppc, Q_REGENERATE, "arena %u: BTT Info header checksum incorrect.|Do you want to " "regenerate BTT Info?", loc->arenap->id); return check_questions_sequence_validate(ppc); } /* * btt_info_gen_fix -- (internal) fix by regenerating BTT Info */ static int btt_info_gen_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *ctx) { LOG(3, NULL); ASSERTeq(ctx, NULL); ASSERTne(loc, NULL); switch (question) { case Q_REGENERATE: CHECK_INFO(ppc, "arena %u: regenerating BTT Info header", loc->arenap->id); /* * We do not have valid BTT Info backup so we get first valid * BTT Info and try to calculate BTT Info for current arena */ uint64_t arena_size = ppc->pool->set_file->size - loc->offset; if (arena_size > BTT_MAX_ARENA) arena_size = BTT_MAX_ARENA; uint64_t space_left = ppc->pool->set_file->size - loc->offset - arena_size; struct btt_info *bttd = &loc->arenap->btt_info; struct btt_info *btts = &loc->pool_valid.btti; btt_info_convert2h(bttd); /* * all valid BTT Info structures have the same signature, UUID, * parent UUID, flags, major, minor, external LBA size, internal * LBA size, nfree, info size and data offset */ memcpy(bttd->sig, btts->sig, BTTINFO_SIG_LEN); memcpy(bttd->uuid, btts->uuid, BTTINFO_UUID_LEN); memcpy(bttd->parent_uuid, btts->parent_uuid, BTTINFO_UUID_LEN); memset(bttd->unused, 0, BTTINFO_UNUSED_LEN); bttd->flags = btts->flags; bttd->major = btts->major; bttd->minor = btts->minor; /* other parameters can be calculated */ if (btt_info_set(bttd, btts->external_lbasize, btts->nfree, arena_size, space_left)) { CHECK_ERR(ppc, "can not restore BTT Info"); return -1; } ASSERTeq(bttd->external_lbasize, btts->external_lbasize); ASSERTeq(bttd->internal_lbasize, btts->internal_lbasize); ASSERTeq(bttd->nfree, btts->nfree); ASSERTeq(bttd->infosize, btts->infosize); ASSERTeq(bttd->dataoff, btts->dataoff); return 0; default: ERR("not implemented question id: %u", question); return -1; } } /* * btt_info_checksum_retry -- (internal) check BTT Info checksum */ static int btt_info_checksum_retry(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); if (loc->valid.btti_header) return 0; btt_info_convert2le(&loc->arenap->btt_info); /* check consistency of BTT Info */ if (pool_btt_info_valid(&loc->arenap->btt_info)) { CHECK_INFO(ppc, "arena %u: BTT Info header checksum correct", loc->arenap->id); loc->valid.btti_header = 1; return 0; } if (CHECK_IS_NOT(ppc, ADVANCED)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; CHECK_INFO(ppc, REQUIRE_ADVANCED); CHECK_ERR(ppc, "arena %u: BTT Info header checksum incorrect", loc->arenap->id); check_end(ppc->data); goto error_cleanup; } CHECK_ASK(ppc, Q_REGENERATE_CHECKSUM, "arena %u: BTT Info header checksum incorrect.|Do you want to " "regenerate BTT Info checksum?", loc->arenap->id); return check_questions_sequence_validate(ppc); error_cleanup: location_release(loc); return -1; } /* * btt_info_checksum_fix -- (internal) fix by regenerating BTT Info checksum */ static int btt_info_checksum_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *ctx) { LOG(3, NULL); ASSERTeq(ctx, NULL); ASSERTne(loc, NULL); switch (question) { case Q_REGENERATE_CHECKSUM: util_checksum(&loc->arenap->btt_info, sizeof(struct btt_info), &loc->arenap->btt_info.checksum, 1, 0); loc->valid.btti_header = 1; break; default: ERR("not implemented question id: %u", question); } return 0; } /* * btt_info_backup_checksum -- (internal) check BTT Info backup checksum */ static int btt_info_backup_checksum(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); ASSERT(loc->valid.btti_header); if (loc->valid.btti_backup) return 0; /* BTT Info backup is not valid so it must be fixed */ if (CHECK_IS_NOT(ppc, REPAIR)) { CHECK_ERR(ppc, "arena %u: BTT Info backup checksum incorrect", loc->arenap->id); ppc->result = CHECK_RESULT_NOT_CONSISTENT; check_end(ppc->data); goto error_cleanup; } CHECK_ASK(ppc, Q_RESTORE_FROM_HEADER, "arena %u: BTT Info backup checksum incorrect.|Do you want to " "restore it from BTT Info header?", loc->arenap->id); return check_questions_sequence_validate(ppc); error_cleanup: location_release(loc); return -1; } /* * btt_info_backup_fix -- (internal) prepare restore BTT Info backup from header */ static int btt_info_backup_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *ctx) { LOG(3, NULL); ASSERTeq(ctx, NULL); ASSERTne(loc, NULL); switch (question) { case Q_RESTORE_FROM_HEADER: /* BTT Info backup would be restored in check_write step */ CHECK_INFO(ppc, "arena %u: restoring BTT Info backup from header", loc->arenap->id); break; default: ERR("not implemented question id: %u", question); } return 0; } struct step { int (*check)(PMEMpoolcheck *, location *); int (*fix)(PMEMpoolcheck *, location *, uint32_t, void *); }; static const struct step steps[] = { { .check = btt_info_checksum, }, { .check = btt_info_backup, }, { .fix = btt_info_from_backup_fix, }, { .check = btt_info_gen, }, { .fix = btt_info_gen_fix, }, { .check = btt_info_checksum_retry, }, { .fix = btt_info_checksum_fix, }, { .check = btt_info_backup_checksum, }, { .fix = btt_info_backup_fix, }, { .check = NULL, .fix = NULL, }, }; /* * step_exe -- (internal) perform single step according to its parameters */ static inline int step_exe(PMEMpoolcheck *ppc, location *loc) { ASSERT(loc->step < ARRAY_SIZE(steps)); const struct step *step = &steps[loc->step++]; if (!step->fix) return step->check(ppc, loc); if (!check_answer_loop(ppc, loc, NULL, 1, step->fix)) return 0; if (check_has_error(ppc->data)) location_release(loc); return -1; } /* * check_btt_info -- entry point for btt info check */ void check_btt_info(PMEMpoolcheck *ppc) { LOG(3, NULL); location *loc = check_get_step_data(ppc->data); uint64_t nextoff = 0; /* initialize check */ if (!loc->offset) { CHECK_INFO(ppc, "checking BTT Info headers"); loc->offset = sizeof(struct pool_hdr); if (ppc->pool->params.type == POOL_TYPE_BLK) loc->offset += ALIGN_UP(sizeof(struct pmemblk) - sizeof(struct pool_hdr), BLK_FORMAT_DATA_ALIGN); loc->pool_valid.btti_offset = pool_get_first_valid_btt( ppc->pool, &loc->pool_valid.btti, loc->offset, NULL); /* Without valid BTT Info we can not proceed */ if (!loc->pool_valid.btti_offset) { if (ppc->pool->params.type == POOL_TYPE_BTT) { CHECK_ERR(ppc, "can not find any valid BTT Info"); ppc->result = CHECK_RESULT_NOT_CONSISTENT; check_end(ppc->data); return; } } else btt_info_convert2h(&loc->pool_valid.btti); } do { /* jump to next offset */ if (ppc->result != CHECK_RESULT_PROCESS_ANSWERS) { loc->offset += nextoff; loc->step = 0; loc->valid.btti_header = 0; loc->valid.btti_backup = 0; } /* do all checks */ while (CHECK_NOT_COMPLETE(loc, steps)) { if (step_exe(ppc, loc) || ppc->pool->blk_no_layout == 1) return; } /* save offset and insert BTT to cache for next steps */ loc->arenap->offset = loc->offset; loc->arenap->valid = true; check_insert_arena(ppc, loc->arenap); nextoff = le64toh(loc->arenap->btt_info.nextoff); } while (nextoff > 0); } pmdk-1.8/src/libpmempool/check_write.c0000664000000000000000000001502713615011243016557 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_write.c -- write fixed data back */ #include #include #include "out.h" #include "btt.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check_util.h" enum questions { Q_REPAIR_MAP, Q_REPAIR_FLOG, }; /* * log_write -- (internal) write all structures for log pool */ static int log_write(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); if (CHECK_WITHOUT_FIXING(ppc)) return 0; /* endianness conversion */ struct pmemlog *log = &ppc->pool->hdr.log; log_convert2le(log); if (pool_write(ppc->pool, log, sizeof(*log), 0)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; return CHECK_ERR(ppc, "writing pmemlog structure failed"); } return 0; } /* * blk_write_flog -- (internal) convert and write flog to file */ static int blk_write_flog(PMEMpoolcheck *ppc, struct arena *arenap) { if (!arenap->flog) { ppc->result = CHECK_RESULT_ERROR; return CHECK_ERR(ppc, "flog is missing"); } uint64_t flogoff = arenap->offset + arenap->btt_info.flogoff; uint8_t *ptr = arenap->flog; uint32_t i; for (i = 0; i < arenap->btt_info.nfree; i++) { struct btt_flog *flog = (struct btt_flog *)ptr; btt_flog_convert2le(&flog[0]); btt_flog_convert2le(&flog[1]); ptr += BTT_FLOG_PAIR_ALIGN; } if (pool_write(ppc->pool, arenap->flog, arenap->flogsize, flogoff)) { CHECK_INFO(ppc, "%s", ppc->path); ppc->result = CHECK_RESULT_CANNOT_REPAIR; return CHECK_ERR(ppc, "arena %u: writing BTT FLOG failed\n", arenap->id); } return 0; } /* * blk_write_map -- (internal) convert and write map to file */ static int blk_write_map(PMEMpoolcheck *ppc, struct arena *arenap) { if (!arenap->map) { ppc->result = CHECK_RESULT_ERROR; return CHECK_ERR(ppc, "map is missing"); } uint64_t mapoff = arenap->offset + arenap->btt_info.mapoff; uint32_t i; for (i = 0; i < arenap->btt_info.external_nlba; i++) arenap->map[i] = htole32(arenap->map[i]); if (pool_write(ppc->pool, arenap->map, arenap->mapsize, mapoff)) { CHECK_INFO(ppc, "%s", ppc->path); ppc->result = CHECK_RESULT_CANNOT_REPAIR; return CHECK_ERR(ppc, "arena %u: writing BTT map failed\n", arenap->id); } return 0; } /* * blk_write -- (internal) write all structures for blk pool */ static int blk_write(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); if (CHECK_WITHOUT_FIXING(ppc)) return 0; /* endianness conversion */ ppc->pool->hdr.blk.bsize = htole32(ppc->pool->hdr.blk.bsize); if (pool_write(ppc->pool, &ppc->pool->hdr.blk, sizeof(ppc->pool->hdr.blk), 0)) { CHECK_INFO(ppc, "%s", ppc->path); ppc->result = CHECK_RESULT_CANNOT_REPAIR; return CHECK_ERR(ppc, "writing pmemblk structure failed"); } return 0; } /* * btt_data_write -- (internal) write BTT data */ static int btt_data_write(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); struct arena *arenap; PMDK_TAILQ_FOREACH(arenap, &ppc->pool->arenas, next) { if (ppc->pool->uuid_op == UUID_NOT_FROM_BTT) { memcpy(arenap->btt_info.parent_uuid, ppc->pool->hdr.pool.poolset_uuid, sizeof(arenap->btt_info.parent_uuid)); util_checksum(&arenap->btt_info, sizeof(arenap->btt_info), &arenap->btt_info.checksum, 1, 0); } if (pool_write(ppc->pool, &arenap->btt_info, sizeof(arenap->btt_info), arenap->offset)) { CHECK_INFO(ppc, "%s", ppc->path); CHECK_ERR(ppc, "arena %u: writing BTT Info failed", arenap->id); goto error; } if (pool_write(ppc->pool, &arenap->btt_info, sizeof(arenap->btt_info), arenap->offset + le64toh(arenap->btt_info.infooff))) { CHECK_INFO(ppc, "%s", ppc->path); CHECK_ERR(ppc, "arena %u: writing BTT Info backup failed", arenap->id); goto error; } if (blk_write_flog(ppc, arenap)) goto error; if (blk_write_map(ppc, arenap)) goto error; } return 0; error: ppc->result = CHECK_RESULT_CANNOT_REPAIR; return -1; } struct step { int (*func)(PMEMpoolcheck *, location *loc); enum pool_type type; }; static const struct step steps[] = { { .func = log_write, .type = POOL_TYPE_LOG, }, { .func = blk_write, .type = POOL_TYPE_BLK, }, { .func = btt_data_write, .type = POOL_TYPE_BLK | POOL_TYPE_BTT, }, { .func = NULL, }, }; /* * step_exe -- (internal) perform single step according to its parameters */ static inline int step_exe(PMEMpoolcheck *ppc, location *loc) { ASSERT(loc->step < ARRAY_SIZE(steps)); const struct step *step = &steps[loc->step++]; /* check step conditions */ if (!(step->type & ppc->pool->params.type)) return 0; return step->func(ppc, loc); } /* * check_write -- write fixed data back */ void check_write(PMEMpoolcheck *ppc) { /* * XXX: Disabling individual checks based on type should be done in the * step structure. This however requires refactor of the step * processing code. */ if (CHECK_IS_NOT(ppc, REPAIR)) return; location *loc = (location *)check_get_step_data(ppc->data); /* do all steps */ while (loc->step != CHECK_STEP_COMPLETE && steps[loc->step].func != NULL) { if (step_exe(ppc, loc)) return; } } pmdk-1.8/src/libpmempool/rm.c0000664000000000000000000001400713615011243014703 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rm.c -- implementation of pmempool_rm() function */ #include #include #include "libpmempool.h" #include "out.h" #include "os.h" #include "util.h" #include "set.h" #include "file.h" #define PMEMPOOL_RM_ALL_FLAGS (\ PMEMPOOL_RM_FORCE |\ PMEMPOOL_RM_POOLSET_LOCAL |\ PMEMPOOL_RM_POOLSET_REMOTE) #define ERR_F(f, ...) do {\ if (CHECK_FLAG((f), FORCE))\ LOG(2, "!(ignored) " __VA_ARGS__);\ else\ ERR(__VA_ARGS__);\ } while (0) #define CHECK_FLAG(f, i) ((f) & PMEMPOOL_RM_##i) struct cb_args { unsigned flags; int error; }; /* * rm_local -- (internal) remove single local file */ static int rm_local(const char *path, unsigned flags, int is_part_file) { int ret = util_unlink_flock(path); if (!ret) { LOG(3, "%s: removed", path); return 0; } int oerrno = errno; os_stat_t buff; ret = os_stat(path, &buff); if (!ret) { if (S_ISDIR(buff.st_mode)) { errno = EISDIR; if (is_part_file) ERR("%s: removing file failed", path); else ERR("removing file failed"); return -1; } } errno = oerrno; if (is_part_file) ERR_F(flags, "%s: removing file failed", path); else ERR_F(flags, "removing file failed"); if (CHECK_FLAG(flags, FORCE)) return 0; return -1; } /* * rm_remote -- (internal) remove remote replica */ static int rm_remote(const char *node, const char *path, unsigned flags) { if (!Rpmem_remove) { ERR_F(flags, "cannot remove remote replica" " -- missing librpmem"); return -1; } int rpmem_flags = 0; if (CHECK_FLAG(flags, FORCE)) rpmem_flags |= RPMEM_REMOVE_FORCE; if (CHECK_FLAG(flags, POOLSET_REMOTE)) rpmem_flags |= RPMEM_REMOVE_POOL_SET; int ret = Rpmem_remove(node, path, rpmem_flags); if (ret) { ERR_F(flags, "%s/%s removing failed", node, path); if (CHECK_FLAG(flags, FORCE)) ret = 0; } else { LOG(3, "%s/%s: removed", node, path); } return ret; } /* * rm_cb -- (internal) foreach part callback */ static int rm_cb(struct part_file *pf, void *arg) { struct cb_args *args = (struct cb_args *)arg; int ret; if (pf->is_remote) { ret = rm_remote(pf->remote->node_addr, pf->remote->pool_desc, args->flags); } else { ret = rm_local(pf->part->path, args->flags, 1); } if (ret) args->error = ret; return 0; } /* * pmempool_rmU -- remove pool files or poolsets */ #ifndef _WIN32 static inline #endif int pmempool_rmU(const char *path, unsigned flags) { LOG(3, "path %s flags %x", path, flags); int ret; if (flags & ~PMEMPOOL_RM_ALL_FLAGS) { ERR("invalid flags specified"); errno = EINVAL; return -1; } int is_poolset = util_is_poolset_file(path); if (is_poolset < 0) { os_stat_t buff; ret = os_stat(path, &buff); if (!ret) { if (S_ISDIR(buff.st_mode)) { errno = EISDIR; ERR("removing file failed"); return -1; } } ERR_F(flags, "removing file failed"); if (CHECK_FLAG(flags, FORCE)) return 0; return -1; } if (!is_poolset) { LOG(2, "%s: not a poolset file", path); return rm_local(path, flags, 0); } LOG(2, "%s: poolset file", path); /* fill up pool_set structure */ struct pool_set *set = NULL; int fd = os_open(path, O_RDONLY); if (fd == -1 || util_poolset_parse(&set, path, fd)) { ERR_F(flags, "parsing poolset file failed"); if (fd != -1) os_close(fd); if (CHECK_FLAG(flags, FORCE)) return 0; return -1; } os_close(fd); if (set->remote) { /* ignore error - it will be handled in rm_remote() */ (void) util_remote_load(); } util_poolset_free(set); struct cb_args args; args.flags = flags; args.error = 0; ret = util_poolset_foreach_part(path, rm_cb, &args); if (ret == -1) { ERR_F(flags, "parsing poolset file failed"); if (CHECK_FLAG(flags, FORCE)) return 0; return ret; } ASSERTeq(ret, 0); if (args.error) return args.error; if (CHECK_FLAG(flags, POOLSET_LOCAL)) { ret = rm_local(path, flags, 0); if (ret) { ERR_F(flags, "removing pool set file failed"); } else { LOG(3, "%s: removed", path); } if (CHECK_FLAG(flags, FORCE)) return 0; return ret; } return 0; } #ifndef _WIN32 /* * pmempool_rm -- remove pool files or poolsets */ int pmempool_rm(const char *path, unsigned flags) { return pmempool_rmU(path, flags); } #else /* * pmempool_rmW -- remove pool files or poolsets in widechar */ int pmempool_rmW(const wchar_t *path, unsigned flags) { char *upath = util_toUTF8(path); if (upath == NULL) { ERR("Invalid poolest/pool file path."); return -1; } int ret = pmempool_rmU(upath, flags); util_free_UTF8(upath); return ret; } #endif pmdk-1.8/src/libpmempool/libpmempool.rc0000664000000000000000000000342213615011243016765 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmempool.rc -- libpmempool resource file */ #include #define FILE_NAME "libpmempool.dll" #define DESCRIPTION "libpmempool - pool management library" #define TYPE VFT_DLL #include pmdk-1.8/src/libpmempool/pool.c0000664000000000000000000006322113615011243015240 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pool.c -- pool processing functions */ #include #include #include #include #include #include #ifndef _WIN32 #include #ifdef __FreeBSD__ #include #define BLKGETSIZE64 DIOCGMEDIASIZE #else #include #endif #endif #include "libpmem.h" #include "libpmemlog.h" #include "libpmemblk.h" #include "libpmempool.h" #include "out.h" #include "pmempool.h" #include "pool.h" #include "lane.h" #include "obj.h" #include "btt.h" #include "file.h" #include "os.h" #include "set.h" #include "check_util.h" #include "util_pmem.h" #include "mmap.h" /* arbitrary size of a maximum file part being read / write at once */ #define RW_BUFFERING_SIZE (128 * 1024 * 1024) /* * pool_btt_lseek -- (internal) perform lseek in BTT file mode */ static inline os_off_t pool_btt_lseek(struct pool_data *pool, os_off_t offset, int whence) { os_off_t result; if ((result = os_lseek(pool->set_file->fd, offset, whence)) == -1) ERR("!lseek"); return result; } /* * pool_btt_read -- (internal) perform read in BTT file mode */ static inline ssize_t pool_btt_read(struct pool_data *pool, void *dst, size_t count) { size_t total = 0; ssize_t nread; while (count > total && (nread = util_read(pool->set_file->fd, dst, count - total))) { if (nread == -1) { ERR("!read"); return total ? (ssize_t)total : -1; } dst = (void *)((ssize_t)dst + nread); total += (size_t)nread; } return (ssize_t)total; } /* * pool_btt_write -- (internal) perform write in BTT file mode */ static inline ssize_t pool_btt_write(struct pool_data *pool, const void *src, size_t count) { ssize_t nwrite = 0; size_t total = 0; while (count > total && (nwrite = util_write(pool->set_file->fd, src, count - total))) { if (nwrite == -1) { ERR("!write"); return total ? (ssize_t)total : -1; } src = (void *)((ssize_t)src + nwrite); total += (size_t)nwrite; } return (ssize_t)total; } /* * pool_set_read_header -- (internal) read a header of a pool set */ static int pool_set_read_header(const char *fname, struct pool_hdr *hdr) { struct pool_set *set; int ret = 0; if (util_poolset_read(&set, fname)) { return -1; } /* open the first part set file to read the pool header values */ const struct pool_set_part *part = PART(REP(set, 0), 0); int fdp = util_file_open(part->path, NULL, 0, O_RDONLY); if (fdp < 0) { ERR("cannot open poolset part file"); ret = -1; goto err_pool_set; } /* read the pool header from first pool set file */ if (pread(fdp, hdr, sizeof(*hdr), 0) != sizeof(*hdr)) { ERR("cannot read pool header from poolset"); ret = -1; goto err_close_part; } err_close_part: os_close(fdp); err_pool_set: util_poolset_free(set); return ret; } /* * pool_set_map -- (internal) map poolset */ static int pool_set_map(const char *fname, struct pool_set **poolset, unsigned flags) { ASSERTeq(util_is_poolset_file(fname), 1); struct pool_hdr hdr; if (pool_set_read_header(fname, &hdr)) return -1; util_convert2h_hdr_nocheck(&hdr); /* parse pool type from first pool set file */ enum pool_type type = pool_hdr_get_type(&hdr); if (type == POOL_TYPE_UNKNOWN) { ERR("cannot determine pool type from poolset"); return -1; } /* * Open the poolset, the values passed to util_pool_open are read * from the first poolset file, these values are then compared with * the values from all headers of poolset files. */ struct pool_attr attr; util_pool_hdr2attr(&attr, &hdr); if (util_pool_open(poolset, fname, 0 /* minpartsize */, &attr, NULL, NULL, flags | POOL_OPEN_IGNORE_SDS | POOL_OPEN_IGNORE_BAD_BLOCKS)) { ERR("opening poolset failed"); return -1; } return 0; } /* * pool_params_from_header -- parse pool params from pool header */ void pool_params_from_header(struct pool_params *params, const struct pool_hdr *hdr) { memcpy(params->signature, hdr->signature, sizeof(params->signature)); memcpy(¶ms->features, &hdr->features, sizeof(params->features)); /* * Check if file is a part of pool set by comparing the UUID with the * next part UUID. If it is the same it means the pool consist of a * single file. */ int uuid_eq_next = uuidcmp(hdr->uuid, hdr->next_part_uuid); int uuid_eq_prev = uuidcmp(hdr->uuid, hdr->prev_part_uuid); params->is_part = !params->is_poolset && (uuid_eq_next || uuid_eq_prev); params->type = pool_hdr_get_type(hdr); } /* * pool_check_type_to_pool_type -- (internal) convert check pool type to * internal pool type value */ static enum pool_type pool_check_type_to_pool_type(enum pmempool_pool_type check_pool_type) { switch (check_pool_type) { case PMEMPOOL_POOL_TYPE_LOG: return POOL_TYPE_LOG; case PMEMPOOL_POOL_TYPE_BLK: return POOL_TYPE_BLK; case PMEMPOOL_POOL_TYPE_OBJ: return POOL_TYPE_OBJ; default: ERR("can not convert pmempool_pool_type %u to pool_type", check_pool_type); return POOL_TYPE_UNKNOWN; } } /* * pool_parse_params -- parse pool type, file size and block size */ static int pool_params_parse(const PMEMpoolcheck *ppc, struct pool_params *params, int check) { LOG(3, NULL); int is_btt = ppc->args.pool_type == PMEMPOOL_POOL_TYPE_BTT; params->type = POOL_TYPE_UNKNOWN; params->is_poolset = util_is_poolset_file(ppc->path) == 1; int fd = util_file_open(ppc->path, NULL, 0, O_RDONLY); if (fd < 0) return -1; int ret = 0; os_stat_t stat_buf; ret = os_fstat(fd, &stat_buf); if (ret) goto out_close; ASSERT(stat_buf.st_size >= 0); params->mode = stat_buf.st_mode; struct pool_set *set; void *addr; if (params->is_poolset) { /* * Need to close the poolset because it will be opened with * flock in the following instructions. */ os_close(fd); fd = -1; if (check) { if (pool_set_map(ppc->path, &set, 0)) return -1; } else { ret = util_poolset_create_set(&set, ppc->path, 0, 0, true); if (ret < 0) { LOG(2, "cannot open pool set -- '%s'", ppc->path); return -1; } if (set->remote) { ERR("poolsets with remote replicas are not " "supported"); return -1; } if (util_pool_open_nocheck(set, POOL_OPEN_IGNORE_BAD_BLOCKS)) return -1; } params->size = set->poolsize; addr = set->replica[0]->part[0].addr; /* * XXX mprotect for device dax with length not aligned to its * page granularity causes SIGBUS on the next page fault. * The length argument of this call should be changed to * set->poolsize once the kernel issue is solved. */ if (mprotect(addr, set->replica[0]->repsize, PROT_READ) < 0) { ERR("!mprotect"); goto out_unmap; } params->is_dev_dax = set->replica[0]->part[0].is_dev_dax; params->is_pmem = set->replica[0]->is_pmem; } else if (is_btt) { params->size = (size_t)stat_buf.st_size; #ifndef _WIN32 if (params->mode & S_IFBLK) if (ioctl(fd, BLKGETSIZE64, ¶ms->size)) { ERR("!ioctl"); goto out_close; } #endif addr = NULL; } else { enum file_type type = util_file_get_type(ppc->path); if (type < 0) { ret = -1; goto out_close; } ssize_t s = util_file_get_size(ppc->path); if (s < 0) { ret = -1; goto out_close; } params->size = (size_t)s; int map_sync; addr = util_map(fd, 0, params->size, MAP_SHARED, 1, 0, &map_sync); if (addr == NULL) { ret = -1; goto out_close; } params->is_dev_dax = type == TYPE_DEVDAX; params->is_pmem = params->is_dev_dax || map_sync || pmem_is_pmem(addr, params->size); } /* stop processing for BTT device */ if (is_btt) { params->type = POOL_TYPE_BTT; params->is_part = false; goto out_close; } struct pool_hdr hdr; memcpy(&hdr, addr, sizeof(hdr)); util_convert2h_hdr_nocheck(&hdr); pool_params_from_header(params, &hdr); if (ppc->args.pool_type != PMEMPOOL_POOL_TYPE_DETECT) { enum pool_type declared_type = pool_check_type_to_pool_type(ppc->args.pool_type); if ((params->type & ~declared_type) != 0) { ERR("declared pool type does not match"); errno = EINVAL; ret = 1; goto out_unmap; } } if (params->type == POOL_TYPE_BLK) { struct pmemblk pbp; memcpy(&pbp, addr, sizeof(pbp)); params->blk.bsize = le32toh(pbp.bsize); } else if (params->type == POOL_TYPE_OBJ) { struct pmemobjpool *pop = addr; memcpy(params->obj.layout, pop->layout, PMEMOBJ_MAX_LAYOUT); } out_unmap: if (params->is_poolset) { ASSERTeq(fd, -1); ASSERTne(addr, NULL); util_poolset_close(set, DO_NOT_DELETE_PARTS); } else if (!is_btt) { ASSERTne(fd, -1); ASSERTne(addr, NULL); munmap(addr, params->size); } out_close: if (fd != -1) os_close(fd); return ret; } /* * pool_set_file_open -- (internal) opens pool set file or regular file */ static struct pool_set_file * pool_set_file_open(const char *fname, struct pool_params *params, int rdonly) { LOG(3, NULL); struct pool_set_file *file = calloc(1, sizeof(*file)); if (!file) return NULL; file->fname = strdup(fname); if (!file->fname) goto err; const char *path = file->fname; if (params->type != POOL_TYPE_BTT) { int ret = util_poolset_create_set(&file->poolset, path, 0, 0, true); if (ret < 0) { LOG(2, "cannot open pool set -- '%s'", path); goto err_free_fname; } unsigned flags = (rdonly ? POOL_OPEN_COW : 0) | POOL_OPEN_IGNORE_BAD_BLOCKS; if (util_pool_open_nocheck(file->poolset, flags)) goto err_free_fname; file->size = file->poolset->poolsize; /* get modification time from the first part of first replica */ path = file->poolset->replica[0]->part[0].path; file->addr = file->poolset->replica[0]->part[0].addr; } else { int oflag = rdonly ? O_RDONLY : O_RDWR; file->fd = util_file_open(fname, NULL, 0, oflag); file->size = params->size; } os_stat_t buf; if (os_stat(path, &buf)) { ERR("%s", path); goto err_close_poolset; } file->mtime = buf.st_mtime; file->mode = buf.st_mode; return file; err_close_poolset: if (params->type != POOL_TYPE_BTT) util_poolset_close(file->poolset, DO_NOT_DELETE_PARTS); else if (file->fd != -1) os_close(file->fd); err_free_fname: free(file->fname); err: free(file); return NULL; } /* * pool_set_parse -- parse poolset file */ int pool_set_parse(struct pool_set **setp, const char *path) { LOG(3, "setp %p path %s", setp, path); int fd = os_open(path, O_RDONLY); int ret = 0; if (fd < 0) return 1; if (util_poolset_parse(setp, path, fd)) { ret = 1; goto err_close; } err_close: os_close(fd); return ret; } /* * pool_data_alloc -- allocate pool data and open set_file */ struct pool_data * pool_data_alloc(PMEMpoolcheck *ppc) { LOG(3, NULL); struct pool_data *pool = calloc(1, sizeof(*pool)); if (!pool) { ERR("!calloc"); return NULL; } PMDK_TAILQ_INIT(&pool->arenas); pool->uuid_op = UUID_NOP; if (pool_params_parse(ppc, &pool->params, 0)) goto error; int rdonly = CHECK_IS_NOT(ppc, REPAIR); int prv = CHECK_IS(ppc, DRY_RUN); if (prv && pool->params.is_dev_dax) { errno = ENOTSUP; ERR("!cannot perform a dry run on dax device"); goto error; } pool->set_file = pool_set_file_open(ppc->path, &pool->params, prv); if (pool->set_file == NULL) goto error; /* * XXX mprotect for device dax with length not aligned to its * page granularity causes SIGBUS on the next page fault. * The length argument of this call should be changed to * pool->set_file->poolsize once the kernel issue is solved. */ if (rdonly && mprotect(pool->set_file->addr, pool->set_file->poolset->replica[0]->repsize, PROT_READ) < 0) goto error; if (pool->params.type != POOL_TYPE_BTT) { if (pool_set_file_map_headers(pool->set_file, rdonly, prv)) goto error; } return pool; error: pool_data_free(pool); return NULL; } /* * pool_set_file_close -- (internal) closes pool set file or regular file */ static void pool_set_file_close(struct pool_set_file *file) { LOG(3, NULL); if (file->poolset) util_poolset_close(file->poolset, DO_NOT_DELETE_PARTS); else if (file->addr) { munmap(file->addr, file->size); os_close(file->fd); } else if (file->fd) os_close(file->fd); free(file->fname); free(file); } /* * pool_data_free -- close set_file and release pool data */ void pool_data_free(struct pool_data *pool) { LOG(3, NULL); if (pool->set_file) { if (pool->params.type != POOL_TYPE_BTT) pool_set_file_unmap_headers(pool->set_file); pool_set_file_close(pool->set_file); } while (!PMDK_TAILQ_EMPTY(&pool->arenas)) { struct arena *arenap = PMDK_TAILQ_FIRST(&pool->arenas); if (arenap->map) free(arenap->map); if (arenap->flog) free(arenap->flog); PMDK_TAILQ_REMOVE(&pool->arenas, arenap, next); free(arenap); } free(pool); } /* * pool_set_file_map -- return mapped address at given offset */ void * pool_set_file_map(struct pool_set_file *file, uint64_t offset) { if (file->addr == MAP_FAILED) return NULL; return (char *)file->addr + offset; } /* * pool_read -- read from pool set file or regular file * * 'buff' has to be a buffer at least 'nbytes' long * 'off' is an offset from the beginning of the pool */ int pool_read(struct pool_data *pool, void *buff, size_t nbytes, uint64_t off) { if (off + nbytes > pool->set_file->size) return -1; if (pool->params.type != POOL_TYPE_BTT) memcpy(buff, (char *)pool->set_file->addr + off, nbytes); else { if (pool_btt_lseek(pool, (os_off_t)off, SEEK_SET) == -1) return -1; if ((size_t)pool_btt_read(pool, buff, nbytes) != nbytes) return -1; } return 0; } /* * pool_write -- write to pool set file or regular file * * 'buff' has to be a buffer at least 'nbytes' long * 'off' is an offset from the beginning of the pool */ int pool_write(struct pool_data *pool, const void *buff, size_t nbytes, uint64_t off) { if (off + nbytes > pool->set_file->size) return -1; if (pool->params.type != POOL_TYPE_BTT) { memcpy((char *)pool->set_file->addr + off, buff, nbytes); util_persist_auto(pool->params.is_pmem, (char *)pool->set_file->addr + off, nbytes); } else { if (pool_btt_lseek(pool, (os_off_t)off, SEEK_SET) == -1) return -1; if ((size_t)pool_btt_write(pool, buff, nbytes) != nbytes) return -1; } return 0; } /* * pool_copy -- make a copy of the pool */ int pool_copy(struct pool_data *pool, const char *dst_path, int overwrite) { struct pool_set_file *file = pool->set_file; int dfd; int exists = util_file_exists(dst_path); if (exists < 0) return -1; if (exists) { if (!overwrite) { errno = EEXIST; return -1; } dfd = util_file_open(dst_path, NULL, 0, O_RDWR); } else { errno = 0; dfd = util_file_create(dst_path, file->size, 0); } if (dfd < 0) return -1; int result = 0; os_stat_t stat_buf; if (os_stat(file->fname, &stat_buf)) { result = -1; goto out_close; } if (fchmod(dfd, stat_buf.st_mode)) { result = -1; goto out_close; } void *daddr = mmap(NULL, file->size, PROT_READ | PROT_WRITE, MAP_SHARED, dfd, 0); if (daddr == MAP_FAILED) { result = -1; goto out_close; } if (pool->params.type != POOL_TYPE_BTT) { void *saddr = pool_set_file_map(file, 0); memcpy(daddr, saddr, file->size); goto out_unmap; } void *buf = malloc(RW_BUFFERING_SIZE); if (buf == NULL) { ERR("!malloc"); result = -1; goto out_unmap; } if (pool_btt_lseek(pool, 0, SEEK_SET) == -1) { result = -1; goto out_free; } ssize_t buf_read = 0; void *dst = daddr; while ((buf_read = pool_btt_read(pool, buf, RW_BUFFERING_SIZE))) { if (buf_read == -1) break; memcpy(dst, buf, (size_t)buf_read); dst = (void *)((ssize_t)dst + buf_read); } out_free: free(buf); out_unmap: munmap(daddr, file->size); out_close: (void) os_close(dfd); return result; } /* * pool_set_part_copy -- make a copy of the poolset part */ int pool_set_part_copy(struct pool_set_part *dpart, struct pool_set_part *spart, int overwrite) { LOG(3, "dpart %p spart %p", dpart, spart); int result = 0; os_stat_t stat_buf; if (os_fstat(spart->fd, &stat_buf)) { ERR("!util_stat"); return -1; } size_t smapped = 0; void *saddr = pmem_map_file(spart->path, 0, 0, S_IREAD, &smapped, NULL); if (!saddr) return -1; size_t dmapped = 0; int is_pmem; void *daddr; int exists = util_file_exists(dpart->path); if (exists < 0) { result = -1; goto out_sunmap; } if (exists) { if (!overwrite) { errno = EEXIST; result = -1; goto out_sunmap; } daddr = pmem_map_file(dpart->path, 0, 0, S_IWRITE, &dmapped, &is_pmem); } else { errno = 0; daddr = pmem_map_file(dpart->path, dpart->filesize, PMEM_FILE_CREATE | PMEM_FILE_EXCL, stat_buf.st_mode, &dmapped, &is_pmem); } if (!daddr) { result = -1; goto out_sunmap; } #ifdef DEBUG /* provide extra logging in case of wrong dmapped/smapped value */ if (dmapped < smapped) { LOG(1, "dmapped < smapped: dmapped = %lu, smapped = %lu", dmapped, smapped); ASSERT(0); } #endif if (is_pmem) { pmem_memcpy_persist(daddr, saddr, smapped); } else { memcpy(daddr, saddr, smapped); pmem_msync(daddr, smapped); } pmem_unmap(daddr, dmapped); out_sunmap: pmem_unmap(saddr, smapped); return result; } /* * pool_memset -- memset pool part described by off and count */ int pool_memset(struct pool_data *pool, uint64_t off, int c, size_t count) { int result = 0; if (pool->params.type != POOL_TYPE_BTT) memset((char *)off, 0, count); else { if (pool_btt_lseek(pool, (os_off_t)off, SEEK_SET) == -1) return -1; size_t zero_size = min(count, RW_BUFFERING_SIZE); void *buf = malloc(zero_size); if (!buf) { ERR("!malloc"); return -1; } memset(buf, c, zero_size); ssize_t nwrite = 0; do { zero_size = min(zero_size, count); nwrite = pool_btt_write(pool, buf, zero_size); if (nwrite < 0) { result = -1; break; } count -= (size_t)nwrite; } while (count > 0); free(buf); } return result; } /* * pool_set_files_count -- get total number of parts of all replicas */ unsigned pool_set_files_count(struct pool_set_file *file) { unsigned ret = 0; unsigned nreplicas = file->poolset->nreplicas; for (unsigned r = 0; r < nreplicas; r++) { struct pool_replica *rep = file->poolset->replica[r]; ret += rep->nparts; } return ret; } /* * pool_set_file_map_headers -- map headers of each pool set part file */ int pool_set_file_map_headers(struct pool_set_file *file, int rdonly, int prv) { if (!file->poolset) return -1; for (unsigned r = 0; r < file->poolset->nreplicas; r++) { struct pool_replica *rep = file->poolset->replica[r]; for (unsigned p = 0; p < rep->nparts; p++) { struct pool_set_part *part = &rep->part[p]; if (util_map_hdr(part, prv ? MAP_PRIVATE : MAP_SHARED, rdonly)) { part->hdr = NULL; goto err; } } } return 0; err: pool_set_file_unmap_headers(file); return -1; } /* * pool_set_file_unmap_headers -- unmap headers of each pool set part file */ void pool_set_file_unmap_headers(struct pool_set_file *file) { if (!file->poolset) return; for (unsigned r = 0; r < file->poolset->nreplicas; r++) { struct pool_replica *rep = file->poolset->replica[r]; for (unsigned p = 0; p < rep->nparts; p++) { struct pool_set_part *part = &rep->part[p]; util_unmap_hdr(part); } } } /* * pool_get_signature -- (internal) return signature of specified pool type */ static const char * pool_get_signature(enum pool_type type) { switch (type) { case POOL_TYPE_LOG: return LOG_HDR_SIG; case POOL_TYPE_BLK: return BLK_HDR_SIG; case POOL_TYPE_OBJ: return OBJ_HDR_SIG; default: return NULL; } } /* * pool_hdr_default -- return default pool header values */ void pool_hdr_default(enum pool_type type, struct pool_hdr *hdrp) { memset(hdrp, 0, sizeof(*hdrp)); const char *sig = pool_get_signature(type); ASSERTne(sig, NULL); memcpy(hdrp->signature, sig, POOL_HDR_SIG_LEN); switch (type) { case POOL_TYPE_LOG: hdrp->major = LOG_FORMAT_MAJOR; hdrp->features = log_format_feat_default; break; case POOL_TYPE_BLK: hdrp->major = BLK_FORMAT_MAJOR; hdrp->features = blk_format_feat_default; break; case POOL_TYPE_OBJ: hdrp->major = OBJ_FORMAT_MAJOR; hdrp->features = obj_format_feat_default; break; default: break; } } /* * pool_hdr_get_type -- return pool type based on pool header data */ enum pool_type pool_hdr_get_type(const struct pool_hdr *hdrp) { if (memcmp(hdrp->signature, LOG_HDR_SIG, POOL_HDR_SIG_LEN) == 0) return POOL_TYPE_LOG; else if (memcmp(hdrp->signature, BLK_HDR_SIG, POOL_HDR_SIG_LEN) == 0) return POOL_TYPE_BLK; else if (memcmp(hdrp->signature, OBJ_HDR_SIG, POOL_HDR_SIG_LEN) == 0) return POOL_TYPE_OBJ; else return POOL_TYPE_UNKNOWN; } /* * pool_get_pool_type_str -- return human-readable pool type string */ const char * pool_get_pool_type_str(enum pool_type type) { switch (type) { case POOL_TYPE_BTT: return "btt"; case POOL_TYPE_LOG: return "pmemlog"; case POOL_TYPE_BLK: return "pmemblk"; case POOL_TYPE_OBJ: return "pmemobj"; default: return "unknown"; } } /* * pool_set_type -- get pool type of a poolset */ enum pool_type pool_set_type(struct pool_set *set) { struct pool_hdr hdr; /* open the first part file to read the pool header values */ const struct pool_set_part *part = PART(REP(set, 0), 0); if (util_file_pread(part->path, &hdr, sizeof(hdr), 0) != sizeof(hdr)) { ERR("cannot read pool header from poolset"); return POOL_TYPE_UNKNOWN; } util_convert2h_hdr_nocheck(&hdr); enum pool_type type = pool_hdr_get_type(&hdr); return type; } /* * pool_btt_info_valid -- check consistency of BTT Info header */ int pool_btt_info_valid(struct btt_info *infop) { if (memcmp(infop->sig, BTTINFO_SIG, BTTINFO_SIG_LEN) != 0) return 0; return util_checksum(infop, sizeof(*infop), &infop->checksum, 0, 0); } /* * pool_blk_get_first_valid_arena -- get first valid BTT Info in arena */ int pool_blk_get_first_valid_arena(struct pool_data *pool, struct arena *arenap) { arenap->zeroed = true; uint64_t offset = pool_get_first_valid_btt(pool, &arenap->btt_info, 2 * BTT_ALIGNMENT, &arenap->zeroed); if (offset != 0) { arenap->offset = offset; arenap->valid = true; return 1; } return 0; } /* * pool_next_arena_offset -- get offset of next arena * * Calculated offset is theoretical. Function does not check if such arena can * exist. */ uint64_t pool_next_arena_offset(struct pool_data *pool, uint64_t offset) { uint64_t lastoff = (pool->set_file->size & ~(BTT_ALIGNMENT - 1)); uint64_t nextoff = min(offset + BTT_MAX_ARENA, lastoff); return nextoff; } /* * pool_get_first_valid_btt -- return offset to first valid BTT Info * * - Return offset to valid BTT Info header in pool file. * - Start looking from given offset. * - Convert BTT Info header to host endianness. * - Return the BTT Info header by pointer. * - If zeroed pointer provided would check if all checked BTT Info are zeroed * which is useful for BLK pools */ uint64_t pool_get_first_valid_btt(struct pool_data *pool, struct btt_info *infop, uint64_t offset, bool *zeroed) { /* if we have valid arena get BTT Info header from it */ if (pool->narenas != 0) { struct arena *arenap = PMDK_TAILQ_FIRST(&pool->arenas); memcpy(infop, &arenap->btt_info, sizeof(*infop)); return arenap->offset; } const size_t info_size = sizeof(*infop); /* theoretical offsets to BTT Info header and backup */ uint64_t offsets[2] = {offset, 0}; while (offsets[0] < pool->set_file->size) { /* calculate backup offset */ offsets[1] = pool_next_arena_offset(pool, offsets[0]) - info_size; /* check both offsets: header and backup */ for (int i = 0; i < 2; ++i) { if (pool_read(pool, infop, info_size, offsets[i])) continue; /* check if all possible BTT Info are zeroed */ if (zeroed) *zeroed &= util_is_zeroed((const void *)infop, info_size); /* check if read BTT Info is valid */ if (pool_btt_info_valid(infop)) { btt_info_convert2h(infop); return offsets[i]; } } /* jump to next arena */ offsets[0] += BTT_MAX_ARENA; } return 0; } /* * pool_get_min_size -- return the minimum pool size of a pool of a given type */ size_t pool_get_min_size(enum pool_type type) { switch (type) { case POOL_TYPE_LOG: return PMEMLOG_MIN_POOL; case POOL_TYPE_BLK: return PMEMBLK_MIN_POOL; case POOL_TYPE_OBJ: return PMEMOBJ_MIN_POOL; default: ERR("unknown type of a pool"); return SIZE_MAX; } } #if FAULT_INJECTION void pmempool_inject_fault_at(enum pmem_allocation_type type, int nth, const char *at) { common_inject_fault_at(type, nth, at); } int pmempool_fault_injection_enabled(void) { return common_fault_injection_enabled(); } #endif pmdk-1.8/src/libpmempool/libpmempool.link.in0000664000000000000000000000362313615011243017726 0ustar rootroot# # Copyright 2016-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # src/libpmempool.link -- linker link file for libpmempool # LIBPMEMPOOL_1.0 { global: pmempool_errormsg; pmempool_check_version; pmempool_check_init; pmempool_check; pmempool_check_end; pmempool_transform; pmempool_sync; pmempool_rm; pmempool_feature_enable; pmempool_feature_disable; pmempool_feature_query; fault_injection; local: *; }; pmdk-1.8/src/libpmempool/check_blk.c0000664000000000000000000001521013615011243016167 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_blk.c -- check pmemblk */ #include #include #include #include "out.h" #include "btt.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check_util.h" enum question { Q_BLK_BSIZE, }; /* * blk_get_max_bsize -- (internal) return maximum size of block for given file * size */ static inline uint32_t blk_get_max_bsize(uint64_t fsize) { LOG(3, NULL); if (fsize == 0) return 0; /* default nfree */ uint32_t nfree = BTT_DEFAULT_NFREE; /* number of blocks must be at least 2 * nfree */ uint32_t internal_nlba = 2 * nfree; /* compute arena size from file size without pmemblk structure */ uint64_t arena_size = fsize - sizeof(struct pmemblk); if (arena_size > BTT_MAX_ARENA) arena_size = BTT_MAX_ARENA; arena_size = btt_arena_datasize(arena_size, nfree); /* compute maximum internal LBA size */ uint64_t internal_lbasize = (arena_size - BTT_ALIGNMENT) / internal_nlba - BTT_MAP_ENTRY_SIZE; ASSERT(internal_lbasize <= UINT32_MAX); if (internal_lbasize < BTT_MIN_LBA_SIZE) internal_lbasize = BTT_MIN_LBA_SIZE; internal_lbasize = roundup(internal_lbasize, BTT_INTERNAL_LBA_ALIGNMENT) - BTT_INTERNAL_LBA_ALIGNMENT; return (uint32_t)internal_lbasize; } /* * blk_read -- (internal) read pmemblk header */ static int blk_read(PMEMpoolcheck *ppc) { /* * Here we want to read the pmemblk header without the pool_hdr as we've * already done it before. * * Take the pointer to fields right after pool_hdr, compute the size and * offset of remaining fields. */ uint8_t *ptr = (uint8_t *)&ppc->pool->hdr.blk; ptr += sizeof(ppc->pool->hdr.blk.hdr); size_t size = sizeof(ppc->pool->hdr.blk) - sizeof(ppc->pool->hdr.blk.hdr); uint64_t offset = sizeof(ppc->pool->hdr.blk.hdr); if (pool_read(ppc->pool, ptr, size, offset)) { return CHECK_ERR(ppc, "cannot read pmemblk structure"); } /* endianness conversion */ ppc->pool->hdr.blk.bsize = le32toh(ppc->pool->hdr.blk.bsize); return 0; } /* * blk_bsize_valid -- (internal) check if block size is valid for given file * size */ static int blk_bsize_valid(uint32_t bsize, uint64_t fsize) { uint32_t max_bsize = blk_get_max_bsize(fsize); return (bsize >= max_bsize); } /* * blk_hdr_check -- (internal) check pmemblk header */ static int blk_hdr_check(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); CHECK_INFO(ppc, "checking pmemblk header"); if (blk_read(ppc)) { ppc->result = CHECK_RESULT_ERROR; return -1; } /* check for valid BTT Info arena as we can take bsize from it */ if (!ppc->pool->bttc.valid) pool_blk_get_first_valid_arena(ppc->pool, &ppc->pool->bttc); if (ppc->pool->bttc.valid) { const uint32_t btt_bsize = ppc->pool->bttc.btt_info.external_lbasize; if (ppc->pool->hdr.blk.bsize != btt_bsize) { CHECK_ASK(ppc, Q_BLK_BSIZE, "invalid pmemblk.bsize.|Do you want to set " "pmemblk.bsize to %u from BTT Info?", btt_bsize); } } else if (!ppc->pool->bttc.zeroed) { if (ppc->pool->hdr.blk.bsize < BTT_MIN_LBA_SIZE || blk_bsize_valid(ppc->pool->hdr.blk.bsize, ppc->pool->set_file->size)) { ppc->result = CHECK_RESULT_CANNOT_REPAIR; return CHECK_ERR(ppc, "invalid pmemblk.bsize"); } } if (ppc->result == CHECK_RESULT_CONSISTENT || ppc->result == CHECK_RESULT_REPAIRED) CHECK_INFO(ppc, "pmemblk header correct"); return check_questions_sequence_validate(ppc); } /* * blk_hdr_fix -- (internal) fix pmemblk header */ static int blk_hdr_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *ctx) { LOG(3, NULL); uint32_t btt_bsize; switch (question) { case Q_BLK_BSIZE: /* * check for valid BTT Info arena as we can take bsize from it */ if (!ppc->pool->bttc.valid) pool_blk_get_first_valid_arena(ppc->pool, &ppc->pool->bttc); btt_bsize = ppc->pool->bttc.btt_info.external_lbasize; CHECK_INFO(ppc, "setting pmemblk.b_size to 0x%x", btt_bsize); ppc->pool->hdr.blk.bsize = btt_bsize; break; default: ERR("not implemented question id: %u", question); } return 0; } struct step { int (*check)(PMEMpoolcheck *, location *); int (*fix)(PMEMpoolcheck *, location *, uint32_t, void *); enum pool_type type; }; static const struct step steps[] = { { .check = blk_hdr_check, .type = POOL_TYPE_BLK }, { .fix = blk_hdr_fix, .type = POOL_TYPE_BLK }, { .check = NULL, .fix = NULL, }, }; /* * step_exe -- (internal) perform single step according to its parameters */ static inline int step_exe(PMEMpoolcheck *ppc, location *loc) { ASSERT(loc->step < ARRAY_SIZE(steps)); ASSERTeq(ppc->pool->params.type, POOL_TYPE_BLK); const struct step *step = &steps[loc->step++]; if (!(step->type & ppc->pool->params.type)) return 0; if (!step->fix) return step->check(ppc, loc); if (blk_read(ppc)) { ppc->result = CHECK_RESULT_ERROR; return -1; } return check_answer_loop(ppc, loc, NULL, 1, step->fix); } /* * check_blk -- entry point for pmemblk checks */ void check_blk(PMEMpoolcheck *ppc) { LOG(3, NULL); location *loc = check_get_step_data(ppc->data); /* do all checks */ while (CHECK_NOT_COMPLETE(loc, steps)) { if (step_exe(ppc, loc)) break; } } pmdk-1.8/src/libpmempool/libpmempool.c0000664000000000000000000002464113615011243016611 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * libpmempool.c -- entry points for libpmempool */ #include #include #include #include #include "pmemcommon.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check.h" #ifdef USE_RPMEM #include "rpmem_common.h" #include "rpmem_util.h" #endif #ifdef _WIN32 #define ANSWER_BUFFSIZE 256 #endif /* * libpmempool_init -- load-time initialization for libpmempool * * Called automatically by the run-time loader. */ ATTR_CONSTRUCTOR void libpmempool_init(void) { common_init(PMEMPOOL_LOG_PREFIX, PMEMPOOL_LOG_LEVEL_VAR, PMEMPOOL_LOG_FILE_VAR, PMEMPOOL_MAJOR_VERSION, PMEMPOOL_MINOR_VERSION); LOG(3, NULL); #ifdef USE_RPMEM util_remote_init(); rpmem_util_cmds_init(); #endif } /* * libpmempool_fini -- libpmempool cleanup routine * * Called automatically when the process terminates. */ ATTR_DESTRUCTOR void libpmempool_fini(void) { LOG(3, NULL); #ifdef USE_RPMEM util_remote_unload(); util_remote_fini(); rpmem_util_cmds_fini(); #endif common_fini(); } /* * pmempool_check_versionU -- see if library meets application version * requirements */ #ifndef _WIN32 static inline #endif const char * pmempool_check_versionU(unsigned major_required, unsigned minor_required) { LOG(3, "major_required %u minor_required %u", major_required, minor_required); if (major_required != PMEMPOOL_MAJOR_VERSION) { ERR("libpmempool major version mismatch (need %u, found %u)", major_required, PMEMPOOL_MAJOR_VERSION); return out_get_errormsg(); } if (minor_required > PMEMPOOL_MINOR_VERSION) { ERR("libpmempool minor version mismatch (need %u, found %u)", minor_required, PMEMPOOL_MINOR_VERSION); return out_get_errormsg(); } return NULL; } #ifndef _WIN32 /* * pmempool_check_version -- see if lib meets application version requirements */ const char * pmempool_check_version(unsigned major_required, unsigned minor_required) { return pmempool_check_versionU(major_required, minor_required); } #else /* * pmempool_check_versionW -- see if library meets application version * requirements as widechar */ const wchar_t * pmempool_check_versionW(unsigned major_required, unsigned minor_required) { if (pmempool_check_versionU(major_required, minor_required) != NULL) return out_get_errormsgW(); else return NULL; } #endif /* * pmempool_errormsgU -- return last error message */ #ifndef _WIN32 static inline #endif const char * pmempool_errormsgU(void) { return out_get_errormsg(); } #ifndef _WIN32 /* * pmempool_errormsg -- return last error message */ const char * pmempool_errormsg(void) { return pmempool_errormsgU(); } #else /* * pmempool_errormsgW -- return last error message as widechar */ const wchar_t * pmempool_errormsgW(void) { return out_get_errormsgW(); } #endif /* * pmempool_ppc_set_default -- (internal) set default values of check context */ static void pmempool_ppc_set_default(PMEMpoolcheck *ppc) { /* all other fields should be zeroed */ const PMEMpoolcheck ppc_default = { .args = { .pool_type = PMEMPOOL_POOL_TYPE_DETECT, }, .result = CHECK_RESULT_CONSISTENT, }; *ppc = ppc_default; } /* * pmempool_check_initU -- initialize check context */ #ifndef _WIN32 static inline #endif PMEMpoolcheck * pmempool_check_initU(struct pmempool_check_argsU *args, size_t args_size) { LOG(3, "path %s backup_path %s pool_type %u flags %x", args->path, args->backup_path, args->pool_type, args->flags); /* * Currently one size of args structure is supported. The version of the * pmempool_check_args structure can be distinguished based on provided * args_size. */ if (args_size < sizeof(struct pmempool_check_args)) { ERR("provided args_size is not supported"); errno = EINVAL; return NULL; } /* * Dry run does not allow to made changes possibly performed during * repair. Advanced allow to perform more complex repairs. Questions * are ask only if repairs are made. So dry run, advanced and always_yes * can be set only if repair is set. */ if (util_flag_isclr(args->flags, PMEMPOOL_CHECK_REPAIR) && util_flag_isset(args->flags, PMEMPOOL_CHECK_DRY_RUN | PMEMPOOL_CHECK_ADVANCED | PMEMPOOL_CHECK_ALWAYS_YES)) { ERR("dry_run, advanced and always_yes are applicable only if " "repair is set"); errno = EINVAL; return NULL; } /* * dry run does not modify anything so performing backup is redundant */ if (util_flag_isset(args->flags, PMEMPOOL_CHECK_DRY_RUN) && args->backup_path != NULL) { ERR("dry run does not allow one to perform backup"); errno = EINVAL; return NULL; } /* * libpmempool uses str format of communication so it must be set */ if (util_flag_isclr(args->flags, PMEMPOOL_CHECK_FORMAT_STR)) { ERR("PMEMPOOL_CHECK_FORMAT_STR flag must be set"); errno = EINVAL; return NULL; } PMEMpoolcheck *ppc = calloc(1, sizeof(*ppc)); if (ppc == NULL) { ERR("!calloc"); return NULL; } pmempool_ppc_set_default(ppc); memcpy(&ppc->args, args, sizeof(ppc->args)); ppc->path = strdup(args->path); if (!ppc->path) { ERR("!strdup"); goto error_path_malloc; } ppc->args.path = ppc->path; if (args->backup_path != NULL) { ppc->backup_path = strdup(args->backup_path); if (!ppc->backup_path) { ERR("!strdup"); goto error_backup_path_malloc; } ppc->args.backup_path = ppc->backup_path; } if (check_init(ppc) != 0) goto error_check_init; return ppc; error_check_init: /* in case errno not set by any of the used functions set its value */ if (errno == 0) errno = EINVAL; free(ppc->backup_path); error_backup_path_malloc: free(ppc->path); error_path_malloc: free(ppc); return NULL; } #ifndef _WIN32 /* * pmempool_check_init -- initialize check context */ PMEMpoolcheck * pmempool_check_init(struct pmempool_check_args *args, size_t args_size) { return pmempool_check_initU(args, args_size); } #else /* * pmempool_check_initW -- initialize check context as widechar */ PMEMpoolcheck * pmempool_check_initW(struct pmempool_check_argsW *args, size_t args_size) { char *upath = util_toUTF8(args->path); if (upath == NULL) return NULL; char *ubackup_path = NULL; if (args->backup_path != NULL) { ubackup_path = util_toUTF8(args->backup_path); if (ubackup_path == NULL) { util_free_UTF8(upath); return NULL; } } struct pmempool_check_argsU uargs = { .path = upath, .backup_path = ubackup_path, .pool_type = args->pool_type, .flags = args->flags }; PMEMpoolcheck *ret = pmempool_check_initU(&uargs, args_size); util_free_UTF8(ubackup_path); util_free_UTF8(upath); return ret; } #endif /* * pmempool_checkU -- continue check till produce status to consume for caller */ #ifndef _WIN32 static inline #endif struct pmempool_check_statusU * pmempool_checkU(PMEMpoolcheck *ppc) { LOG(3, NULL); ASSERTne(ppc, NULL); struct check_status *result; do { result = check_step(ppc); if (check_is_end(ppc->data) && result == NULL) return NULL; } while (result == NULL); return check_status_get(result); } #ifndef _WIN32 /* * pmempool_check -- continue check till produce status to consume for caller */ struct pmempool_check_status * pmempool_check(PMEMpoolcheck *ppc) { return pmempool_checkU(ppc); } #else /* * pmempool_checkW -- continue check till produce status to consume for caller */ struct pmempool_check_statusW * pmempool_checkW(PMEMpoolcheck *ppc) { LOG(3, NULL); ASSERTne(ppc, NULL); /* check the cache and convert msg and answer */ char buf[ANSWER_BUFFSIZE]; memset(buf, 0, ANSWER_BUFFSIZE); convert_status_cache(ppc, buf, ANSWER_BUFFSIZE); struct check_status *uresult; do { uresult = check_step(ppc); if (check_is_end(ppc->data) && uresult == NULL) return NULL; } while (uresult == NULL); struct pmempool_check_statusU *uret_res = check_status_get(uresult); const wchar_t *wmsg = util_toUTF16(uret_res->str.msg); if (wmsg == NULL) FATAL("!malloc"); struct pmempool_check_statusW *wret_res = (struct pmempool_check_statusW *)uret_res; /* pointer to old message is freed in next check step */ wret_res->str.msg = wmsg; return wret_res; } #endif /* * pmempool_check_end -- end check and release check context */ enum pmempool_check_result pmempool_check_end(PMEMpoolcheck *ppc) { LOG(3, NULL); const enum check_result result = ppc->result; const unsigned sync_required = ppc->sync_required; check_fini(ppc); free(ppc->path); free(ppc->backup_path); free(ppc); if (sync_required) { switch (result) { case CHECK_RESULT_CONSISTENT: case CHECK_RESULT_REPAIRED: return PMEMPOOL_CHECK_RESULT_SYNC_REQ; default: /* other results require fixing prior to sync */ ; } } switch (result) { case CHECK_RESULT_CONSISTENT: return PMEMPOOL_CHECK_RESULT_CONSISTENT; case CHECK_RESULT_NOT_CONSISTENT: return PMEMPOOL_CHECK_RESULT_NOT_CONSISTENT; case CHECK_RESULT_REPAIRED: return PMEMPOOL_CHECK_RESULT_REPAIRED; case CHECK_RESULT_CANNOT_REPAIR: return PMEMPOOL_CHECK_RESULT_CANNOT_REPAIR; default: return PMEMPOOL_CHECK_RESULT_ERROR; } } pmdk-1.8/src/libpmempool/pmempool.h0000664000000000000000000000461213615011243016123 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmempool.h -- internal definitions for libpmempool */ #ifndef PMEMPOOL_H #define PMEMPOOL_H #ifdef __cplusplus extern "C" { #endif #define PMEMPOOL_LOG_PREFIX "libpmempool" #define PMEMPOOL_LOG_LEVEL_VAR "PMEMPOOL_LOG_LEVEL" #define PMEMPOOL_LOG_FILE_VAR "PMEMPOOL_LOG_FILE" enum check_result { CHECK_RESULT_CONSISTENT, CHECK_RESULT_NOT_CONSISTENT, CHECK_RESULT_ASK_QUESTIONS, CHECK_RESULT_PROCESS_ANSWERS, CHECK_RESULT_REPAIRED, CHECK_RESULT_CANNOT_REPAIR, CHECK_RESULT_ERROR, CHECK_RESULT_INTERNAL_ERROR }; /* * pmempool_check_ctx -- context and arguments for check command */ struct pmempool_check_ctx { struct pmempool_check_args args; char *path; char *backup_path; struct check_data *data; struct pool_data *pool; enum check_result result; unsigned sync_required; }; #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmempool/Makefile0000664000000000000000000000526213615011243015564 0ustar rootroot# Copyright 2016-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # src/libpmempool/Makefile -- Makefile for libpmempool # LIBRARY_NAME = pmempool LIBRARY_SO_VERSION = 1 LIBRARY_VERSION = 0.0 INCS += -I$(TOP)/src/libpmemlog INCS += -I$(TOP)/src/libpmemblk INCS += -I$(TOP)/src/libpmemobj INCS += -I$(TOP)/src/rpmem_common INCS += -I$(TOP)/src/librpmem vpath %.c ../librpmem include ../common/pmemcommon.inc SOURCE +=\ libpmempool.c\ check.c\ check_bad_blocks.c\ check_backup.c\ check_btt_info.c\ check_btt_map_flog.c\ check_log.c\ check_blk.c\ check_pool_hdr.c\ check_sds.c\ check_util.c\ check_write.c\ pool.c\ replica.c\ feature.c\ $(RPMEM_COMMON)/rpmem_common.c\ rpmem_ssh.c\ rpmem_cmd.c\ rpmem_util.c\ sync.c\ transform.c\ rm.c LIBPMEMBLK_PRIV_FUNCS=btt_info_set btt_arena_datasize btt_flog_size\ btt_map_size btt_flog_get_valid map_entry_is_initial btt_info_convert2h\ btt_info_convert2le btt_flog_convert2h btt_flog_convert2le include ../Makefile.inc CFLAGS += $(LIBNDCTL_CFLAGS) LIBS += -pthread -lpmem $(LIBDL) $(LIBNDCTL_LIBS) CFLAGS += -DUSE_LIBDL CFLAGS += -DUSE_RPMEM pmemblk_priv_funcs.o: $(PMEMBLK_PRIV_OBJ) $(OBJCOPY) --localize-hidden $(addprefix -G, $(LIBPMEMBLK_PRIV_FUNCS)) \ $< $@ pmdk-1.8/src/libpmempool/check.h0000664000000000000000000000411213615011243015343 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check.h -- internal definitions for logic performing check */ #ifndef CHECK_H #define CHECK_H #ifdef __cplusplus extern "C" { #endif int check_init(PMEMpoolcheck *ppc); struct check_status *check_step(PMEMpoolcheck *ppc); void check_fini(PMEMpoolcheck *ppc); int check_is_end(struct check_data *data); struct pmempool_check_status *check_status_get(struct check_status *status); #ifdef _WIN32 void convert_status_cache(PMEMpoolcheck *ppc, char *buf, size_t size); #endif #ifdef __cplusplus } #endif #endif pmdk-1.8/src/libpmempool/check_log.c0000664000000000000000000001420313615011243016201 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check_log.c -- check pmemlog */ #include #include #include #include "out.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check_util.h" enum question { Q_LOG_START_OFFSET, Q_LOG_END_OFFSET, Q_LOG_WRITE_OFFSET, }; /* * log_read -- (internal) read pmemlog header */ static int log_read(PMEMpoolcheck *ppc) { /* * Here we want to read the pmemlog header without the pool_hdr as we've * already done it before. * * Take the pointer to fields right after pool_hdr, compute the size and * offset of remaining fields. */ uint8_t *ptr = (uint8_t *)&ppc->pool->hdr.log; ptr += sizeof(ppc->pool->hdr.log.hdr); size_t size = sizeof(ppc->pool->hdr.log) - sizeof(ppc->pool->hdr.log.hdr); uint64_t offset = sizeof(ppc->pool->hdr.log.hdr); if (pool_read(ppc->pool, ptr, size, offset)) return CHECK_ERR(ppc, "cannot read pmemlog structure"); /* endianness conversion */ log_convert2h(&ppc->pool->hdr.log); return 0; } /* * log_hdr_check -- (internal) check pmemlog header */ static int log_hdr_check(PMEMpoolcheck *ppc, location *loc) { LOG(3, NULL); CHECK_INFO(ppc, "checking pmemlog header"); if (log_read(ppc)) { ppc->result = CHECK_RESULT_ERROR; return -1; } /* determine constant values for pmemlog */ const uint64_t d_start_offset = roundup(sizeof(ppc->pool->hdr.log), LOG_FORMAT_DATA_ALIGN); if (ppc->pool->hdr.log.start_offset != d_start_offset) { if (CHECK_ASK(ppc, Q_LOG_START_OFFSET, "invalid pmemlog.start_offset: 0x%jx.|Do you " "want to set pmemlog.start_offset to default " "0x%jx?", ppc->pool->hdr.log.start_offset, d_start_offset)) goto error; } if (ppc->pool->hdr.log.end_offset != ppc->pool->set_file->size) { if (CHECK_ASK(ppc, Q_LOG_END_OFFSET, "invalid pmemlog.end_offset: 0x%jx.|Do you " "want to set pmemlog.end_offset to 0x%jx?", ppc->pool->hdr.log.end_offset, ppc->pool->set_file->size)) goto error; } if (ppc->pool->hdr.log.write_offset < d_start_offset || ppc->pool->hdr.log.write_offset > ppc->pool->set_file->size) { if (CHECK_ASK(ppc, Q_LOG_WRITE_OFFSET, "invalid pmemlog.write_offset: 0x%jx.|Do you " "want to set pmemlog.write_offset to " "pmemlog.end_offset?", ppc->pool->hdr.log.write_offset)) goto error; } if (ppc->result == CHECK_RESULT_CONSISTENT || ppc->result == CHECK_RESULT_REPAIRED) CHECK_INFO(ppc, "pmemlog header correct"); return check_questions_sequence_validate(ppc); error: ppc->result = CHECK_RESULT_NOT_CONSISTENT; check_end(ppc->data); return -1; } /* * log_hdr_fix -- (internal) fix pmemlog header */ static int log_hdr_fix(PMEMpoolcheck *ppc, location *loc, uint32_t question, void *ctx) { LOG(3, NULL); uint64_t d_start_offset; switch (question) { case Q_LOG_START_OFFSET: /* determine constant values for pmemlog */ d_start_offset = roundup(sizeof(ppc->pool->hdr.log), LOG_FORMAT_DATA_ALIGN); CHECK_INFO(ppc, "setting pmemlog.start_offset to 0x%jx", d_start_offset); ppc->pool->hdr.log.start_offset = d_start_offset; break; case Q_LOG_END_OFFSET: CHECK_INFO(ppc, "setting pmemlog.end_offset to 0x%jx", ppc->pool->set_file->size); ppc->pool->hdr.log.end_offset = ppc->pool->set_file->size; break; case Q_LOG_WRITE_OFFSET: CHECK_INFO(ppc, "setting pmemlog.write_offset to " "pmemlog.end_offset"); ppc->pool->hdr.log.write_offset = ppc->pool->set_file->size; break; default: ERR("not implemented question id: %u", question); } return 0; } struct step { int (*check)(PMEMpoolcheck *, location *); int (*fix)(PMEMpoolcheck *, location *, uint32_t, void *); enum pool_type type; }; static const struct step steps[] = { { .check = log_hdr_check, .type = POOL_TYPE_LOG }, { .fix = log_hdr_fix, .type = POOL_TYPE_LOG }, { .check = NULL, .fix = NULL, }, }; /* * step_exe -- (internal) perform single step according to its parameters */ static inline int step_exe(PMEMpoolcheck *ppc, location *loc) { ASSERT(loc->step < ARRAY_SIZE(steps)); ASSERTeq(ppc->pool->params.type, POOL_TYPE_LOG); const struct step *step = &steps[loc->step++]; if (!(step->type & ppc->pool->params.type)) return 0; if (!step->fix) return step->check(ppc, loc); if (log_read(ppc)) { ppc->result = CHECK_RESULT_ERROR; return -1; } return check_answer_loop(ppc, loc, NULL, 1, step->fix); } /* * check_log -- entry point for pmemlog checks */ void check_log(PMEMpoolcheck *ppc) { LOG(3, NULL); location *loc = check_get_step_data(ppc->data); /* do all checks */ while (CHECK_NOT_COMPLETE(loc, steps)) { if (step_exe(ppc, loc)) break; } } pmdk-1.8/src/libpmempool/check.c0000664000000000000000000001402613615011243015343 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * check.c -- functions performing checks in proper order */ #include #include "out.h" #include "libpmempool.h" #include "pmempool.h" #include "pool.h" #include "check.h" #include "check_util.h" #define CHECK_RESULT_IS_STOP(result)\ ((result) == CHECK_RESULT_ERROR ||\ (result) == CHECK_RESULT_INTERNAL_ERROR ||\ ((result) == CHECK_RESULT_CANNOT_REPAIR) ||\ ((result) == CHECK_RESULT_NOT_CONSISTENT)) struct step { void (*func)(PMEMpoolcheck *); enum pool_type type; bool part; }; static const struct step steps[] = { { .type = POOL_TYPE_ANY, .func = check_bad_blocks, .part = true, }, { .type = POOL_TYPE_ANY, .func = check_backup, .part = true, }, { .type = POOL_TYPE_BLK | POOL_TYPE_LOG | POOL_TYPE_OBJ, .func = check_sds, .part = true, }, { .type = POOL_TYPE_BLK | POOL_TYPE_LOG | POOL_TYPE_OBJ | POOL_TYPE_UNKNOWN, .func = check_pool_hdr, .part = true, }, { .type = POOL_TYPE_BLK | POOL_TYPE_LOG | POOL_TYPE_OBJ | POOL_TYPE_UNKNOWN, .func = check_pool_hdr_uuids, .part = true, }, { .type = POOL_TYPE_LOG, .func = check_log, .part = false, }, { .type = POOL_TYPE_BLK, .func = check_blk, .part = false, }, { .type = POOL_TYPE_BLK | POOL_TYPE_BTT, .func = check_btt_info, .part = false, }, { .type = POOL_TYPE_BLK | POOL_TYPE_BTT, .func = check_btt_map_flog, .part = false, }, { .type = POOL_TYPE_BLK | POOL_TYPE_LOG | POOL_TYPE_BTT, .func = check_write, .part = false, }, { .func = NULL, }, }; /* * check_init -- initialize check process */ int check_init(PMEMpoolcheck *ppc) { LOG(3, NULL); if (!(ppc->data = check_data_alloc())) goto error_data_malloc; if (!(ppc->pool = pool_data_alloc(ppc))) goto error_pool_malloc; return 0; error_pool_malloc: check_data_free(ppc->data); error_data_malloc: return -1; } #ifdef _WIN32 void convert_status_cache(PMEMpoolcheck *ppc, char *buf, size_t size) { cache_to_utf8(ppc->data, buf, size); } #endif /* * status_get -- (internal) get next check_status * * The assumed order of check_statuses is: all info messages, error or question. */ static struct check_status * status_get(PMEMpoolcheck *ppc) { struct check_status *status = NULL; /* clear cache if exists */ check_clear_status_cache(ppc->data); /* return next info if exists */ if ((status = check_pop_info(ppc->data))) return status; /* return error if exists */ if ((status = check_pop_error(ppc->data))) return status; if (ppc->result == CHECK_RESULT_ASK_QUESTIONS) { /* * push answer for previous question and return info if answer * is not valid */ if (check_push_answer(ppc)) if ((status = check_pop_info(ppc->data))) return status; /* if has next question ask it */ if ((status = check_pop_question(ppc->data))) return status; /* process answers otherwise */ ppc->result = CHECK_RESULT_PROCESS_ANSWERS; } else if (CHECK_RESULT_IS_STOP(ppc->result)) check_end(ppc->data); return NULL; } /* * check_step -- perform single check step */ struct check_status * check_step(PMEMpoolcheck *ppc) { LOG(3, NULL); struct check_status *status = NULL; /* return if we have information or questions to ask or check ended */ if ((status = status_get(ppc)) || check_is_end(ppc->data)) return status; /* get next step and check if exists */ const struct step *step = &steps[check_step_get(ppc->data)]; if (step->func == NULL) { check_end(ppc->data); return status; } /* * step would be performed if pool type is one of the required pool type * and it is not part if parts are excluded from current step */ if (!(step->type & ppc->pool->params.type) || (ppc->pool->params.is_part && !step->part)) { /* skip test */ check_step_inc(ppc->data); return NULL; } /* perform step */ step->func(ppc); /* move on to next step if no questions were generated */ if (ppc->result != CHECK_RESULT_ASK_QUESTIONS) check_step_inc(ppc->data); /* get current status and return */ return status_get(ppc); } /* * check_fini -- stop check process */ void check_fini(PMEMpoolcheck *ppc) { LOG(3, NULL); pool_data_free(ppc->pool); check_data_free(ppc->data); } /* * check_is_end -- return if check has ended */ int check_is_end(struct check_data *data) { return check_is_end_util(data); } /* * check_status_get -- extract pmempool_check_status from check_status */ struct pmempool_check_status * check_status_get(struct check_status *status) { return check_status_get_util(status); } pmdk-1.8/src/benchmarks/0000775000000000000000000000000013615011243013715 5ustar rootrootpmdk-1.8/src/benchmarks/pmembench.vcxproj.filters0000664000000000000000000001366613615011243020753 0ustar rootroot {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {e2b109e4-df13-4e15-a231-6125a1c8de19} {eb18140e-4f6a-4d43-8f76-7ce1d000c8c8} pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj pmemobj Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files pmemobj pmemobj Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files pmdk-1.8/src/benchmarks/scenario.cpp0000664000000000000000000001235713615011243016234 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * scenario.cpp -- scenario module definitions */ #include #include #include #include "queue.h" #include "scenario.hpp" /* * kv_alloc -- allocate key/value structure */ struct kv * kv_alloc(const char *key, const char *value) { struct kv *kv = (struct kv *)malloc(sizeof(*kv)); assert(kv != nullptr); kv->key = strdup(key); assert(kv->key != nullptr); kv->value = strdup(value); assert(kv->value != nullptr); return kv; } /* * kv_free -- free the key/value structure */ void kv_free(struct kv *kv) { assert(kv != nullptr); free(kv->key); free(kv->value); free(kv); } /* * scenario_alloc -- allocate scenario structure */ struct scenario * scenario_alloc(const char *name, const char *bench) { struct scenario *s = (struct scenario *)malloc(sizeof(*s)); assert(s != nullptr); PMDK_TAILQ_INIT(&s->head); s->name = strdup(name); assert(s->name != nullptr); s->benchmark = strdup(bench); assert(s->benchmark != nullptr); s->group = nullptr; return s; } /* * scenario_free -- free the scenario structure and all its content */ void scenario_free(struct scenario *s) { assert(s != nullptr); while (!PMDK_TAILQ_EMPTY(&s->head)) { struct kv *kv = PMDK_TAILQ_FIRST(&s->head); PMDK_TAILQ_REMOVE(&s->head, kv, next); kv_free(kv); } free(s->group); free(s->name); free(s->benchmark); free(s); } /* * scenario_set_group -- set group of scenario */ void scenario_set_group(struct scenario *s, const char *group) { assert(s != nullptr); s->group = strdup(group); } /* * scenarios_alloc -- allocate scenarios structure */ struct scenarios * scenarios_alloc(void) { struct scenarios *scenarios = (struct scenarios *)malloc(sizeof(*scenarios)); assert(nullptr != scenarios); PMDK_TAILQ_INIT(&scenarios->head); return scenarios; } /* * scenarios_free -- free scenarios structure and all its content */ void scenarios_free(struct scenarios *scenarios) { assert(scenarios != nullptr); while (!PMDK_TAILQ_EMPTY(&scenarios->head)) { struct scenario *sce = PMDK_TAILQ_FIRST(&scenarios->head); PMDK_TAILQ_REMOVE(&scenarios->head, sce, next); scenario_free(sce); } free(scenarios); } /* * scenarios_get_scenario -- get scenario of given name */ struct scenario * scenarios_get_scenario(struct scenarios *ss, const char *name) { struct scenario *scenario; FOREACH_SCENARIO(scenario, ss) { if (strcmp(scenario->name, name) == 0) return scenario; } return nullptr; } /* * contains_scenarios -- check if cmd line args contain any scenarios from ss */ bool contains_scenarios(int argc, char **argv, struct scenarios *ss) { assert(argv != nullptr); assert(argc > 0); assert(ss != nullptr); for (int i = 0; i < argc; i++) { if (scenarios_get_scenario(ss, argv[i])) return true; } return false; } /* * clone_scenario -- alloc a new scenario and copy all data from src scenario */ struct scenario * clone_scenario(struct scenario *src_scenario) { assert(src_scenario != nullptr); struct scenario *new_scenario = scenario_alloc(src_scenario->name, src_scenario->benchmark); assert(new_scenario != nullptr); struct kv *src_kv; FOREACH_KV(src_kv, src_scenario) { struct kv *new_kv = kv_alloc(src_kv->key, src_kv->value); assert(new_kv != nullptr); PMDK_TAILQ_INSERT_TAIL(&new_scenario->head, new_kv, next); } return new_scenario; } /* * find_kv_in_scenario - find a kv in the given scenario with the given key * value. Function returns the pointer to the kv structure containing the key or * nullptr if it is not found */ struct kv * find_kv_in_scenario(const char *key, const struct scenario *scenario) { struct kv *kv; FOREACH_KV(kv, scenario) { if (strcmp(kv->key, key) == 0) return kv; } return nullptr; } pmdk-1.8/src/benchmarks/perf.cfg0000664000000000000000000000454713615011243015344 0ustar rootroot[global] file = testfile repeats = 3 # pmemobj_tx_alloc(size = 256) vs threads [obj_tx_alloc_small_v_thread] bench = obj_tx_alloc group = pmemobj ops-per-thread = 200000 threads = 1:+1:32 type-number = per-thread data-size = 256 # pmemobj_tx_alloc(size = 128k) vs threads [obj_tx_alloc_huge_v_threads] bench = obj_tx_alloc group = pmemobj ops-per-thread = 11000 threads = 1:+1:32 type-number = per-thread data-size = 131072 # pmemobj_tx_alloc vs data-size [obj_tx_alloc_v_data_size] bench = obj_tx_alloc group = pmemobj ops-per-thread = 50000 threads = 1 type-number = one data-size = 64:*2:32768 # pmalloc (size = 100) vs threads [obj_pmalloc_small_v_threads] bench = pmalloc group = pmemobj ops-per-thread = 200000 threads = 1:+1:32 data-size = 100 # pmalloc (size = 128k) vs threads [obj_pmalloc_huge_v_threads] bench = pmalloc group = pmemobj ops-per-thread = 11000 threads = 1:+1:32 data-size = 131072 # pmalloc vs data-size [obj_pmalloc_v_data_size] bench = pmalloc group = pmemobj ops-per-thread = 50000 threads = 1 data-size = 64:*2:32768 # btree_map_insert [obj_btree_map_insert] bench = map_insert group = pmemobj ops-per-thread = 5000000 type = btree threads = 1 # ctree_map_insert [obj_ctree_map_insert] bench = map_insert group = pmemobj ops-per-thread = 5000000 type = ctree threads = 1 # rtree_map_insert [obj_rtree_map_insert] bench = map_insert group = pmemobj ops-per-thread = 1000000 type = rtree threads = 1 # rbtree_map_insert [obj_rbtree_map_insert] bench = map_insert group = pmemobj ops-per-thread = 5000000 type = rbtree threads = 1 # hashmap_tx_map_insert [obj_hashmap_tx_map_insert] bench = map_insert group = pmemobj ops-per-thread = 5000000 type = hashmap_tx threads = 1 # hashmap_atomic_map_insert [obj_hashmap_atomic_map_insert] bench = map_insert group = pmemobj ops-per-thread = 5000000 type = hashmap_atomic threads = 1 # pmemblk_write(size = 512, random) vs threads [blk_write_v_threads] bench = blk_write group = pmemblk file-size = 1073741824 ops-per-thread = 1000000 threads = 1:+1:32 data-size = 512 mode = rand # pmemblk_read(size = 512, random) vs threads [blk_read_v_threads] bench = blk_read group = pmemblk file-size = 1073741824 ops-per-thread = 1000000 threads = 1:+1:32 data-size = 512 mode = rand # log_append vs data-size [log_append_v_data_size] bench = log_append group = pmemlog ops-per-thread = 1000000 threads = 1 data-size = 32:*2:32768 pmdk-1.8/src/benchmarks/pmem_memset.cpp0000664000000000000000000003041213615011243016731 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmem_memset.cpp -- benchmark for pmem_memset function */ #include #include #include #include #include #include #include #include #include "benchmark.hpp" #include "file.h" #include "os.h" #define MAX_OFFSET 63 #define CONST_B 0xFF struct memset_bench; typedef int (*operation_fn)(void *dest, int c, size_t len); /* * memset_args -- benchmark specific command line options */ struct memset_args { char *mode; /* operation mode: stat, seq, rand */ bool memset; /* use libc memset function */ bool persist; /* perform persist operation */ bool msync; /* perform msync operation */ bool no_warmup; /* do not do warmup */ size_t chunk_size; /* elementary chunk size */ size_t dest_off; /* destination address offset */ unsigned seed; /* seed for random numbers */ }; /* * memset_bench -- benchmark context */ struct memset_bench { struct memset_args *pargs; /* benchmark specific arguments */ uint64_t *offsets; /* random/sequential address offsets */ size_t n_offsets; /* number of random elements */ int const_b; /* memset() value */ size_t fsize; /* file size */ void *pmem_addr; /* mapped file address */ operation_fn func_op; /* operation function */ }; /* * operation_mode -- mode of operation of memset() */ enum operation_mode { OP_MODE_UNKNOWN, OP_MODE_STAT, /* always use the same chunk */ OP_MODE_SEQ, /* use consecutive chunks */ OP_MODE_RAND /* use random chunks */ }; /* * parse_op_mode -- parse operation mode from string */ static enum operation_mode parse_op_mode(const char *arg) { if (strcmp(arg, "stat") == 0) return OP_MODE_STAT; else if (strcmp(arg, "seq") == 0) return OP_MODE_SEQ; else if (strcmp(arg, "rand") == 0) return OP_MODE_RAND; else return OP_MODE_UNKNOWN; } /* * init_offsets -- initialize offsets[] array depending on the selected mode */ static int init_offsets(struct benchmark_args *args, struct memset_bench *mb, enum operation_mode op_mode) { unsigned n_threads = args->n_threads; size_t n_ops = args->n_ops_per_thread; mb->n_offsets = n_ops * n_threads; assert(mb->n_offsets != 0); mb->offsets = (uint64_t *)malloc(mb->n_offsets * sizeof(*mb->offsets)); if (!mb->offsets) { perror("malloc"); return -1; } unsigned seed = mb->pargs->seed; for (unsigned i = 0; i < n_threads; i++) { for (size_t j = 0; j < n_ops; j++) { size_t o; switch (op_mode) { case OP_MODE_STAT: o = i; break; case OP_MODE_SEQ: o = i * n_ops + j; break; case OP_MODE_RAND: o = i * n_ops + os_rand_r(&seed) % n_ops; break; default: assert(0); return -1; } mb->offsets[i * n_ops + j] = o * mb->pargs->chunk_size; } } return 0; } /* * libpmem_memset_persist -- perform operation using libpmem * pmem_memset_persist(). */ static int libpmem_memset_persist(void *dest, int c, size_t len) { pmem_memset_persist(dest, c, len); return 0; } /* * libpmem_memset_nodrain -- perform operation using libpmem * pmem_memset_nodrain(). */ static int libpmem_memset_nodrain(void *dest, int c, size_t len) { pmem_memset_nodrain(dest, c, len); return 0; } /* * libc_memset_persist -- perform operation using libc memset() function * followed by pmem_persist(). */ static int libc_memset_persist(void *dest, int c, size_t len) { memset(dest, c, len); pmem_persist(dest, len); return 0; } /* * libc_memset_msync -- perform operation using libc memset() function * followed by pmem_msync(). */ static int libc_memset_msync(void *dest, int c, size_t len) { memset(dest, c, len); return pmem_msync(dest, len); } /* * libc_memset -- perform operation using libc memset() function * followed by pmem_flush(). */ static int libc_memset(void *dest, int c, size_t len) { memset(dest, c, len); pmem_flush(dest, len); return 0; } /* * warmup_persist -- does the warmup by writing the whole pool area */ static int warmup_persist(struct memset_bench *mb) { void *dest = mb->pmem_addr; int c = mb->const_b; size_t len = mb->fsize; pmem_memset_persist(dest, c, len); return 0; } /* * warmup_msync -- does the warmup by writing the whole pool area */ static int warmup_msync(struct memset_bench *mb) { void *dest = mb->pmem_addr; int c = mb->const_b; size_t len = mb->fsize; return libc_memset_msync(dest, c, len); } /* * memset_op -- actual benchmark operation. It can have one of the four * functions assigned: * libc_memset, * libc_memset_persist, * libpmem_memset_nodrain, * libpmem_memset_persist. */ static int memset_op(struct benchmark *bench, struct operation_info *info) { auto *mb = (struct memset_bench *)pmembench_get_priv(bench); assert(info->index < mb->n_offsets); size_t idx = info->worker->index * info->args->n_ops_per_thread + info->index; void *dest = (char *)mb->pmem_addr + mb->offsets[idx] + mb->pargs->dest_off; int c = mb->const_b; size_t len = mb->pargs->chunk_size; mb->func_op(dest, c, len); return 0; } /* * memset_init -- initialization function */ static int memset_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); assert(args->opts != nullptr); int ret = 0; size_t size; size_t large; size_t little; size_t file_size = 0; int flags = 0; enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } int (*warmup_func)(struct memset_bench *) = warmup_persist; auto *mb = (struct memset_bench *)malloc(sizeof(struct memset_bench)); if (!mb) { perror("malloc"); return -1; } mb->pargs = (struct memset_args *)args->opts; mb->pargs->chunk_size = args->dsize; enum operation_mode op_mode = parse_op_mode(mb->pargs->mode); if (op_mode == OP_MODE_UNKNOWN) { fprintf(stderr, "Invalid operation mode argument '%s'\n", mb->pargs->mode); ret = -1; goto err_free_mb; } size = MAX_OFFSET + mb->pargs->chunk_size; large = size * args->n_ops_per_thread * args->n_threads; little = size * args->n_threads; mb->fsize = (op_mode == OP_MODE_STAT) ? little : large; /* initialize offsets[] array depending on benchmark args */ if (init_offsets(args, mb, op_mode) < 0) { ret = -1; goto err_free_mb; } /* initialize memset() value */ mb->const_b = CONST_B; if (type != TYPE_DEVDAX) { file_size = mb->fsize; flags = PMEM_FILE_CREATE | PMEM_FILE_EXCL; } /* create a pmem file and memory map it */ if ((mb->pmem_addr = pmem_map_file(args->fname, file_size, flags, args->fmode, nullptr, nullptr)) == nullptr) { perror(args->fname); ret = -1; goto err_free_offsets; } if (mb->pargs->memset) { if (mb->pargs->persist && mb->pargs->msync) { fprintf(stderr, "Invalid benchmark parameters: persist and msync cannot be specified together\n"); ret = -1; goto err_free_offsets; } if (mb->pargs->persist) { mb->func_op = libc_memset_persist; } else if (mb->pargs->msync) { mb->func_op = libc_memset_msync; warmup_func = warmup_msync; } else { mb->func_op = libc_memset; } } else { mb->func_op = (mb->pargs->persist) ? libpmem_memset_persist : libpmem_memset_nodrain; } if (!mb->pargs->no_warmup && type != TYPE_DEVDAX) { ret = warmup_func(mb); if (ret) { perror("Pool warmup failed"); goto err_free_offsets; } } pmembench_set_priv(bench, mb); return ret; err_free_offsets: free(mb->offsets); err_free_mb: free(mb); return ret; } /* * memset_exit -- benchmark cleanup function */ static int memset_exit(struct benchmark *bench, struct benchmark_args *args) { auto *mb = (struct memset_bench *)pmembench_get_priv(bench); pmem_unmap(mb->pmem_addr, mb->fsize); free(mb->offsets); free(mb); return 0; } static struct benchmark_clo memset_clo[7]; /* Stores information about benchmark. */ static struct benchmark_info memset_info; CONSTRUCTOR(pmem_memset_constructor) void pmem_memset_constructor(void) { memset_clo[0].opt_short = 'M'; memset_clo[0].opt_long = "mem-mode"; memset_clo[0].descr = "Memory writing mode - " "stat, seq, rand"; memset_clo[0].def = "seq"; memset_clo[0].off = clo_field_offset(struct memset_args, mode); memset_clo[0].type = CLO_TYPE_STR; memset_clo[1].opt_short = 'm'; memset_clo[1].opt_long = "memset"; memset_clo[1].descr = "Use libc memset()"; memset_clo[1].def = "false"; memset_clo[1].off = clo_field_offset(struct memset_args, memset); memset_clo[1].type = CLO_TYPE_FLAG; memset_clo[2].opt_short = 'p'; memset_clo[2].opt_long = "persist"; memset_clo[2].descr = "Use pmem_persist()"; memset_clo[2].def = "true"; memset_clo[2].off = clo_field_offset(struct memset_args, persist); memset_clo[2].type = CLO_TYPE_FLAG; memset_clo[3].opt_short = 'D'; memset_clo[3].opt_long = "dest-offset"; memset_clo[3].descr = "Destination cache line alignment " "offset"; memset_clo[3].def = "0"; memset_clo[3].off = clo_field_offset(struct memset_args, dest_off); memset_clo[3].type = CLO_TYPE_UINT; memset_clo[3].type_uint.size = clo_field_size(struct memset_args, dest_off); memset_clo[3].type_uint.base = CLO_INT_BASE_DEC; memset_clo[3].type_uint.min = 0; memset_clo[3].type_uint.max = MAX_OFFSET; memset_clo[4].opt_short = 'w'; memset_clo[4].opt_long = "no-warmup"; memset_clo[4].descr = "Don't do warmup"; memset_clo[4].def = "false"; memset_clo[4].type = CLO_TYPE_FLAG; memset_clo[4].off = clo_field_offset(struct memset_args, no_warmup); memset_clo[5].opt_short = 'S'; memset_clo[5].opt_long = "seed"; memset_clo[5].descr = "seed for random numbers"; memset_clo[5].def = "1"; memset_clo[5].off = clo_field_offset(struct memset_args, seed); memset_clo[5].type = CLO_TYPE_UINT; memset_clo[5].type_uint.size = clo_field_size(struct memset_args, seed); memset_clo[5].type_uint.base = CLO_INT_BASE_DEC; memset_clo[5].type_uint.min = 1; memset_clo[5].type_uint.max = UINT_MAX; memset_clo[6].opt_short = 's'; memset_clo[6].opt_long = "msync"; memset_clo[6].descr = "Use pmem_msync()"; memset_clo[6].def = "false"; memset_clo[6].off = clo_field_offset(struct memset_args, msync); memset_clo[6].type = CLO_TYPE_FLAG; memset_info.name = "pmem_memset"; memset_info.brief = "Benchmark for pmem_memset_persist() " "and pmem_memset_nodrain() operations"; memset_info.init = memset_init; memset_info.exit = memset_exit; memset_info.multithread = true; memset_info.multiops = true; memset_info.operation = memset_op; memset_info.measure_time = true; memset_info.clos = memset_clo; memset_info.nclos = ARRAY_SIZE(memset_clo); memset_info.opts_size = sizeof(struct memset_args); memset_info.rm_file = true; memset_info.allow_poolset = false; memset_info.print_bandwidth = true; REGISTER_BENCHMARK(memset_info); }; pmdk-1.8/src/benchmarks/pmembench.vcxproj0000664000000000000000000003500613615011243017274 0ustar rootroot Debug x64 Release x64 {492baa3d-0d5d-478e-9765-500463ae69aa} {49a7cc5a-d5e7-4a07-917f-c6918b982be8} {f7c6c6b6-4142-4c82-8699-4a9d8183181b} {0b1818eb-bdc8-4865-964f-db8bf05cfd86} {1baa1617-93ae-4196-8a1a-bd492fb18aef} {cf9a0883-6334-44c7-ac29-349468c78e27} {9186eac4-2f34-4f17-b940-6585d7869bcd} CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC CompileAsC true true {CB906E89-1313-4929-AFF7-86FBF1CC301F} Win32Proj pmembench 10.0.16299.0 Application true v140 NotSet Application false v140 false NotSet $(SolutionDir)\examples\libpmemobj\hashmap;$(SolutionDir)\examples\libpmemobj\map;$(SolutionDir)\examples\libpmemobj\tree_map;$(SolutionDir)\common;$(SolutionDir)\test\unittest;$(SolutionDir)\windows\include;$(SolutionDir)\libpmemobj\;$(SolutionDir)\include;$(SolutionDir)\libpmemblk;$(SolutionDir)\windows\getopt;$(IncludePath) $(SolutionDir)\examples\libpmemobj\hashmap;$(SolutionDir)\examples\libpmemobj\map;$(SolutionDir)\examples\libpmemobj\tree_map;$(SolutionDir)\common;$(SolutionDir)\test\unittest;$(SolutionDir)\windows\include;$(SolutionDir)\libpmemobj\;$(SolutionDir)\include;$(SolutionDir)\libpmemblk;$(SolutionDir)\windows\getopt;$(IncludePath) NotUsing Level3 PMDK_UTF8_API;_CONSOLE;%(PreprocessorDefinitions) platform.h CompileAsCpp true false false 4200 Console DbgHelp.lib;Shlwapi.lib;%(AdditionalDependencies) false Debug 8000000 Level3 NotUsing true PMDK_UTF8_API;NDEBUG;_CONSOLE;%(PreprocessorDefinitions) CompileAsCpp platform.h true 4200 Console DbgHelp.lib;Shlwapi.lib;%(AdditionalDependencies) false DebugFastLink 8000000 pmdk-1.8/src/benchmarks/pmembench_log.cfg0000664000000000000000000000313613615011243017200 0ustar rootroot# # pmembench_log.cfg -- this is an example config file for pmembench # with scenarios for pmemlog benchmark # # Global parameters [global] group = pmemlog file = testfile.log ops-per-thread = 1000 # log_append benchmark with variable number of threads [log_append_threads] bench = log_append threads = 1:+1:31 data-size = 512 # log_append benchmark with variable data sizes # from 32 to 8k bytes [log_append_data_size_huge] bench = log_append threads = 1 data-size = 32:*2:8192 min-size = 16 random = true seed = 1 # log_append benchmark with variable data sizes # from 1 to 32 bytes [log_append_data_size_small] bench = log_append threads = 1 data-size = 2:+1:32 min-size = 1 random = true seed = 1 # log_read benchmark with variable data sizes [log_read_variable_chunk_sizes] bench = log_read threads = 1 data-size = 64:*2:8192 # log_append benchmark with multiple threads and variable # vector sizes [log_append_threads_vector] bench = log_append threads = 8 data-size = 512 vector = 2:*2:32 # log_append benchmark with multiple threads, variable # vector sizes and random sizes [log_append_threads_random_vector] bench = log_append threads = 8 data-size = 512 random = true min-size = 32 vector = 2:*2:32 # fileio benchmark with multiple threads and variable # vector sizes [fileio_append_threads_vector] bench = log_append file-io = true threads = 8 data-size = 512 vector = 2:*2:32 # log_append benchmark with multiple threads, variable # vector sizes and random sizes [fileio_append_threads_random_vector] bench = log_append file-io = true threads = 8 data-size = 512 random = true min-size = 32 vector = 2:*2:32 pmdk-1.8/src/benchmarks/pmembench_memcpy.cfg0000664000000000000000000000267013615011243017713 0ustar rootroot# Global parameters [global] group = pmem file = testfile.memcpy ops-per-thread=50000 # pmem_memcpy benchmark with variable chunk sizes # from 64 to 8k bytes [pmcpy_chunk_sizes_aligned] bench = pmem_memcpy threads = 1 data-size = 64:*2:8192 # pmem_memcpy benchmark # with variable destination address offsets [pmcpy_destination_offsets] bench = pmem_memcpy threads = 1 data-size = 512 src-offset = 0 dest-offset = 1:+1:63 # pmem_memcpy pmem_memcpy_persist # copy mode: random # from 64 to 8k bytes [pmcpy_chunk_sizes_rand] bench = pmem_memcpy threads = 1 data-size = 64:*2:8192 src-mode = rand dest-mode = rand # pmem_memcpy pmem_memcpy_persist() # copy mode: sequential # from 64 to 8k bytes [pmcpy_pmem_memcpy_persist] bench = pmem_memcpy threads = 1 data-size = 64:*2:8192 libc-memcpy = false persist = true # pmem_memcpy pmem_memcpy_nodrain() # copy mode: sequential # from 64 to 8k bytes [pmcpy_pmem_memcpy_nodrain] bench = pmem_memcpy threads = 1 data-size = 64:*2:8192 libc-memcpy = false persist = false # pmem_memcpy standard memcpy() # followed by pmem_persist() # copy mode: sequential # from 64 to 8k bytes [pmcpy_libc_memcpy_persist] bench = pmem_memcpy threads = 1 data-size = 64:*2:8192 libc-memcpy = true persist = true # pmem_memcpy standard memcpy() # followed by pmem_flush() # copy mode: sequential # from 64 to 8k bytes [pmcpy_libc_memcpy_flush] bench = pmem_memcpy threads = 1 data-size = 64:*2:8192 libc-memcpy = true persist = false pmdk-1.8/src/benchmarks/obj_locks.cpp0000664000000000000000000005430613615011243016376 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * obj_locks.cpp -- main source file for PMEM locks benchmark */ #include #include #include "benchmark.hpp" #include "libpmemobj.h" #include "file.h" #include "lane.h" #include "list.h" #include "memops.h" #include "obj.h" #include "os_thread.h" #include "out.h" #include "pmalloc.h" #include "sync.h" struct prog_args { bool use_system_threads; /* use system locks instead of PMEM locks */ unsigned n_locks; /* number of mutex/rwlock objects */ bool run_id_increment; /* increment run_id after each lock/unlock */ uint64_t runid_initial_value; /* initial value of run_id */ char *lock_mode; /* "1by1" or "all-lock" */ char *lock_type; /* "mutex", "rwlock" or "ram-mutex" */ bool use_rdlock; /* use read lock, instead of write lock */ }; /* * mutex similar to PMEMmutex, but with os_mutex_t in RAM */ typedef union padded_volatile_pmemmutex { char padding[_POBJ_CL_SIZE]; struct { uint64_t runid; os_mutex_t *mutexp; /* pointer to os_thread mutex in RAM */ } volatile_pmemmutex; } PMEM_volatile_mutex; typedef union lock_union { PMEMmutex pm_mutex; PMEMrwlock pm_rwlock; PMEM_volatile_mutex pm_vmutex; os_mutex_t pt_mutex; os_rwlock_t pt_rwlock; } lock_t; POBJ_LAYOUT_BEGIN(pmembench_lock_layout); POBJ_LAYOUT_ROOT(pmembench_lock_layout, struct my_root); POBJ_LAYOUT_TOID(pmembench_lock_layout, lock_t); POBJ_LAYOUT_END(pmembench_lock_layout); /* * my_root -- root object structure */ struct my_root { TOID(lock_t) locks; /* an array of locks */ }; /* * lock usage */ enum operation_mode { OP_MODE_1BY1, /* lock and unlock one lock at a time */ OP_MODE_ALL_LOCK, /* grab all locks, then unlock them all */ OP_MODE_MAX, }; /* * lock type */ enum benchmark_mode { BENCH_MODE_MUTEX, /* PMEMmutex vs. os_mutex_t */ BENCH_MODE_RWLOCK, /* PMEMrwlock vs. os_rwlock_t */ BENCH_MODE_VOLATILE_MUTEX, /* PMEMmutex with os_thread mutex in RAM */ BENCH_MODE_MAX }; struct mutex_bench; struct bench_ops { int (*bench_init)(struct mutex_bench *); int (*bench_exit)(struct mutex_bench *); int (*bench_op)(struct mutex_bench *); }; /* * mutex_bench -- stores variables used in benchmark, passed within functions */ struct mutex_bench { PMEMobjpool *pop; /* pointer to the persistent pool */ TOID(struct my_root) root; /* OID of the root object */ struct prog_args *pa; /* prog_args structure */ enum operation_mode lock_mode; /* lock usage mode */ enum benchmark_mode lock_type; /* lock type */ lock_t *locks; /* pointer to the array of locks */ struct bench_ops *ops; }; #define GET_VOLATILE_MUTEX(pop, mutexp) \ (os_mutex_t *)get_lock( \ (pop)->run_id, &(mutexp)->volatile_pmemmutex.runid, \ (mutexp)->volatile_pmemmutex.mutexp, \ (int (*)(void **lock, void *arg))volatile_mutex_init) typedef int (*lock_fun_wrapper)(PMEMobjpool *pop, void *lock); /* * bench_operation_1by1 -- acquire lock and unlock release locks */ static void bench_operation_1by1(lock_fun_wrapper flock, lock_fun_wrapper funlock, struct mutex_bench *mb, PMEMobjpool *pop) { for (unsigned i = 0; i < (mb)->pa->n_locks; (i)++) { auto *o = (void *)(&(mb)->locks[i]); flock(pop, o); funlock(pop, o); } } /* * bench_operation_all_lock -- acquire all locks and release all locks */ static void bench_operation_all_lock(lock_fun_wrapper flock, lock_fun_wrapper funlock, struct mutex_bench *mb, PMEMobjpool *pop) { for (unsigned i = 0; i < (mb)->pa->n_locks; (i)++) { auto *o = (void *)(&(mb)->locks[i]); flock(pop, o); } for (unsigned i = 0; i < (mb)->pa->n_locks; i++) { auto *o = (void *)(&(mb)->locks[i]); funlock(pop, o); } } /* * get_lock -- atomically initialize and return a lock */ static void * get_lock(uint64_t pop_runid, volatile uint64_t *runid, void *lock, int (*init_lock)(void **lock, void *arg)) { uint64_t tmp_runid; while ((tmp_runid = *runid) != pop_runid) { if ((tmp_runid != (pop_runid - 1))) { if (util_bool_compare_and_swap64(runid, tmp_runid, (pop_runid - 1))) { if (init_lock(&lock, nullptr)) { util_fetch_and_and64(runid, 0); return nullptr; } if (util_bool_compare_and_swap64( runid, (pop_runid - 1), pop_runid) == 0) { return nullptr; } } } } return lock; } /* * volatile_mutex_init -- initialize the volatile mutex object * * Allocate memory for the os_thread mutex and initialize it. * Set the runid to the same value as in the memory pool. */ static int volatile_mutex_init(os_mutex_t **mutexp, void *attr) { if (*mutexp == nullptr) { *mutexp = (os_mutex_t *)malloc(sizeof(os_mutex_t)); if (*mutexp == nullptr) { perror("volatile_mutex_init alloc"); return ENOMEM; } } return os_mutex_init(*mutexp); } /* * volatile_mutex_lock -- initialize the mutex object if needed and lock it */ static int volatile_mutex_lock(PMEMobjpool *pop, PMEM_volatile_mutex *mutexp) { auto *mutex = GET_VOLATILE_MUTEX(pop, mutexp); if (mutex == nullptr) return EINVAL; return os_mutex_lock(mutex); } /* * volatile_mutex_unlock -- unlock the mutex */ static int volatile_mutex_unlock(PMEMobjpool *pop, PMEM_volatile_mutex *mutexp) { auto *mutex = (os_mutex_t *)GET_VOLATILE_MUTEX(pop, mutexp); if (mutex == nullptr) return EINVAL; return os_mutex_unlock(mutex); } /* * volatile_mutex_destroy -- destroy os_thread mutex and release memory */ static int volatile_mutex_destroy(PMEMobjpool *pop, PMEM_volatile_mutex *mutexp) { auto *mutex = (os_mutex_t *)GET_VOLATILE_MUTEX(pop, mutexp); if (mutex == nullptr) return EINVAL; int ret = os_mutex_destroy(mutex); if (ret != 0) return ret; free(mutex); return 0; } /* * os_mutex_lock_wrapper -- wrapper for os_mutex_lock */ static int os_mutex_lock_wrapper(PMEMobjpool *pop, void *lock) { return os_mutex_lock((os_mutex_t *)lock); } /* * os_mutex_unlock_wrapper -- wrapper for os_mutex_unlock */ static int os_mutex_unlock_wrapper(PMEMobjpool *pop, void *lock) { return os_mutex_unlock((os_mutex_t *)lock); } /* * pmemobj_mutex_lock_wrapper -- wrapper for pmemobj_mutex_lock */ static int pmemobj_mutex_lock_wrapper(PMEMobjpool *pop, void *lock) { return pmemobj_mutex_lock(pop, (PMEMmutex *)lock); } /* * pmemobj_mutex_unlock_wrapper -- wrapper for pmemobj_mutex_unlock */ static int pmemobj_mutex_unlock_wrapper(PMEMobjpool *pop, void *lock) { return pmemobj_mutex_unlock(pop, (PMEMmutex *)lock); } /* * os_rwlock_wrlock_wrapper -- wrapper for os_rwlock_wrlock */ static int os_rwlock_wrlock_wrapper(PMEMobjpool *pop, void *lock) { return os_rwlock_wrlock((os_rwlock_t *)lock); } /* * os_rwlock_rdlock_wrapper -- wrapper for os_rwlock_rdlock */ static int os_rwlock_rdlock_wrapper(PMEMobjpool *pop, void *lock) { return os_rwlock_rdlock((os_rwlock_t *)lock); } /* * os_rwlock_unlock_wrapper -- wrapper for os_rwlock_unlock */ static int os_rwlock_unlock_wrapper(PMEMobjpool *pop, void *lock) { return os_rwlock_unlock((os_rwlock_t *)lock); } /* * pmemobj_rwlock_wrlock_wrapper -- wrapper for pmemobj_rwlock_wrlock */ static int pmemobj_rwlock_wrlock_wrapper(PMEMobjpool *pop, void *lock) { return pmemobj_rwlock_wrlock(pop, (PMEMrwlock *)lock); } /* * pmemobj_rwlock_rdlock_wrapper -- wrapper for pmemobj_rwlock_rdlock */ static int pmemobj_rwlock_rdlock_wrapper(PMEMobjpool *pop, void *lock) { return pmemobj_rwlock_rdlock(pop, (PMEMrwlock *)lock); } /* * pmemobj_rwlock_unlock_wrapper -- wrapper for pmemobj_rwlock_unlock */ static int pmemobj_rwlock_unlock_wrapper(PMEMobjpool *pop, void *lock) { return pmemobj_rwlock_unlock(pop, (PMEMrwlock *)lock); } /* * volatile_mutex_lock_wrapper -- wrapper for volatile_mutex_lock */ static int volatile_mutex_lock_wrapper(PMEMobjpool *pop, void *lock) { return volatile_mutex_lock(pop, (PMEM_volatile_mutex *)lock); } /* * volatile_mutex_unlock_wrapper -- wrapper for volatile_mutex_unlock */ static int volatile_mutex_unlock_wrapper(PMEMobjpool *pop, void *lock) { return volatile_mutex_unlock(pop, (PMEM_volatile_mutex *)lock); } /* * init_bench_mutex -- allocate and initialize mutex objects */ static int init_bench_mutex(struct mutex_bench *mb) { POBJ_ZALLOC(mb->pop, &D_RW(mb->root)->locks, lock_t, mb->pa->n_locks * sizeof(lock_t)); if (TOID_IS_NULL(D_RO(mb->root)->locks)) { perror("POBJ_ZALLOC"); return -1; } struct my_root *root = D_RW(mb->root); assert(root != nullptr); mb->locks = D_RW(root->locks); assert(mb->locks != nullptr); if (!mb->pa->use_system_threads) { /* initialize PMEM mutexes */ for (unsigned i = 0; i < mb->pa->n_locks; i++) { auto *p = (PMEMmutex_internal *)&mb->locks[i]; p->pmemmutex.runid = mb->pa->runid_initial_value; os_mutex_init(&p->PMEMmutex_lock); } } else { /* initialize os_thread mutexes */ for (unsigned i = 0; i < mb->pa->n_locks; i++) { auto *p = (os_mutex_t *)&mb->locks[i]; os_mutex_init(p); } } return 0; } /* * exit_bench_mutex -- destroy the mutex objects and release memory */ static int exit_bench_mutex(struct mutex_bench *mb) { if (mb->pa->use_system_threads) { /* deinitialize os_thread mutex objects */ for (unsigned i = 0; i < mb->pa->n_locks; i++) { auto *p = (os_mutex_t *)&mb->locks[i]; os_mutex_destroy(p); } } POBJ_FREE(&D_RW(mb->root)->locks); return 0; } /* * op_bench_mutex -- lock and unlock the mutex object * * If requested, increment the run_id of the memory pool. In case of PMEMmutex * this will force the rwlock object(s) reinitialization at the lock operation. */ static int op_bench_mutex(struct mutex_bench *mb) { if (!mb->pa->use_system_threads) { if (mb->lock_mode == OP_MODE_1BY1) { bench_operation_1by1(pmemobj_mutex_lock_wrapper, pmemobj_mutex_unlock_wrapper, mb, mb->pop); } else { bench_operation_all_lock(pmemobj_mutex_lock_wrapper, pmemobj_mutex_unlock_wrapper, mb, mb->pop); } if (mb->pa->run_id_increment) mb->pop->run_id += 2; /* must be a multiple of 2 */ } else { if (mb->lock_mode == OP_MODE_1BY1) { bench_operation_1by1(os_mutex_lock_wrapper, os_mutex_unlock_wrapper, mb, nullptr); } else { bench_operation_all_lock(os_mutex_lock_wrapper, os_mutex_unlock_wrapper, mb, nullptr); } } return 0; } /* * init_bench_rwlock -- allocate and initialize rwlock objects */ static int init_bench_rwlock(struct mutex_bench *mb) { struct my_root *root = D_RW(mb->root); assert(root != nullptr); POBJ_ZALLOC(mb->pop, &root->locks, lock_t, mb->pa->n_locks * sizeof(lock_t)); if (TOID_IS_NULL(root->locks)) { perror("POBJ_ZALLOC"); return -1; } mb->locks = D_RW(root->locks); assert(mb->locks != nullptr); if (!mb->pa->use_system_threads) { /* initialize PMEM rwlocks */ for (unsigned i = 0; i < mb->pa->n_locks; i++) { auto *p = (PMEMrwlock_internal *)&mb->locks[i]; p->pmemrwlock.runid = mb->pa->runid_initial_value; os_rwlock_init(&p->PMEMrwlock_lock); } } else { /* initialize os_thread rwlocks */ for (unsigned i = 0; i < mb->pa->n_locks; i++) { auto *p = (os_rwlock_t *)&mb->locks[i]; os_rwlock_init(p); } } return 0; } /* * exit_bench_rwlock -- destroy the rwlocks and release memory */ static int exit_bench_rwlock(struct mutex_bench *mb) { if (mb->pa->use_system_threads) { /* deinitialize os_thread mutex objects */ for (unsigned i = 0; i < mb->pa->n_locks; i++) { auto *p = (os_rwlock_t *)&mb->locks[i]; os_rwlock_destroy(p); } } POBJ_FREE(&D_RW(mb->root)->locks); return 0; } /* * op_bench_rwlock -- lock and unlock the rwlock object * * If requested, increment the run_id of the memory pool. In case of PMEMrwlock * this will force the rwlock object(s) reinitialization at the lock operation. */ static int op_bench_rwlock(struct mutex_bench *mb) { if (!mb->pa->use_system_threads) { if (mb->lock_mode == OP_MODE_1BY1) { bench_operation_1by1( !mb->pa->use_rdlock ? pmemobj_rwlock_wrlock_wrapper : pmemobj_rwlock_rdlock_wrapper, pmemobj_rwlock_unlock_wrapper, mb, mb->pop); } else { bench_operation_all_lock( !mb->pa->use_rdlock ? pmemobj_rwlock_wrlock_wrapper : pmemobj_rwlock_rdlock_wrapper, pmemobj_rwlock_unlock_wrapper, mb, mb->pop); } if (mb->pa->run_id_increment) mb->pop->run_id += 2; /* must be a multiple of 2 */ } else { if (mb->lock_mode == OP_MODE_1BY1) { bench_operation_1by1( !mb->pa->use_rdlock ? os_rwlock_wrlock_wrapper : os_rwlock_rdlock_wrapper, os_rwlock_unlock_wrapper, mb, nullptr); } else { bench_operation_all_lock( !mb->pa->use_rdlock ? os_rwlock_wrlock_wrapper : os_rwlock_rdlock_wrapper, os_rwlock_unlock_wrapper, mb, nullptr); } } return 0; } /* * init_bench_vmutex -- allocate and initialize mutexes */ static int init_bench_vmutex(struct mutex_bench *mb) { struct my_root *root = D_RW(mb->root); assert(root != nullptr); POBJ_ZALLOC(mb->pop, &root->locks, lock_t, mb->pa->n_locks * sizeof(lock_t)); if (TOID_IS_NULL(root->locks)) { perror("POBJ_ZALLOC"); return -1; } mb->locks = D_RW(root->locks); assert(mb->locks != nullptr); /* initialize PMEM volatile mutexes */ for (unsigned i = 0; i < mb->pa->n_locks; i++) { auto *p = (PMEM_volatile_mutex *)&mb->locks[i]; p->volatile_pmemmutex.runid = mb->pa->runid_initial_value; volatile_mutex_init(&p->volatile_pmemmutex.mutexp, nullptr); } return 0; } /* * exit_bench_vmutex -- destroy the mutex objects and release their * memory */ static int exit_bench_vmutex(struct mutex_bench *mb) { for (unsigned i = 0; i < mb->pa->n_locks; i++) { auto *p = (PMEM_volatile_mutex *)&mb->locks[i]; volatile_mutex_destroy(mb->pop, p); } POBJ_FREE(&D_RW(mb->root)->locks); return 0; } /* * op_bench_volatile_mutex -- lock and unlock the mutex object */ static int op_bench_vmutex(struct mutex_bench *mb) { if (mb->lock_mode == OP_MODE_1BY1) { bench_operation_1by1(volatile_mutex_lock_wrapper, volatile_mutex_unlock_wrapper, mb, mb->pop); } else { bench_operation_all_lock(volatile_mutex_lock_wrapper, volatile_mutex_unlock_wrapper, mb, mb->pop); } if (mb->pa->run_id_increment) mb->pop->run_id += 2; /* must be a multiple of 2 */ return 0; } struct bench_ops benchmark_ops[BENCH_MODE_MAX] = { {init_bench_mutex, exit_bench_mutex, op_bench_mutex}, {init_bench_rwlock, exit_bench_rwlock, op_bench_rwlock}, {init_bench_vmutex, exit_bench_vmutex, op_bench_vmutex}}; /* * operation_mode -- parses command line "--mode" and returns * proper operation mode */ static enum operation_mode parse_op_mode(const char *arg) { if (strcmp(arg, "1by1") == 0) return OP_MODE_1BY1; else if (strcmp(arg, "all-lock") == 0) return OP_MODE_ALL_LOCK; else return OP_MODE_MAX; } /* * benchmark_mode -- parses command line "--bench_type" and returns * proper benchmark ops */ static struct bench_ops * parse_benchmark_mode(const char *arg) { if (strcmp(arg, "mutex") == 0) return &benchmark_ops[BENCH_MODE_MUTEX]; else if (strcmp(arg, "rwlock") == 0) return &benchmark_ops[BENCH_MODE_RWLOCK]; else if (strcmp(arg, "volatile-mutex") == 0) return &benchmark_ops[BENCH_MODE_VOLATILE_MUTEX]; else return nullptr; } /* * locks_init -- allocates persistent memory, maps it, creates the appropriate * objects in the allocated memory and initializes them */ static int locks_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } int ret = 0; size_t poolsize; struct mutex_bench *mb = (struct mutex_bench *)malloc(sizeof(*mb)); if (mb == nullptr) { perror("malloc"); return -1; } mb->pa = (struct prog_args *)args->opts; mb->lock_mode = parse_op_mode(mb->pa->lock_mode); if (mb->lock_mode >= OP_MODE_MAX) { fprintf(stderr, "Invalid mutex mode: %s\n", mb->pa->lock_mode); errno = EINVAL; goto err_free_mb; } mb->ops = parse_benchmark_mode(mb->pa->lock_type); if (mb->ops == nullptr) { fprintf(stderr, "Invalid benchmark type: %s\n", mb->pa->lock_type); errno = EINVAL; goto err_free_mb; } /* reserve some space for metadata */ poolsize = mb->pa->n_locks * sizeof(lock_t) + PMEMOBJ_MIN_POOL; if (args->is_poolset || type == TYPE_DEVDAX) { if (args->fsize < poolsize) { fprintf(stderr, "file size too large\n"); goto err_free_mb; } poolsize = 0; } mb->pop = pmemobj_create(args->fname, POBJ_LAYOUT_NAME(pmembench_lock_layout), poolsize, args->fmode); if (mb->pop == nullptr) { ret = -1; perror("pmemobj_create"); goto err_free_mb; } mb->root = POBJ_ROOT(mb->pop, struct my_root); assert(!TOID_IS_NULL(mb->root)); ret = mb->ops->bench_init(mb); if (ret != 0) goto err_free_pop; pmembench_set_priv(bench, mb); return 0; err_free_pop: pmemobj_close(mb->pop); err_free_mb: free(mb); return ret; } /* * locks_exit -- destroys allocated objects and release memory */ static int locks_exit(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); auto *mb = (struct mutex_bench *)pmembench_get_priv(bench); assert(mb != nullptr); mb->ops->bench_exit(mb); pmemobj_close(mb->pop); free(mb); return 0; } /* * locks_op -- actual benchmark operation * * Performs lock and unlock as by the program arguments. */ static int locks_op(struct benchmark *bench, struct operation_info *info) { auto *mb = (struct mutex_bench *)pmembench_get_priv(bench); assert(mb != nullptr); assert(mb->pop != nullptr); assert(!TOID_IS_NULL(mb->root)); assert(mb->locks != nullptr); assert(mb->lock_mode < OP_MODE_MAX); mb->ops->bench_op(mb); return 0; } /* structure to define command line arguments */ static struct benchmark_clo locks_clo[7]; static struct benchmark_info locks_info; CONSTRUCTOR(pmem_locks_constructor) void pmem_locks_constructor(void) { locks_clo[0].opt_short = 'p'; locks_clo[0].opt_long = "use_system_threads"; locks_clo[0].descr = "Use os_thread locks instead of PMEM, " "does not matter for volatile mutex"; locks_clo[0].def = "false"; locks_clo[0].off = clo_field_offset(struct prog_args, use_system_threads); locks_clo[0].type = CLO_TYPE_FLAG; locks_clo[1].opt_short = 'm'; locks_clo[1].opt_long = "numlocks"; locks_clo[1].descr = "The number of lock objects used " "for benchmark"; locks_clo[1].def = "1"; locks_clo[1].off = clo_field_offset(struct prog_args, n_locks); locks_clo[1].type = CLO_TYPE_UINT; locks_clo[1].type_uint.size = clo_field_size(struct prog_args, n_locks); locks_clo[1].type_uint.base = CLO_INT_BASE_DEC; locks_clo[1].type_uint.min = 1; locks_clo[1].type_uint.max = UINT_MAX; locks_clo[2].opt_short = 0; locks_clo[2].opt_long = "mode"; locks_clo[2].descr = "Locking mode"; locks_clo[2].type = CLO_TYPE_STR; locks_clo[2].off = clo_field_offset(struct prog_args, lock_mode); locks_clo[2].def = "1by1"; locks_clo[3].opt_short = 'r'; locks_clo[3].opt_long = "run_id"; locks_clo[3].descr = "Increment the run_id of PMEM object " "pool after each operation"; locks_clo[3].def = "false"; locks_clo[3].off = clo_field_offset(struct prog_args, run_id_increment); locks_clo[3].type = CLO_TYPE_FLAG; locks_clo[4].opt_short = 'i'; locks_clo[4].opt_long = "run_id_init_val"; locks_clo[4].descr = "Use this value for initializing the " "run_id of each PMEMmutex object"; locks_clo[4].def = "2"; locks_clo[4].off = clo_field_offset(struct prog_args, runid_initial_value); locks_clo[4].type = CLO_TYPE_UINT; locks_clo[4].type_uint.size = clo_field_size(struct prog_args, runid_initial_value); locks_clo[4].type_uint.base = CLO_INT_BASE_DEC; locks_clo[4].type_uint.min = 0; locks_clo[4].type_uint.max = UINT64_MAX; locks_clo[5].opt_short = 'b'; locks_clo[5].opt_long = "bench_type"; locks_clo[5].descr = "The Benchmark type: mutex, " "rwlock or volatile-mutex"; locks_clo[5].type = CLO_TYPE_STR; locks_clo[5].off = clo_field_offset(struct prog_args, lock_type); locks_clo[5].def = "mutex"; locks_clo[6].opt_short = 'R'; locks_clo[6].opt_long = "rdlock"; locks_clo[6].descr = "Select read over write lock, only " "valid when lock_type is \"rwlock\""; locks_clo[6].type = CLO_TYPE_FLAG; locks_clo[6].off = clo_field_offset(struct prog_args, use_rdlock); locks_info.name = "obj_locks"; locks_info.brief = "Benchmark for pmem locks operations"; locks_info.init = locks_init; locks_info.exit = locks_exit; locks_info.multithread = false; locks_info.multiops = true; locks_info.operation = locks_op; locks_info.measure_time = true; locks_info.clos = locks_clo; locks_info.nclos = ARRAY_SIZE(locks_clo); locks_info.opts_size = sizeof(struct prog_args); locks_info.rm_file = true; locks_info.allow_poolset = true; REGISTER_BENCHMARK(locks_info); }; pmdk-1.8/src/benchmarks/benchdiff0000775000000000000000000001240713615011243015557 0ustar rootroot#!/usr/bin/perl -w # # Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # A simple tool to compare the results of two benchmarks. # Results assume a terminal, you can also install "colorized-logs" then pipe # through ansi2txt or ansi2html. # # Numbers which improved are in green (bright if > 2×), ones which worsened # in red, white means same (within 5%). Fields in light gray don't have an # assigned direction. # # This tool needs to be taught which fields go up and which go down; # please update the table below accordingly. use Term::ANSIColor; # +1 means "more is better" my %dir=( "total-avg[sec]" => -1, "ops-per-second[1/sec]" => +1, "total-max[sec]" => -1, "total-min[sec]" => -1, "total-median[sec]" => -1, "total-std-dev[sec]" => -1, "latency-avg[nsec]" => -1, "latency-min[nsec]" => -1, "latency-max[nsec]" => -1, "latency-std-dev[nsec]" => -1, "latency-pctl-50.0%[nsec]" => -1, "latency-pctl-99.0%[nsec]" => -1, "latency-pctl-99.9%[nsec]" => -1, "threads" => 0, "ops-per-thread" => 0, "data-size" => 0, "seed" => 0, "repeats" => 0, "thread-affinity" => 0, "main-affinity" => 0, "min-exe-time" => 0, "type-number" => 0, "min-size" => 0, "one-pool" => 0, "one-object" => 0, "operation" => 0, "lib" => 0, "nestings" => 0, # ? "position" => 0, "list-len" => 0, "queue" => 0, "no-warmup" => 0, "mode" => 0, "file-size" => 0, "random" => 0, "file-io" => 0, "vector" => 0, "src-offset" => 0, "dest-offset" => 0, "src-mode" => 0, "dest-mode" => 0, "libc-memcpy" => 0, "persist" => 0, "bandwidth[MiB/s]" => +1, "mem-mode" => 0, "memset" => 0, "msync" => 0, "objects" => 0, "use_system_threads" => 0, "numlocks" => 0, "run_id" => 0, "run_id_init_val" => 0, "bench_type" => 0, "rdlock" => 0, "min-rsize" => 0, "realloc-size" => 0, "changed-type" => 0, "stdlib-alloc" => 0, "pool-per-thread" => 0, "alloc-min" => 0, "realloc-min" => 0, "mix-thread" => 0, ); my %head_colors = ( +1 => color('green'), -1 => color('red'), 0 => "", ); my %arrows = ( +1 => "↑", -1 => "↓", 0 => "", ); scalar @ARGV==2 or die "Usage: benchdiff \n"; open FA, "<", "$ARGV[0]" or die "Can't open ｢$ARGV[0]｣: $!\n"; open FB, "<", "$ARGV[1]" or die "Can't open ｢$ARGV[1]｣: $!\n"; my ($A, $B, @head); while (defined($A = )) { defined ($B = ) or die "Benchmarks have unequal length!\n"; chomp $A; chomp $B; my @A = split /;/, $A; my @B = split /;/, $B; $#A == $#B or die "Different # of fields: ｢$A｣ vs ｢$B｣\n"; unless ($#A) { $A eq $B or die "Title/error differs: ｢$A｣ vs ｢$B｣\n"; print colored("$A\n", "bright_cyan"); next; } if ($A[0] eq 'total-avg[sec]') { $A eq $B or die "Header differs: ｢$A｣ vs ｢$B｣\n"; @head = @A; print join(colored(";", 'bright_black'), map { "$_$head_colors{$dir{$_}//0}$arrows{$dir{$_}//0}" . color('reset') } @head), "\n"; next; } for (0..$#head) { print colored(";", 'bright_black') if $_; my ($a, $b, $dir) = ($A[$_], $B[$_], $dir{$head[$_]}); unless (defined $dir) { print STDERR colored("Unknown direction for ｢" . $head[$_] . "｣\n", 'bright_red'); $dir = $dir{$head[$_]} = 0; } if ($dir) { if (1.0 * ($dir > 0 ? $a : $b)) { my $d = $dir > 0 ? $b / $a : $a / $b; print $d < 0.5 ? color('bright_red') : $d < 0.95 ? color('red') : $d < 1/0.95 ? color('bright_white') : $d < 2 ? color('green') : color('bright_green'); printf "%1.2f".color('reset'), $d; } else { # 0 or non-numeric -> non-zero print $a eq $b ? $a : colored("$a↔$b", 'on_magenta'); } } else { # All fields without a known direction are assumed to stay # constant, warn if they're not. print $a eq $b ? $a : colored($a, 'bright_red on_magenta') . "↔" . colored($b, 'bright_green on_magenta'); } } print "\n"; } pmdk-1.8/src/benchmarks/pmem_memcpy.cpp0000664000000000000000000004133313615011243016735 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmem_memcpy.cpp -- benchmark implementation for pmem_memcpy */ #include #include #include #include #include #include #include #include #include #include #include "benchmark.hpp" #include "file.h" #define FLUSH_ALIGN 64 #define MAX_OFFSET (FLUSH_ALIGN - 1) struct pmem_bench; typedef size_t (*offset_fn)(struct pmem_bench *pmb, struct operation_info *info); /* * pmem_args -- benchmark specific arguments */ struct pmem_args { /* * Defines the copy operation direction. Whether it is * writing from RAM to PMEM (for argument value "write") * or PMEM to RAM (for argument value "read"). */ char *operation; /* * The source address offset used to test pmem_memcpy() * performance when source address is not aligned. */ size_t src_off; /* * The destination address offset used to test * pmem_memcpy() performance when destination address * is not aligned. */ size_t dest_off; /* The size of data chunk. */ size_t chunk_size; /* * Specifies the order in which data chunks are selected * to be copied. There are three modes supported: * stat, seq, rand. */ char *src_mode; /* * Specifies the order in which data chunks are written * to the destination address. There are three modes * supported: stat, seq, rand. */ char *dest_mode; /* * When this flag is set to true, PMEM is not used. * This option is useful, when comparing performance * of pmem_memcpy() function to regular memcpy(). */ bool memcpy; /* * When this flag is set to true, pmem_persist() * function is used, otherwise pmem_flush() is performed. */ bool persist; /* do not do warmup */ bool no_warmup; }; /* * pmem_bench -- benchmark context */ struct pmem_bench { /* random offsets */ unsigned *rand_offsets; /* number of elements in randoms array */ size_t n_rand_offsets; /* The size of the allocated PMEM */ size_t fsize; /* The size of the allocated buffer */ size_t bsize; /* Pointer to the allocated volatile memory */ unsigned char *buf; /* Pointer to the allocated PMEM */ unsigned char *pmem_addr; /* * This field gets 'buf' or 'pmem_addr' fields assigned, * depending on the prog_args operation direction. */ unsigned char *src_addr; /* * This field gets 'buf' or 'pmem_addr' fields assigned, * depending on the prog_args operation direction. */ unsigned char *dest_addr; /* Stores prog_args structure */ struct pmem_args *pargs; /* * Function which returns src offset. Matches src_mode. */ offset_fn func_src; /* * Function which returns dst offset. Matches dst_mode. */ offset_fn func_dest; /* * The actual operation performed based on benchmark specific * arguments. */ int (*func_op)(void *dest, void *source, size_t len); }; /* * operation_type -- type of operation relative to persistent memory */ enum operation_type { OP_TYPE_UNKNOWN, OP_TYPE_READ, OP_TYPE_WRITE }; /* * operation_mode -- the mode of the copy process * * * static - read/write always the same chunk, * * sequential - read/write chunk by chunk, * * random - read/write to chunks selected randomly. * * It is used to determine source mode as well as the destination mode. */ enum operation_mode { OP_MODE_UNKNOWN, OP_MODE_STAT, OP_MODE_SEQ, OP_MODE_RAND }; /* * parse_op_type -- parses command line "--operation" argument * and returns proper operation type. */ static enum operation_type parse_op_type(const char *arg) { if (strcmp(arg, "read") == 0) return OP_TYPE_READ; else if (strcmp(arg, "write") == 0) return OP_TYPE_WRITE; else return OP_TYPE_UNKNOWN; } /* * parse_op_mode -- parses command line "--src-mode" or "--dest-mode" * and returns proper operation mode. */ static enum operation_mode parse_op_mode(const char *arg) { if (strcmp(arg, "stat") == 0) return OP_MODE_STAT; else if (strcmp(arg, "seq") == 0) return OP_MODE_SEQ; else if (strcmp(arg, "rand") == 0) return OP_MODE_RAND; else return OP_MODE_UNKNOWN; } /* * mode_seq -- if copy mode is sequential mode_seq() returns * index of a chunk. */ static uint64_t mode_seq(struct pmem_bench *pmb, struct operation_info *info) { return info->args->n_ops_per_thread * info->worker->index + info->index; } /* * mode_stat -- if mode is static, the offset is always 0, * as only one block is used. */ static uint64_t mode_stat(struct pmem_bench *pmb, struct operation_info *info) { return 0; } /* * mode_rand -- if mode is random returns index of a random chunk */ static uint64_t mode_rand(struct pmem_bench *pmb, struct operation_info *info) { assert(info->index < pmb->n_rand_offsets); return info->args->n_ops_per_thread * info->worker->index + pmb->rand_offsets[info->index]; } /* * assign_mode_func -- parses "--src-mode" and "--dest-mode" command line * arguments and returns one of the above mode functions. */ static offset_fn assign_mode_func(char *option) { enum operation_mode op_mode = parse_op_mode(option); switch (op_mode) { case OP_MODE_STAT: return mode_stat; case OP_MODE_SEQ: return mode_seq; case OP_MODE_RAND: return mode_rand; default: return nullptr; } } /* * libc_memcpy -- copy using libc memcpy() function * followed by pmem_flush(). */ static int libc_memcpy(void *dest, void *source, size_t len) { memcpy(dest, source, len); pmem_flush(dest, len); return 0; } /* * libc_memcpy_persist -- copy using libc memcpy() function * followed by pmem_persist(). */ static int libc_memcpy_persist(void *dest, void *source, size_t len) { memcpy(dest, source, len); pmem_persist(dest, len); return 0; } /* * lipmem_memcpy_nodrain -- copy using libpmem pmem_memcpy_no_drain() * function without pmem_persist(). */ static int libpmem_memcpy_nodrain(void *dest, void *source, size_t len) { pmem_memcpy_nodrain(dest, source, len); return 0; } /* * libpmem_memcpy_persist -- copy using libpmem pmem_memcpy_persist() function. */ static int libpmem_memcpy_persist(void *dest, void *source, size_t len) { pmem_memcpy_persist(dest, source, len); return 0; } /* * assign_size -- assigns file and buffer size * depending on the operation mode and type. */ static int assign_size(struct pmem_bench *pmb, struct benchmark_args *args, enum operation_type *op_type) { *op_type = parse_op_type(pmb->pargs->operation); if (*op_type == OP_TYPE_UNKNOWN) { fprintf(stderr, "Invalid operation argument '%s'", pmb->pargs->operation); return -1; } enum operation_mode op_mode_src = parse_op_mode(pmb->pargs->src_mode); if (op_mode_src == OP_MODE_UNKNOWN) { fprintf(stderr, "Invalid source mode argument '%s'", pmb->pargs->src_mode); return -1; } enum operation_mode op_mode_dest = parse_op_mode(pmb->pargs->dest_mode); if (op_mode_dest == OP_MODE_UNKNOWN) { fprintf(stderr, "Invalid destination mode argument '%s'", pmb->pargs->dest_mode); return -1; } size_t large = args->n_ops_per_thread * pmb->pargs->chunk_size * args->n_threads; size_t little = pmb->pargs->chunk_size; if (*op_type == OP_TYPE_WRITE) { pmb->bsize = op_mode_src == OP_MODE_STAT ? little : large; pmb->fsize = op_mode_dest == OP_MODE_STAT ? little : large; if (pmb->pargs->src_off != 0) pmb->bsize += MAX_OFFSET; if (pmb->pargs->dest_off != 0) pmb->fsize += MAX_OFFSET; } else { pmb->fsize = op_mode_src == OP_MODE_STAT ? little : large; pmb->bsize = op_mode_dest == OP_MODE_STAT ? little : large; if (pmb->pargs->src_off != 0) pmb->fsize += MAX_OFFSET; if (pmb->pargs->dest_off != 0) pmb->bsize += MAX_OFFSET; } return 0; } /* * pmem_memcpy_init -- benchmark initialization * * Parses command line arguments, allocates persistent memory, and maps it. */ static int pmem_memcpy_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); int ret = 0; size_t file_size = 0; int flags = 0; enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } auto *pmb = (struct pmem_bench *)malloc(sizeof(struct pmem_bench)); assert(pmb != nullptr); pmb->pargs = (struct pmem_args *)args->opts; assert(pmb->pargs != nullptr); pmb->pargs->chunk_size = args->dsize; enum operation_type op_type; /* * Assign file and buffer size depending on the operation type * (READ from PMEM or WRITE to PMEM) */ if (assign_size(pmb, args, &op_type) != 0) { ret = -1; goto err_free_pmb; } pmb->buf = (unsigned char *)util_aligned_malloc(FLUSH_ALIGN, pmb->bsize); if (pmb->buf == nullptr) { perror("posix_memalign"); ret = -1; goto err_free_pmb; } pmb->n_rand_offsets = args->n_ops_per_thread * args->n_threads; assert(pmb->n_rand_offsets != 0); pmb->rand_offsets = (unsigned *)malloc(pmb->n_rand_offsets * sizeof(*pmb->rand_offsets)); if (pmb->rand_offsets == nullptr) { perror("malloc"); ret = -1; goto err_free_pmb_buf; } for (size_t i = 0; i < pmb->n_rand_offsets; ++i) pmb->rand_offsets[i] = rand() % args->n_ops_per_thread; if (type != TYPE_DEVDAX) { file_size = pmb->fsize; flags = PMEM_FILE_CREATE | PMEM_FILE_EXCL; } /* create a pmem file and memory map it */ pmb->pmem_addr = (unsigned char *)pmem_map_file( args->fname, file_size, flags, args->fmode, nullptr, nullptr); if (pmb->pmem_addr == nullptr) { perror(args->fname); ret = -1; goto err_free_pmb_rand_offsets; } if (op_type == OP_TYPE_READ) { pmb->src_addr = pmb->pmem_addr; pmb->dest_addr = pmb->buf; } else { pmb->src_addr = pmb->buf; pmb->dest_addr = pmb->pmem_addr; } /* set proper func_src() and func_dest() depending on benchmark args */ if ((pmb->func_src = assign_mode_func(pmb->pargs->src_mode)) == nullptr) { fprintf(stderr, "wrong src_mode parameter -- '%s'", pmb->pargs->src_mode); ret = -1; goto err_unmap; } if ((pmb->func_dest = assign_mode_func(pmb->pargs->dest_mode)) == nullptr) { fprintf(stderr, "wrong dest_mode parameter -- '%s'", pmb->pargs->dest_mode); ret = -1; goto err_unmap; } if (pmb->pargs->memcpy) { pmb->func_op = pmb->pargs->persist ? libc_memcpy_persist : libc_memcpy; } else { pmb->func_op = pmb->pargs->persist ? libpmem_memcpy_persist : libpmem_memcpy_nodrain; } if (!pmb->pargs->no_warmup) { memset(pmb->buf, 0, pmb->bsize); pmem_memset_persist(pmb->pmem_addr, 0, pmb->fsize); } pmembench_set_priv(bench, pmb); return 0; err_unmap: pmem_unmap(pmb->pmem_addr, pmb->fsize); err_free_pmb_rand_offsets: free(pmb->rand_offsets); err_free_pmb_buf: util_aligned_free(pmb->buf); err_free_pmb: free(pmb); return ret; } /* * pmem_memcpy_operation -- actual benchmark operation * * Depending on the memcpy flag "-m" tested operation will be memcpy() * or pmem_memcpy_persist(). */ static int pmem_memcpy_operation(struct benchmark *bench, struct operation_info *info) { auto *pmb = (struct pmem_bench *)pmembench_get_priv(bench); size_t src_index = pmb->func_src(pmb, info); size_t dest_index = pmb->func_dest(pmb, info); void *source = pmb->src_addr + src_index * pmb->pargs->chunk_size + pmb->pargs->src_off; void *dest = pmb->dest_addr + dest_index * pmb->pargs->chunk_size + pmb->pargs->dest_off; size_t len = pmb->pargs->chunk_size; pmb->func_op(dest, source, len); return 0; } /* * pmem_memcpy_exit -- benchmark cleanup */ static int pmem_memcpy_exit(struct benchmark *bench, struct benchmark_args *args) { auto *pmb = (struct pmem_bench *)pmembench_get_priv(bench); pmem_unmap(pmb->pmem_addr, pmb->fsize); util_aligned_free(pmb->buf); free(pmb->rand_offsets); free(pmb); return 0; } /* structure to define command line arguments */ static struct benchmark_clo pmem_memcpy_clo[8]; /* Stores information about benchmark. */ static struct benchmark_info pmem_memcpy_bench; CONSTRUCTOR(pmem_memcpy_constructor) void pmem_memcpy_constructor(void) { pmem_memcpy_clo[0].opt_short = 'o'; pmem_memcpy_clo[0].opt_long = "operation"; pmem_memcpy_clo[0].descr = "Operation type - write, read"; pmem_memcpy_clo[0].type = CLO_TYPE_STR; pmem_memcpy_clo[0].off = clo_field_offset(struct pmem_args, operation); pmem_memcpy_clo[0].def = "write"; pmem_memcpy_clo[1].opt_short = 'S'; pmem_memcpy_clo[1].opt_long = "src-offset"; pmem_memcpy_clo[1].descr = "Source cache line alignment" " offset"; pmem_memcpy_clo[1].type = CLO_TYPE_UINT; pmem_memcpy_clo[1].off = clo_field_offset(struct pmem_args, src_off); pmem_memcpy_clo[1].def = "0"; pmem_memcpy_clo[1].type_uint.size = clo_field_size(struct pmem_args, src_off); pmem_memcpy_clo[1].type_uint.base = CLO_INT_BASE_DEC; pmem_memcpy_clo[1].type_uint.min = 0; pmem_memcpy_clo[1].type_uint.max = MAX_OFFSET; pmem_memcpy_clo[2].opt_short = 'D'; pmem_memcpy_clo[2].opt_long = "dest-offset"; pmem_memcpy_clo[2].descr = "Destination cache line " "alignment offset"; pmem_memcpy_clo[2].type = CLO_TYPE_UINT; pmem_memcpy_clo[2].off = clo_field_offset(struct pmem_args, dest_off); pmem_memcpy_clo[2].def = "0"; pmem_memcpy_clo[2].type_uint.size = clo_field_size(struct pmem_args, dest_off); pmem_memcpy_clo[2].type_uint.base = CLO_INT_BASE_DEC; pmem_memcpy_clo[2].type_uint.min = 0; pmem_memcpy_clo[2].type_uint.max = MAX_OFFSET; pmem_memcpy_clo[3].opt_short = 0; pmem_memcpy_clo[3].opt_long = "src-mode"; pmem_memcpy_clo[3].descr = "Source reading mode"; pmem_memcpy_clo[3].type = CLO_TYPE_STR; pmem_memcpy_clo[3].off = clo_field_offset(struct pmem_args, src_mode); pmem_memcpy_clo[3].def = "seq"; pmem_memcpy_clo[4].opt_short = 0; pmem_memcpy_clo[4].opt_long = "dest-mode"; pmem_memcpy_clo[4].descr = "Destination writing mode"; pmem_memcpy_clo[4].type = CLO_TYPE_STR; pmem_memcpy_clo[4].off = clo_field_offset(struct pmem_args, dest_mode); pmem_memcpy_clo[4].def = "seq"; pmem_memcpy_clo[5].opt_short = 'm'; pmem_memcpy_clo[5].opt_long = "libc-memcpy"; pmem_memcpy_clo[5].descr = "Use libc memcpy()"; pmem_memcpy_clo[5].type = CLO_TYPE_FLAG; pmem_memcpy_clo[5].off = clo_field_offset(struct pmem_args, memcpy); pmem_memcpy_clo[5].def = "false"; pmem_memcpy_clo[6].opt_short = 'p'; pmem_memcpy_clo[6].opt_long = "persist"; pmem_memcpy_clo[6].descr = "Use pmem_persist()"; pmem_memcpy_clo[6].type = CLO_TYPE_FLAG; pmem_memcpy_clo[6].off = clo_field_offset(struct pmem_args, persist); pmem_memcpy_clo[6].def = "true"; pmem_memcpy_clo[7].opt_short = 'w'; pmem_memcpy_clo[7].opt_long = "no-warmup"; pmem_memcpy_clo[7].descr = "Don't do warmup"; pmem_memcpy_clo[7].def = "false"; pmem_memcpy_clo[7].type = CLO_TYPE_FLAG; pmem_memcpy_clo[7].off = clo_field_offset(struct pmem_args, no_warmup); pmem_memcpy_bench.name = "pmem_memcpy"; pmem_memcpy_bench.brief = "Benchmark for" "pmem_memcpy_persist() and " "pmem_memcpy_nodrain()" "operations"; pmem_memcpy_bench.init = pmem_memcpy_init; pmem_memcpy_bench.exit = pmem_memcpy_exit; pmem_memcpy_bench.multithread = true; pmem_memcpy_bench.multiops = true; pmem_memcpy_bench.operation = pmem_memcpy_operation; pmem_memcpy_bench.measure_time = true; pmem_memcpy_bench.clos = pmem_memcpy_clo; pmem_memcpy_bench.nclos = ARRAY_SIZE(pmem_memcpy_clo); pmem_memcpy_bench.opts_size = sizeof(struct pmem_args); pmem_memcpy_bench.rm_file = true; pmem_memcpy_bench.allow_poolset = false; pmem_memcpy_bench.print_bandwidth = true; REGISTER_BENCHMARK(pmem_memcpy_bench); }; pmdk-1.8/src/benchmarks/obj_pmalloc.cpp0000664000000000000000000003432313615011243016707 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * obj_pmalloc.cpp -- pmalloc benchmarks definition */ #include #include #include #include #include #include #include #include #include "benchmark.hpp" #include "file.h" #include "libpmemobj.h" #include "memops.h" #include "os.h" #include "pmalloc.h" #include "poolset_util.hpp" #include "valgrind_internal.h" /* * The factor used for PMEM pool size calculation, accounts for metadata, * fragmentation and etc. */ #define FACTOR 1.2f /* The minimum allocation size that pmalloc can perform */ #define ALLOC_MIN_SIZE 64 /* OOB and allocation header size */ #define OOB_HEADER_SIZE 64 /* * prog_args - command line parsed arguments */ struct prog_args { size_t minsize; /* minimum size for random allocation size */ bool use_random_size; /* if set, use random size allocations */ unsigned seed; /* PRNG seed */ }; POBJ_LAYOUT_BEGIN(pmalloc_layout); POBJ_LAYOUT_ROOT(pmalloc_layout, struct my_root); POBJ_LAYOUT_TOID(pmalloc_layout, uint64_t); POBJ_LAYOUT_END(pmalloc_layout); /* * my_root - root object */ struct my_root { TOID(uint64_t) offs; /* vector of the allocated object offsets */ }; /* * obj_bench - variables used in benchmark, passed within functions */ struct obj_bench { PMEMobjpool *pop; /* persistent pool handle */ struct prog_args *pa; /* prog_args structure */ size_t *sizes; /* sizes for allocations */ TOID(struct my_root) root; /* root object's OID */ uint64_t *offs; /* pointer to the vector of offsets */ }; /* * obj_init -- common part of the benchmark initialization for pmalloc and * pfree. It allocates the PMEM memory pool and the necessary offset vector. */ static int obj_init(struct benchmark *bench, struct benchmark_args *args) { struct my_root *root = nullptr; assert(bench != nullptr); assert(args != nullptr); assert(args->opts != nullptr); char path[PATH_MAX]; if (util_safe_strcpy(path, args->fname, sizeof(path)) != 0) return -1; enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } if (((struct prog_args *)(args->opts))->minsize >= args->dsize) { fprintf(stderr, "Wrong params - allocation size\n"); return -1; } auto *ob = (struct obj_bench *)malloc(sizeof(struct obj_bench)); if (ob == nullptr) { perror("malloc"); return -1; } pmembench_set_priv(bench, ob); ob->pa = (struct prog_args *)args->opts; size_t n_ops_total = args->n_ops_per_thread * args->n_threads; assert(n_ops_total != 0); /* Create pmemobj pool. */ size_t alloc_size = args->dsize; if (alloc_size < ALLOC_MIN_SIZE) alloc_size = ALLOC_MIN_SIZE; /* For data objects */ size_t poolsize = PMEMOBJ_MIN_POOL + (n_ops_total * (alloc_size + OOB_HEADER_SIZE)) /* for offsets */ + n_ops_total * sizeof(uint64_t); /* multiply by FACTOR for metadata, fragmentation, etc. */ poolsize = (size_t)(poolsize * FACTOR); if (args->is_poolset || type == TYPE_DEVDAX) { if (args->fsize < poolsize) { fprintf(stderr, "file size too large\n"); goto free_ob; } poolsize = 0; } else if (poolsize < PMEMOBJ_MIN_POOL) { poolsize = PMEMOBJ_MIN_POOL; } if (args->is_dynamic_poolset) { int ret = dynamic_poolset_create(args->fname, poolsize); if (ret == -1) goto free_ob; if (util_safe_strcpy(path, POOLSET_PATH, sizeof(path)) != 0) goto free_ob; poolsize = 0; } ob->pop = pmemobj_create(path, POBJ_LAYOUT_NAME(pmalloc_layout), poolsize, args->fmode); if (ob->pop == nullptr) { fprintf(stderr, "%s\n", pmemobj_errormsg()); goto free_ob; } ob->root = POBJ_ROOT(ob->pop, struct my_root); if (TOID_IS_NULL(ob->root)) { fprintf(stderr, "POBJ_ROOT: %s\n", pmemobj_errormsg()); goto free_pop; } root = D_RW(ob->root); assert(root != nullptr); POBJ_ZALLOC(ob->pop, &root->offs, uint64_t, n_ops_total * sizeof(PMEMoid)); if (TOID_IS_NULL(root->offs)) { fprintf(stderr, "POBJ_ZALLOC off_vect: %s\n", pmemobj_errormsg()); goto free_pop; } ob->offs = D_RW(root->offs); ob->sizes = (size_t *)malloc(n_ops_total * sizeof(size_t)); if (ob->sizes == nullptr) { fprintf(stderr, "malloc rand size vect err\n"); goto free_pop; } if (ob->pa->use_random_size) { size_t width = args->dsize - ob->pa->minsize; for (size_t i = 0; i < n_ops_total; i++) { auto hr = (uint32_t)os_rand_r(&ob->pa->seed); auto lr = (uint32_t)os_rand_r(&ob->pa->seed); uint64_t r64 = (uint64_t)hr << 32 | lr; ob->sizes[i] = r64 % width + ob->pa->minsize; } } else { for (size_t i = 0; i < n_ops_total; i++) ob->sizes[i] = args->dsize; } return 0; free_pop: pmemobj_close(ob->pop); free_ob: free(ob); return -1; } /* * obj_exit -- common part for the exit function for pmalloc and pfree * benchmarks. It frees the allocated offset vector and the memory pool. */ static int obj_exit(struct benchmark *bench, struct benchmark_args *args) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); free(ob->sizes); POBJ_FREE(&D_RW(ob->root)->offs); pmemobj_close(ob->pop); return 0; } /* * pmalloc_init -- initialization for the pmalloc benchmark. Performs only the * common initialization. */ static int pmalloc_init(struct benchmark *bench, struct benchmark_args *args) { return obj_init(bench, args); } /* * pmalloc_op -- actual benchmark operation. Performs the pmalloc allocations. */ static int pmalloc_op(struct benchmark *bench, struct operation_info *info) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); uint64_t i = info->index + info->worker->index * info->args->n_ops_per_thread; int ret = pmalloc(ob->pop, &ob->offs[i], ob->sizes[i], 0, 0); if (ret) { fprintf(stderr, "pmalloc ret: %d\n", ret); return ret; } return 0; } struct pmix_worker { size_t nobjects; size_t shuffle_start; unsigned seed; }; /* * pmix_worker_init -- initialization of the worker structure */ static int pmix_worker_init(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { struct pmix_worker *w = (struct pmix_worker *)calloc(1, sizeof(*w)); auto *ob = (struct obj_bench *)pmembench_get_priv(bench); if (w == nullptr) return -1; w->seed = ob->pa->seed; worker->priv = w; return 0; } /* * pmix_worker_fini -- destruction of the worker structure */ static void pmix_worker_fini(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *w = (struct pmix_worker *)worker->priv; free(w); } /* * shuffle_objects -- randomly shuffle elements on a list * * Ideally, we wouldn't count the time this function takes, but for all * practical purposes this is fast enough and isn't visible on the results. * Just make sure the amount of objects to shuffle is not large. */ static void shuffle_objects(uint64_t *objects, size_t start, size_t nobjects, unsigned *seed) { uint64_t tmp; size_t dest; for (size_t n = start; n < nobjects; ++n) { dest = RRAND_R(seed, nobjects - 1, 0); tmp = objects[n]; objects[n] = objects[dest]; objects[dest] = tmp; } } #define FREE_PCT 10 #define FREE_OPS 10 /* * pmix_op -- mixed workload benchmark */ static int pmix_op(struct benchmark *bench, struct operation_info *info) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); auto *w = (struct pmix_worker *)info->worker->priv; uint64_t idx = info->worker->index * info->args->n_ops_per_thread; uint64_t *objects = &ob->offs[idx]; if (w->nobjects > FREE_OPS && FREE_PCT > RRAND_R(&w->seed, 100, 0)) { shuffle_objects(objects, w->shuffle_start, w->nobjects, &w->seed); for (int i = 0; i < FREE_OPS; ++i) { uint64_t off = objects[--w->nobjects]; pfree(ob->pop, &off); } w->shuffle_start = w->nobjects; } else { int ret = pmalloc(ob->pop, &objects[w->nobjects++], ob->sizes[idx + info->index], 0, 0); if (ret) { fprintf(stderr, "pmalloc ret: %d\n", ret); return ret; } } return 0; } /* * pmalloc_exit -- the end of the pmalloc benchmark. Frees the memory allocated * during pmalloc_op and performs the common exit operations. */ static int pmalloc_exit(struct benchmark *bench, struct benchmark_args *args) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); for (size_t i = 0; i < args->n_ops_per_thread * args->n_threads; i++) { if (ob->offs[i]) pfree(ob->pop, &ob->offs[i]); } return obj_exit(bench, args); } /* * pfree_init -- initialization for the pfree benchmark. Performs the common * initialization and allocates the memory to be freed during pfree_op. */ static int pfree_init(struct benchmark *bench, struct benchmark_args *args) { int ret = obj_init(bench, args); if (ret) return ret; auto *ob = (struct obj_bench *)pmembench_get_priv(bench); for (size_t i = 0; i < args->n_ops_per_thread * args->n_threads; i++) { ret = pmalloc(ob->pop, &ob->offs[i], ob->sizes[i], 0, 0); if (ret) { fprintf(stderr, "pmalloc at idx %" PRIu64 " ret: %s\n", i, pmemobj_errormsg()); /* free the allocated memory */ while (i != 0) { pfree(ob->pop, &ob->offs[i - 1]); i--; } obj_exit(bench, args); return ret; } } return 0; } /* * pmalloc_op -- actual benchmark operation. Performs the pfree operation. */ static int pfree_op(struct benchmark *bench, struct operation_info *info) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); uint64_t i = info->index + info->worker->index * info->args->n_ops_per_thread; pfree(ob->pop, &ob->offs[i]); return 0; } /* command line options definition */ static struct benchmark_clo pmalloc_clo[3]; /* * Stores information about pmalloc benchmark. */ static struct benchmark_info pmalloc_info; /* * Stores information about pfree benchmark. */ static struct benchmark_info pfree_info; /* * Stores information about pmix benchmark. */ static struct benchmark_info pmix_info; CONSTRUCTOR(obj_pmalloc_constructor) void obj_pmalloc_constructor(void) { pmalloc_clo[0].opt_short = 'r'; pmalloc_clo[0].opt_long = "random"; pmalloc_clo[0].descr = "Use random size allocations - " "from min-size to data-size"; pmalloc_clo[0].off = clo_field_offset(struct prog_args, use_random_size); pmalloc_clo[0].type = CLO_TYPE_FLAG; pmalloc_clo[1].opt_short = 'm'; pmalloc_clo[1].opt_long = "min-size"; pmalloc_clo[1].descr = "Minimum size of allocation for " "random mode"; pmalloc_clo[1].type = CLO_TYPE_UINT; pmalloc_clo[1].off = clo_field_offset(struct prog_args, minsize); pmalloc_clo[1].def = "1"; pmalloc_clo[1].type_uint.size = clo_field_size(struct prog_args, minsize); pmalloc_clo[1].type_uint.base = CLO_INT_BASE_DEC; pmalloc_clo[1].type_uint.min = 1; pmalloc_clo[1].type_uint.max = UINT64_MAX; pmalloc_clo[2].opt_short = 'S'; pmalloc_clo[2].opt_long = "seed"; pmalloc_clo[2].descr = "Random mode seed value"; pmalloc_clo[2].off = clo_field_offset(struct prog_args, seed); pmalloc_clo[2].def = "1"; pmalloc_clo[2].type = CLO_TYPE_UINT; pmalloc_clo[2].type_uint.size = clo_field_size(struct prog_args, seed); pmalloc_clo[2].type_uint.base = CLO_INT_BASE_DEC; pmalloc_clo[2].type_uint.min = 1; pmalloc_clo[2].type_uint.max = UINT_MAX; pmalloc_info.name = "pmalloc", pmalloc_info.brief = "Benchmark for internal pmalloc() " "operation"; pmalloc_info.init = pmalloc_init; pmalloc_info.exit = pmalloc_exit; pmalloc_info.multithread = true; pmalloc_info.multiops = true; pmalloc_info.operation = pmalloc_op; pmalloc_info.measure_time = true; pmalloc_info.clos = pmalloc_clo; pmalloc_info.nclos = ARRAY_SIZE(pmalloc_clo); pmalloc_info.opts_size = sizeof(struct prog_args); pmalloc_info.rm_file = true; pmalloc_info.allow_poolset = true; REGISTER_BENCHMARK(pmalloc_info); pfree_info.name = "pfree"; pfree_info.brief = "Benchmark for internal pfree() " "operation"; pfree_info.init = pfree_init; pfree_info.exit = pmalloc_exit; /* same as for pmalloc */ pfree_info.multithread = true; pfree_info.multiops = true; pfree_info.operation = pfree_op; pfree_info.measure_time = true; pfree_info.clos = pmalloc_clo; pfree_info.nclos = ARRAY_SIZE(pmalloc_clo); pfree_info.opts_size = sizeof(struct prog_args); pfree_info.rm_file = true; pfree_info.allow_poolset = true; REGISTER_BENCHMARK(pfree_info); pmix_info.name = "pmix"; pmix_info.brief = "Benchmark for mixed alloc/free workload"; pmix_info.init = pmalloc_init; pmix_info.exit = pmalloc_exit; /* same as for pmalloc */ pmix_info.multithread = true; pmix_info.multiops = true; pmix_info.operation = pmix_op; pmix_info.init_worker = pmix_worker_init; pmix_info.free_worker = pmix_worker_fini; pmix_info.measure_time = true; pmix_info.clos = pmalloc_clo; pmix_info.nclos = ARRAY_SIZE(pmalloc_clo); pmix_info.opts_size = sizeof(struct prog_args); pmix_info.rm_file = true; pmix_info.allow_poolset = true; REGISTER_BENCHMARK(pmix_info); }; pmdk-1.8/src/benchmarks/benchmark_time.hpp0000664000000000000000000000424513615011243017403 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * benchmark_time.hpp -- declarations of benchmark_time module */ #include typedef struct timespec benchmark_time_t; void benchmark_time_get(benchmark_time_t *time); void benchmark_time_diff(benchmark_time_t *d, benchmark_time_t *t1, benchmark_time_t *t2); double benchmark_time_get_secs(benchmark_time_t *t); unsigned long long benchmark_time_get_nsecs(benchmark_time_t *t); int benchmark_time_compare(const benchmark_time_t *t1, const benchmark_time_t *t2); void benchmark_time_set(benchmark_time_t *time, unsigned long long nsecs); unsigned long long benchmark_get_avg_get_time(void); pmdk-1.8/src/benchmarks/clo.hpp0000664000000000000000000000435713615011243015214 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * clo.hpp -- command line options module declarations */ int benchmark_clo_parse(int argc, char *argv[], struct benchmark_clo *clos, ssize_t nclo, struct clo_vec *clovec); int benchmark_clo_parse_scenario(struct scenario *scenario, struct benchmark_clo *clos, size_t nclo, struct clo_vec *clovec); const char *benchmark_clo_str(struct benchmark_clo *clo, void *args, size_t size); int clo_get_scenarios(int argc, char *argv[], struct scenarios *available_scenarios, struct scenarios *found_scenarios); int benchmark_override_clos_in_scenario(struct scenario *scenario, int argc, char *argv[], struct benchmark_clo *clos, int nclos); pmdk-1.8/src/benchmarks/config_reader_win.cpp0000664000000000000000000002110513615011243020064 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * config_reader_win.cpp -- config reader module definitions */ #include #include #include #include #include #include "config_reader.hpp" #include "queue.h" #include "scenario.hpp" #define SECTION_GLOBAL TEXT("global") #define KEY_BENCHMARK TEXT("bench") #define KEY_GROUP TEXT("group") /* * Maximum section size according to MSDN documentation */ #define SIZEOF_SECTION 32767 #define NULL_LIST_EMPTY(x) (_tcslen(x) == 0) #define NULL_LIST_NEXT(x) ((x) += (_tcslen(x) + 1)) #define KV_LIST_EMPTY(x) (_tcslen(x) == 0) #define KV_FIRST(x) #define KV_LIST_NEXT(x) \ ((x) += (_tcslen(x) + 1), (x) += (_tcslen(x) + 1), \ (x) = kv_list_skip_comment(x)) #define KV_LIST_KEY(x) (x) #define KV_LIST_VALUE(x) ((x) + _tcslen(x) + 1) #define KV_LIST_INIT(x) kv_list_init(x) #define LIST LPTSTR #define KV_LIST LPTSTR /* * kv_list_skip_comment -- skip comment lines in ini file */ static KV_LIST kv_list_skip_comment(KV_LIST list) { while (list[0] == TEXT('#')) list += (_tcslen(list) + 1); return list; } /* * kv_list_init -- init KV list */ static KV_LIST kv_list_init(LPTSTR list) { list = kv_list_skip_comment(list); for (KV_LIST it = list; !KV_LIST_EMPTY(it); KV_LIST_NEXT(it)) { LPTSTR c = _tcsstr(it, TEXT("=")); if (c == NULL) return NULL; *c = TEXT('\0'); } return list; } /* * config_reader -- handle structure */ struct config_reader { LPTSTR lpFileName; }; /* * config_reader_alloc -- allocate config reader */ struct config_reader * config_reader_alloc(void) { struct config_reader *cr = (struct config_reader *)malloc(sizeof(*cr)); if (cr == NULL) return NULL; return cr; } /* * config_reader_read -- read config file */ int config_reader_read(struct config_reader *cr, const char *fname) { DWORD len = 0; LPTSTR buf = TEXT(" "); /* get the length of the full pathname incl. terminating null char */ len = GetFullPathName((LPTSTR)fname, 0, buf, NULL); if (len == 0) { /* the function failed */ return -1; } else { /* allocate a buffer large enough to store the pathname */ LPTSTR buffer = (LPTSTR)malloc(len * sizeof(TCHAR)); DWORD ret = GetFullPathName((LPTSTR)fname, len, buffer, NULL); if (_taccess(buffer, 0) != 0) { printf("%s", strerror(errno)); return -1; } cr->lpFileName = (LPTSTR)buffer; } return 0; } /* * config_reader_free -- free config reader */ void config_reader_free(struct config_reader *cr) { free(cr); } /* * is_scenario -- (internal) return true if _name_ is scenario name * * This filters out the _global_ and _config_ sections. */ static int is_scenario(LPTSTR name) { return _tcscmp(name, SECTION_GLOBAL); } /* * is_argument -- (internal) return true if _name_ is argument name * * This filters out the _benchmark_ key. */ static int is_argument(LPTSTR name) { return _tcscmp(name, KEY_BENCHMARK) != 0 && _tcscmp(name, KEY_GROUP) != 0; } /* * config_reader_get_scenarios -- return scenarios from config file * * This function reads the config file and returns a list of scenarios. * Each scenario contains a list of key/value arguments. * The scenario's arguments are merged with arguments from global section. */ int config_reader_get_scenarios(struct config_reader *cr, struct scenarios **scenarios) { /* * Read all groups. * The config file must have at least one group, otherwise * it is considered as invalid. */ int ret = 0; TCHAR *sections = (TCHAR *)malloc(sizeof(TCHAR) * SIZEOF_SECTION); if (!sections) return -1; GetPrivateProfileSectionNames(sections, SIZEOF_SECTION, cr->lpFileName); if (NULL_LIST_EMPTY(sections)) { ret = -1; goto err_sections; } /* * Check if global section is present and read it. */ TCHAR *global = (TCHAR *)malloc(sizeof(TCHAR) * SIZEOF_SECTION); if (!global) return -1; GetPrivateProfileSection(SECTION_GLOBAL, global, SIZEOF_SECTION, cr->lpFileName); KV_LIST global_kv = KV_LIST_INIT(global); int has_global = !KV_LIST_EMPTY(global_kv); struct scenarios *s = scenarios_alloc(); assert(NULL != s); if (!s) { ret = -1; goto err_gkeys; } LPTSTR global_group = NULL; for (KV_LIST it = global_kv; !KV_LIST_EMPTY(it); KV_LIST_NEXT(it)) { if (_tcscmp(KV_LIST_KEY(it), KEY_GROUP) == 0) { global_group = KV_LIST_VALUE(it); break; } } TCHAR *section; for (LPTSTR group_name = sections; !NULL_LIST_EMPTY(group_name); group_name = NULL_LIST_NEXT(group_name)) { /* * Check whether a group is a scenario * or global section. */ if (!is_scenario(group_name)) continue; /* * Check for KEY_BENCHMARK which contains benchmark name. * If not present the benchmark name is the same as the * name of the section. */ section = (TCHAR *)malloc(sizeof(TCHAR) * SIZEOF_SECTION); if (!section) ret = -1; GetPrivateProfileSection(group_name, section, SIZEOF_SECTION, cr->lpFileName); KV_LIST section_kv = KV_LIST_INIT(section); struct scenario *scenario = NULL; LPTSTR name = NULL; LPTSTR group = NULL; for (KV_LIST it = section_kv; !KV_LIST_EMPTY(it); KV_LIST_NEXT(it)) { if (_tcscmp(KV_LIST_KEY(it), KEY_BENCHMARK) == 0) { name = KV_LIST_VALUE(it); } if (_tcscmp(KV_LIST_KEY(it), KEY_GROUP) == 0) { group = KV_LIST_VALUE(it); } } if (name == NULL) { scenario = scenario_alloc((const char *)group_name, (const char *)group_name); } else { scenario = scenario_alloc((const char *)group_name, (const char *)name); } assert(scenario != NULL); if (has_global) { /* * Merge key/values from global section. */ for (KV_LIST it = global_kv; !KV_LIST_EMPTY(it); KV_LIST_NEXT(it)) { LPTSTR key = KV_LIST_KEY(it); if (!is_argument(key)) continue; LPTSTR value = KV_LIST_VALUE(it); assert(NULL != value); if (!value) { ret = -1; goto err_scenarios; } struct kv *kv = kv_alloc((const char *)key, (const char *)value); assert(NULL != kv); if (!kv) { ret = -1; goto err_scenarios; } PMDK_TAILQ_INSERT_TAIL(&scenario->head, kv, next); } } /* check for group name */ if (group) { scenario_set_group(scenario, (const char *)group); } else if (global_group) { scenario_set_group(scenario, (const char *)global_group); } for (KV_LIST it = section_kv; !KV_LIST_EMPTY(it); KV_LIST_NEXT(it)) { LPTSTR key = KV_LIST_KEY(it); if (!is_argument(key)) continue; LPTSTR value = KV_LIST_VALUE(it); assert(NULL != value); if (!value) { ret = -1; goto err_scenarios; } struct kv *kv = kv_alloc((const char *)key, (const char *)value); assert(NULL != kv); if (!kv) { ret = -1; goto err_scenarios; } PMDK_TAILQ_INSERT_TAIL(&scenario->head, kv, next); } PMDK_TAILQ_INSERT_TAIL(&s->head, scenario, next); free(section); } *scenarios = s; free(global); free(sections); return 0; err_scenarios: free(section); scenarios_free(s); err_gkeys: free(global); err_sections: free(sections); return ret; } pmdk-1.8/src/benchmarks/pmemobj_tx.cpp0000664000000000000000000010614213615011243016571 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmemobj_tx.cpp -- pmemobj_tx_alloc(), pmemobj_tx_free(), * pmemobj_tx_realloc(), pmemobj_tx_add_range() benchmarks. */ #include #include #include #include #include #include #include #include "benchmark.hpp" #include "file.h" #include "libpmemobj.h" #include "poolset_util.hpp" #define LAYOUT_NAME "benchmark" #define FACTOR 1.2f #define ALLOC_OVERHEAD 64 /* * operations number is limited to prevent stack overflow during * performing recursive functions. */ #define MAX_OPS 10000 TOID_DECLARE(struct item, 0); struct obj_tx_bench; struct obj_tx_worker; int obj_tx_init(struct benchmark *bench, struct benchmark_args *args); int obj_tx_exit(struct benchmark *bench, struct benchmark_args *args); /* * type_num_mode -- type number mode */ enum type_num_mode { NUM_MODE_ONE, NUM_MODE_PER_THREAD, NUM_MODE_RAND, NUM_MODE_UNKNOWN }; /* * op_mode -- operation type */ enum op_mode { OP_MODE_COMMIT, OP_MODE_ABORT, OP_MODE_ABORT_NESTED, OP_MODE_ONE_OBJ, OP_MODE_ONE_OBJ_NESTED, OP_MODE_ONE_OBJ_RANGE, OP_MODE_ONE_OBJ_NESTED_RANGE, OP_MODE_ALL_OBJ, OP_MODE_ALL_OBJ_NESTED, OP_MODE_UNKNOWN }; /* * lib_mode -- operation type */ enum lib_mode { LIB_MODE_DRAM, LIB_MODE_OBJ_TX, LIB_MODE_OBJ_ATOMIC, LIB_MODE_NONE, }; /* * nesting_mode -- nesting type */ enum nesting_mode { NESTING_MODE_SIM, NESTING_MODE_TX, NESTING_MODE_UNKNOWN, }; /* * add_range_mode -- operation type for obj_add_range benchmark */ enum add_range_mode { ADD_RANGE_MODE_ONE_TX, ADD_RANGE_MODE_NESTED_TX }; /* * parse_mode -- parsing function type */ enum parse_mode { PARSE_OP_MODE, PARSE_OP_MODE_ADD_RANGE }; typedef size_t (*fn_type_num_t)(struct obj_tx_bench *obj_bench, size_t worker_idx, size_t op_idx); typedef size_t (*fn_num_t)(size_t idx); typedef int (*fn_op_t)(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx); typedef struct offset (*fn_os_off_t)(struct obj_tx_bench *obj_bench, size_t idx); typedef enum op_mode (*fn_parse_t)(const char *arg); /* * obj_tx_args -- stores command line parsed arguments. */ struct obj_tx_args { /* * operation which will be performed when flag io set to false. * modes for obj_tx_alloc, obj_tx_free and obj_tx_realloc: * - basic - transaction will be committed * - abort - 'external' transaction will be aborted. * - abort-nested - all nested transactions will be * aborted. * * modes for obj_tx_add_range benchmark: * - basic - one object is added to undo log many times in * one transaction. * - range - fields of one object are added to undo * log many times in one transaction. * - all-obj - all objects are added to undo log in * one transaction. * - range-nested - fields of one object are added to undo * log many times in many nested transactions. * - one-obj-nested - one object is added to undo log many * times in many nested transactions. * - all-obj-nested - all objects are added to undo log in * many separate, nested transactions. */ char *operation; /* * type number for each persistent object. There are three modes: * - one - all of objects have the same type number * - per-thread - all of object allocated by the same * thread have the same type number * - rand - type numbers are assigned randomly for * each persistent object */ char *type_num; /* * define s which library will be used in main operations There are * three modes in which benchmark can be run: * - tx - uses PMEM transactions * - pmem - uses PMEM without transactions * - dram - does not use PMEM */ char *lib; unsigned nested; /* number of nested transactions */ unsigned min_size; /* minimum allocation size */ unsigned min_rsize; /* minimum reallocation size */ unsigned rsize; /* reallocation size */ bool change_type; /* change type number in reallocation */ size_t obj_size; /* size of each allocated object */ size_t n_ops; /* number of operations */ int parse_mode; /* type of parsing function */ }; /* * obj_tx_bench -- stores variables used in benchmark, passed within functions. */ static struct obj_tx_bench { PMEMobjpool *pop; /* handle to persistent pool */ struct obj_tx_args *obj_args; /* pointer to benchmark arguments */ size_t *random_types; /* array to store random type numbers */ size_t *sizes; /* array to store size of each allocation */ size_t *resizes; /* array to store size of each reallocation */ size_t n_objs; /* number of objects to allocate */ int type_mode; /* type number mode */ int op_mode; /* type of operation */ int lib_mode; /* type of operation used in initialization */ int lib_op; /* type of main operation */ int lib_op_free; /* type of main operation */ int nesting_mode; /* type of nesting in main operation */ fn_num_t n_oid; /* returns object's number in array */ fn_os_off_t fn_off; /* returns offset for proper operation */ /* * fn_type_num gets proper function assigned, depending on the * value of the type_mode argument, which returns proper type number for * each persistent object. Possible functions are: * - type_mode_one, * - type_mode_rand. */ fn_type_num_t fn_type_num; /* * fn_op gets proper array with functions pointer assigned, depending on * function which is tested by benchmark. Possible arrays are: * -alloc_op * -free_op * -realloc_op */ fn_op_t *fn_op; } obj_bench; /* * item -- TOID's structure */ struct item; /* * obj_tx_worker - stores variables used by one thread. */ struct obj_tx_worker { TOID(struct item) * oids; char **items; unsigned tx_level; unsigned max_level; }; /* * offset - stores offset data used in pmemobj_tx_add_range() */ struct offset { uint64_t off; size_t size; }; /* * alloc_dram -- main operations for obj_tx_alloc benchmark in dram mode */ static int alloc_dram(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { auto *obj_worker = (struct obj_tx_worker *)worker->priv; obj_worker->items[idx] = (char *)malloc(obj_bench->sizes[idx]); if (obj_worker->items[idx] == nullptr) { perror("malloc"); return -1; } return 0; } /* * alloc_pmem -- main operations for obj_tx_alloc benchmark in pmem mode */ static int alloc_pmem(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { size_t type_num = obj_bench->fn_type_num(obj_bench, worker->index, idx); auto *obj_worker = (struct obj_tx_worker *)worker->priv; if (pmemobj_alloc(obj_bench->pop, &obj_worker->oids[idx].oid, obj_bench->sizes[idx], type_num, nullptr, nullptr) != 0) { perror("pmemobj_alloc"); return -1; } return 0; } /* * alloc_tx -- main operations for obj_tx_alloc benchmark in tx mode */ static int alloc_tx(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { size_t type_num = obj_bench->fn_type_num(obj_bench, worker->index, idx); auto *obj_worker = (struct obj_tx_worker *)worker->priv; obj_worker->oids[idx].oid = pmemobj_tx_xalloc( obj_bench->sizes[idx], type_num, POBJ_XALLOC_NO_FLUSH); if (OID_IS_NULL(obj_worker->oids[idx].oid)) { perror("pmemobj_tx_alloc"); return -1; } return 0; } /* * free_dram -- main operations for obj_tx_free benchmark in dram mode */ static int free_dram(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { auto *obj_worker = (struct obj_tx_worker *)worker->priv; free(obj_worker->items[idx]); return 0; } /* * free_pmem -- main operations for obj_tx_free benchmark in pmem mode */ static int free_pmem(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { auto *obj_worker = (struct obj_tx_worker *)worker->priv; POBJ_FREE(&obj_worker->oids[idx]); return 0; } /* * free_tx -- main operations for obj_tx_free benchmark in tx mode */ static int free_tx(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { auto *obj_worker = (struct obj_tx_worker *)worker->priv; TX_FREE(obj_worker->oids[idx]); return 0; } /* * no_free -- exit operation for benchmarks obj_tx_alloc and obj_tx_free * if there is no need to free memory */ static int no_free(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { return 0; } /* * realloc_dram -- main operations for obj_tx_realloc benchmark in dram mode */ static int realloc_dram(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { auto *obj_worker = (struct obj_tx_worker *)worker->priv; auto *tmp = (char *)realloc(obj_worker->items[idx], obj_bench->resizes[idx]); if (tmp == nullptr) { perror("realloc"); return -1; } obj_worker->items[idx] = tmp; return 0; } /* * realloc_pmem -- main operations for obj_tx_realloc benchmark in pmem mode */ static int realloc_pmem(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { auto *obj_worker = (struct obj_tx_worker *)worker->priv; size_t type_num = obj_bench->fn_type_num(obj_bench, worker->index, idx); if (obj_bench->obj_args->change_type) type_num++; if (pmemobj_realloc(obj_bench->pop, &obj_worker->oids[idx].oid, obj_bench->resizes[idx], type_num) != 0) { perror("pmemobj_realloc"); return -1; } return 0; } /* * realloc_tx -- main operations for obj_tx_realloc benchmark in tx mode */ static int realloc_tx(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { auto *obj_worker = (struct obj_tx_worker *)worker->priv; size_t type_num = obj_bench->fn_type_num(obj_bench, worker->index, idx); if (obj_bench->obj_args->change_type) type_num++; obj_worker->oids[idx].oid = pmemobj_tx_realloc( obj_worker->oids[idx].oid, obj_bench->sizes[idx], type_num); if (OID_IS_NULL(obj_worker->oids[idx].oid)) { perror("pmemobj_tx_realloc"); return -1; } return 0; } /* * add_range_nested_tx -- main operations of the obj_tx_add_range with nesting. */ static int add_range_nested_tx(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { int ret = 0; auto *obj_worker = (struct obj_tx_worker *)worker->priv; TX_BEGIN(obj_bench->pop) { if (obj_bench->obj_args->n_ops != obj_worker->tx_level) { size_t n_oid = obj_bench->n_oid(obj_worker->tx_level); struct offset offset = obj_bench->fn_off( obj_bench, obj_worker->tx_level); pmemobj_tx_add_range(obj_worker->oids[n_oid].oid, offset.off, offset.size); obj_worker->tx_level++; ret = add_range_nested_tx(obj_bench, worker, idx); } } TX_ONABORT { fprintf(stderr, "transaction failed\n"); ret = -1; } TX_END return ret; } /* * add_range_tx -- main operations of the obj_tx_add_range without nesting. */ static int add_range_tx(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { int ret = 0; size_t i = 0; auto *obj_worker = (struct obj_tx_worker *)worker->priv; TX_BEGIN(obj_bench->pop) { for (i = 0; i < obj_bench->obj_args->n_ops; i++) { size_t n_oid = obj_bench->n_oid(i); struct offset offset = obj_bench->fn_off(obj_bench, i); ret = pmemobj_tx_add_range(obj_worker->oids[n_oid].oid, offset.off, offset.size); } } TX_ONABORT { fprintf(stderr, "transaction failed\n"); ret = -1; } TX_END return ret; } /* * obj_op_sim -- main function for benchmarks which simulates nested * transactions on dram or pmemobj atomic API by calling function recursively. */ static int obj_op_sim(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { int ret = 0; auto *obj_worker = (struct obj_tx_worker *)worker->priv; if (obj_worker->max_level == obj_worker->tx_level) { ret = obj_bench->fn_op[obj_bench->lib_op](obj_bench, worker, idx); } else { obj_worker->tx_level++; ret = obj_op_sim(obj_bench, worker, idx); } return ret; } /* * obj_op_tx -- main recursive function for transactional benchmarks */ static int obj_op_tx(struct obj_tx_bench *obj_bench, struct worker_info *worker, size_t idx) { volatile int ret = 0; auto *obj_worker = (struct obj_tx_worker *)worker->priv; TX_BEGIN(obj_bench->pop) { if (obj_worker->max_level == obj_worker->tx_level) { ret = obj_bench->fn_op[obj_bench->lib_op](obj_bench, worker, idx); if (obj_bench->op_mode == OP_MODE_ABORT_NESTED) pmemobj_tx_abort(-1); } else { obj_worker->tx_level++; ret = obj_op_tx(obj_bench, worker, idx); if (--obj_worker->tx_level == 0 && obj_bench->op_mode == OP_MODE_ABORT) pmemobj_tx_abort(-1); } } TX_ONABORT { if (obj_bench->op_mode != OP_MODE_ABORT && obj_bench->op_mode != OP_MODE_ABORT_NESTED) { fprintf(stderr, "transaction failed\n"); ret = -1; } } TX_END return ret; } /* * type_mode_one -- always returns 0, as in the mode NUM_MODE_ONE * all of the persistent objects have the same type_number value. */ static size_t type_mode_one(struct obj_tx_bench *obj_bench, size_t worker_idx, size_t op_idx) { return 0; } /* * type_mode_per_thread -- always returns worker index to all of the persistent * object allocated by the same thread have the same type number. */ static size_t type_mode_per_thread(struct obj_tx_bench *obj_bench, size_t worker_idx, size_t op_idx) { return worker_idx; } /* * type_mode_rand -- returns the value from the random_types array assigned * for the specific operation in a specific thread. */ static size_t type_mode_rand(struct obj_tx_bench *obj_bench, size_t worker_idx, size_t op_idx) { return obj_bench->random_types[op_idx]; } /* * parse_op_mode_add_range -- parses command line "--operation" argument * and returns proper op_mode enum value for obj_tx_add_range. */ static enum op_mode parse_op_mode_add_range(const char *arg) { if (strcmp(arg, "basic") == 0) return OP_MODE_ONE_OBJ; else if (strcmp(arg, "one-obj-nested") == 0) return OP_MODE_ONE_OBJ_NESTED; else if (strcmp(arg, "range") == 0) return OP_MODE_ONE_OBJ_RANGE; else if (strcmp(arg, "range-nested") == 0) return OP_MODE_ONE_OBJ_NESTED_RANGE; else if (strcmp(arg, "all-obj") == 0) return OP_MODE_ALL_OBJ; else if (strcmp(arg, "all-obj-nested") == 0) return OP_MODE_ALL_OBJ_NESTED; else return OP_MODE_UNKNOWN; } /* * parse_op_mode -- parses command line "--operation" argument * and returns proper op_mode enum value. */ static enum op_mode parse_op_mode(const char *arg) { if (strcmp(arg, "basic") == 0) return OP_MODE_COMMIT; else if (strcmp(arg, "abort") == 0) return OP_MODE_ABORT; else if (strcmp(arg, "abort-nested") == 0) return OP_MODE_ABORT_NESTED; else return OP_MODE_UNKNOWN; } static fn_op_t alloc_op[] = {alloc_dram, alloc_tx, alloc_pmem}; static fn_op_t free_op[] = {free_dram, free_tx, free_pmem, no_free}; static fn_op_t realloc_op[] = {realloc_dram, realloc_tx, realloc_pmem}; static fn_op_t add_range_op[] = {add_range_tx, add_range_nested_tx}; static fn_parse_t parse_op[] = {parse_op_mode, parse_op_mode_add_range}; static fn_op_t nestings[] = {obj_op_sim, obj_op_tx}; /* * parse_type_num_mode -- converts string to type_num_mode enum */ static enum type_num_mode parse_type_num_mode(const char *arg) { if (strcmp(arg, "one") == 0) return NUM_MODE_ONE; else if (strcmp(arg, "per-thread") == 0) return NUM_MODE_PER_THREAD; else if (strcmp(arg, "rand") == 0) return NUM_MODE_RAND; fprintf(stderr, "unknown type number\n"); return NUM_MODE_UNKNOWN; } /* * parse_lib_mode -- converts string to type_num_mode enum */ static enum lib_mode parse_lib_mode(const char *arg) { if (strcmp(arg, "dram") == 0) return LIB_MODE_DRAM; else if (strcmp(arg, "pmem") == 0) return LIB_MODE_OBJ_ATOMIC; else if (strcmp(arg, "tx") == 0) return LIB_MODE_OBJ_TX; fprintf(stderr, "unknown lib mode\n"); return LIB_MODE_NONE; } static fn_type_num_t type_num_fn[] = {type_mode_one, type_mode_per_thread, type_mode_rand, nullptr}; /* * one_num -- returns always the same number. */ static size_t one_num(size_t idx) { return 0; } /* * diff_num -- returns number given as argument. */ static size_t diff_num(size_t idx) { return idx; } /* * off_entire -- returns zero offset. */ static struct offset off_entire(struct obj_tx_bench *obj_bench, size_t idx) { struct offset offset; offset.off = 0; offset.size = obj_bench->sizes[obj_bench->n_oid(idx)]; return offset; } /* * off_range -- returns offset for range in object. */ static struct offset off_range(struct obj_tx_bench *obj_bench, size_t idx) { struct offset offset; offset.size = obj_bench->sizes[0] / obj_bench->obj_args->n_ops; offset.off = offset.size * idx; return offset; } /* * rand_values -- allocates array and if range mode calculates random * values as allocation sizes for each object otherwise populates whole array * with max value. Used only when range flag set. */ static size_t * rand_values(size_t min, size_t max, size_t n_ops) { size_t size = max - min; auto *sizes = (size_t *)calloc(n_ops, sizeof(size_t)); if (sizes == nullptr) { perror("calloc"); return nullptr; } for (size_t i = 0; i < n_ops; i++) sizes[i] = max; if (min) { if (min > max) { fprintf(stderr, "Invalid size\n"); free(sizes); return nullptr; } for (size_t i = 0; i < n_ops; i++) sizes[i] = (rand() % size) + min; } return sizes; } /* * obj_tx_add_range_op -- main operations of the obj_tx_add_range benchmark. */ static int obj_tx_add_range_op(struct benchmark *bench, struct operation_info *info) { auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); auto *obj_worker = (struct obj_tx_worker *)info->worker->priv; if (add_range_op[obj_bench->lib_op](obj_bench, info->worker, info->index) != 0) return -1; obj_worker->tx_level = 0; return 0; } /* * obj_tx_op -- main operation for obj_tx_alloc(), obj_tx_free() and * obj_tx_realloc() benchmarks. */ static int obj_tx_op(struct benchmark *bench, struct operation_info *info) { auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); auto *obj_worker = (struct obj_tx_worker *)info->worker->priv; int ret = nestings[obj_bench->nesting_mode](obj_bench, info->worker, info->index); obj_worker->tx_level = 0; return ret; } /* * obj_tx_init_worker -- common part for the worker initialization functions * for transactional benchmarks. */ static int obj_tx_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); auto *obj_worker = (struct obj_tx_worker *)calloc(1, sizeof(struct obj_tx_worker)); if (obj_worker == nullptr) { perror("calloc"); return -1; } worker->priv = obj_worker; obj_worker->tx_level = 0; obj_worker->max_level = obj_bench->obj_args->nested; if (obj_bench->lib_mode != LIB_MODE_DRAM) obj_worker->oids = (TOID(struct item) *)calloc( obj_bench->n_objs, sizeof(TOID(struct item))); else obj_worker->items = (char **)calloc(obj_bench->n_objs, sizeof(char *)); if (obj_worker->oids == nullptr && obj_worker->items == nullptr) { free(obj_worker); perror("calloc"); return -1; } return 0; } /* * obj_tx_free_init_worker_alloc_obj -- special part for the worker * initialization function for benchmarks which needs allocated objects * before operation. */ static int obj_tx_init_worker_alloc_obj(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { unsigned i; if (obj_tx_init_worker(bench, args, worker) != 0) return -1; auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); auto *obj_worker = (struct obj_tx_worker *)worker->priv; for (i = 0; i < obj_bench->n_objs; i++) { if (alloc_op[obj_bench->lib_mode](obj_bench, worker, i) != 0) goto out; } return 0; out: for (; i > 0; i--) free_op[obj_bench->lib_mode](obj_bench, worker, i - 1); if (obj_bench->lib_mode == LIB_MODE_DRAM) free(obj_worker->items); else free(obj_worker->oids); free(obj_worker); return -1; } /* * obj_tx_exit_worker -- common part for the worker de-initialization. */ static void obj_tx_exit_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); auto *obj_worker = (struct obj_tx_worker *)worker->priv; for (unsigned i = 0; i < obj_bench->n_objs; i++) free_op[obj_bench->lib_op_free](obj_bench, worker, i); if (obj_bench->lib_mode == LIB_MODE_DRAM) free(obj_worker->items); else free(obj_worker->oids); free(obj_worker); } /* * obj_tx_add_range_init -- specific part of the obj_tx_add_range * benchmark initialization. */ static int obj_tx_add_range_init(struct benchmark *bench, struct benchmark_args *args) { auto *obj_args = (struct obj_tx_args *)args->opts; obj_args->parse_mode = PARSE_OP_MODE_ADD_RANGE; if (args->n_ops_per_thread > MAX_OPS) args->n_ops_per_thread = MAX_OPS; if (obj_tx_init(bench, args) != 0) return -1; auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); obj_bench->n_oid = diff_num; if (obj_bench->op_mode < OP_MODE_ALL_OBJ) { obj_bench->n_oid = one_num; obj_bench->n_objs = 1; } obj_bench->fn_off = off_entire; if (obj_bench->op_mode == OP_MODE_ONE_OBJ_RANGE || obj_bench->op_mode == OP_MODE_ONE_OBJ_NESTED_RANGE) { obj_bench->fn_off = off_range; if (args->n_ops_per_thread > args->dsize) args->dsize = args->n_ops_per_thread; obj_bench->sizes[0] = args->dsize; } obj_bench->lib_op = (obj_bench->op_mode == OP_MODE_ONE_OBJ || obj_bench->op_mode == OP_MODE_ALL_OBJ) ? ADD_RANGE_MODE_ONE_TX : ADD_RANGE_MODE_NESTED_TX; return 0; } /* * obj_tx_free_init -- specific part of the obj_tx_free initialization. */ static int obj_tx_free_init(struct benchmark *bench, struct benchmark_args *args) { if (obj_tx_init(bench, args) != 0) return -1; auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); obj_bench->fn_op = free_op; /* * Generally all objects which were allocated during worker * initialization are released in main operation so there is no need to * free them in exit operation. Only exception is situation where * transaction (inside which object is releasing) is aborted. * Then object is not released so there there is necessary to free it * in exit operation. */ if (!(obj_bench->lib_op == LIB_MODE_OBJ_TX && obj_bench->op_mode != OP_MODE_COMMIT)) obj_bench->lib_op_free = LIB_MODE_NONE; return 0; } /* * obj_tx_alloc_init -- specific part of the obj_tx_alloc initialization. */ static int obj_tx_alloc_init(struct benchmark *bench, struct benchmark_args *args) { if (obj_tx_init(bench, args) != 0) return -1; auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); obj_bench->fn_op = alloc_op; /* * Generally all objects which will be allocated during main operation * need to be released. Only exception is situation where transaction * (inside which object is allocating) is aborted. Then object is not * allocated so there is no need to free it in exit operation. */ if (obj_bench->lib_op == LIB_MODE_OBJ_TX && obj_bench->op_mode != OP_MODE_COMMIT) obj_bench->lib_op_free = LIB_MODE_NONE; return 0; } /* * obj_tx_realloc_init -- specific part of the obj_tx_realloc initialization. */ static int obj_tx_realloc_init(struct benchmark *bench, struct benchmark_args *args) { if (obj_tx_init(bench, args) != 0) return -1; auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); obj_bench->resizes = rand_values(obj_bench->obj_args->min_rsize, obj_bench->obj_args->rsize, args->n_ops_per_thread); if (obj_bench->resizes == nullptr) { obj_tx_exit(bench, args); return -1; } obj_bench->fn_op = realloc_op; return 0; } /* * obj_tx_init -- common part of the benchmark initialization for transactional * benchmarks in their init functions. Parses command line arguments, set * variables and creates persistent pool. */ int obj_tx_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); assert(args->opts != nullptr); char path[PATH_MAX]; if (util_safe_strcpy(path, args->fname, sizeof(path)) != 0) return -1; enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } pmembench_set_priv(bench, &obj_bench); obj_bench.obj_args = (struct obj_tx_args *)args->opts; obj_bench.obj_args->obj_size = args->dsize; obj_bench.obj_args->n_ops = args->n_ops_per_thread; obj_bench.n_objs = args->n_ops_per_thread; obj_bench.lib_op = obj_bench.obj_args->lib != nullptr ? parse_lib_mode(obj_bench.obj_args->lib) : LIB_MODE_OBJ_ATOMIC; if (obj_bench.lib_op == LIB_MODE_NONE) return -1; obj_bench.lib_mode = obj_bench.lib_op == LIB_MODE_DRAM ? LIB_MODE_DRAM : LIB_MODE_OBJ_ATOMIC; obj_bench.lib_op_free = obj_bench.lib_mode; obj_bench.nesting_mode = obj_bench.lib_op == LIB_MODE_OBJ_TX ? NESTING_MODE_TX : NESTING_MODE_SIM; /* * Multiplication by FACTOR prevents from out of memory error * as the actual size of the allocated persistent objects * is always larger than requested. */ size_t dsize = obj_bench.obj_args->rsize > args->dsize ? obj_bench.obj_args->rsize : args->dsize; size_t psize = args->n_ops_per_thread * (dsize + ALLOC_OVERHEAD) * args->n_threads; psize += PMEMOBJ_MIN_POOL; psize = (size_t)(psize * FACTOR); /* * When adding all allocated objects to undo log there is necessary * to prepare larger pool to prevent out of memory error. */ if (obj_bench.op_mode == OP_MODE_ALL_OBJ || obj_bench.op_mode == OP_MODE_ALL_OBJ_NESTED) psize *= 2; obj_bench.op_mode = parse_op[obj_bench.obj_args->parse_mode]( obj_bench.obj_args->operation); if (obj_bench.op_mode == OP_MODE_UNKNOWN) { fprintf(stderr, "operation mode unknown\n"); return -1; } obj_bench.type_mode = parse_type_num_mode(obj_bench.obj_args->type_num); if (obj_bench.type_mode == NUM_MODE_UNKNOWN) return -1; obj_bench.fn_type_num = type_num_fn[obj_bench.type_mode]; if (obj_bench.type_mode == NUM_MODE_RAND) { obj_bench.random_types = rand_values(1, UINT32_MAX, args->n_ops_per_thread); if (obj_bench.random_types == nullptr) return -1; } obj_bench.sizes = rand_values(obj_bench.obj_args->min_size, obj_bench.obj_args->obj_size, args->n_ops_per_thread); if (obj_bench.sizes == nullptr) goto free_random_types; if (obj_bench.lib_mode == LIB_MODE_DRAM) return 0; /* Create pmemobj pool. */ if (args->is_poolset || type == TYPE_DEVDAX) { if (args->fsize < psize) { fprintf(stderr, "file size too large\n"); goto free_all; } psize = 0; } else if (args->is_dynamic_poolset) { int ret = dynamic_poolset_create(args->fname, psize); if (ret == -1) goto free_all; if (util_safe_strcpy(path, POOLSET_PATH, sizeof(path)) != 0) goto free_all; psize = 0; } obj_bench.pop = pmemobj_create(path, LAYOUT_NAME, psize, args->fmode); if (obj_bench.pop == nullptr) { perror("pmemobj_create"); goto free_all; } return 0; free_all: free(obj_bench.sizes); free_random_types: if (obj_bench.type_mode == NUM_MODE_RAND) free(obj_bench.random_types); return -1; } /* * obj_tx_exit -- common part for the exit function of the transactional * benchmarks in their exit functions. */ int obj_tx_exit(struct benchmark *bench, struct benchmark_args *args) { auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); if (obj_bench->lib_mode != LIB_MODE_DRAM) pmemobj_close(obj_bench->pop); free(obj_bench->sizes); if (obj_bench->type_mode == NUM_MODE_RAND) free(obj_bench->random_types); return 0; } /* * obj_tx_realloc_exit -- common part for the exit function of the transactional * benchmarks in their exit functions. */ static int obj_tx_realloc_exit(struct benchmark *bench, struct benchmark_args *args) { auto *obj_bench = (struct obj_tx_bench *)pmembench_get_priv(bench); free(obj_bench->resizes); return obj_tx_exit(bench, args); } /* Array defining common command line arguments. */ static struct benchmark_clo obj_tx_clo[8]; static struct benchmark_info obj_tx_alloc; static struct benchmark_info obj_tx_free; static struct benchmark_info obj_tx_realloc; static struct benchmark_info obj_tx_add_range; CONSTRUCTOR(pmemobj_tx_constructor) void pmemobj_tx_constructor(void) { obj_tx_clo[0].opt_short = 'T'; obj_tx_clo[0].opt_long = "type-number"; obj_tx_clo[0].descr = "Type number - one, rand, per-thread"; obj_tx_clo[0].def = "one"; obj_tx_clo[0].type = CLO_TYPE_STR; obj_tx_clo[0].off = clo_field_offset(struct obj_tx_args, type_num); obj_tx_clo[1].opt_short = 'O'; obj_tx_clo[1].opt_long = "operation"; obj_tx_clo[1].descr = "Type of operation"; obj_tx_clo[1].def = "basic"; obj_tx_clo[1].off = clo_field_offset(struct obj_tx_args, operation); obj_tx_clo[1].type = CLO_TYPE_STR; obj_tx_clo[2].opt_short = 'm'; obj_tx_clo[2].opt_long = "min-size"; obj_tx_clo[2].type = CLO_TYPE_UINT; obj_tx_clo[2].descr = "Minimum allocation size"; obj_tx_clo[2].off = clo_field_offset(struct obj_tx_args, min_size); obj_tx_clo[2].def = "0"; obj_tx_clo[2].type_uint.size = clo_field_size(struct obj_tx_args, min_size); obj_tx_clo[2].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; obj_tx_clo[2].type_uint.min = 0; obj_tx_clo[2].type_uint.max = UINT_MAX; /* * nclos field in benchmark_info structures is decremented to make this * options available only for obj_tx_alloc, obj_tx_free and * obj_tx_realloc benchmarks. */ obj_tx_clo[3].opt_short = 'L'; obj_tx_clo[3].opt_long = "lib"; obj_tx_clo[3].descr = "Type of library"; obj_tx_clo[3].def = "tx"; obj_tx_clo[3].off = clo_field_offset(struct obj_tx_args, lib); obj_tx_clo[3].type = CLO_TYPE_STR; obj_tx_clo[4].opt_short = 'N'; obj_tx_clo[4].opt_long = "nestings"; obj_tx_clo[4].type = CLO_TYPE_UINT; obj_tx_clo[4].descr = "Number of nested transactions"; obj_tx_clo[4].off = clo_field_offset(struct obj_tx_args, nested); obj_tx_clo[4].def = "0"; obj_tx_clo[4].type_uint.size = clo_field_size(struct obj_tx_args, nested); obj_tx_clo[4].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; obj_tx_clo[4].type_uint.min = 0; obj_tx_clo[4].type_uint.max = MAX_OPS; obj_tx_clo[5].opt_short = 'r'; obj_tx_clo[5].opt_long = "min-rsize"; obj_tx_clo[5].type = CLO_TYPE_UINT; obj_tx_clo[5].descr = "Minimum reallocation size"; obj_tx_clo[5].off = clo_field_offset(struct obj_tx_args, min_rsize); obj_tx_clo[5].def = "0"; obj_tx_clo[5].type_uint.size = clo_field_size(struct obj_tx_args, min_rsize); obj_tx_clo[5].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; obj_tx_clo[5].type_uint.min = 0; obj_tx_clo[5].type_uint.max = UINT_MAX; obj_tx_clo[6].opt_short = 'R'; obj_tx_clo[6].opt_long = "realloc-size"; obj_tx_clo[6].type = CLO_TYPE_UINT; obj_tx_clo[6].descr = "Reallocation size"; obj_tx_clo[6].off = clo_field_offset(struct obj_tx_args, rsize); obj_tx_clo[6].def = "1"; obj_tx_clo[6].type_uint.size = clo_field_size(struct obj_tx_args, rsize); obj_tx_clo[6].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; obj_tx_clo[6].type_uint.min = 1; obj_tx_clo[6].type_uint.max = ULONG_MAX; obj_tx_clo[7].opt_short = 'c'; obj_tx_clo[7].opt_long = "changed-type"; obj_tx_clo[7].descr = "Use another type number in " "reallocation than in allocation"; obj_tx_clo[7].type = CLO_TYPE_FLAG; obj_tx_clo[7].off = clo_field_offset(struct obj_tx_args, change_type); obj_tx_alloc.name = "obj_tx_alloc"; obj_tx_alloc.brief = "pmemobj_tx_alloc() benchmark"; obj_tx_alloc.init = obj_tx_alloc_init; obj_tx_alloc.exit = obj_tx_exit; obj_tx_alloc.multithread = true; obj_tx_alloc.multiops = true; obj_tx_alloc.init_worker = obj_tx_init_worker; obj_tx_alloc.free_worker = obj_tx_exit_worker; obj_tx_alloc.operation = obj_tx_op; obj_tx_alloc.measure_time = true; obj_tx_alloc.clos = obj_tx_clo; obj_tx_alloc.nclos = ARRAY_SIZE(obj_tx_clo) - 3; obj_tx_alloc.opts_size = sizeof(struct obj_tx_args); obj_tx_alloc.rm_file = true; obj_tx_alloc.allow_poolset = true; REGISTER_BENCHMARK(obj_tx_alloc); obj_tx_free.name = "obj_tx_free"; obj_tx_free.brief = "pmemobj_tx_free() benchmark"; obj_tx_free.init = obj_tx_free_init; obj_tx_free.exit = obj_tx_exit; obj_tx_free.multithread = true; obj_tx_free.multiops = true; obj_tx_free.init_worker = obj_tx_init_worker_alloc_obj; obj_tx_free.free_worker = obj_tx_exit_worker; obj_tx_free.operation = obj_tx_op; obj_tx_free.measure_time = true; obj_tx_free.clos = obj_tx_clo; obj_tx_free.nclos = ARRAY_SIZE(obj_tx_clo) - 3; obj_tx_free.opts_size = sizeof(struct obj_tx_args); obj_tx_free.rm_file = true; obj_tx_free.allow_poolset = true; REGISTER_BENCHMARK(obj_tx_free); obj_tx_realloc.name = "obj_tx_realloc"; obj_tx_realloc.brief = "pmemobj_tx_realloc() benchmark"; obj_tx_realloc.init = obj_tx_realloc_init; obj_tx_realloc.exit = obj_tx_realloc_exit; obj_tx_realloc.multithread = true; obj_tx_realloc.multiops = true; obj_tx_realloc.init_worker = obj_tx_init_worker_alloc_obj; obj_tx_realloc.free_worker = obj_tx_exit_worker; obj_tx_realloc.operation = obj_tx_op; obj_tx_realloc.measure_time = true; obj_tx_realloc.clos = obj_tx_clo; obj_tx_realloc.nclos = ARRAY_SIZE(obj_tx_clo); obj_tx_realloc.opts_size = sizeof(struct obj_tx_args); obj_tx_realloc.rm_file = true; obj_tx_realloc.allow_poolset = true; REGISTER_BENCHMARK(obj_tx_realloc); obj_tx_add_range.name = "obj_tx_add_range"; obj_tx_add_range.brief = "pmemobj_tx_add_range() benchmark"; obj_tx_add_range.init = obj_tx_add_range_init; obj_tx_add_range.exit = obj_tx_exit; obj_tx_add_range.multithread = true; obj_tx_add_range.multiops = false; obj_tx_add_range.init_worker = obj_tx_init_worker_alloc_obj; obj_tx_add_range.free_worker = obj_tx_exit_worker; obj_tx_add_range.operation = obj_tx_add_range_op; obj_tx_add_range.measure_time = true; obj_tx_add_range.clos = obj_tx_clo; obj_tx_add_range.nclos = ARRAY_SIZE(obj_tx_clo) - 5; obj_tx_add_range.opts_size = sizeof(struct obj_tx_args); obj_tx_add_range.rm_file = true; obj_tx_add_range.allow_poolset = true; REGISTER_BENCHMARK(obj_tx_add_range); } pmdk-1.8/src/benchmarks/pmembench_blk.cfg0000664000000000000000000000346113615011243017170 0ustar rootroot# Global parameters [global] group = pmemblk file = testfile.blk ops-per-thread=1000 # blk_read benchmark using blk with variable number of threads # from 1 to 32 [blk_blk_read_threads] bench = blk_read mode = rand operation = blk file-size = 536870912 threads = 1:+1:32 data-size = 512 # blk_read benchmark without using blk with variable number of threads # from 1 to 32 [blk_non_blk_read_threads] bench = blk_read mode = rand operation = file file-size = 536870912 threads = 1:+1:32 data-size = 512 # blk_read benchmark using blk with variable data size (block size) # from 512 to 1048576 bytes [blk_blk_read_data_size] bench = blk_read mode = rand operation = blk threads = 1 data-size = 512:*2:524288 file-size = 536870912 # blk_read benchmark without using blk with variable data size # from 512 to 1048576 bytes [blk_non_blk_read_data_size] bench = blk_read mode = rand operation = file threads = 1 data-size = 512:*2:524288 file-size = 536870912 # blk_write benchmark using blk with variable number of threads # from 1 to 32 [blk_blk_write_threads] bench = blk_write mode = rand operation = blk file-size = 536870912 threads = 1:+1:32 data-size = 512 # blk_write benchmark without using blk with variable number of threads # from 1 to 32 [blk_non_blk_write_threads] bench = blk_write mode = rand operation = file file-size = 536870912 threads = 1:+1:32 data-size = 512 # blk_write benchmark using blk with variable data size (block size) # from 512 to 1048576 bytes [blk_blk_write_data_size] bench = blk_write mode = seq operation = blk threads = 1 data-size = 512:*2:524288 file-size = 536870912 # blk_write benchmark without using blk with variable data size # from 512 to 1048576 bytes [blk_non_blk_write_data_size] bench = blk_write mode = seq operation = file threads = 1 data-size = 512:*2:524288 file-size = 536870912 pmdk-1.8/src/benchmarks/clo_vec.cpp0000664000000000000000000001472713615011243016046 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * clo_vec.cpp -- command line options vector definitions */ #include #include #include #include "clo_vec.hpp" /* * clo_vec_alloc -- allocate new CLO vector */ struct clo_vec * clo_vec_alloc(size_t size) { struct clo_vec *clovec = (struct clo_vec *)malloc(sizeof(*clovec)); assert(clovec != nullptr); /* init list of arguments and allocations */ PMDK_TAILQ_INIT(&clovec->allocs); PMDK_TAILQ_INIT(&clovec->args); clovec->nallocs = 0; /* size of each struct */ clovec->size = size; /* add first struct to list */ struct clo_vec_args *args = (struct clo_vec_args *)malloc(sizeof(*args)); assert(args != nullptr); args->args = calloc(1, size); assert(args->args != nullptr); PMDK_TAILQ_INSERT_TAIL(&clovec->args, args, next); clovec->nargs = 1; return clovec; } /* * clo_vec_free -- free CLO vector and all allocations */ void clo_vec_free(struct clo_vec *clovec) { assert(clovec != nullptr); /* free all allocations */ while (!PMDK_TAILQ_EMPTY(&clovec->allocs)) { struct clo_vec_alloc *alloc = PMDK_TAILQ_FIRST(&clovec->allocs); PMDK_TAILQ_REMOVE(&clovec->allocs, alloc, next); free(alloc->ptr); free(alloc); } /* free all arguments */ while (!PMDK_TAILQ_EMPTY(&clovec->args)) { struct clo_vec_args *args = PMDK_TAILQ_FIRST(&clovec->args); PMDK_TAILQ_REMOVE(&clovec->args, args, next); free(args->args); free(args); } free(clovec); } /* * clo_vec_get_args -- return pointer to CLO arguments at specified index */ void * clo_vec_get_args(struct clo_vec *clovec, size_t i) { if (i >= clovec->nargs) return nullptr; size_t c = 0; struct clo_vec_args *args; PMDK_TAILQ_FOREACH(args, &clovec->args, next) { if (c == i) return args->args; c++; } return nullptr; } /* * clo_vec_add_alloc -- add allocation to CLO vector */ int clo_vec_add_alloc(struct clo_vec *clovec, void *ptr) { struct clo_vec_alloc *alloc = (struct clo_vec_alloc *)malloc(sizeof(*alloc)); assert(alloc != nullptr); alloc->ptr = ptr; PMDK_TAILQ_INSERT_TAIL(&clovec->allocs, alloc, next); clovec->nallocs++; return 0; } /* * clo_vec_grow -- (internal) grow in size the CLO vector */ static void clo_vec_grow(struct clo_vec *clovec, size_t new_len) { size_t nargs = new_len - clovec->nargs; size_t i; for (i = 0; i < nargs; i++) { struct clo_vec_args *args = (struct clo_vec_args *)calloc(1, sizeof(*args)); assert(args != nullptr); PMDK_TAILQ_INSERT_TAIL(&clovec->args, args, next); args->args = malloc(clovec->size); assert(args->args != nullptr); void *argscpy = clo_vec_get_args(clovec, i % clovec->nargs); assert(argscpy != nullptr); memcpy(args->args, argscpy, clovec->size); } clovec->nargs = new_len; } /* * clo_vec_vlist_alloc -- allocate list of values */ struct clo_vec_vlist * clo_vec_vlist_alloc(void) { struct clo_vec_vlist *list = (struct clo_vec_vlist *)malloc(sizeof(*list)); assert(list != nullptr); list->nvalues = 0; PMDK_TAILQ_INIT(&list->head); return list; } /* * clo_vec_vlist_free -- release list of values */ void clo_vec_vlist_free(struct clo_vec_vlist *list) { assert(list != nullptr); while (!PMDK_TAILQ_EMPTY(&list->head)) { struct clo_vec_value *val = PMDK_TAILQ_FIRST(&list->head); PMDK_TAILQ_REMOVE(&list->head, val, next); free(val->ptr); free(val); } free(list); } /* * clo_vec_vlist_add -- add value to list */ void clo_vec_vlist_add(struct clo_vec_vlist *list, void *ptr, size_t size) { struct clo_vec_value *val = (struct clo_vec_value *)malloc(sizeof(*val)); assert(val != nullptr); val->ptr = malloc(size); assert(val->ptr != nullptr); memcpy(val->ptr, ptr, size); PMDK_TAILQ_INSERT_TAIL(&list->head, val, next); list->nvalues++; } /* * clo_vec_memcpy -- copy value to CLO vector * * - clovec - CLO vector * - off - offset to value in structure * - size - size of value field * - ptr - pointer to value */ int clo_vec_memcpy(struct clo_vec *clovec, size_t off, size_t size, void *ptr) { if (off + size > clovec->size) return -1; size_t i; for (i = 0; i < clovec->nargs; i++) { auto *args = (char *)clo_vec_get_args(clovec, i); char *dptr = args + off; memcpy(dptr, ptr, size); } return 0; } /* * clo_vec_memcpy_list -- copy values from list to CLO vector * * - clovec - CLO vector * - off - offset to value in structure * - size - size of value field * - list - list of values */ int clo_vec_memcpy_list(struct clo_vec *clovec, size_t off, size_t size, struct clo_vec_vlist *list) { if (off + size > clovec->size) return -1; size_t len = clovec->nargs; if (list->nvalues > 1) clo_vec_grow(clovec, clovec->nargs * list->nvalues); struct clo_vec_value *value; size_t value_i = 0; size_t i; PMDK_TAILQ_FOREACH(value, &list->head, next) { for (i = value_i * len; i < (value_i + 1) * len; i++) { auto *args = (char *)clo_vec_get_args(clovec, i); char *dptr = args + off; memcpy(dptr, value->ptr, size); } value_i++; } return 0; } pmdk-1.8/src/benchmarks/blk.cpp0000664000000000000000000004164313615011243015201 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * blk.cpp -- pmemblk benchmarks definitions */ #include "benchmark.hpp" #include "file.h" #include "libpmem.h" #include "libpmemblk.h" #include "libpmempool.h" #include "os.h" #include "poolset_util.hpp" #include #include #include #include #include #include #include #include struct blk_bench; struct blk_worker; /* * op_type -- type of operation */ enum op_type { OP_TYPE_UNKNOWN, OP_TYPE_BLK, OP_TYPE_FILE, OP_TYPE_MEMCPY, }; /* * op_mode -- mode of the copy process */ enum op_mode { OP_MODE_UNKNOWN, OP_MODE_STAT, /* read/write always the same chunk */ OP_MODE_SEQ, /* read/write chunk by chunk */ OP_MODE_RAND /* read/write to chunks selected randomly */ }; /* * typedef for the worker function */ typedef int (*worker_fn)(struct blk_bench *, struct benchmark_args *, struct blk_worker *, os_off_t); /* * blk_args -- benchmark specific arguments */ struct blk_args { size_t fsize; /* requested file size */ bool no_warmup; /* don't do warmup */ unsigned seed; /* seed for randomization */ char *type_str; /* type: blk, file, memcpy */ char *mode_str; /* mode: stat, seq, rand */ }; /* * blk_bench -- pmemblk benchmark context */ struct blk_bench { PMEMblkpool *pbp; /* pmemblk handle */ char *addr; /* address of user data (memcpy) */ int fd; /* file descr. for file io */ size_t nblocks; /* actual number of blocks */ size_t blocks_per_thread; /* number of blocks per thread */ worker_fn worker; /* worker function */ enum op_type type; enum op_mode mode; }; /* * struct blk_worker -- pmemblk worker context */ struct blk_worker { os_off_t *blocks; /* array with block numbers */ char *buff; /* buffer for read/write */ unsigned seed; /* worker seed */ }; /* * parse_op_type -- parse command line "--operation" argument * * Returns proper operation type. */ static enum op_type parse_op_type(const char *arg) { if (strcmp(arg, "blk") == 0) return OP_TYPE_BLK; else if (strcmp(arg, "file") == 0) return OP_TYPE_FILE; else if (strcmp(arg, "memcpy") == 0) return OP_TYPE_MEMCPY; else return OP_TYPE_UNKNOWN; } /* * parse_op_mode -- parse command line "--mode" argument * * Returns proper operation mode. */ static enum op_mode parse_op_mode(const char *arg) { if (strcmp(arg, "stat") == 0) return OP_MODE_STAT; else if (strcmp(arg, "seq") == 0) return OP_MODE_SEQ; else if (strcmp(arg, "rand") == 0) return OP_MODE_RAND; else return OP_MODE_UNKNOWN; } /* * blk_do_warmup -- perform warm-up by writing to each block */ static int blk_do_warmup(struct blk_bench *bb, struct benchmark_args *args) { size_t lba; int ret = 0; auto *buff = (char *)calloc(1, args->dsize); if (!buff) { perror("calloc"); return -1; } for (lba = 0; lba < bb->nblocks; ++lba) { switch (bb->type) { case OP_TYPE_FILE: { size_t off = lba * args->dsize; if (pwrite(bb->fd, buff, args->dsize, off) != (ssize_t)args->dsize) { perror("pwrite"); ret = -1; goto out; } } break; case OP_TYPE_BLK: if (pmemblk_write(bb->pbp, buff, lba) < 0) { perror("pmemblk_write"); ret = -1; goto out; } break; case OP_TYPE_MEMCPY: { size_t off = lba * args->dsize; pmem_memcpy_persist((char *)bb->addr + off, buff, args->dsize); } break; default: perror("unknown type"); ret = -1; goto out; } } out: free(buff); return ret; } /* * blk_read -- read function for pmemblk */ static int blk_read(struct blk_bench *bb, struct benchmark_args *ba, struct blk_worker *bworker, os_off_t off) { if (pmemblk_read(bb->pbp, bworker->buff, off) < 0) { perror("pmemblk_read"); return -1; } return 0; } /* * fileio_read -- read function for file io */ static int fileio_read(struct blk_bench *bb, struct benchmark_args *ba, struct blk_worker *bworker, os_off_t off) { os_off_t file_off = off * ba->dsize; if (pread(bb->fd, bworker->buff, ba->dsize, file_off) != (ssize_t)ba->dsize) { perror("pread"); return -1; } return 0; } /* * memcpy_read -- read function for memcpy */ static int memcpy_read(struct blk_bench *bb, struct benchmark_args *ba, struct blk_worker *bworker, os_off_t off) { os_off_t file_off = off * ba->dsize; memcpy(bworker->buff, (char *)bb->addr + file_off, ba->dsize); return 0; } /* * blk_write -- write function for pmemblk */ static int blk_write(struct blk_bench *bb, struct benchmark_args *ba, struct blk_worker *bworker, os_off_t off) { if (pmemblk_write(bb->pbp, bworker->buff, off) < 0) { perror("pmemblk_write"); return -1; } return 0; } /* * memcpy_write -- write function for memcpy */ static int memcpy_write(struct blk_bench *bb, struct benchmark_args *ba, struct blk_worker *bworker, os_off_t off) { os_off_t file_off = off * ba->dsize; pmem_memcpy_persist((char *)bb->addr + file_off, bworker->buff, ba->dsize); return 0; } /* * fileio_write -- write function for file io */ static int fileio_write(struct blk_bench *bb, struct benchmark_args *ba, struct blk_worker *bworker, os_off_t off) { os_off_t file_off = off * ba->dsize; if (pwrite(bb->fd, bworker->buff, ba->dsize, file_off) != (ssize_t)ba->dsize) { perror("pwrite"); return -1; } return 0; } /* * blk_operation -- main operations for blk_read and blk_write benchmark */ static int blk_operation(struct benchmark *bench, struct operation_info *info) { auto *bb = (struct blk_bench *)pmembench_get_priv(bench); auto *bworker = (struct blk_worker *)info->worker->priv; os_off_t off = bworker->blocks[info->index]; return bb->worker(bb, info->args, bworker, off); } /* * blk_init_worker -- initialize worker */ static int blk_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { struct blk_worker *bworker = (struct blk_worker *)malloc(sizeof(*bworker)); if (!bworker) { perror("malloc"); return -1; } auto *bb = (struct blk_bench *)pmembench_get_priv(bench); auto *bargs = (struct blk_args *)args->opts; bworker->seed = os_rand_r(&bargs->seed); bworker->buff = (char *)malloc(args->dsize); if (!bworker->buff) { perror("malloc"); goto err_buff; } /* fill buffer with some random data */ memset(bworker->buff, bworker->seed, args->dsize); assert(args->n_ops_per_thread != 0); bworker->blocks = (os_off_t *)malloc(sizeof(*bworker->blocks) * args->n_ops_per_thread); if (!bworker->blocks) { perror("malloc"); goto err_blocks; } switch (bb->mode) { case OP_MODE_RAND: for (size_t i = 0; i < args->n_ops_per_thread; i++) { bworker->blocks[i] = worker->index * bb->blocks_per_thread + os_rand_r(&bworker->seed) % bb->blocks_per_thread; } break; case OP_MODE_SEQ: for (size_t i = 0; i < args->n_ops_per_thread; i++) bworker->blocks[i] = i % bb->blocks_per_thread; break; case OP_MODE_STAT: for (size_t i = 0; i < args->n_ops_per_thread; i++) bworker->blocks[i] = 0; break; default: perror("unknown mode"); goto err_mode; } worker->priv = bworker; return 0; err_mode: free(bworker->blocks); err_blocks: free(bworker->buff); err_buff: free(bworker); return -1; } /* * blk_free_worker -- cleanup worker */ static void blk_free_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *bworker = (struct blk_worker *)worker->priv; free(bworker->blocks); free(bworker->buff); free(bworker); } /* * blk_init -- function for initialization benchmark */ static int blk_init(struct blk_bench *bb, struct benchmark_args *args) { auto *ba = (struct blk_args *)args->opts; assert(ba != nullptr); char path[PATH_MAX]; if (util_safe_strcpy(path, args->fname, sizeof(path)) != 0) return -1; bb->type = parse_op_type(ba->type_str); if (bb->type == OP_TYPE_UNKNOWN) { fprintf(stderr, "Invalid operation argument '%s'", ba->type_str); return -1; } enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } if (bb->type == OP_TYPE_FILE && type == TYPE_DEVDAX) { fprintf(stderr, "fileio not supported on device dax\n"); return -1; } bb->mode = parse_op_mode(ba->mode_str); if (bb->mode == OP_MODE_UNKNOWN) { fprintf(stderr, "Invalid mode argument '%s'", ba->mode_str); return -1; } if (ba->fsize == 0) ba->fsize = PMEMBLK_MIN_POOL; size_t req_fsize = ba->fsize; if (ba->fsize / args->dsize < args->n_threads || ba->fsize < PMEMBLK_MIN_POOL) { fprintf(stderr, "too small file size\n"); return -1; } if (args->dsize >= ba->fsize) { fprintf(stderr, "block size bigger than file size\n"); return -1; } if (args->is_poolset || type == TYPE_DEVDAX) { if (args->fsize < ba->fsize) { fprintf(stderr, "file size too large\n"); return -1; } ba->fsize = 0; } else if (args->is_dynamic_poolset) { int ret = dynamic_poolset_create(args->fname, ba->fsize); if (ret == -1) return -1; if (util_safe_strcpy(path, POOLSET_PATH, sizeof(path)) != 0) return -1; ba->fsize = 0; } bb->fd = -1; /* * Create pmemblk in order to get the number of blocks * even for file-io mode. */ bb->pbp = pmemblk_create(path, args->dsize, ba->fsize, args->fmode); if (bb->pbp == nullptr) { perror("pmemblk_create"); return -1; } bb->nblocks = pmemblk_nblock(bb->pbp); /* limit the number of used blocks */ if (bb->nblocks > req_fsize / args->dsize) bb->nblocks = req_fsize / args->dsize; if (bb->nblocks < args->n_threads) { fprintf(stderr, "too small file size"); goto out_close; } if (bb->type == OP_TYPE_FILE) { pmemblk_close(bb->pbp); bb->pbp = nullptr; int flags = O_RDWR | O_CREAT | O_SYNC; #ifdef _WIN32 flags |= O_BINARY; #endif bb->fd = os_open(args->fname, flags, args->fmode); if (bb->fd < 0) { perror("open"); return -1; } } else if (bb->type == OP_TYPE_MEMCPY) { /* skip pool header, so addr points to the first block */ bb->addr = (char *)bb->pbp + 8192; } bb->blocks_per_thread = bb->nblocks / args->n_threads; if (!ba->no_warmup) { if (blk_do_warmup(bb, args) != 0) goto out_close; } return 0; out_close: if (bb->type == OP_TYPE_FILE) os_close(bb->fd); else pmemblk_close(bb->pbp); return -1; } /* * blk_read_init - function for initializing blk_read benchmark */ static int blk_read_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); int ret; auto *bb = (struct blk_bench *)malloc(sizeof(struct blk_bench)); if (bb == nullptr) { perror("malloc"); return -1; } pmembench_set_priv(bench, bb); ret = blk_init(bb, args); if (ret != 0) { free(bb); return ret; } switch (bb->type) { case OP_TYPE_FILE: bb->worker = fileio_read; break; case OP_TYPE_BLK: bb->worker = blk_read; break; case OP_TYPE_MEMCPY: bb->worker = memcpy_read; break; default: perror("unknown operation type"); return -1; } return ret; } /* * blk_write_init - function for initializing blk_write benchmark */ static int blk_write_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); int ret; auto *bb = (struct blk_bench *)malloc(sizeof(struct blk_bench)); if (bb == nullptr) { perror("malloc"); return -1; } pmembench_set_priv(bench, bb); ret = blk_init(bb, args); if (ret != 0) { free(bb); return ret; } switch (bb->type) { case OP_TYPE_FILE: bb->worker = fileio_write; break; case OP_TYPE_BLK: bb->worker = blk_write; break; case OP_TYPE_MEMCPY: bb->worker = memcpy_write; break; default: perror("unknown operation type"); return -1; } return ret; } /* * blk_exit -- function for de-initialization benchmark */ static int blk_exit(struct benchmark *bench, struct benchmark_args *args) { auto *bb = (struct blk_bench *)pmembench_get_priv(bench); char path[PATH_MAX]; if (util_safe_strcpy(path, args->fname, sizeof(path)) != 0) return -1; if (args->is_dynamic_poolset) { if (util_safe_strcpy(path, POOLSET_PATH, sizeof(path)) != 0) return -1; } int result; switch (bb->type) { case OP_TYPE_FILE: os_close(bb->fd); break; case OP_TYPE_BLK: pmemblk_close(bb->pbp); result = pmemblk_check(path, args->dsize); if (result < 0) { perror("pmemblk_check error"); return -1; } else if (result == 0) { perror("pmemblk_check: not consistent"); return -1; } break; case OP_TYPE_MEMCPY: pmemblk_close(bb->pbp); break; default: perror("unknown operation type"); return -1; } free(bb); return 0; } static struct benchmark_clo blk_clo[5]; static struct benchmark_info blk_read_info; static struct benchmark_info blk_write_info; CONSTRUCTOR(blk_constructor) void blk_constructor(void) { blk_clo[0].opt_short = 'o'; blk_clo[0].opt_long = "operation"; blk_clo[0].descr = "Operation type - blk, file, memcpy"; blk_clo[0].type = CLO_TYPE_STR; blk_clo[0].off = clo_field_offset(struct blk_args, type_str); blk_clo[0].def = "blk"; blk_clo[1].opt_short = 'w'; blk_clo[1].opt_long = "no-warmup"; blk_clo[1].descr = "Don't do warmup"; blk_clo[1].type = CLO_TYPE_FLAG; blk_clo[1].off = clo_field_offset(struct blk_args, no_warmup); blk_clo[2].opt_short = 'm'; blk_clo[2].opt_long = "mode"; blk_clo[2].descr = "Reading/writing mode - stat, seq, rand"; blk_clo[2].type = CLO_TYPE_STR; blk_clo[2].off = clo_field_offset(struct blk_args, mode_str); blk_clo[2].def = "seq"; blk_clo[3].opt_short = 'S'; blk_clo[3].opt_long = "seed"; blk_clo[3].descr = "Random seed"; blk_clo[3].off = clo_field_offset(struct blk_args, seed); blk_clo[3].def = "1"; blk_clo[3].type = CLO_TYPE_UINT; blk_clo[3].type_uint.size = clo_field_size(struct blk_args, seed); blk_clo[3].type_uint.base = CLO_INT_BASE_DEC; blk_clo[3].type_uint.min = 1; blk_clo[3].type_uint.max = UINT_MAX; blk_clo[4].opt_short = 's'; blk_clo[4].opt_long = "file-size"; blk_clo[4].descr = "Requested file size in bytes - 0 means minimum"; blk_clo[4].type = CLO_TYPE_UINT; blk_clo[4].off = clo_field_offset(struct blk_args, fsize); blk_clo[4].def = "0"; blk_clo[4].type_uint.size = clo_field_size(struct blk_args, fsize); blk_clo[4].type_uint.base = CLO_INT_BASE_DEC; blk_clo[4].type_uint.min = 0; blk_clo[4].type_uint.max = ~0; blk_read_info.name = "blk_read"; blk_read_info.brief = "Benchmark for blk_read() operation"; blk_read_info.init = blk_read_init; blk_read_info.exit = blk_exit; blk_read_info.multithread = true; blk_read_info.multiops = true; blk_read_info.init_worker = blk_init_worker; blk_read_info.free_worker = blk_free_worker; blk_read_info.operation = blk_operation; blk_read_info.clos = blk_clo; blk_read_info.nclos = ARRAY_SIZE(blk_clo); blk_read_info.opts_size = sizeof(struct blk_args); blk_read_info.rm_file = true; blk_read_info.allow_poolset = true; REGISTER_BENCHMARK(blk_read_info); blk_write_info.name = "blk_write"; blk_write_info.brief = "Benchmark for blk_write() operation"; blk_write_info.init = blk_write_init; blk_write_info.exit = blk_exit; blk_write_info.multithread = true; blk_write_info.multiops = true; blk_write_info.init_worker = blk_init_worker; blk_write_info.free_worker = blk_free_worker; blk_write_info.operation = blk_operation; blk_write_info.clos = blk_clo; blk_write_info.nclos = ARRAY_SIZE(blk_clo); blk_write_info.opts_size = sizeof(struct blk_args); blk_write_info.rm_file = true; blk_write_info.allow_poolset = true; REGISTER_BENCHMARK(blk_write_info); } pmdk-1.8/src/benchmarks/clo.cpp0000664000000000000000000006013113615011243015177 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * clo.cpp -- command line options module definitions */ #include #include #include #include #include #include #include "benchmark.hpp" #include "clo.hpp" #include "clo_vec.hpp" #include "queue.h" #include "scenario.hpp" #ifndef min #define min(a, b) ((a) < (b) ? (a) : (b)) #endif #ifndef max #define max(a, b) ((a) > (b) ? (a) : (b)) #endif typedef int (*clo_parse_fn)(struct benchmark_clo *clo, const char *arg, struct clo_vec *clovec); typedef int (*clo_parse_single_fn)(struct benchmark_clo *clo, const char *arg, void *ptr); typedef int (*clo_eval_range_fn)(struct benchmark_clo *clo, void *first, void *step, void *last, char type, struct clo_vec_vlist *vlist); typedef const char *(*clo_str_fn)(struct benchmark_clo *clo, void *addr, size_t size); #define STR_BUFF_SIZE 1024 static char str_buff[STR_BUFF_SIZE]; /* * clo_parse_flag -- (internal) parse flag */ static int clo_parse_flag(struct benchmark_clo *clo, const char *arg, struct clo_vec *clovec) { bool flag = true; if (arg != nullptr) { if (strcmp(arg, "true") == 0) flag = true; else if (strcmp(arg, "false") == 0) flag = false; else return -1; } return clo_vec_memcpy(clovec, clo->off, sizeof(flag), &flag); } /* * clo_parse_str -- (internal) parse string value */ static int clo_parse_str(struct benchmark_clo *clo, const char *arg, struct clo_vec *clovec) { struct clo_vec_vlist *vlist = clo_vec_vlist_alloc(); assert(vlist != nullptr); char *str = strdup(arg); assert(str != nullptr); clo_vec_add_alloc(clovec, str); char *next = strtok(str, ","); while (next) { clo_vec_vlist_add(vlist, &next, sizeof(next)); next = strtok(nullptr, ","); } int ret = clo_vec_memcpy_list(clovec, clo->off, sizeof(str), vlist); clo_vec_vlist_free(vlist); return ret; } /* * is_oct -- check if string may be octal number */ static int is_oct(const char *arg, size_t len) { return (arg[0] == '0' || (len > 1 && arg[0] == '-' && arg[1] == '0')); } /* * is_hex -- check if string may be hexadecimal number */ static int is_hex(const char *arg, size_t len) { if (arg[0] == '-') { arg++; len--; } return (len > 2 && arg[0] == '0' && (arg[1] == 'x' || arg[1] == 'X')); } /* * parse_number_base -- parse string as integer of given sign and base */ static int parse_number_base(const char *arg, void *value, int s, int base) { char *end; errno = 0; if (s) { auto *v = (int64_t *)value; *v = strtoll(arg, &end, base); } else { auto *v = (uint64_t *)value; *v = strtoull(arg, &end, base); } if (errno || *end != '\0') return -1; return 0; } /* * parse_number -- parse string as integer of given sign and allowed bases */ static int parse_number(const char *arg, size_t len, void *value, int s, int base) { if ((base & CLO_INT_BASE_HEX) && is_hex(arg, len)) { if (!parse_number_base(arg, value, s, 16)) return 0; } if ((base & CLO_INT_BASE_OCT) && is_oct(arg, len)) { if (!parse_number_base(arg, value, s, 8)) return 0; } if (base & CLO_INT_BASE_DEC) { if (!parse_number_base(arg, value, s, 10)) return 0; } return -1; } /* * clo_parse_single_int -- (internal) parse single int value */ static int clo_parse_single_int(struct benchmark_clo *clo, const char *arg, void *ptr) { int64_t value = 0; size_t len = strlen(arg); if (parse_number(arg, len, &value, 1, clo->type_int.base)) { errno = EINVAL; return -1; } int64_t tmax = ((int64_t)1 << (8 * clo->type_int.size - 1)) - 1; int64_t tmin = -((int64_t)1 << (8 * clo->type_int.size - 1)); tmax = min(tmax, clo->type_int.max); tmin = max(tmin, clo->type_int.min); if (value > tmax || value < tmin) { errno = ERANGE; return -1; } memcpy(ptr, &value, clo->type_int.size); return 0; } /* * clo_parse_single_uint -- (internal) parse single uint value */ static int clo_parse_single_uint(struct benchmark_clo *clo, const char *arg, void *ptr) { if (arg[0] == '-') { errno = EINVAL; return -1; } uint64_t value = 0; size_t len = strlen(arg); if (parse_number(arg, len, &value, 0, clo->type_uint.base)) { errno = EINVAL; return -1; } uint64_t tmax = ~0 >> (64 - 8 * clo->type_uint.size); uint64_t tmin = 0; tmax = min(tmax, clo->type_uint.max); tmin = max(tmin, clo->type_uint.min); if (value > tmax || value < tmin) { errno = ERANGE; return -1; } memcpy(ptr, &value, clo->type_uint.size); return 0; } /* * clo_eval_range_uint -- (internal) evaluate range for uint values */ static int clo_eval_range_uint(struct benchmark_clo *clo, void *first, void *step, void *last, char type, struct clo_vec_vlist *vlist) { uint64_t curr = *(uint64_t *)first; uint64_t l = *(uint64_t *)last; int64_t s = *(int64_t *)step; while (1) { clo_vec_vlist_add(vlist, &curr, clo->type_uint.size); switch (type) { case '+': curr += s; if (curr > l) return 0; break; case '-': if (curr < (uint64_t)s) return 0; curr -= s; if (curr < l) return 0; break; case '*': curr *= s; if (curr > l) return 0; break; case '/': curr /= s; if (curr < l) return 0; break; default: return -1; } } return -1; } /* * clo_eval_range_int -- (internal) evaluate range for int values */ static int clo_eval_range_int(struct benchmark_clo *clo, void *first, void *step, void *last, char type, struct clo_vec_vlist *vlist) { int64_t curr = *(int64_t *)first; int64_t l = *(int64_t *)last; uint64_t s = *(uint64_t *)step; while (1) { clo_vec_vlist_add(vlist, &curr, clo->type_int.size); switch (type) { case '+': curr += s; if (curr > l) return 0; break; case '-': curr -= s; if (curr < l) return 0; break; case '*': curr *= s; if (curr > l) return 0; break; case '/': curr /= s; if (curr < l) return 0; break; default: return -1; } } return -1; } /* * clo_check_range_params -- (internal) validate step and step type */ static int clo_check_range_params(uint64_t step, char step_type) { switch (step_type) { /* * Cannot construct range with step equal to 0 * for '+' or '-' range. */ case '+': case '-': if (step == 0) return -1; break; /* * Cannot construct range with step equal to 0 or 1 * for '*' or '/' range. */ case '*': case '/': if (step == 0 || step == 1) return -1; break; default: return -1; } return 0; } /* * clo_parse_range -- (internal) parse range or value * * The range may be in the following format: * :: * * Step type must be one of the following: +, -, *, /. */ static int clo_parse_range(struct benchmark_clo *clo, const char *arg, clo_parse_single_fn parse_single, clo_eval_range_fn eval_range, struct clo_vec_vlist *vlist) { auto *str_first = (char *)malloc(strlen(arg) + 1); assert(str_first != nullptr); auto *str_step = (char *)malloc(strlen(arg) + 1); assert(str_step != nullptr); char step_type = '\0'; auto *str_last = (char *)malloc(strlen(arg) + 1); assert(str_last != nullptr); int ret = sscanf(arg, "%[^:]:%c%[^:]:%[^:]", str_first, &step_type, str_step, str_last); if (ret == 1) { /* single value */ uint64_t value; if (parse_single(clo, arg, &value)) { ret = -1; } else { if (clo->type == CLO_TYPE_UINT) clo_vec_vlist_add(vlist, &value, clo->type_uint.size); else clo_vec_vlist_add(vlist, &value, clo->type_int.size); ret = 0; } } else if (ret == 4) { /* range */ uint64_t first = 0; uint64_t last = 0; uint64_t step = 0; if (parse_single(clo, str_first, &first)) { ret = -1; goto out; } char *end; errno = 0; step = strtoull(str_step, &end, 10); if (errno || !end || *end != '\0') { ret = -1; goto out; } if (parse_single(clo, str_last, &last)) { ret = -1; goto out; } if (clo_check_range_params(step, step_type)) { ret = -1; goto out; } /* evaluate the range */ if (eval_range(clo, &first, &step, &last, step_type, vlist)) { ret = -1; goto out; } ret = 0; } else { ret = -1; } out: free(str_first); free(str_step); free(str_last); return ret; } /* * clo_parse_ranges -- (internal) parse ranges/values separated by commas */ static int clo_parse_ranges(struct benchmark_clo *clo, const char *arg, struct clo_vec *clovec, clo_parse_single_fn parse_single, clo_eval_range_fn eval_range) { struct clo_vec_vlist *vlist = clo_vec_vlist_alloc(); assert(vlist != nullptr); int ret = 0; char *args = strdup(arg); assert(args != nullptr); char *curr = args; char *next; /* iterate through all values separated by comma */ while ((next = strchr(curr, ',')) != nullptr) { *next = '\0'; next++; /* parse each comma separated value as range or single value */ if ((ret = clo_parse_range(clo, curr, parse_single, eval_range, vlist))) goto out; curr = next; } /* parse each comma separated value as range or single value */ if ((ret = clo_parse_range(clo, curr, parse_single, eval_range, vlist))) goto out; /* add list of values to CLO vector */ if (clo->type == CLO_TYPE_UINT) ret = clo_vec_memcpy_list(clovec, clo->off, clo->type_uint.size, vlist); else ret = clo_vec_memcpy_list(clovec, clo->off, clo->type_int.size, vlist); out: free(args); clo_vec_vlist_free(vlist); return ret; } /* * clo_parse_int -- (internal) parse int value */ static int clo_parse_int(struct benchmark_clo *clo, const char *arg, struct clo_vec *clovec) { return clo_parse_ranges(clo, arg, clovec, clo_parse_single_int, clo_eval_range_int); } /* * clo_parse_uint -- (internal) parse uint value */ static int clo_parse_uint(struct benchmark_clo *clo, const char *arg, struct clo_vec *clovec) { return clo_parse_ranges(clo, arg, clovec, clo_parse_single_uint, clo_eval_range_uint); } /* * clo_str_flag -- (internal) convert flag value to string */ static const char * clo_str_flag(struct benchmark_clo *clo, void *addr, size_t size) { if (clo->off + sizeof(bool) > size) return nullptr; bool flag = *(bool *)((char *)addr + clo->off); return flag ? "true" : "false"; } /* * clo_str_str -- (internal) convert str value to string */ static const char * clo_str_str(struct benchmark_clo *clo, void *addr, size_t size) { if (clo->off + sizeof(char *) > size) return nullptr; return *(char **)((char *)addr + clo->off); } /* * clo_str_int -- (internal) convert int value to string */ static const char * clo_str_int(struct benchmark_clo *clo, void *addr, size_t size) { if (clo->off + clo->type_int.size > size) return nullptr; void *val = (char *)addr + clo->off; int ret = 0; switch (clo->type_int.size) { case 1: ret = snprintf(str_buff, STR_BUFF_SIZE, "%" PRId8, *(int8_t *)val); break; case 2: ret = snprintf(str_buff, STR_BUFF_SIZE, "%" PRId16, *(int16_t *)val); break; case 4: ret = snprintf(str_buff, STR_BUFF_SIZE, "%" PRId32, *(int32_t *)val); break; case 8: ret = snprintf(str_buff, STR_BUFF_SIZE, "%" PRId64, *(int64_t *)val); break; default: return nullptr; } if (ret < 0) return nullptr; return str_buff; } /* * clo_str_uint -- (internal) convert uint value to string */ static const char * clo_str_uint(struct benchmark_clo *clo, void *addr, size_t size) { if (clo->off + clo->type_uint.size > size) return nullptr; void *val = (char *)addr + clo->off; int ret = 0; switch (clo->type_uint.size) { case 1: ret = snprintf(str_buff, STR_BUFF_SIZE, "%" PRIu8, *(uint8_t *)val); break; case 2: ret = snprintf(str_buff, STR_BUFF_SIZE, "%" PRIu16, *(uint16_t *)val); break; case 4: ret = snprintf(str_buff, STR_BUFF_SIZE, "%" PRIu32, *(uint32_t *)val); break; case 8: ret = snprintf(str_buff, STR_BUFF_SIZE, "%" PRIu64, *(uint64_t *)val); break; default: return nullptr; } if (ret < 0) return nullptr; return str_buff; } /* * clo_parse -- (internal) array with functions for parsing CLOs */ static clo_parse_fn clo_parse[CLO_TYPE_MAX] = { /* [CLO_TYPE_FLAG] = */ clo_parse_flag, /* [CLO_TYPE_STR] = */ clo_parse_str, /* [CLO_TYPE_INT] = */ clo_parse_int, /* [CLO_TYPE_UINT] = */ clo_parse_uint, }; /* * clo_str -- (internal) array with functions for converting to string */ static clo_str_fn clo_str[CLO_TYPE_MAX] = { /* [CLO_TYPE_FLAG] = */ clo_str_flag, /* [CLO_TYPE_STR] = */ clo_str_str, /* [CLO_TYPE_INT] = */ clo_str_int, /* [CLO_TYPE_UINT] = */ clo_str_uint, }; /* * clo_get_by_short -- (internal) return CLO with specified short opt */ static struct benchmark_clo * clo_get_by_short(struct benchmark_clo *clos, size_t nclo, char opt_short) { size_t i; for (i = 0; i < nclo; i++) { if (clos[i].opt_short == opt_short) return &clos[i]; } return nullptr; } /* * clo_get_by_long -- (internal) return CLO with specified long opt */ static struct benchmark_clo * clo_get_by_long(struct benchmark_clo *clos, size_t nclo, const char *opt_long) { size_t i; for (i = 0; i < nclo; i++) { if (strcmp(clos[i].opt_long, opt_long) == 0) return &clos[i]; } return nullptr; } /* * clo_get_optstr -- (internal) returns option string from CLOs * * This function returns option string which contains all short * options from CLO structure. * The returned value must be freed by caller. */ static char * clo_get_optstr(struct benchmark_clo *clos, size_t nclo) { size_t i; char *optstr; char *ptr; /* * In worst case every option requires an argument * so we need space for ':' character + terminating * NULL. */ size_t optstrlen = nclo * 2 + 1; optstr = (char *)calloc(1, optstrlen); assert(optstr != nullptr); ptr = optstr; for (i = 0; i < nclo; i++) { if (clos[i].opt_short) { *(ptr++) = clos[i].opt_short; if (clos[i].type != CLO_TYPE_FLAG) *(ptr++) = ':'; } } return optstr; } /* * clo_get_long_options -- (internal) allocate long options structure * * This function allocates structure for long options and fills all * entries according to values from becnhmark_clo. This is essentially * conversion from struct benchmark_clo to struct option. * The returned value must be freed by caller. */ static struct option * clo_get_long_options(struct benchmark_clo *clos, size_t nclo) { size_t i; struct option *options; options = (struct option *)calloc(nclo + 1, sizeof(struct option)); assert(options != nullptr); for (i = 0; i < nclo; i++) { options[i].name = clos[i].opt_long; options[i].val = clos[i].opt_short; /* no optional arguments */ if (clos[i].type == CLO_TYPE_FLAG) { options[i].has_arg = no_argument; } else { options[i].has_arg = required_argument; } } return options; } /* * clo_set_defaults -- (internal) set default values * * Default values are stored as strings in CLO * structure so this function parses default values in * the same manner as values passed by user. Returns -1 * if argument was not passed by user and default value * is missing. */ static int clo_set_defaults(struct benchmark_clo *clos, size_t nclo, struct clo_vec *clovec) { size_t i; for (i = 0; i < nclo; i++) { if (clos[i].used) continue; /* * If option was not used and default value * is not specified, return error. Otherwise * parse the default value in the same way as * values passed by user. Except for the flag. * If the flag default value was not specified * assign "false" value to it. */ if (clos[i].def) { if (clo_parse[clos[i].type](&clos[i], clos[i].def, clovec)) return -1; } else if (clos[i].type == CLO_TYPE_FLAG) { if (clo_parse[clos[i].type](&clos[i], "false", clovec)) return -1; } else { printf("'%s' is required option\n", clos[i].opt_long); return -1; } } return 0; } /* * benchmark_clo_parse -- parse CLOs and store values in desired structure * * This function parses command line arguments according to information * from CLOs structure. The parsed values are stored in CLO vector * pointed by clovec. If any of command line options are not passed by user, * the default value is stored if exists. Otherwise it means the argument is * required and error is returned. * * - argc - number of command line options passed by user * - argv - command line options passed by user * - clos - description of available command line options * - nclos - number of available command line options * - clovec - vector of arguments */ int benchmark_clo_parse(int argc, char *argv[], struct benchmark_clo *clos, ssize_t nclos, struct clo_vec *clovec) { char *optstr; struct option *options; int ret = 0; int opt; int optindex; /* convert CLOs to option string and long options structure */ optstr = clo_get_optstr(clos, nclos); options = clo_get_long_options(clos, nclos); /* parse CLOs as long and/or short options */ while ((opt = getopt_long(argc, argv, optstr, options, &optindex)) != -1) { struct benchmark_clo *clo = nullptr; if (opt) { clo = clo_get_by_short(clos, nclos, opt); } else { assert(optindex < nclos); clo = &clos[optindex]; } if (!clo) { ret = -1; goto out; } /* invoke parser according to type of CLO */ assert(clo->type < CLO_TYPE_MAX); ret = clo_parse[clo->type](clo, optarg, clovec); if (ret) goto out; /* mark CLO as used */ clo->used = optarg != nullptr || clo->type == CLO_TYPE_FLAG; } if (optind < argc) { fprintf(stderr, "Unknown option: %s\n", argv[optind]); ret = -1; goto out; } /* parse unused CLOs with default values */ ret = clo_set_defaults(clos, nclos, clovec); out: free(options); free(optstr); if (ret) errno = EINVAL; return ret; } /* * benchmark_clo_parse_scenario -- parse CLOs from scenario * * This function parses command line arguments according to information * from CLOs structure. The parsed values are stored in CLO vector * pointed by clovec. If any of command line options are not passed by user, * the default value is stored if exists. Otherwise it means the argument is * required and error is returned. * * - scenario - scenario with key value arguments * - clos - description of available command line options * - nclos - number of available command line options * - clovec - vector of arguments */ int benchmark_clo_parse_scenario(struct scenario *scenario, struct benchmark_clo *clos, size_t nclos, struct clo_vec *clovec) { struct kv *kv; FOREACH_KV(kv, scenario) { struct benchmark_clo *clo = clo_get_by_long(clos, nclos, kv->key); if (!clo) { fprintf(stderr, "unrecognized option -- '%s'\n", kv->key); return -1; } assert(clo->type < CLO_TYPE_MAX); if (clo_parse[clo->type](clo, kv->value, clovec)) { fprintf(stderr, "parsing option -- '%s' failed\n", kv->value); return -1; } /* mark CLO as used */ clo->used = 1; } return clo_set_defaults(clos, nclos, clovec); } /* * benchmark_override_clos_in_scenario - parse the command line arguments and * override/add the parameters in/to the scenario by replacing/adding the kv * struct in/to the scenario. * * - scenario - scenario with key value arguments * - argc - command line arguments number * - argv - command line arguments vector * - clos - description of available command line options * - nclos - number of available command line options */ int benchmark_override_clos_in_scenario(struct scenario *scenario, int argc, char *argv[], struct benchmark_clo *clos, int nclos) { char *optstr; struct option *options; int ret = 0; int opt; int optindex; const char *true_str = "true"; /* convert CLOs to option string and long options structure */ optstr = clo_get_optstr(clos, nclos); options = clo_get_long_options(clos, nclos); /* parse CLOs as long and/or short options */ while ((opt = getopt_long(argc, argv, optstr, options, &optindex)) != -1) { struct benchmark_clo *clo = nullptr; if (opt) { clo = clo_get_by_short(clos, nclos, opt); } else { assert(optindex < nclos); clo = &clos[optindex]; } if (!clo) { ret = -1; goto out; } /* Check if the given clo is defined in the scenario */ struct kv *kv = find_kv_in_scenario(clo->opt_long, scenario); if (kv) { /* replace the value in the scenario */ if (optarg != nullptr && clo->type != CLO_TYPE_FLAG) { free(kv->value); kv->value = strdup(optarg); } else if (optarg == nullptr && clo->type == CLO_TYPE_FLAG) { free(kv->value); kv->value = strdup(true_str); } else { ret = -1; goto out; } } else { /* add a new param to the scenario */ if (optarg != nullptr && clo->type != CLO_TYPE_FLAG) { kv = kv_alloc(clo->opt_long, optarg); PMDK_TAILQ_INSERT_TAIL(&scenario->head, kv, next); } else if (optarg == nullptr && clo->type == CLO_TYPE_FLAG) { kv = kv_alloc(clo->opt_long, true_str); PMDK_TAILQ_INSERT_TAIL(&scenario->head, kv, next); } else { ret = -1; goto out; } } } if (optind < argc) { fprintf(stderr, "Unknown option: %s\n", argv[optind]); ret = -1; goto out; } out: free(options); free(optstr); if (ret) errno = EINVAL; return ret; } /* * benchmark_clo_str -- converts command line option to string * * According to command line option type and parameters, converts * the value from structure pointed by args of size size. */ const char * benchmark_clo_str(struct benchmark_clo *clo, void *args, size_t size) { assert(clo->type < CLO_TYPE_MAX); return clo_str[clo->type](clo, args, size); } /* * clo_get_scenarios - search the command line arguments for scenarios listed in * available_scenarios and put them in found_scenarios. Returns the number of * found scenarios in the cmd line or -1 on error. The passed cmd line * args should contain the scenario name(s) as the first argument(s) - starting * from index 0 */ int clo_get_scenarios(int argc, char *argv[], struct scenarios *available_scenarios, struct scenarios *found_scenarios) { assert(argv != nullptr); assert(available_scenarios != nullptr); assert(found_scenarios != nullptr); if (argc <= 0) { fprintf(stderr, "clo get scenarios, argc invalid value: %d\n", argc); return -1; } int tmp_argc = argc; char **tmp_argv = argv; do { struct scenario *scenario = scenarios_get_scenario(available_scenarios, *tmp_argv); if (!scenario) { fprintf(stderr, "unknown scenario: %s\n", *tmp_argv); return -1; } struct scenario *new_scenario = clone_scenario(scenario); assert(new_scenario != nullptr); PMDK_TAILQ_INSERT_TAIL(&found_scenarios->head, new_scenario, next); tmp_argc--; tmp_argv++; } while (tmp_argc && contains_scenarios(tmp_argc, tmp_argv, available_scenarios)); return argc - tmp_argc; } pmdk-1.8/src/benchmarks/README0000664000000000000000000000103713615011243014576 0ustar rootrootPersistent Memory Development Kit This is benchmarks/README. This directory contains benchmarks for Persistent Memory Development Kit. Benchmarks may be built and run from this directory using: $ make $ make run See how to run benchmarks manually using: $ LD_LIBRARY_PATH=../nondebug ./pmembench --help ** DEPENDENCIES: ** In order to build benchmarks you need to install glib-2.0 development package. rpm-based systems : glibX-devel (where X is the API/ABI version) dpkg-based systems: libglibX-dev (where X is the API/ABI version) pmdk-1.8/src/benchmarks/clo_vec.hpp0000664000000000000000000000531413615011243016043 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * clo_vec.hpp -- command line options vector declarations */ #include "queue.h" #include struct clo_vec_args { PMDK_TAILQ_ENTRY(clo_vec_args) next; void *args; }; struct clo_vec_alloc { PMDK_TAILQ_ENTRY(clo_vec_alloc) next; void *ptr; }; struct clo_vec_value { PMDK_TAILQ_ENTRY(clo_vec_value) next; void *ptr; }; struct clo_vec_vlist { PMDK_TAILQ_HEAD(valueshead, clo_vec_value) head; size_t nvalues; }; struct clo_vec { size_t size; PMDK_TAILQ_HEAD(argshead, clo_vec_args) args; size_t nargs; PMDK_TAILQ_HEAD(allochead, clo_vec_alloc) allocs; size_t nallocs; }; struct clo_vec *clo_vec_alloc(size_t size); void clo_vec_free(struct clo_vec *clovec); void *clo_vec_get_args(struct clo_vec *clovec, size_t i); int clo_vec_add_alloc(struct clo_vec *clovec, void *ptr); int clo_vec_memcpy(struct clo_vec *clovec, size_t off, size_t size, void *ptr); int clo_vec_memcpy_list(struct clo_vec *clovec, size_t off, size_t size, struct clo_vec_vlist *list); struct clo_vec_vlist *clo_vec_vlist_alloc(void); void clo_vec_vlist_free(struct clo_vec_vlist *list); void clo_vec_vlist_add(struct clo_vec_vlist *list, void *ptr, size_t size); pmdk-1.8/src/benchmarks/obj_lanes.cpp0000664000000000000000000001035513615011243016361 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * obj_lanes.cpp -- lane benchmark definition */ #include #include #include #include "benchmark.hpp" #include "file.h" #include "libpmemobj.h" /* an internal libpmemobj code */ #include "lane.h" /* * The number of times to repeat the operation, used to get more accurate * results, because the operation time was minimal compared to the framework * overhead. */ #define OPERATION_REPEAT_COUNT 10000 /* * obj_bench - variables used in benchmark, passed within functions */ struct obj_bench { PMEMobjpool *pop; /* persistent pool handle */ struct prog_args *pa; /* prog_args structure */ }; /* * lanes_init -- benchmark initialization */ static int lanes_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); assert(args->opts != nullptr); enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } auto *ob = (struct obj_bench *)malloc(sizeof(struct obj_bench)); if (ob == nullptr) { perror("malloc"); return -1; } pmembench_set_priv(bench, ob); ob->pa = (struct prog_args *)args->opts; size_t psize; if (args->is_poolset || type == TYPE_DEVDAX) psize = 0; else psize = PMEMOBJ_MIN_POOL; /* create pmemobj pool */ ob->pop = pmemobj_create(args->fname, "obj_lanes", psize, args->fmode); if (ob->pop == nullptr) { fprintf(stderr, "%s\n", pmemobj_errormsg()); goto err; } return 0; err: free(ob); return -1; } /* * lanes_exit -- benchmark clean up */ static int lanes_exit(struct benchmark *bench, struct benchmark_args *args) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); pmemobj_close(ob->pop); free(ob); return 0; } /* * lanes_op -- performs the lane hold and release operations */ static int lanes_op(struct benchmark *bench, struct operation_info *info) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); struct lane *lane; for (int i = 0; i < OPERATION_REPEAT_COUNT; i++) { lane_hold(ob->pop, &lane); lane_release(ob->pop); } return 0; } static struct benchmark_info lanes_info; CONSTRUCTOR(obj_lines_constructor) void obj_lines_constructor(void) { lanes_info.name = "obj_lanes"; lanes_info.brief = "Benchmark for internal lanes " "operation"; lanes_info.init = lanes_init; lanes_info.exit = lanes_exit; lanes_info.multithread = true; lanes_info.multiops = true; lanes_info.operation = lanes_op; lanes_info.measure_time = true; lanes_info.clos = NULL; lanes_info.nclos = 0; lanes_info.opts_size = 0; lanes_info.rm_file = true; lanes_info.allow_poolset = true; REGISTER_BENCHMARK(lanes_info); } pmdk-1.8/src/benchmarks/pmemobj_persist.cpp0000664000000000000000000001716113615011243017631 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmemobj_persist.cpp -- pmemobj persist benchmarks definition */ #include #include #include #include #include #include #include #include #include #include #include "benchmark.hpp" #include "file.h" #include "libpmemobj.h" #include "util.h" /* * The factor used for PMEM pool size calculation, accounts for metadata, * fragmentation and etc. */ #define FACTOR 3 /* The minimum allocation size that pmalloc can perform */ #define ALLOC_MIN_SIZE 64 /* OOB and allocation header size */ #define OOB_HEADER_SIZE 64 #define CONST_B 0xFF /* * prog_args -- benchmark specific command line options */ struct prog_args { size_t minsize; /* minimum size for random allocation size */ bool use_random_size; /* if set, use random size allocations */ bool no_warmup; /* do not do warmup */ unsigned seed; /* seed for random numbers */ }; /* * obj_bench -- benchmark context */ struct obj_bench { PMEMobjpool *pop; /* persistent pool handle */ struct prog_args *pa; /* prog_args structure */ PMEMoid *oids; /* vector of allocated objects */ void **ptrs; /* pointers to allocated objects */ uint64_t nobjs; /* number of allocated objects */ size_t obj_size; /* size of each allocated objects */ int const_b; /* memset() value */ }; /* * init_objects -- allocate persistent objects and obtain direct pointers */ static int init_objects(struct obj_bench *ob) { assert(ob->nobjs != 0); ob->oids = (PMEMoid *)malloc(ob->nobjs * sizeof(*ob->oids)); if (!ob->oids) { perror("malloc"); return -1; } ob->ptrs = (void **)malloc(ob->nobjs * sizeof(*ob->ptrs)); if (!ob->ptrs) { perror("malloc"); goto err_malloc; } for (uint64_t i = 0; i < ob->nobjs; i++) { PMEMoid oid; void *ptr; if (pmemobj_alloc(ob->pop, &oid, ob->obj_size, 0, nullptr, nullptr)) { perror("pmemobj_alloc"); goto err_palloc; } ptr = pmemobj_direct(oid); if (!ptr) { perror("pmemobj_direct"); goto err_palloc; } ob->oids[i] = oid; ob->ptrs[i] = ptr; } return 0; err_palloc: free(ob->ptrs); err_malloc: free(ob->oids); return -1; } /* * do_warmup -- does the warmup by writing the whole pool area */ static void do_warmup(struct obj_bench *ob) { for (uint64_t i = 0; i < ob->nobjs; ++i) { memset(ob->ptrs[i], 0, ob->obj_size); pmemobj_persist(ob->pop, ob->ptrs[i], ob->obj_size); } } /* * obj_persist_op -- actual benchmark operation */ static int obj_persist_op(struct benchmark *bench, struct operation_info *info) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); uint64_t idx = info->worker->index * info->args->n_ops_per_thread + info->index; assert(idx < ob->nobjs); void *ptr = ob->ptrs[idx]; memset(ptr, ob->const_b, ob->obj_size); pmemobj_persist(ob->pop, ptr, ob->obj_size); return 0; } /* * obj_persist_init -- initialization function */ static int obj_persist_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); assert(args->opts != nullptr); enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } auto *pa = (struct prog_args *)args->opts; size_t poolsize; if (pa->minsize >= args->dsize) { fprintf(stderr, "Wrong params - allocation size\n"); return -1; } auto *ob = (struct obj_bench *)malloc(sizeof(struct obj_bench)); if (ob == nullptr) { perror("malloc"); return -1; } pmembench_set_priv(bench, ob); ob->pa = pa; /* initialize memset() value */ ob->const_b = CONST_B; ob->nobjs = args->n_ops_per_thread * args->n_threads; /* Create pmemobj pool. */ ob->obj_size = args->dsize; if (ob->obj_size < ALLOC_MIN_SIZE) ob->obj_size = ALLOC_MIN_SIZE; /* For data objects */ poolsize = ob->nobjs * (ob->obj_size + OOB_HEADER_SIZE); /* multiply by FACTOR for metadata, fragmentation, etc. */ poolsize = poolsize * FACTOR; if (args->is_poolset || type == TYPE_DEVDAX) { if (args->fsize < poolsize) { fprintf(stderr, "file size too large\n"); goto free_ob; } poolsize = 0; } else if (poolsize < PMEMOBJ_MIN_POOL) { poolsize = PMEMOBJ_MIN_POOL; } poolsize = PAGE_ALIGNED_UP_SIZE(poolsize); ob->pop = pmemobj_create(args->fname, nullptr, poolsize, args->fmode); if (ob->pop == nullptr) { fprintf(stderr, "%s\n", pmemobj_errormsg()); goto free_ob; } if (init_objects(ob)) { goto free_pop; } if (!ob->pa->no_warmup) { do_warmup(ob); } return 0; free_pop: pmemobj_close(ob->pop); free_ob: free(ob); return -1; } /* * obj_persist_exit -- benchmark cleanup function */ static int obj_persist_exit(struct benchmark *bench, struct benchmark_args *args) { auto *ob = (struct obj_bench *)pmembench_get_priv(bench); for (uint64_t i = 0; i < ob->nobjs; ++i) { pmemobj_free(&ob->oids[i]); } pmemobj_close(ob->pop); free(ob->oids); free(ob->ptrs); free(ob); return 0; } static struct benchmark_clo obj_persist_clo[1]; /* Stores information about benchmark. */ static struct benchmark_info obj_persist_info; CONSTRUCTOR(pmemobj_persist_constructor) void pmemobj_persist_constructor(void) { obj_persist_clo[0].opt_short = 'w'; obj_persist_clo[0].opt_long = "no-warmup"; obj_persist_clo[0].descr = "Don't do warmup"; obj_persist_clo[0].def = "false"; obj_persist_clo[0].type = CLO_TYPE_FLAG; obj_persist_clo[0].off = clo_field_offset(struct prog_args, no_warmup); obj_persist_info.name = "pmemobj_persist"; obj_persist_info.brief = "Benchmark for pmemobj_persist() " "operation"; obj_persist_info.init = obj_persist_init; obj_persist_info.exit = obj_persist_exit; obj_persist_info.multithread = true; obj_persist_info.multiops = true; obj_persist_info.operation = obj_persist_op; obj_persist_info.measure_time = true; obj_persist_info.clos = obj_persist_clo; obj_persist_info.nclos = ARRAY_SIZE(obj_persist_clo); obj_persist_info.opts_size = sizeof(struct prog_args); obj_persist_info.rm_file = true; obj_persist_info.allow_poolset = true; REGISTER_BENCHMARK(obj_persist_info); }; pmdk-1.8/src/benchmarks/pmem_flush.cpp0000664000000000000000000003450213615011243016564 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmem_flush.cpp -- benchmark implementation for pmem_persist and pmem_msync */ #include #include #include #include #include #include #include #include #include #include #include "benchmark.hpp" #include "file.h" #define PAGE_4K ((uintptr_t)1 << 12) #define PAGE_2M ((uintptr_t)1 << 21) /* * align_addr -- round addr down to given boundary */ static void * align_addr(void *addr, uintptr_t align) { return (char *)((uintptr_t)addr & ~(align - 1)); } /* * align_len -- increase len by the amount we gain when we round addr down */ static size_t align_len(size_t len, void *addr, uintptr_t align) { return len + ((uintptr_t)addr & (align - 1)); } /* * roundup_len -- increase len by the amount we gain when we round addr down, * then round up to the nearest multiple of 4K */ static size_t roundup_len(size_t len, void *addr, uintptr_t align) { return (align_len(len, addr, align) + align - 1) & ~(align - 1); } /* * pmem_args -- benchmark specific arguments */ struct pmem_args { char *operation; /* msync, dummy_msync, persist, ... */ char *mode; /* stat, seq, rand */ bool no_warmup; /* don't do warmup */ }; /* * pmem_bench -- benchmark context */ struct pmem_bench { uint64_t *offsets; /* write offsets */ size_t n_offsets; /* number of elements in offsets array */ size_t fsize; /* The size of the allocated PMEM */ struct pmem_args *pargs; /* prog_args structure */ void *pmem_addr; /* PMEM base address */ size_t pmem_len; /* length of PMEM mapping */ void *invalid_addr; /* invalid pages */ void *nondirty_addr; /* non-dirty pages */ void *pmem_addr_aligned; /* PMEM pages - 2M aligned */ void *invalid_addr_aligned; /* invalid pages - 2M aligned */ void *nondirty_addr_aligned; /* non-dirty pages - 2M aligned */ /* the actual benchmark operation */ int (*func_op)(struct pmem_bench *pmb, void *addr, size_t len); }; /* * mode_seq -- if copy mode is sequential, returns index of a chunk. */ static uint64_t mode_seq(struct pmem_bench *pmb, uint64_t index) { return index; } /* * mode_stat -- if mode is static, the offset is always 0 */ static uint64_t mode_stat(struct pmem_bench *pmb, uint64_t index) { return 0; } /* * mode_rand -- if mode is random, returns index of a random chunk */ static uint64_t mode_rand(struct pmem_bench *pmb, uint64_t index) { return rand() % pmb->n_offsets; } /* * operation_mode -- the mode of the copy process * * * static - write always the same chunk, * * sequential - write chunk by chunk, * * random - write to chunks selected randomly. */ struct op_mode { const char *mode; uint64_t (*func_mode)(struct pmem_bench *pmb, uint64_t index); }; static struct op_mode modes[] = { {"stat", mode_stat}, {"seq", mode_seq}, {"rand", mode_rand}, }; #define MODES (sizeof(modes) / sizeof(modes[0])) /* * parse_op_mode -- parses command line "--mode" * and returns proper operation mode index. */ static int parse_op_mode(const char *arg) { for (unsigned i = 0; i < MODES; i++) { if (strcmp(arg, modes[i].mode) == 0) return i; } return -1; } /* * flush_noop -- dummy flush, does nothing */ static int flush_noop(struct pmem_bench *pmb, void *addr, size_t len) { return 0; } /* * flush_persist -- flush data to persistence using pmem_persist() */ static int flush_persist(struct pmem_bench *pmb, void *addr, size_t len) { pmem_persist(addr, len); return 0; } /* * flush_persist_4K -- always flush entire 4K page(s) using pmem_persist() */ static int flush_persist_4K(struct pmem_bench *pmb, void *addr, size_t len) { void *ptr = align_addr(addr, PAGE_4K); len = roundup_len(len, addr, PAGE_4K); pmem_persist(ptr, len); return 0; } /* * flush_persist_2M -- always flush entire 2M page(s) using pmem_persist() */ static int flush_persist_2M(struct pmem_bench *pmb, void *addr, size_t len) { void *ptr = align_addr(addr, PAGE_2M); len = roundup_len(len, addr, PAGE_2M); pmem_persist(ptr, len); return 0; } /* * flush_msync -- flush data to persistence using pmem_msync() */ static int flush_msync(struct pmem_bench *pmb, void *addr, size_t len) { pmem_msync(addr, len); return 0; } /* * flush_msync_async -- emulate dummy msync() using MS_ASYNC flag */ static int flush_msync_async(struct pmem_bench *pmb, void *addr, size_t len) { void *ptr = align_addr(addr, PAGE_4K); len = align_len(len, addr, PAGE_4K); msync(ptr, len, MS_ASYNC); return 0; } /* * flush_msync_0 -- emulate dummy msync() using zero length */ static int flush_msync_0(struct pmem_bench *pmb, void *addr, size_t len) { void *ptr = align_addr(addr, PAGE_4K); (void)len; msync(ptr, 0, MS_SYNC); return 0; } /* * flush_persist_4K_msync_0 -- emulate msync() that only flushes CPU cache * * Do flushing in user space (4K pages) + dummy syscall. */ static int flush_persist_4K_msync_0(struct pmem_bench *pmb, void *addr, size_t len) { void *ptr = align_addr(addr, PAGE_4K); len = roundup_len(len, addr, PAGE_4K); pmem_persist(ptr, len); msync(ptr, 0, MS_SYNC); return 0; } /* * flush_persist_2M_msync_0 -- emulate msync() that only flushes CPU cache * * Do flushing in user space (2M pages) + dummy syscall. */ static int flush_persist_2M_msync_0(struct pmem_bench *pmb, void *addr, size_t len) { void *ptr = align_addr(addr, PAGE_2M); len = roundup_len(len, addr, PAGE_2M); pmem_persist(ptr, len); msync(ptr, 0, MS_SYNC); return 0; } /* * flush_msync_err -- emulate dummy msync() using invalid flags */ static int flush_msync_err(struct pmem_bench *pmb, void *addr, size_t len) { void *ptr = align_addr(addr, PAGE_4K); len = align_len(len, addr, PAGE_4K); msync(ptr, len, MS_SYNC | MS_ASYNC); return 0; } /* * flush_msync_nodirty -- call msync() on non-dirty pages */ static int flush_msync_nodirty(struct pmem_bench *pmb, void *addr, size_t len) { uintptr_t uptr = (uintptr_t)addr - (uintptr_t)pmb->pmem_addr_aligned; uptr += (uintptr_t)pmb->nondirty_addr_aligned; void *ptr = align_addr((void *)uptr, PAGE_4K); len = align_len(len, (void *)uptr, PAGE_4K); pmem_msync(ptr, len); return 0; } /* * flush_msync_invalid -- emulate dummy msync() using invalid address */ static int flush_msync_invalid(struct pmem_bench *pmb, void *addr, size_t len) { uintptr_t uptr = (uintptr_t)addr - (uintptr_t)pmb->pmem_addr_aligned; uptr += (uintptr_t)pmb->invalid_addr_aligned; void *ptr = align_addr((void *)uptr, PAGE_4K); len = align_len(len, (void *)uptr, PAGE_4K); pmem_msync(ptr, len); return 0; } struct op { const char *opname; int (*func_op)(struct pmem_bench *pmb, void *addr, size_t len); }; static struct op ops[] = { {"noop", flush_noop}, {"persist", flush_persist}, {"persist_4K", flush_persist_4K}, {"persist_2M", flush_persist_2M}, {"msync", flush_msync}, {"msync_0", flush_msync_0}, {"msync_err", flush_msync_err}, {"persist_4K_msync_0", flush_persist_4K_msync_0}, {"persist_2M_msync_0", flush_persist_2M_msync_0}, {"msync_async", flush_msync_async}, {"msync_nodirty", flush_msync_nodirty}, {"msync_invalid", flush_msync_invalid}, }; #define NOPS (sizeof(ops) / sizeof(ops[0])) /* * parse_op_type -- parses command line "--operation" argument * and returns proper operation type. */ static int parse_op_type(const char *arg) { for (unsigned i = 0; i < NOPS; i++) { if (strcmp(arg, ops[i].opname) == 0) return i; } return -1; } /* * pmem_flush_init -- benchmark initialization * * Parses command line arguments, allocates persistent memory, and maps it. */ static int pmem_flush_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); size_t file_size = 0; int flags = 0; enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } uint64_t (*func_mode)(struct pmem_bench * pmb, uint64_t index); auto *pmb = (struct pmem_bench *)malloc(sizeof(struct pmem_bench)); assert(pmb != nullptr); pmb->pargs = (struct pmem_args *)args->opts; assert(pmb->pargs != nullptr); int i = parse_op_type(pmb->pargs->operation); if (i == -1) { fprintf(stderr, "wrong operation: %s\n", pmb->pargs->operation); goto err_free_pmb; } pmb->func_op = ops[i].func_op; pmb->n_offsets = args->n_ops_per_thread * args->n_threads; pmb->fsize = pmb->n_offsets * args->dsize + (2 * PAGE_2M); /* round up to 2M boundary */ pmb->fsize = (pmb->fsize + PAGE_2M - 1) & ~(PAGE_2M - 1); i = parse_op_mode(pmb->pargs->mode); if (i == -1) { fprintf(stderr, "wrong mode: %s\n", pmb->pargs->mode); goto err_free_pmb; } func_mode = modes[i].func_mode; /* populate offsets array */ assert(pmb->n_offsets != 0); pmb->offsets = (size_t *)malloc(pmb->n_offsets * sizeof(*pmb->offsets)); assert(pmb->offsets != nullptr); for (size_t i = 0; i < pmb->n_offsets; ++i) pmb->offsets[i] = func_mode(pmb, i); if (type != TYPE_DEVDAX) { file_size = pmb->fsize; flags = PMEM_FILE_CREATE | PMEM_FILE_EXCL; } /* create a pmem file and memory map it */ pmb->pmem_addr = pmem_map_file(args->fname, file_size, flags, args->fmode, &pmb->pmem_len, nullptr); if (pmb->pmem_addr == nullptr) { perror("pmem_map_file"); goto err_free_pmb; } pmb->nondirty_addr = mmap(nullptr, pmb->fsize, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); if (pmb->nondirty_addr == MAP_FAILED) { perror("mmap(1)"); goto err_unmap1; } pmb->invalid_addr = mmap(nullptr, pmb->fsize, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); if (pmb->invalid_addr == MAP_FAILED) { perror("mmap(2)"); goto err_unmap2; } munmap(pmb->invalid_addr, pmb->fsize); pmb->pmem_addr_aligned = (void *)(((uintptr_t)pmb->pmem_addr + PAGE_2M - 1) & ~(PAGE_2M - 1)); pmb->nondirty_addr_aligned = (void *)(((uintptr_t)pmb->nondirty_addr + PAGE_2M - 1) & ~(PAGE_2M - 1)); pmb->invalid_addr_aligned = (void *)(((uintptr_t)pmb->invalid_addr + PAGE_2M - 1) & ~(PAGE_2M - 1)); pmembench_set_priv(bench, pmb); if (!pmb->pargs->no_warmup) { size_t off; for (off = 0; off < pmb->fsize - PAGE_2M; off += PAGE_4K) { *(int *)((char *)pmb->pmem_addr_aligned + off) = 0; *(int *)((char *)pmb->nondirty_addr_aligned + off) = 0; } } return 0; err_unmap2: munmap(pmb->nondirty_addr, pmb->fsize); err_unmap1: pmem_unmap(pmb->pmem_addr, pmb->pmem_len); err_free_pmb: free(pmb); return -1; } /* * pmem_flush_exit -- benchmark cleanup */ static int pmem_flush_exit(struct benchmark *bench, struct benchmark_args *args) { auto *pmb = (struct pmem_bench *)pmembench_get_priv(bench); pmem_unmap(pmb->pmem_addr, pmb->fsize); munmap(pmb->nondirty_addr, pmb->fsize); free(pmb); return 0; } /* * pmem_flush_operation -- actual benchmark operation */ static int pmem_flush_operation(struct benchmark *bench, struct operation_info *info) { auto *pmb = (struct pmem_bench *)pmembench_get_priv(bench); size_t op_idx = info->index; assert(op_idx < pmb->n_offsets); uint64_t chunk_idx = pmb->offsets[op_idx]; void *addr = (char *)pmb->pmem_addr_aligned + chunk_idx * info->args->dsize; /* store + flush */ *(int *)addr = *(int *)addr + 1; pmb->func_op(pmb, addr, info->args->dsize); return 0; } /* structure to define command line arguments */ static struct benchmark_clo pmem_flush_clo[3]; /* Stores information about benchmark. */ static struct benchmark_info pmem_flush_bench; CONSTRUCTOR(pmem_flush_constructor) void pmem_flush_constructor(void) { pmem_flush_clo[0].opt_short = 'o'; pmem_flush_clo[0].opt_long = "operation"; pmem_flush_clo[0].descr = "Operation type - persist," " msync, ..."; pmem_flush_clo[0].type = CLO_TYPE_STR; pmem_flush_clo[0].off = clo_field_offset(struct pmem_args, operation); pmem_flush_clo[0].def = "noop"; pmem_flush_clo[1].opt_short = 0; pmem_flush_clo[1].opt_long = "mode"; pmem_flush_clo[1].descr = "mode - stat, seq or rand"; pmem_flush_clo[1].type = CLO_TYPE_STR; pmem_flush_clo[1].off = clo_field_offset(struct pmem_args, mode); pmem_flush_clo[1].def = "stat"; pmem_flush_clo[2].opt_short = 'w'; pmem_flush_clo[2].opt_long = "no-warmup"; pmem_flush_clo[2].descr = "Don't do warmup"; pmem_flush_clo[2].type = CLO_TYPE_FLAG; pmem_flush_clo[2].off = clo_field_offset(struct pmem_args, no_warmup); pmem_flush_bench.name = "pmem_flush"; pmem_flush_bench.brief = "Benchmark for pmem_msync() " "and pmem_persist()"; pmem_flush_bench.init = pmem_flush_init; pmem_flush_bench.exit = pmem_flush_exit; pmem_flush_bench.multithread = true; pmem_flush_bench.multiops = true; pmem_flush_bench.operation = pmem_flush_operation; pmem_flush_bench.measure_time = true; pmem_flush_bench.clos = pmem_flush_clo; pmem_flush_bench.nclos = ARRAY_SIZE(pmem_flush_clo); pmem_flush_bench.opts_size = sizeof(struct pmem_args); pmem_flush_bench.rm_file = true; pmem_flush_bench.allow_poolset = false; REGISTER_BENCHMARK(pmem_flush_bench); } pmdk-1.8/src/benchmarks/config_reader.cpp0000664000000000000000000001646413615011243017223 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * config_reader.cpp -- config reader module definitions */ #include #include #include #include #include #include #include "config_reader.hpp" #include "scenario.hpp" #define SECTION_GLOBAL "global" #define KEY_BENCHMARK "bench" #define KEY_GROUP "group" /* * config_reader -- handle structure */ struct config_reader { GKeyFile *key_file; }; /* * config_reader_alloc -- allocate config reader */ struct config_reader * config_reader_alloc(void) { struct config_reader *cr = (struct config_reader *)malloc(sizeof(*cr)); assert(cr != nullptr); cr->key_file = g_key_file_new(); if (!cr->key_file) goto err; return cr; err: free(cr); return nullptr; } /* * config_reader_read -- read config file */ int config_reader_read(struct config_reader *cr, const char *fname) { if (g_key_file_load_from_file(cr->key_file, fname, G_KEY_FILE_NONE, nullptr) != TRUE) return -1; return 0; } /* * config_reader_free -- free config reader */ void config_reader_free(struct config_reader *cr) { g_key_file_free(cr->key_file); free(cr); } /* * is_scenario -- (internal) return true if _name_ is scenario name * * This filters out the _global_ and _config_ sections. */ static int is_scenario(const char *name) { return strcmp(name, SECTION_GLOBAL); } /* * is_argument -- (internal) return true if _name_ is argument name * * This filters out the _benchmark_ key. */ static int is_argument(const char *name) { return strcmp(name, KEY_BENCHMARK) != 0 && strcmp(name, KEY_GROUP) != 0; } /* * config_reader_get_scenarios -- return scenarios from config file * * This function reads the config file and returns a list of scenarios. * Each scenario contains a list of key/value arguments. * The scenario's arguments are merged with arguments from global section. */ int config_reader_get_scenarios(struct config_reader *cr, struct scenarios **scenarios) { /* * Read all groups. * The config file must have at least one group, otherwise * it is considered as invalid. */ gsize ngroups; gsize g; gchar **groups = g_key_file_get_groups(cr->key_file, &ngroups); assert(nullptr != groups); if (!groups) return -1; /* * Check if global section is present and read keys from it. */ int ret = 0; int has_global = g_key_file_has_group(cr->key_file, SECTION_GLOBAL) == TRUE; gsize ngkeys; gchar **gkeys = nullptr; struct scenarios *s; if (has_global) { gkeys = g_key_file_get_keys(cr->key_file, SECTION_GLOBAL, &ngkeys, nullptr); assert(nullptr != gkeys); if (!gkeys) { ret = -1; goto err_groups; } } s = scenarios_alloc(); assert(nullptr != s); if (!s) { ret = -1; goto err_gkeys; } for (g = 0; g < ngroups; g++) { /* * Check whether a group is a scenario * or global section. */ if (!is_scenario(groups[g])) continue; /* * Check for KEY_BENCHMARK which contains benchmark name. * If not present the benchmark name is the same as the * name of the section. */ struct scenario *scenario = nullptr; if (g_key_file_has_key(cr->key_file, groups[g], KEY_BENCHMARK, nullptr) == FALSE) { scenario = scenario_alloc(groups[g], groups[g]); assert(scenario != nullptr); } else { gchar *benchmark = g_key_file_get_value(cr->key_file, groups[g], KEY_BENCHMARK, nullptr); assert(benchmark != nullptr); if (!benchmark) { ret = -1; goto err_scenarios; } scenario = scenario_alloc(groups[g], benchmark); assert(scenario != nullptr); free(benchmark); } gsize k; if (has_global) { /* * Merge key/values from global section. */ for (k = 0; k < ngkeys; k++) { if (g_key_file_has_key(cr->key_file, groups[g], gkeys[k], nullptr) == TRUE) continue; if (!is_argument(gkeys[k])) continue; char *value = g_key_file_get_value( cr->key_file, SECTION_GLOBAL, gkeys[k], nullptr); assert(nullptr != value); if (!value) { ret = -1; goto err_scenarios; } struct kv *kv = kv_alloc(gkeys[k], value); assert(nullptr != kv); free(value); if (!kv) { ret = -1; goto err_scenarios; } PMDK_TAILQ_INSERT_TAIL(&scenario->head, kv, next); } } /* check for group name */ if (g_key_file_has_key(cr->key_file, groups[g], KEY_GROUP, nullptr) != FALSE) { gchar *group = g_key_file_get_value( cr->key_file, groups[g], KEY_GROUP, nullptr); assert(group != nullptr); scenario_set_group(scenario, group); } else if (g_key_file_has_key(cr->key_file, SECTION_GLOBAL, KEY_GROUP, nullptr) != FALSE) { gchar *group = g_key_file_get_value(cr->key_file, SECTION_GLOBAL, KEY_GROUP, nullptr); scenario_set_group(scenario, group); } gsize nkeys; gchar **keys = g_key_file_get_keys(cr->key_file, groups[g], &nkeys, nullptr); assert(nullptr != keys); if (!keys) { ret = -1; goto err_scenarios; } /* * Read key/values from the scenario's section. */ for (k = 0; k < nkeys; k++) { if (!is_argument(keys[k])) continue; char *value = g_key_file_get_value( cr->key_file, groups[g], keys[k], nullptr); assert(nullptr != value); if (!value) { ret = -1; g_strfreev(keys); goto err_scenarios; } struct kv *kv = kv_alloc(keys[k], value); assert(nullptr != kv); free(value); if (!kv) { g_strfreev(keys); ret = -1; goto err_scenarios; } PMDK_TAILQ_INSERT_TAIL(&scenario->head, kv, next); } g_strfreev(keys); PMDK_TAILQ_INSERT_TAIL(&s->head, scenario, next); } g_strfreev(gkeys); g_strfreev(groups); *scenarios = s; return 0; err_scenarios: scenarios_free(s); err_gkeys: g_strfreev(gkeys); err_groups: g_strfreev(groups); return ret; } pmdk-1.8/src/benchmarks/benchmark_worker.hpp0000664000000000000000000000777313615011243017767 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * benchmark_worker.hpp -- benchmark_worker module declarations */ #include "benchmark.hpp" #include "os_thread.h" /* * * The following table shows valid state transitions upon specified * API calls and operations performed by the worker thread: * * +========================+==========================+=============+ * | Application | State | Worker | * +========================+==========================+=============+ * | benchmark_worker_alloc | WORKER_STATE_IDLE | wait | * +------------------------+--------------------------+-------------+ * | benchmark_worker_init | WORKER_STATE_INIT | invoke init | * +------------------------+--------------------------+-------------+ * | wait | WORKER_STATE_INITIALIZED | end of init | * +------------------------+--------------------------+-------------+ * | benchmark_worker_run | WORKER_STATE_RUN | invoke func | * +------------------------+--------------------------+-------------+ * | benchmark_worker_join | WORKER_STATE_END | end of func | * +------------------------+--------------------------+-------------+ * | benchmark_worker_exit | WORKER_STATE_EXIT | invoke exit | * +------------------------+--------------------------+-------------+ * | wait | WORKER_STATE_DONE | end of exit | * +------------------------+--------------------------+-------------+ */ enum benchmark_worker_state { WORKER_STATE_IDLE, WORKER_STATE_INIT, WORKER_STATE_INITIALIZED, WORKER_STATE_RUN, WORKER_STATE_END, WORKER_STATE_EXIT, WORKER_STATE_DONE, MAX_WORKER_STATE, }; struct benchmark_worker { os_thread_t thread; struct benchmark *bench; struct benchmark_args *args; struct worker_info info; int ret; int ret_init; int (*func)(struct benchmark *bench, struct worker_info *info); int (*init)(struct benchmark *bench, struct benchmark_args *args, struct worker_info *info); void (*exit)(struct benchmark *bench, struct benchmark_args *args, struct worker_info *info); os_cond_t cond; os_mutex_t lock; enum benchmark_worker_state state; }; struct benchmark_worker *benchmark_worker_alloc(void); void benchmark_worker_free(struct benchmark_worker *); int benchmark_worker_init(struct benchmark_worker *); void benchmark_worker_exit(struct benchmark_worker *); int benchmark_worker_run(struct benchmark_worker *); int benchmark_worker_join(struct benchmark_worker *); pmdk-1.8/src/benchmarks/map_bench.cpp0000664000000000000000000005265513615011243016352 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_bench.cpp -- benchmarks for: ctree, btree, rtree, rbtree, hashmap_atomic * and hashmap_tx from examples. */ #include #include "benchmark.hpp" #include "file.h" #include "os.h" #include "os_thread.h" #include "poolset_util.hpp" #include "map.h" #include "map_btree.h" #include "map_ctree.h" #include "map_hashmap_atomic.h" #include "map_hashmap_rp.h" #include "map_hashmap_tx.h" #include "map_rbtree.h" #include "map_rtree.h" /* Values less than 3 is not suitable for current rtree implementation */ #define FACTOR 3 #define ALLOC_OVERHEAD 64 TOID_DECLARE_ROOT(struct root); struct root { TOID(struct map) map; }; #define OBJ_TYPE_NUM 1 #define swap(a, b) \ do { \ __typeof__(a) _tmp = (a); \ (a) = (b); \ (b) = _tmp; \ } while (0) /* Values less than 2048 is not suitable for current rtree implementation */ #define SIZE_PER_KEY 2048 static const struct { const char *str; const struct map_ops *ops; } map_types[] = { {"ctree", MAP_CTREE}, {"btree", MAP_BTREE}, {"rtree", MAP_RTREE}, {"rbtree", MAP_RBTREE}, {"hashmap_tx", MAP_HASHMAP_TX}, {"hashmap_atomic", MAP_HASHMAP_ATOMIC}, {"hashmap_rp", MAP_HASHMAP_RP}}; #define MAP_TYPES_NUM (sizeof(map_types) / sizeof(map_types[0])) struct map_bench_args { unsigned seed; uint64_t max_key; char *type; bool ext_tx; bool alloc; }; struct map_bench_worker { uint64_t *keys; size_t nkeys; }; struct map_bench { struct map_ctx *mapc; os_mutex_t lock; PMEMobjpool *pop; size_t pool_size; size_t nkeys; size_t init_nkeys; uint64_t *keys; struct benchmark_args *args; struct map_bench_args *margs; TOID(struct root) root; PMEMoid root_oid; TOID(struct map) map; int (*insert)(struct map_bench *, uint64_t); int (*remove)(struct map_bench *, uint64_t); int (*get)(struct map_bench *, uint64_t); }; /* * mutex_lock_nofail -- locks mutex and aborts if locking failed */ static void mutex_lock_nofail(os_mutex_t *lock) { errno = os_mutex_lock(lock); if (errno) { perror("os_mutex_lock"); abort(); } } /* * mutex_unlock_nofail -- unlocks mutex and aborts if unlocking failed */ static void mutex_unlock_nofail(os_mutex_t *lock) { errno = os_mutex_unlock(lock); if (errno) { perror("os_mutex_unlock"); abort(); } } /* * get_key -- return 64-bit random key */ static uint64_t get_key(unsigned *seed, uint64_t max_key) { unsigned key_lo = os_rand_r(seed); unsigned key_hi = os_rand_r(seed); uint64_t key = (((uint64_t)key_hi) << 32) | ((uint64_t)key_lo); if (max_key) key = key % max_key; return key; } /* * parse_map_type -- parse type of map */ static const struct map_ops * parse_map_type(const char *str) { for (unsigned i = 0; i < MAP_TYPES_NUM; i++) { if (strcmp(str, map_types[i].str) == 0) return map_types[i].ops; } return nullptr; } /* * map_remove_free_op -- remove and free object from map */ static int map_remove_free_op(struct map_bench *map_bench, uint64_t key) { volatile int ret = 0; TX_BEGIN(map_bench->pop) { PMEMoid val = map_remove(map_bench->mapc, map_bench->map, key); if (OID_IS_NULL(val)) ret = -1; else pmemobj_tx_free(val); } TX_ONABORT { ret = -1; } TX_END return ret; } /* * map_remove_root_op -- remove root object from map */ static int map_remove_root_op(struct map_bench *map_bench, uint64_t key) { PMEMoid val = map_remove(map_bench->mapc, map_bench->map, key); return !OID_EQUALS(val, map_bench->root_oid); } /* * map_remove_op -- main operation for map_remove benchmark */ static int map_remove_op(struct benchmark *bench, struct operation_info *info) { auto *map_bench = (struct map_bench *)pmembench_get_priv(bench); auto *tworker = (struct map_bench_worker *)info->worker->priv; uint64_t key = tworker->keys[info->index]; mutex_lock_nofail(&map_bench->lock); int ret = map_bench->remove(map_bench, key); mutex_unlock_nofail(&map_bench->lock); return ret; } /* * map_insert_alloc_op -- allocate an object and insert to map */ static int map_insert_alloc_op(struct map_bench *map_bench, uint64_t key) { int ret = 0; TX_BEGIN(map_bench->pop) { PMEMoid oid = pmemobj_tx_alloc(map_bench->args->dsize, OBJ_TYPE_NUM); ret = map_insert(map_bench->mapc, map_bench->map, key, oid); } TX_ONABORT { ret = -1; } TX_END return ret; } /* * map_insert_root_op -- insert root object to map */ static int map_insert_root_op(struct map_bench *map_bench, uint64_t key) { return map_insert(map_bench->mapc, map_bench->map, key, map_bench->root_oid); } /* * map_insert_op -- main operation for map_insert benchmark */ static int map_insert_op(struct benchmark *bench, struct operation_info *info) { auto *map_bench = (struct map_bench *)pmembench_get_priv(bench); auto *tworker = (struct map_bench_worker *)info->worker->priv; uint64_t key = tworker->keys[info->index]; mutex_lock_nofail(&map_bench->lock); int ret = map_bench->insert(map_bench, key); mutex_unlock_nofail(&map_bench->lock); return ret; } /* * map_get_obj_op -- get object from map at specified key */ static int map_get_obj_op(struct map_bench *map_bench, uint64_t key) { PMEMoid val = map_get(map_bench->mapc, map_bench->map, key); return OID_IS_NULL(val); } /* * map_get_root_op -- get root object from map at specified key */ static int map_get_root_op(struct map_bench *map_bench, uint64_t key) { PMEMoid val = map_get(map_bench->mapc, map_bench->map, key); return !OID_EQUALS(val, map_bench->root_oid); } /* * map_get_op -- main operation for map_get benchmark */ static int map_get_op(struct benchmark *bench, struct operation_info *info) { auto *map_bench = (struct map_bench *)pmembench_get_priv(bench); auto *tworker = (struct map_bench_worker *)info->worker->priv; uint64_t key = tworker->keys[info->index]; mutex_lock_nofail(&map_bench->lock); int ret = map_bench->get(map_bench, key); mutex_unlock_nofail(&map_bench->lock); return ret; } /* * map_common_init_worker -- common init worker function for map_* benchmarks */ static int map_common_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { struct map_bench_worker *tworker = (struct map_bench_worker *)calloc(1, sizeof(*tworker)); struct map_bench *tree; struct map_bench_args *targs; if (!tworker) { perror("calloc"); return -1; } tworker->nkeys = args->n_ops_per_thread; tworker->keys = (uint64_t *)malloc(tworker->nkeys * sizeof(*tworker->keys)); if (!tworker->keys) { perror("malloc"); goto err_free_worker; } tree = (struct map_bench *)pmembench_get_priv(bench); targs = (struct map_bench_args *)args->opts; if (targs->ext_tx) { int ret = pmemobj_tx_begin(tree->pop, nullptr); if (ret) { (void)pmemobj_tx_end(); goto err_free_keys; } } worker->priv = tworker; return 0; err_free_keys: free(tworker->keys); err_free_worker: free(tworker); return -1; } /* * map_common_free_worker -- common cleanup worker function for map_* * benchmarks */ static void map_common_free_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *tworker = (struct map_bench_worker *)worker->priv; auto *targs = (struct map_bench_args *)args->opts; if (targs->ext_tx) { pmemobj_tx_commit(); (void)pmemobj_tx_end(); } free(tworker->keys); free(tworker); } /* * map_insert_init_worker -- init worker function for map_insert benchmark */ static int map_insert_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { int ret = map_common_init_worker(bench, args, worker); if (ret) return ret; auto *targs = (struct map_bench_args *)args->opts; assert(targs); auto *tworker = (struct map_bench_worker *)worker->priv; assert(tworker); for (size_t i = 0; i < tworker->nkeys; i++) tworker->keys[i] = get_key(&targs->seed, targs->max_key); return 0; } /* * map_global_rand_keys_init -- assign random keys from global keys array */ static int map_global_rand_keys_init(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *tree = (struct map_bench *)pmembench_get_priv(bench); assert(tree); auto *targs = (struct map_bench_args *)args->opts; assert(targs); auto *tworker = (struct map_bench_worker *)worker->priv; assert(tworker); assert(tree->init_nkeys); /* * Assign random keys from global tree->keys array without repetitions. */ for (size_t i = 0; i < tworker->nkeys; i++) { uint64_t index = get_key(&targs->seed, tree->init_nkeys); tworker->keys[i] = tree->keys[index]; swap(tree->keys[index], tree->keys[tree->init_nkeys - 1]); tree->init_nkeys--; } return 0; } /* * map_remove_init_worker -- init worker function for map_remove benchmark */ static int map_remove_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { int ret = map_common_init_worker(bench, args, worker); if (ret) return ret; ret = map_global_rand_keys_init(bench, args, worker); if (ret) goto err_common_free_worker; return 0; err_common_free_worker: map_common_free_worker(bench, args, worker); return -1; } /* * map_bench_get_init_worker -- init worker function for map_get benchmark */ static int map_bench_get_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { int ret = map_common_init_worker(bench, args, worker); if (ret) return ret; ret = map_global_rand_keys_init(bench, args, worker); if (ret) goto err_common_free_worker; return 0; err_common_free_worker: map_common_free_worker(bench, args, worker); return -1; } /* * map_common_init -- common init function for map_* benchmarks */ static int map_common_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench); assert(args); assert(args->opts); char path[PATH_MAX]; if (util_safe_strcpy(path, args->fname, sizeof(path)) != 0) return -1; enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } size_t size_per_key; struct map_bench *map_bench = (struct map_bench *)calloc(1, sizeof(*map_bench)); if (!map_bench) { perror("calloc"); return -1; } map_bench->args = args; map_bench->margs = (struct map_bench_args *)args->opts; const struct map_ops *ops = parse_map_type(map_bench->margs->type); if (!ops) { fprintf(stderr, "invalid map type value specified -- '%s'\n", map_bench->margs->type); goto err_free_bench; } if (map_bench->margs->ext_tx && args->n_threads > 1) { fprintf(stderr, "external transaction requires single thread\n"); goto err_free_bench; } if (map_bench->margs->alloc) { map_bench->insert = map_insert_alloc_op; map_bench->remove = map_remove_free_op; map_bench->get = map_get_obj_op; } else { map_bench->insert = map_insert_root_op; map_bench->remove = map_remove_root_op; map_bench->get = map_get_root_op; } map_bench->nkeys = args->n_threads * args->n_ops_per_thread; map_bench->init_nkeys = map_bench->nkeys; size_per_key = map_bench->margs->alloc ? SIZE_PER_KEY + map_bench->args->dsize + ALLOC_OVERHEAD : SIZE_PER_KEY; map_bench->pool_size = map_bench->nkeys * size_per_key * FACTOR; if (args->is_poolset || type == TYPE_DEVDAX) { if (args->fsize < map_bench->pool_size) { fprintf(stderr, "file size too large\n"); goto err_free_bench; } map_bench->pool_size = 0; } else if (map_bench->pool_size < 2 * PMEMOBJ_MIN_POOL) { map_bench->pool_size = 2 * PMEMOBJ_MIN_POOL; } if (args->is_dynamic_poolset) { int ret = dynamic_poolset_create(args->fname, map_bench->pool_size); if (ret == -1) goto err_free_bench; if (util_safe_strcpy(path, POOLSET_PATH, sizeof(path)) != 0) goto err_free_bench; map_bench->pool_size = 0; } map_bench->pop = pmemobj_create(path, "map_bench", map_bench->pool_size, args->fmode); if (!map_bench->pop) { fprintf(stderr, "pmemobj_create: %s\n", pmemobj_errormsg()); goto err_free_bench; } errno = os_mutex_init(&map_bench->lock); if (errno) { perror("os_mutex_init"); goto err_close; } map_bench->mapc = map_ctx_init(ops, map_bench->pop); if (!map_bench->mapc) { perror("map_ctx_init"); goto err_destroy_lock; } map_bench->root = POBJ_ROOT(map_bench->pop, struct root); if (TOID_IS_NULL(map_bench->root)) { fprintf(stderr, "pmemobj_root: %s\n", pmemobj_errormsg()); goto err_free_map; } map_bench->root_oid = map_bench->root.oid; if (map_create(map_bench->mapc, &D_RW(map_bench->root)->map, nullptr)) { perror("map_new"); goto err_free_map; } map_bench->map = D_RO(map_bench->root)->map; pmembench_set_priv(bench, map_bench); return 0; err_free_map: map_ctx_free(map_bench->mapc); err_destroy_lock: os_mutex_destroy(&map_bench->lock); err_close: pmemobj_close(map_bench->pop); err_free_bench: free(map_bench); return -1; } /* * map_common_exit -- common cleanup function for map_* benchmarks */ static int map_common_exit(struct benchmark *bench, struct benchmark_args *args) { auto *tree = (struct map_bench *)pmembench_get_priv(bench); os_mutex_destroy(&tree->lock); map_ctx_free(tree->mapc); pmemobj_close(tree->pop); free(tree); return 0; } /* * map_keys_init -- initialize array with keys */ static int map_keys_init(struct benchmark *bench, struct benchmark_args *args) { auto *map_bench = (struct map_bench *)pmembench_get_priv(bench); assert(map_bench); auto *targs = (struct map_bench_args *)args->opts; assert(targs); assert(map_bench->nkeys != 0); map_bench->keys = (uint64_t *)malloc(map_bench->nkeys * sizeof(*map_bench->keys)); if (!map_bench->keys) { perror("malloc"); return -1; } int ret = 0; mutex_lock_nofail(&map_bench->lock); TX_BEGIN(map_bench->pop) { for (size_t i = 0; i < map_bench->nkeys; i++) { uint64_t key; PMEMoid oid; do { key = get_key(&targs->seed, targs->max_key); oid = map_get(map_bench->mapc, map_bench->map, key); } while (!OID_IS_NULL(oid)); if (targs->alloc) oid = pmemobj_tx_alloc(args->dsize, OBJ_TYPE_NUM); else oid = map_bench->root_oid; ret = map_insert(map_bench->mapc, map_bench->map, key, oid); if (ret) break; map_bench->keys[i] = key; } } TX_ONABORT { ret = -1; } TX_END mutex_unlock_nofail(&map_bench->lock); if (!ret) return 0; free(map_bench->keys); return ret; } /* * map_keys_exit -- cleanup of keys array */ static int map_keys_exit(struct benchmark *bench, struct benchmark_args *args) { auto *tree = (struct map_bench *)pmembench_get_priv(bench); free(tree->keys); return 0; } /* * map_remove_init -- init function for map_remove benchmark */ static int map_remove_init(struct benchmark *bench, struct benchmark_args *args) { int ret = map_common_init(bench, args); if (ret) return ret; ret = map_keys_init(bench, args); if (ret) goto err_exit_common; return 0; err_exit_common: map_common_exit(bench, args); return -1; } /* * map_remove_exit -- cleanup function for map_remove benchmark */ static int map_remove_exit(struct benchmark *bench, struct benchmark_args *args) { map_keys_exit(bench, args); return map_common_exit(bench, args); } /* * map_bench_get_init -- init function for map_get benchmark */ static int map_bench_get_init(struct benchmark *bench, struct benchmark_args *args) { int ret = map_common_init(bench, args); if (ret) return ret; ret = map_keys_init(bench, args); if (ret) goto err_exit_common; return 0; err_exit_common: map_common_exit(bench, args); return -1; } /* * map_get_exit -- exit function for map_get benchmark */ static int map_get_exit(struct benchmark *bench, struct benchmark_args *args) { map_keys_exit(bench, args); return map_common_exit(bench, args); } static struct benchmark_clo map_bench_clos[5]; static struct benchmark_info map_insert_info; static struct benchmark_info map_remove_info; static struct benchmark_info map_get_info; CONSTRUCTOR(map_bench_constructor) void map_bench_constructor(void) { map_bench_clos[0].opt_short = 'T'; map_bench_clos[0].opt_long = "type"; map_bench_clos[0].descr = "Type of container " "[ctree|btree|rtree|rbtree|hashmap_tx|hashmap_atomic]"; map_bench_clos[0].off = clo_field_offset(struct map_bench_args, type); map_bench_clos[0].type = CLO_TYPE_STR; map_bench_clos[0].def = "ctree"; map_bench_clos[1].opt_short = 's'; map_bench_clos[1].opt_long = "seed"; map_bench_clos[1].descr = "PRNG seed"; map_bench_clos[1].off = clo_field_offset(struct map_bench_args, seed); map_bench_clos[1].type = CLO_TYPE_UINT; map_bench_clos[1].def = "1"; map_bench_clos[1].type_uint.size = clo_field_size(struct map_bench_args, seed); map_bench_clos[1].type_uint.base = CLO_INT_BASE_DEC; map_bench_clos[1].type_uint.min = 1; map_bench_clos[1].type_uint.max = UINT_MAX; map_bench_clos[2].opt_short = 'M'; map_bench_clos[2].opt_long = "max-key"; map_bench_clos[2].descr = "maximum key (0 means no limit)"; map_bench_clos[2].off = clo_field_offset(struct map_bench_args, max_key); map_bench_clos[2].type = CLO_TYPE_UINT; map_bench_clos[2].def = "0"; map_bench_clos[2].type_uint.size = clo_field_size(struct map_bench_args, seed); map_bench_clos[2].type_uint.base = CLO_INT_BASE_DEC; map_bench_clos[2].type_uint.min = 0; map_bench_clos[2].type_uint.max = UINT64_MAX; map_bench_clos[3].opt_short = 'x'; map_bench_clos[3].opt_long = "external-tx"; map_bench_clos[3].descr = "Use external transaction for all " "operations (works with single " "thread only)"; map_bench_clos[3].off = clo_field_offset(struct map_bench_args, ext_tx); map_bench_clos[3].type = CLO_TYPE_FLAG; map_bench_clos[4].opt_short = 'A'; map_bench_clos[4].opt_long = "alloc"; map_bench_clos[4].descr = "Allocate object of specified size " "when inserting"; map_bench_clos[4].off = clo_field_offset(struct map_bench_args, alloc); map_bench_clos[4].type = CLO_TYPE_FLAG; map_insert_info.name = "map_insert"; map_insert_info.brief = "Inserting to tree map"; map_insert_info.init = map_common_init; map_insert_info.exit = map_common_exit; map_insert_info.multithread = true; map_insert_info.multiops = true; map_insert_info.init_worker = map_insert_init_worker; map_insert_info.free_worker = map_common_free_worker; map_insert_info.operation = map_insert_op; map_insert_info.measure_time = true; map_insert_info.clos = map_bench_clos; map_insert_info.nclos = ARRAY_SIZE(map_bench_clos); map_insert_info.opts_size = sizeof(struct map_bench_args); map_insert_info.rm_file = true; map_insert_info.allow_poolset = true; REGISTER_BENCHMARK(map_insert_info); map_remove_info.name = "map_remove"; map_remove_info.brief = "Removing from tree map"; map_remove_info.init = map_remove_init; map_remove_info.exit = map_remove_exit; map_remove_info.multithread = true; map_remove_info.multiops = true; map_remove_info.init_worker = map_remove_init_worker; map_remove_info.free_worker = map_common_free_worker; map_remove_info.operation = map_remove_op; map_remove_info.measure_time = true; map_remove_info.clos = map_bench_clos; map_remove_info.nclos = ARRAY_SIZE(map_bench_clos); map_remove_info.opts_size = sizeof(struct map_bench_args); map_remove_info.rm_file = true; map_remove_info.allow_poolset = true; REGISTER_BENCHMARK(map_remove_info); map_get_info.name = "map_get"; map_get_info.brief = "Tree lookup"; map_get_info.init = map_bench_get_init; map_get_info.exit = map_get_exit; map_get_info.multithread = true; map_get_info.multiops = true; map_get_info.init_worker = map_bench_get_init_worker; map_get_info.free_worker = map_common_free_worker; map_get_info.operation = map_get_op; map_get_info.measure_time = true; map_get_info.clos = map_bench_clos; map_get_info.nclos = ARRAY_SIZE(map_bench_clos); map_get_info.opts_size = sizeof(struct map_bench_args); map_get_info.rm_file = true; map_get_info.allow_poolset = true; REGISTER_BENCHMARK(map_get_info); } pmdk-1.8/src/benchmarks/benchmark_worker.cpp0000664000000000000000000001307413615011243017751 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * benchmark_worker.cpp -- benchmark_worker module definitions */ #include #include #include "benchmark_worker.hpp" #include "sys_util.h" /* * worker_state_wait_for_transition -- wait for transition from and to * specified states */ static void worker_state_wait_for_transition(struct benchmark_worker *worker, enum benchmark_worker_state state, enum benchmark_worker_state new_state) { while (worker->state == state) os_cond_wait(&worker->cond, &worker->lock); assert(worker->state == new_state); } /* * worker_state_transition -- change worker state from and to specified states */ static void worker_state_transition(struct benchmark_worker *worker, enum benchmark_worker_state old_state, enum benchmark_worker_state new_state) { assert(worker->state == old_state); worker->state = new_state; os_cond_signal(&worker->cond); } /* * thread_func -- (internal) callback for os_thread */ static void * thread_func(void *arg) { assert(arg != nullptr); auto *worker = (struct benchmark_worker *)arg; util_mutex_lock(&worker->lock); worker_state_wait_for_transition(worker, WORKER_STATE_IDLE, WORKER_STATE_INIT); if (worker->init) worker->ret_init = worker->init(worker->bench, worker->args, &worker->info); worker_state_transition(worker, WORKER_STATE_INIT, WORKER_STATE_INITIALIZED); if (worker->ret_init) { util_mutex_unlock(&worker->lock); return nullptr; } worker_state_wait_for_transition(worker, WORKER_STATE_INITIALIZED, WORKER_STATE_RUN); worker->ret = worker->func(worker->bench, &worker->info); worker_state_transition(worker, WORKER_STATE_RUN, WORKER_STATE_END); worker_state_wait_for_transition(worker, WORKER_STATE_END, WORKER_STATE_EXIT); if (worker->exit) worker->exit(worker->bench, worker->args, &worker->info); worker_state_transition(worker, WORKER_STATE_EXIT, WORKER_STATE_DONE); util_mutex_unlock(&worker->lock); return nullptr; } /* * benchmark_worker_alloc -- allocate benchmark worker */ struct benchmark_worker * benchmark_worker_alloc(void) { struct benchmark_worker *w = (struct benchmark_worker *)calloc(1, sizeof(*w)); if (!w) return nullptr; util_mutex_init(&w->lock); if (os_cond_init(&w->cond)) goto err_destroy_mutex; if (os_thread_create(&w->thread, nullptr, thread_func, w)) goto err_destroy_cond; return w; err_destroy_cond: os_cond_destroy(&w->cond); err_destroy_mutex: util_mutex_destroy(&w->lock); free(w); return nullptr; } /* * benchmark_worker_free -- release benchmark worker */ void benchmark_worker_free(struct benchmark_worker *w) { os_thread_join(&w->thread, nullptr); os_cond_destroy(&w->cond); util_mutex_destroy(&w->lock); free(w); } /* * benchmark_worker_init -- call init function for worker */ int benchmark_worker_init(struct benchmark_worker *worker) { util_mutex_lock(&worker->lock); worker_state_transition(worker, WORKER_STATE_IDLE, WORKER_STATE_INIT); worker_state_wait_for_transition(worker, WORKER_STATE_INIT, WORKER_STATE_INITIALIZED); int ret = worker->ret_init; util_mutex_unlock(&worker->lock); return ret; } /* * benchmark_worker_exit -- call exit function for worker */ void benchmark_worker_exit(struct benchmark_worker *worker) { util_mutex_lock(&worker->lock); worker_state_transition(worker, WORKER_STATE_END, WORKER_STATE_EXIT); worker_state_wait_for_transition(worker, WORKER_STATE_EXIT, WORKER_STATE_DONE); util_mutex_unlock(&worker->lock); } /* * benchmark_worker_run -- run benchmark worker */ int benchmark_worker_run(struct benchmark_worker *worker) { int ret = 0; util_mutex_lock(&worker->lock); worker_state_transition(worker, WORKER_STATE_INITIALIZED, WORKER_STATE_RUN); util_mutex_unlock(&worker->lock); return ret; } /* * benchmark_worker_join -- join benchmark worker */ int benchmark_worker_join(struct benchmark_worker *worker) { util_mutex_lock(&worker->lock); worker_state_wait_for_transition(worker, WORKER_STATE_RUN, WORKER_STATE_END); util_mutex_unlock(&worker->lock); return 0; } pmdk-1.8/src/benchmarks/.gitignore0000664000000000000000000000003313615011243015701 0ustar rootrootpmembench *.csv testfile.* pmdk-1.8/src/benchmarks/scenario.hpp0000664000000000000000000000534213615011243016235 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * scenario.hpp -- scenario module declaration */ #include "queue.h" #include struct kv { PMDK_TAILQ_ENTRY(kv) next; char *key; char *value; }; struct scenario { PMDK_TAILQ_ENTRY(scenario) next; PMDK_TAILQ_HEAD(scenariohead, kv) head; char *name; char *benchmark; char *group; }; struct scenarios { PMDK_TAILQ_HEAD(scenarioshead, scenario) head; }; #define FOREACH_SCENARIO(s, ss) PMDK_TAILQ_FOREACH((s), &(ss)->head, next) #define FOREACH_KV(kv, s) PMDK_TAILQ_FOREACH((kv), &(s)->head, next) struct kv *kv_alloc(const char *key, const char *value); void kv_free(struct kv *kv); struct scenario *scenario_alloc(const char *name, const char *bench); void scenario_free(struct scenario *s); void scenario_set_group(struct scenario *s, const char *group); struct scenarios *scenarios_alloc(void); void scenarios_free(struct scenarios *scenarios); struct scenario *scenarios_get_scenario(struct scenarios *ss, const char *name); bool contains_scenarios(int argc, char **argv, struct scenarios *ss); struct scenario *clone_scenario(struct scenario *src_scenario); struct kv *find_kv_in_scenario(const char *key, const struct scenario *scenario); pmdk-1.8/src/benchmarks/pmembench_memset.cfg0000664000000000000000000000367313615011243017717 0ustar rootroot# # pmembench_memset.cfg -- this is an example config file for pmembench # with scenarios for pmem_memset benchmark # # Global parameters [global] group = pmem file = testfile.memset ops-per-thread = 10000 # memset benchmark with variable data sizes # from 1 to 32 bytes [pmem_memset_data_sizes_small] bench = pmem_memset threads = 1 data-size = 1:+1:32 # memset benchmark with variable data sizes # from 64 to 8k bytes # mode random [pmem_memset_data_sizes_rand] bench = pmem_memset threads = 1 data-size = 64:*2:8192 mem-mode = rand # memset benchmark with variable data sizes # from 64 to 8k bytes # offset 10 [pmem_memset_data_sizes_offset] bench = pmem_memset threads = 1 data-size = 64:*2:8192 dest-offset = 10 # memset benchmark with variable data sizes # from 64 to 8k bytes # mode sequential # pmem_memset_persist() [pmem_memset_pmem_memset_persist] bench = pmem_memset threads = 1 data-size = 64:*2:8192 memset = false persist = true mem-mode = seq # memset benchmark with variable data sizes # from 64 to 8k bytes # mode sequential # pmem_memset_nodrain() [pmem_memset_pmem_memset_nodrain] bench = pmem_memset threads = 1 data-size = 64:*2:8192 memset = false persist = false mem-mode = seq # memset benchmark with variable data sizes # from 64 to 8k bytes # mode sequential # memset followed by pmem_persist() [pmem_memset_libc_memset_persist] bench = pmem_memset threads = 1 data-size = 64:*2:8192 memset = true persist = true mem-mode = seq # memset benchmark with variable data sizes # from 64 to 8k bytes # mode sequential # memset followed by pmem_flush() [pmem_memset_libc_memset_flush] bench = pmem_memset threads = 1 data-size = 64:*2:8192 memset = true persist = false mem-mode = seq # memset benchmark with variable data sizes # from 64 to 8k bytes # mode sequential # memset followed by pmem_msync() [pmem_memset_libc_memset_persist] bench = pmem_memset threads = 1 data-size = 64:*2:8192 memset = true persist = false msync = true mem-mode = seq pmdk-1.8/src/benchmarks/pmembench_obj_lanes.cfg0000664000000000000000000000021113615011243020342 0ustar rootroot# Global parameters [global] group = pmemobj file = ./testfile.lane ops-per-thread = 100000 threads = 1:*2:32 [lanes] bench = obj_lanes pmdk-1.8/src/benchmarks/pmembench_atomic_lists.cfg0000664000000000000000000001207513615011243021113 0ustar rootroot# This is an example config file for pmembench # Global parameters [global] group = pmemobj file = testfile.list ops-per-thread = 10000 # obj_insert benchmark # variable number of threads # random type number, position and allocation size [obj_list_insert_threads] bench = obj_insert threads = 1:+1:10 data-size = 512 position = rand type-number = rand min-size = 1 # obj_insert benchmark # use circle queue from queue.h # variable number of threads # random type number, position and allocation size [queue_list_insert_threads] bench = obj_insert threads = 1:+1:10 data-size = 512 position = rand min-size = 1 queue = true # obj_insert benchmark # variable allocation size # insert tail # one type number per thread [obj_list_insert_data_size] bench = obj_insert threads = 1 data-size = 2:*2:8192 position = tail type-number = per-thread # obj_insert benchmark # use circle queue from queue.h # variable allocation size # insert tail # one type number per thread [queue_list_insert_data_size] bench = obj_insert threads = 1 data-size = 2:*2:8192 position = tail type-number = per-thread queue = true # obj_insert benchmark # variable number of threads # insert int the middle # one type-number [obj_list_insert_list_len] bench = obj_insert threads = 1:+1:10 data-size = 512 position = middle type-number = one list-len = 1000 # obj_insert benchmark # use circle queue from queue.h # variable number of threads # insert int the middle # one type-number [queue_list_insert_list_len] bench = obj_insert threads = 1:+1:10 data-size = 512 position = middle type-number = one list-len = 1000 queue = true # obj_insert_new benchmark # variable number of threads # random type number, position and allocation size [obj_list_insert_new_threads] bench = obj_insert_new threads = 1:+1:10 data-size = 512 position = rand type-number = rand min-size = 1 # obj_insert_new benchmark # variable allocation size # insert tail # one type number per thread [obj_list_insert_new_data_size] bench = obj_insert_new threads = 1 data-size = 2:*2:8192 position = tail type-number = per-thread # obj_insert_new benchmark # variable number of threads # insert int the middle # one type-number [obj_list_insert_new_list_len] bench = obj_insert_new threads = 1:+1:10 data-size = 512 position = middle type-number = one list-len = 1000 # obj_remove benchmark # variable number of threads # random type number, position and allocation size [obj_list_remove_threads] bench = obj_remove threads = 1:+1:10 data-size = 512 position = rand type-number = rand min-size = 1 # obj_remove benchmark # use circle queue from queue.h # variable number of threads # random type number, position and allocation size [queue_list_remove_threads] bench = obj_remove threads = 1:+1:10 data-size = 512 position = rand type-number = rand min-size = 1 queue = true # obj_remove benchmark # variable allocation size # insert tail # one type number per thread [obj_list_remove_data_size] bench = obj_remove threads = 1 data-size = 2:*2:8192 position = tail type-number = per-thread # obj_remove benchmark # use circle queue from queue.h # variable allocation size # insert tail # one type number per thread [queue_list_remove_data_size] bench = obj_remove threads = 1 data-size = 2:*2:8192 position = tail type-number = per-thread queue = true # obj_remove benchmark # variable number of threads # insert int the middle # one type-number [obj_list_remove_list_len] bench = obj_remove threads = 1:+1:10 data-size = 512 position = middle type-number = one list-len = 1000 # obj_remove benchmark # use circle queue from queue.h # variable number of threads # insert int the middle # one type-number [queue_list_remove_list_len] bench = obj_remove threads = 1:+1:10 data-size = 512 position = middle type-number = one list-len = 1000 queue = true # obj_remove_free benchmark # variable number of threads # random type number, position and allocation size [obj_list_remove_free_threads] bench = obj_remove_free threads = 1:+1:10 data-size = 512 position = rand type-number = rand min-size = 1 # obj_remove_free benchmark # variable allocation size # insert tail # one type number per thread [obj_list_remove_free_data_size] bench = obj_remove_free threads = 1 data-size = 2:*2:8192 position = tail type-number = per-thread # obj_remove_free benchmark # variable number of threads # insert int the middle # one type-number [obj_list_remove_free_list_len] bench = obj_remove_free threads = 1:+1:10 data-size = 512 position = middle type-number = one list-len = 1000 # obj_move benchmark # variable number of threads # random type number, position and allocation size [obj_list_move_threads] bench = obj_move threads = 1:+1:10 data-size = 512 position = rand type-number = rand min-size = 1 # obj_move benchmark # variable allocation size # insert tail # one type number per thread [obj_list_move_data_size] bench = obj_move threads = 1 data-size = 2:*2:8192 position = tail type-number = per-thread # obj_move benchmark # variable number of threads # insert int the middle # one type-number [obj_list_move_list_len] bench = obj_move threads = 1:+1:10 data-size = 512 position = middle type-number = one list-len = 1000 pmdk-1.8/src/benchmarks/config_reader.hpp0000664000000000000000000000363713615011243017226 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * config_reader.hpp -- config reader module declarations */ struct config_reader; struct config_reader *config_reader_alloc(void); int config_reader_read(struct config_reader *cr, const char *fname); void config_reader_free(struct config_reader *cr); int config_reader_get_scenarios(struct config_reader *cr, struct scenarios **scenarios); pmdk-1.8/src/benchmarks/pmembench_flush.cfg0000664000000000000000000000204113615011243017532 0ustar rootroot# # pmembench_flush.cfg -- this is an example config file for pmembench # with scenarios for pmem_persist & pmem_msync benchmark # # Global parameters [global] group = pmem file = testfile.flush ops-per-thread = 100000 repeats = 3 threads = 1:*2:16 data-size = 64:*2:8192 mode = rand no-warmup = false [flush_noop] bench = pmem_flush operation = noop [flush_persist] bench = pmem_flush operation = persist [flush_persist_4K] bench = pmem_flush operation = persist_4K [flush_persist_2M] bench = pmem_flush operation = persist_2M [flush_persist_4K_msync_0] bench = pmem_flush operation = persist_4K_msync_0 [flush_persist_2M_msync_0] bench = pmem_flush operation = persist_2M_msync_0 [flush_msync] bench = pmem_flush operation = msync [flush_msync_0] bench = pmem_flush operation = msync_0 [flush_msync_async] bench = pmem_flush operation = msync_async [flush_msync_err] bench = pmem_flush operation = msync_err [flush_msync_invalid] bench = pmem_flush operation = msync_invalid [flush_msync_nodirty] bench = pmem_flush operation = msync_nodirty pmdk-1.8/src/benchmarks/benchmark.hpp0000664000000000000000000003037713615011243016372 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * benchmark.hpp -- This file contains interface for creating benchmarks to the * pmembench framework. The _most_ important data structure is * struct benchmark_info which should be properly filled and registered by the * benchmark. Some fields should be filled by meta-data and information about * the benchmark like: name, brief description, supported operation modes etc. * The other group of fields are function callbacks which may be implemented by * the benchmark. Some callbacks are required, others are optional. This is * indicated in the structure description. * * To register a benchmark you can use the special macro * REGISTER_BENCHMARK() which takes static benchmark_info data structure as an * argument. You can also use the pmembench_register() function. Please note * that registering a benchmark should be done at initialization time. You can * achieve this by specifying pmembench_init macro in function attributes: * * static void pmembench_init my_benchmark_init() * { * pmembench_register(&my_benchmark); * } * * However using the REGISTER_BENCHMARK() macro is recommended. */ #ifndef _BENCHMARK_H #define _BENCHMARK_H #include #include #include #include #include #include "benchmark_time.hpp" #include "os.h" #include "util.h" #ifndef ARRAY_SIZE #define ARRAY_SIZE(x) (sizeof(x) / sizeof(x[0])) #endif #define RRAND(max, min) (rand() % ((max) - (min)) + (min)) #define RRAND_R(seed, max, min) (os_rand_r(seed) % ((max) - (min)) + (min)) struct benchmark; /* * benchmark_args - Arguments for benchmark. * * It contains set of common arguments and pointer to benchmark's specific * arguments which are automatically processed by framework according to * clos, nclos and opt_size in benchmark_info structure. */ struct benchmark_args { const char *fname; /* path to test file */ size_t fsize; /* size of test file */ bool is_poolset; /* test file is a poolset */ bool is_dynamic_poolset; /* test file is directory in which benchmark creates reusable files */ mode_t fmode; /* test file's permissions */ unsigned n_threads; /* number of working threads */ size_t n_ops_per_thread; /* number of operations per thread */ bool thread_affinity; /* set worker threads CPU affinity mask */ ssize_t main_affinity; /* main thread affinity */ char *affinity_list; /* set CPU affinity order */ size_t dsize; /* data size */ unsigned seed; /* PRNG seed */ unsigned repeats; /* number of repeats of one scenario */ unsigned min_exe_time; /* minimal execution time */ bool help; /* print help for benchmark */ void *opts; /* benchmark specific arguments */ }; /* * benchmark_results - Benchmark's execution results. */ struct benchmark_results { uint64_t nbytes; /* number of bytes processed */ uint64_t nops; /* number of operations executed */ benchmark_time_t time; /* total execution time */ }; /* * struct results -- statistics for total measurements */ struct results { double min; double max; double avg; double std_dev; double med; }; /* * struct latency -- statistics for latency measurements */ struct latency { uint64_t max; uint64_t min; uint64_t avg; double std_dev; uint64_t pctl50_0p; uint64_t pctl99_0p; uint64_t pctl99_9p; }; /* * struct thread_results -- results of a single thread */ struct thread_results { benchmark_time_t beg; benchmark_time_t end; benchmark_time_t end_op[]; }; /* * struct bench_results -- results of the whole benchmark */ struct bench_results { struct thread_results **thres; }; /* * struct total_results -- results and statistics of the whole benchmark */ struct total_results { size_t nrepeats; size_t nthreads; size_t nops; double nopsps; struct results total; struct latency latency; struct bench_results *res; }; /* * Command Line Option integer value base. */ #define CLO_INT_BASE_NONE 0x0 #define CLO_INT_BASE_DEC 0x1 #define CLO_INT_BASE_HEX 0x2 #define CLO_INT_BASE_OCT 0x4 /* * Command Line Option type. */ enum clo_type { CLO_TYPE_FLAG, CLO_TYPE_STR, CLO_TYPE_INT, CLO_TYPE_UINT, CLO_TYPE_MAX, }; /* * Description of command line option. * * This structure is used to declare command line options by the benchmark * which will be automatically parsed by the framework. * * opt_short : Short option char. If there is no short option write 0. * opt_long : Long option string. * descr : Description of command line option. * off : Offset in data structure in which the value should be stored. * type : Type of command line option. * def : Default value. If set to NULL, this options is required. * ignore_in_res: Do not print in results. * check : Optional callback for checking the command line option value. * type_int : Parameters for signed integer. * type_uint : Parameters for unsigned integer. * type_str : Parameters for string. * * size : Size of integer value. Valid values: 1, 2, 4, 8. * base : Integer base system from which the parsing should be * performed. This field may be used as bit mask by logically * adding different base types. * limit_min : Indicates whether value should be limited by the minimum * value. * limit_max : Indicates whether value should be limited by the maximum * value. * min : Minimum value when limit_min is set. * max : Maximum value when limit_min is set. * * alloc : If set to true the framework should allocate memory for the * value. The memory will be freed by the framework at the end of * execution. Otherwise benchmark must provide valid pointer in * opt_var and max_size parameter must be set properly. * max_size : Maximum size of string. */ struct benchmark_clo { int opt_short; const char *opt_long; enum clo_type type; const char *descr; size_t off; const char *def; bool ignore_in_res; struct { size_t size; int base; int64_t min; int64_t max; } type_int; struct { size_t size; int base; uint64_t min; uint64_t max; } type_uint; int used; }; #define clo_field_offset(s, f) ((size_t) & ((s *)0)->f) #define clo_field_size(s, f) (sizeof(((s *)0)->f)) /* * worker_info - Worker thread's information structure. */ struct worker_info { size_t index; /* index of worker thread */ struct operation_info *opinfo; /* operation info structure */ size_t nops; /* number of operations */ void *priv; /* worker's private data */ benchmark_time_t beg; /* start time */ benchmark_time_t end; /* end time */ }; /* * operation_info - Information about operation. */ struct operation_info { struct worker_info *worker; /* worker's info */ struct benchmark_args *args; /* benchmark arguments */ size_t index; /* operation's index */ benchmark_time_t end; /* operation's end time */ }; /* * struct benchmark_info -- benchmark descriptor * name : Name of benchmark. * brief : Brief description of benchmark. * clos : Command line options which will be automatically parsed by * framework. * nclos : Number of command line options. * opts_size : Size of data structure where the parsed values should be * stored in. * print_help : Callback for printing help message. * pre_init : Function for initialization of the benchmark before parsing * command line arguments. * init : Function for initialization of the benchmark after parsing * command line arguments. * exit : Function for de-initialization of the benchmark. * multithread : Indicates whether the benchmark operation function may be * run in many threads. * multiops : Indicates whether the benchmark operation function may be * run many time in a loop. * measure_time : Indicates whether the benchmark framework should measure the * execution time of operation function. If set to false, the * benchmark must report the execution time by itself. * init_worker : Callback for initialization thread specific data. Invoked in * the worker thread but globally serialized. * operation : Callback function which does the main job of benchmark. * rm_file : Indicates whether the test file should be removed by * framework before the init function will be called. * allow_poolset: Indicates whether benchmark may use poolset files. * If set to false and fname points to a poolset, an error * will be returned. * According to multithread and single_operation flags it may be * invoked in different ways: * +-------------+----------+-------------------------------------+ * | multithread | multiops | description | * +-------------+----------+-------------------------------------+ * | false | false | invoked once, in one thread | * +-------------+----------+-------------------------------------+ * | false | true | invoked many times, in one thread | * +-------------+----------+-------------------------------------+ * | true | false | invoked once, in many threads | * +-------------+----------+-------------------------------------+ * | true | true | invoked many times, in many threads | * +-------------+----------+-------------------------------------+ * */ struct benchmark_info { const char *name; const char *brief; struct benchmark_clo *clos; size_t nclos; size_t opts_size; void (*print_help)(struct benchmark *bench); int (*pre_init)(struct benchmark *bench); int (*init)(struct benchmark *bench, struct benchmark_args *args); int (*exit)(struct benchmark *bench, struct benchmark_args *args); int (*init_worker)(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker); void (*free_worker)(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker); int (*operation)(struct benchmark *bench, struct operation_info *info); void (*print_extra_headers)(); void (*print_extra_values)(struct benchmark *bench, struct benchmark_args *args, struct total_results *res); bool multithread; bool multiops; bool measure_time; bool rm_file; bool allow_poolset; bool print_bandwidth; }; void *pmembench_get_priv(struct benchmark *bench); void pmembench_set_priv(struct benchmark *bench, void *priv); struct benchmark_info *pmembench_get_info(struct benchmark *bench); int pmembench_register(struct benchmark_info *bench_info); #define REGISTER_BENCHMARK(bench) \ if (pmembench_register(&(bench))) { \ fprintf(stderr, "Unable to register benchmark '%s'\n", \ (bench).name); \ } #endif /* _BENCHMARK_H */ pmdk-1.8/src/benchmarks/pmembench_map.cfg0000664000000000000000000000036013615011243017170 0ustar rootroot[global] group = pmemobj file = testfile.map ops-per-thread=1000000 threads=1 type = ctree,btree,rtree,rbtree,hashmap_atomic,hashmap_tx,hashmap_rp [map_insert] bench = map_insert [map_remove] bench = map_remove [map_get] bench = map_get pmdk-1.8/src/benchmarks/poolset_util.hpp0000664000000000000000000000351713615011243017156 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * poolset_util.hpp -- this file provides interface for creating * poolsets of specified size */ #ifndef POOLSET_UTIL_HPP #define POOLSET_UTIL_HPP #include #define POOLSET_PATH "pool.set" int dynamic_poolset_create(const char *path, size_t size); #endif pmdk-1.8/src/benchmarks/pmembench_rpmem.cfg0000664000000000000000000000122513615011243017534 0ustar rootroot# # pmembench_rpmem.cfg -- this is an example config file for pmembench # with scenarios for rpmem benchmark # # Global parameters [global] group = rpmem file = testfile.poolset.rpmem ops-per-thread = 10000 threads = 32 mem-mode = rand-wrap [rpmem_persist_DS8] bench = rpmem_persist persist-relaxed = true data-size = 8 [rpmem_flush_drain_DS512] bench = rpmem_flush_drain flushes-per-drain = 10 flush-relaxed = true data-size = 512 [rpmem_mixed_DS1024] bench = rpmem_mixed # p = rpmem_persist() # r = rpmem_persist() + RPMEM_PERSIST_RELAXED # f = rpmem_flush() # g = rpmem_flush() + RPMEM_FLUSH_RELAXED # d = rpmem_drain() workload = rfgdp data-size = 1024 pmdk-1.8/src/benchmarks/pmembench_obj_locks.cfg0000664000000000000000000001203213615011243020357 0ustar rootroot# Global parameters [global] group = pmemobj file = ./testfile.locks ops-per-thread = 1000000:*10:10000000 # Single mutex benchmarks [single_pmem_locks] bench = obj_locks [single_pmem_locks_uninitialized] bench = obj_locks run_id = true run_id_init_val = 4 [single_system_mutex] bench = obj_locks use_system_threads = true # Multiple mutex benchmarks - 1by1 [multiple_pmem_locks_locked_once_1by1] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 [multiple_pmem_locks_uninitialized_locked_once_1by1] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 run_id_init_val = 4 [multiple_system_mutex_locked_once_1by1] bench = obj_locks ops-per-thread = 1 use_system_threads = true numlocks = 1000000:*10:10000000 [multiple_pmem_locks_1by1] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 [multiple_pmem_locks_uninitialized_1by1] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 run_id = true run_id_init_val = 4 [multiple_system_mutex_1by1] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 use_system_threads = true # Multiple mutex benchmarks - all-lock [multiple_pmem_locks_locked_once_alllock] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 mode = all-lock [multiple_pmem_locks_uninitialized_locked_once_alllock] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 mode = all-lock run_id = true run_id_init_val = 4 [multiple_system_mutex_locked_once_alllock] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 use_system_threads = true mode = all-lock [multiple_pmem_locks_alllock] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 mode = all-lock [multiple_pmem_locks_uninitialized_alllock] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 run_id = true run_id_init_val = 4 mode = all-lock [multiple_system_mutex_alllock] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 use_system_threads = true mode = all-lock #rwlock benchmarks [single_pmem_wrlock] bench = obj_locks bench_type = rwlock [single_pmem_rdlock] bench = obj_locks bench_type = rwlock rdlock = true [single_system_wrlock] bench = obj_locks bench_type = rwlock use_system_threads = true [single_system_rdlock] bench = obj_locks bench_type = rwlock rdlock = true use_system_threads = true [multiple_pmem_wrlock_locked_once_1by1] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 bench_type = rwlock [multiple_pmem_wrlock_uninitialized_locked_once_1by1] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 run_id_init_val = 4 bench_type = rwlock [multiple_system_wrlock_locked_once_1by1] bench = obj_locks ops-per-thread = 1 use_system_threads = true numlocks = 1000000:*10:10000000 bench_type = rwlock [multiple_pmem_wrlock_1by1] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 bench_type = rwlock [multiple_pmem_wrlock_uninitialized_1by1] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 run_id = true run_id_init_val = 4 bench_type = rwlock [multiple_system_wrlock_1by1] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 use_system_threads = true bench_type = rwlock [multiple_pmem_wrlock_locked_once_alllock] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 bench_type = rwlock mode = all-lock [multiple_pmem_wrlock_uninitialized_locked_once_alllock] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 run_id_init_val = 4 bench_type = rwlock mode = all-lock [multiple_system_wrlock_locked_once_alllock] bench = obj_locks ops-per-thread = 1 use_system_threads = true numlocks = 1000000:*10:10000000 bench_type = rwlock mode = all-lock [multiple_pmem_wrlock_alllock] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 bench_type = rwlock mode = all-lock [multiple_pmem_wrlock_uninitialized_alllock] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 run_id = true run_id_init_val = 4 bench_type = rwlock mode = all-lock [multiple_system_wrlock_alllock] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 use_system_threads = true bench_type = rwlock mode = all-lock # volatile mutex - only for testing # it is an alternate implementation of PMEMmutex, which keeps # the system mutex in RAM [single_volatile_mutex] bench = obj_locks bench_type = volatile-mutex [multiple_volatile_mutex_locked_once_1by1] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 bench_type = volatile-mutex [multiple_volatile_mutex_1by1] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 bench_type = volatile-mutex [multiple_volatile_mutex_locked_once_alllock] bench = obj_locks ops-per-thread = 1 numlocks = 1000000:*10:10000000 mode = all-lock bench_type = volatile-mutex [multiple_volatile_mutex_alllock] bench = obj_locks numlocks = 10000:*10:100000 ops-per-thread = 10000:/10:100 mode = all-lock bench_type = volatile-mutex pmdk-1.8/src/benchmarks/pmembench_obj_pmalloc.cfg0000664000000000000000000000107413615011243020677 0ustar rootroot# Global parameters [global] group = pmemobj file = ./testfile.pmalloc ops-per-thread = 1000 data-size = 4096 threads = 1 #Single thread benchmarks [pmalloc_single_thread_size] bench = pmalloc data-size = 64:*2:262144 [pmalloc_single_thread_ops] bench = pmalloc ops-per-thread = 10:*10:100000 [pfree_single_thread_size] bench = pfree data-size = 64:*2:262144 [pfree_single_thread_ops] bench = pfree ops-per-thread = 10:*10:100000 #Multithreaded benchmarks [pmalloc_multi_thread] bench = pmalloc threads = 2:*2:32 [pfree_multi_thread] bench = pfree threads = 2:*2:32 pmdk-1.8/src/benchmarks/pmemobj_atomic_lists.cpp0000664000000000000000000010031413615011243020623 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmemobj_atomic_lists.cpp -- benchmark for pmemobj atomic list API */ #include "benchmark.hpp" #include "file.h" #include "libpmemobj.h" #include "queue.h" #include #include #include #include #include #include #include #define FACTOR 8 #define LAYOUT_NAME "benchmark" struct obj_bench; struct obj_worker; struct element; TOID_DECLARE(struct item, 0); TOID_DECLARE(struct list, 1); typedef size_t (*fn_type_num_t)(size_t worker_idx, size_t op_idx); typedef struct element (*fn_position_t)(struct obj_worker *obj_worker, size_t op_idx); typedef int (*fn_init_t)(struct worker_info *worker, size_t n_elm, size_t list_len); /* * args -- stores command line parsed arguments. */ struct obj_list_args { char *type_num; /* type_number mode - one, per-thread, rand */ char *position; /* position - head, tail, middle, rand */ unsigned list_len; /* initial list length */ bool queue; /* use circle queue from */ bool range; /* use random allocation size */ unsigned min_size; /* minimum random allocation size */ unsigned seed; /* seed value */ }; /* * obj_bench -- stores variables used in benchmark, passed within functions. */ static struct obj_bench { /* handle to persistent pool */ PMEMobjpool *pop; /* pointer to benchmark specific arguments */ struct obj_list_args *args; /* array to store random type_number values */ size_t *random_types; /* * fn_rpositions array stores random functions returning proper element * from list, if position where operation is performed is random. * Possible function which can be in array are: * - position_head, * - position_tail, * - position_middle. */ size_t *alloc_sizes; /* array to store random sizes of each object */ size_t max_len; /* maximum list length */ size_t min_len; /* initial list length */ int type_mode; /* type_number mode */ int position_mode; /* list destination mode */ /* * fn_type_num gets proper function assigned, depending on the * value of the type_mode argument, which returns proper type number for * each persistent object. Possible functions are: * - type_mode_one, * - type_mode_per_thread, * - type_mode_rand. */ fn_type_num_t fn_type_num; /* * fn_position gets proper function assigned, depending on the value * of the position argument, which returns handle to proper element on * the list. Possible functions are: * - position_head, * - position_tail, * - position_middle, * - position_rand. */ fn_position_t fn_position; /* * fn_init gets proper function assigned, depending on the file_io * flag, which allocates objects and initializes proper list. Possible * functions are: * - obj_init_list, * - queue_init_list. */ fn_init_t fn_init; } obj_bench; /* * item -- structure used to connect elements in lists. */ struct item { POBJ_LIST_ENTRY(struct item) field; PMDK_CIRCLEQ_ENTRY(item) fieldq; }; /* * element -- struct contains one item from list with proper type. */ struct element { struct item *itemq; TOID(struct item) itemp; bool before; }; /* * obj_worker -- stores variables used by one thread, concerning one list. */ struct obj_worker { /* head of the pmemobj list */ POBJ_LIST_HEAD(plist, struct item) head; /* head of the circular queue */ PMDK_CIRCLEQ_HEAD(qlist, item) headq; TOID(struct item) * oids; /* persistent pmemobj list elements */ struct item **items; /* volatile elements */ size_t n_elm; /* number of elements in array */ fn_position_t *fn_positions; /* element access functions */ struct element elm; /* pointer to current element */ /* * list_move is a pointer to structure storing variables used by * second list (used only for obj_move benchmark). */ struct obj_worker *list_move; }; /* * position_mode -- list destination type */ enum position_mode { /* object inserted/removed/moved to/from head of list */ POSITION_MODE_HEAD, /* object inserted/removed/moved to/from tail of list */ POSITION_MODE_TAIL, /* * object inserted/removed/moved to/from second element of the list * or to/from head if list length equal to one */ POSITION_MODE_MIDDLE, /* object inserted/removed/moved to/from head, tail or middle */ POSITION_MODE_RAND, POSITION_MODE_UNKNOWN, }; /* * type_mode -- type number type */ enum type_mode { TYPE_MODE_ONE, /* one type number for all of objects */ /* one type number for objects allocated by the same thread */ TYPE_MODE_PER_THREAD, TYPE_MODE_RAND, /* random type number for each object */ TYPE_MODE_UNKNOWN, }; /* * position_head -- returns head of the persistent list or volatile queue. */ static struct element position_head(struct obj_worker *obj_worker, size_t op_idx) { struct element head = {nullptr, OID_NULL, false}; head.before = true; if (!obj_bench.args->queue) head.itemp = POBJ_LIST_FIRST(&obj_worker->head); else head.itemq = PMDK_CIRCLEQ_FIRST(&obj_worker->headq); return head; } /* * position_tail -- returns tail of the persistent list or volatile queue. */ static struct element position_tail(struct obj_worker *obj_worker, size_t op_idx) { struct element tail = {nullptr, OID_NULL, false}; tail.before = false; if (!obj_bench.args->queue) tail.itemp = POBJ_LIST_LAST(&obj_worker->head, field); else tail.itemq = PMDK_CIRCLEQ_LAST(&obj_worker->headq); return tail; } /* * position_middle -- returns second or first element from the persistent list * or volatile queue. */ static struct element position_middle(struct obj_worker *obj_worker, size_t op_idx) { struct element elm = position_head(obj_worker, op_idx); elm.before = true; if (!obj_bench.args->queue) elm.itemp = POBJ_LIST_NEXT(elm.itemp, field); else elm.itemq = PMDK_CIRCLEQ_NEXT(elm.itemq, fieldq); return elm; } /* * position_rand -- returns first, second or last element from the persistent * list or volatile queue based on r_positions array. */ static struct element position_rand(struct obj_worker *obj_worker, size_t op_idx) { struct element elm; elm = obj_worker->fn_positions[op_idx](obj_worker, op_idx); elm.before = true; return elm; } /* * type_mode_one -- always returns 0, as in the mode TYPE_MODE_ONE * all of the persistent objects have the same type_number value. */ static size_t type_mode_one(size_t worker_idx, size_t op_idx) { return 0; } /* * type_mode_per_thread -- always returns the index of the worker, * as in the TYPE_MODE_PER_THREAD the value of the persistent object * type_number is specific to the thread. */ static size_t type_mode_per_thread(size_t worker_idx, size_t op_idx) { return worker_idx; } /* * type_mode_rand -- returns the value from the random_types array assigned * for the specific operation in a specific thread. */ static size_t type_mode_rand(size_t worker_idx, size_t op_idx) { return obj_bench.random_types[op_idx]; } const char *type_num_names[] = {"one", "per-thread", "rand"}; const char *position_names[] = {"head", "tail", "middle", "rand"}; static fn_type_num_t type_num_modes[] = {type_mode_one, type_mode_per_thread, type_mode_rand}; static fn_position_t positions[] = {position_head, position_tail, position_middle, position_rand}; /* function pointers randomly picked when using rand mode */ static fn_position_t rand_positions[] = {position_head, position_tail, position_middle}; /* * get_item -- common part of initial operation of the all benchmarks. * It gets pointer to element on the list where object will * be inserted/removed/moved to/from. */ static void get_item(struct benchmark *bench, struct operation_info *info) { auto *obj_worker = (struct obj_worker *)info->worker->priv; obj_worker->elm = obj_bench.fn_position(obj_worker, info->index); } /* * get_move_item -- special part of initial operation of the obj_move * benchmarks. It gets pointer to element on the list where object will be * inserted/removed/moved to/from. */ static void get_move_item(struct benchmark *bench, struct operation_info *info) { auto *obj_worker = (struct obj_worker *)info->worker->priv; obj_worker->list_move->elm = obj_bench.fn_position(obj_worker->list_move, info->index); get_item(bench, info); } /* * parse_args -- parse command line string argument */ static int parse_args(char *arg, int max, const char **names) { int i = 0; for (; i < max && strcmp(names[i], arg) != 0; i++) ; if (i == max) fprintf(stderr, "Invalid argument\n"); return i; } /* * obj_init_list -- special part of worker initialization, performed only if * queue flag set false. Allocates proper number of items, and inserts proper * part of them to the pmemobj list. */ static int obj_init_list(struct worker_info *worker, size_t n_oids, size_t list_len) { size_t i; auto *obj_worker = (struct obj_worker *)worker->priv; obj_worker->oids = (TOID(struct item) *)calloc(n_oids, sizeof(TOID(struct item))); if (obj_worker->oids == nullptr) { perror("calloc"); return -1; } for (i = 0; i < n_oids; i++) { size_t type_num = obj_bench.fn_type_num(worker->index, i); size_t size = obj_bench.alloc_sizes[i]; auto *tmp = (PMEMoid *)&obj_worker->oids[i]; if (pmemobj_alloc(obj_bench.pop, tmp, size, type_num, nullptr, nullptr) != 0) goto err_oids; } for (i = 0; i < list_len; i++) POBJ_LIST_INSERT_TAIL(obj_bench.pop, &obj_worker->head, obj_worker->oids[i], field); return 0; err_oids: for (; i > 0; i--) POBJ_FREE(&obj_worker->oids[i - 1]); free(obj_worker->oids); return -1; } /* * queue_init_list -- special part of worker initialization, performed only if * queue flag set. Initiates circle queue, allocates proper number of items and * inserts proper part of them to the queue. */ static int queue_init_list(struct worker_info *worker, size_t n_items, size_t list_len) { size_t i; auto *obj_worker = (struct obj_worker *)worker->priv; PMDK_CIRCLEQ_INIT(&obj_worker->headq); obj_worker->items = (struct item **)malloc(n_items * sizeof(struct item *)); if (obj_worker->items == nullptr) { perror("malloc"); return -1; } for (i = 0; i < n_items; i++) { size_t size = obj_bench.alloc_sizes[i]; obj_worker->items[i] = (struct item *)calloc(1, size); if (obj_worker->items[i] == nullptr) { perror("calloc"); goto err; } } for (i = 0; i < list_len; i++) PMDK_CIRCLEQ_INSERT_TAIL(&obj_worker->headq, obj_worker->items[i], fieldq); return 0; err: for (; i > 0; i--) free(obj_worker->items[i - 1]); free(obj_worker->items); return -1; } /* * queue_free_worker_list -- special part for the worker de-initialization when * queue flag is true. Releases items directly from atomic list. */ static void queue_free_worker_list(struct obj_worker *obj_worker) { while (!PMDK_CIRCLEQ_EMPTY(&obj_worker->headq)) { struct item *tmp = PMDK_CIRCLEQ_LAST(&obj_worker->headq); PMDK_CIRCLEQ_REMOVE(&obj_worker->headq, tmp, fieldq); free(tmp); } free(obj_worker->items); } /* * obj_free_worker_list -- special part for the worker de-initialization when * queue flag is false. Releases items directly from atomic list. */ static void obj_free_worker_list(struct obj_worker *obj_worker) { while (!POBJ_LIST_EMPTY(&obj_worker->head)) { TOID(struct item) tmp = POBJ_LIST_FIRST(&obj_worker->head); POBJ_LIST_REMOVE_FREE(obj_bench.pop, &obj_worker->head, tmp, field); } free(obj_worker->oids); } /* * obj_free_worker_items -- special part for the worker de-initialization when * queue flag is false. Releases items used for create pmemobj list. */ static void obj_free_worker_items(struct obj_worker *obj_worker) { for (size_t i = 0; i < obj_worker->n_elm; i++) POBJ_FREE(&obj_worker->oids[i]); free(obj_worker->oids); } /* * queue_free_worker_items -- special part for the worker de-initialization * when queue flag set. Releases used for create circle queue. */ static void queue_free_worker_items(struct obj_worker *obj_worker) { for (size_t i = 0; i < obj_worker->n_elm; i++) free(obj_worker->items[i]); free(obj_worker->items); } /* * random_positions -- allocates array and calculates random values for * defining positions where each operation will be performed. Used only * in POSITION_MODE_RAND */ static fn_position_t * random_positions(void) { auto *positions = (fn_position_t *)calloc(obj_bench.max_len, sizeof(fn_position_t)); if (positions == nullptr) { perror("calloc"); return nullptr; } if (obj_bench.args->seed != 0) srand(obj_bench.args->seed); size_t rmax = ARRAY_SIZE(rand_positions); for (size_t i = 0; i < obj_bench.max_len; i++) { size_t id = RRAND(rmax, 0); positions[i] = rand_positions[id]; } return positions; } /* * rand_values -- allocates array and if range mode calculates random * values as allocation sizes for each object otherwise populates whole array * with max value. Used only when range flag set. */ static size_t * random_values(size_t min, size_t max, size_t n_ops, size_t min_range) { auto *randoms = (size_t *)calloc(n_ops, sizeof(size_t)); if (randoms == nullptr) { perror("calloc"); return nullptr; } for (size_t i = 0; i < n_ops; i++) randoms[i] = max; if (min > min_range) { if (min > max) { fprintf(stderr, "Invalid size\n"); free(randoms); return nullptr; } for (size_t i = 0; i < n_ops; i++) randoms[i] = RRAND(max, min); } return randoms; } /* * queue_insert_op -- main operations of the obj_insert benchmark when queue * flag set to true. */ static int queue_insert_op(struct operation_info *info) { auto *obj_worker = (struct obj_worker *)info->worker->priv; PMDK_CIRCLEQ_INSERT_AFTER( &obj_worker->headq, obj_worker->elm.itemq, obj_worker->items[info->index + obj_bench.min_len], fieldq); return 0; } /* * obj_insert_op -- main operations of the obj_insert benchmark when queue flag * set to false. */ static int obj_insert_op(struct operation_info *info) { auto *obj_worker = (struct obj_worker *)info->worker->priv; POBJ_LIST_INSERT_AFTER( obj_bench.pop, &obj_worker->head, obj_worker->elm.itemp, obj_worker->oids[info->index + obj_bench.min_len], field); return 0; } /* * queue_remove_op -- main operations of the obj_remove benchmark when queue * flag set to true. */ static int queue_remove_op(struct operation_info *info) { auto *obj_worker = (struct obj_worker *)info->worker->priv; PMDK_CIRCLEQ_REMOVE(&obj_worker->headq, obj_worker->elm.itemq, fieldq); return 0; } /* * obj_remove_op -- main operations of the obj_remove benchmark when queue flag * set to false. */ static int obj_remove_op(struct operation_info *info) { auto *obj_worker = (struct obj_worker *)info->worker->priv; POBJ_LIST_REMOVE(obj_bench.pop, &obj_worker->head, obj_worker->elm.itemp, field); return 0; } /* * insert_op -- main operations of the obj_insert benchmark. */ static int insert_op(struct benchmark *bench, struct operation_info *info) { get_item(bench, info); return obj_bench.args->queue ? queue_insert_op(info) : obj_insert_op(info); } /* * obj_insert_new_op -- main operations of the obj_insert_new benchmark. */ static int obj_insert_new_op(struct benchmark *bench, struct operation_info *info) { get_item(bench, info); auto *obj_worker = (struct obj_worker *)info->worker->priv; PMEMoid tmp; size_t size = obj_bench.alloc_sizes[info->index]; size_t type_num = obj_bench.fn_type_num(info->worker->index, info->index); tmp = pmemobj_list_insert_new( obj_bench.pop, offsetof(struct item, field), &obj_worker->head, obj_worker->elm.itemp.oid, obj_worker->elm.before, size, type_num, nullptr, nullptr); if (OID_IS_NULL(tmp)) { perror("pmemobj_list_insert_new"); return -1; } return 0; } /* * remove_op -- main operations of the obj_remove benchmark. */ static int remove_op(struct benchmark *bench, struct operation_info *info) { get_item(bench, info); return obj_bench.args->queue ? queue_remove_op(info) : obj_remove_op(info); } /* * obj_remove_free_op -- main operation of the obj_remove_free benchmark. */ static int obj_remove_free_op(struct benchmark *bench, struct operation_info *info) { get_item(bench, info); auto *obj_worker = (struct obj_worker *)info->worker->priv; POBJ_LIST_REMOVE_FREE(obj_bench.pop, &obj_worker->head, obj_worker->elm.itemp, field); return 0; } /* * obj_move_op -- main operation of the obj_move benchmark. */ static int obj_move_op(struct benchmark *bench, struct operation_info *info) { get_move_item(bench, info); auto *obj_worker = (struct obj_worker *)info->worker->priv; POBJ_LIST_MOVE_ELEMENT_BEFORE(obj_bench.pop, &obj_worker->head, &obj_worker->list_move->head, obj_worker->list_move->elm.itemp, obj_worker->elm.itemp, field, field); return 0; } /* * free_worker -- free common worker state */ static void free_worker(struct obj_worker *obj_worker) { if (obj_bench.position_mode == POSITION_MODE_RAND) free(obj_worker->fn_positions); free(obj_worker); } /* * free_worker_list -- worker de-initialization function for: obj_insert_new, * obj_remove_free, obj_move. Requires releasing objects directly from list. */ static void free_worker_list(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *obj_worker = (struct obj_worker *)worker->priv; obj_bench.args->queue ? queue_free_worker_list(obj_worker) : obj_free_worker_list(obj_worker); free_worker(obj_worker); } /* * obj_free_worker_items -- worker de-initialization function of obj_insert and * obj_remove benchmarks, where deallocation can't be performed directly on the * list and where is possibility of using queue flag. */ static void free_worker_items(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *obj_worker = (struct obj_worker *)worker->priv; auto *obj_args = (struct obj_list_args *)args->opts; obj_args->queue ? queue_free_worker_items(obj_worker) : obj_free_worker_items(obj_worker); free_worker(obj_worker); } /* * obj_move_free_worker -- special part for the worker de-initialization * function of obj_move benchmarks. */ static void obj_move_free_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *obj_worker = (struct obj_worker *)worker->priv; while (!POBJ_LIST_EMPTY(&obj_worker->list_move->head)) POBJ_LIST_REMOVE_FREE( obj_bench.pop, &obj_worker->list_move->head, POBJ_LIST_LAST(&obj_worker->list_move->head, field), field); if (obj_bench.position_mode == POSITION_MODE_RAND) free(obj_worker->list_move->fn_positions); free(obj_worker->list_move); free_worker_list(bench, args, worker); } /* * obj_init_worker -- common part for the worker initialization for: * obj_insert, obj_insert_new, obj_remove obj_remove_free and obj_move. */ static int obj_init_worker(struct worker_info *worker, size_t n_elm, size_t list_len) { auto *obj_worker = (struct obj_worker *)calloc(1, sizeof(struct obj_worker)); if (obj_worker == nullptr) { perror("calloc"); return -1; } worker->priv = obj_worker; obj_worker->n_elm = obj_bench.max_len; obj_worker->list_move = nullptr; if (obj_bench.position_mode == POSITION_MODE_RAND) { obj_worker->fn_positions = random_positions(); if (obj_worker->fn_positions == nullptr) goto err; } if (obj_bench.fn_init(worker, n_elm, list_len) != 0) goto err_positions; return 0; err_positions: free(obj_worker->fn_positions); err: free(obj_worker); return -1; } /* * obj_insert_init_worker -- worker initialization functions of the obj_insert * benchmark. */ static int obj_insert_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { return obj_init_worker(worker, obj_bench.max_len, obj_bench.min_len); } /* * obj_insert_new_init_worker -- worker initialization functions of the * obj_insert_new benchmark. */ static int obj_insert_new_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { return obj_init_worker(worker, obj_bench.min_len, obj_bench.min_len); } /* * obj_remove_init_worker -- worker initialization functions of the obj_remove * and obj_remove_free benchmarks. */ static int obj_remove_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { return obj_init_worker(worker, obj_bench.max_len, obj_bench.max_len); } /* * obj_move_init_worker -- worker initialization functions of the obj_move * benchmark. */ static int obj_move_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { if (obj_init_worker(worker, obj_bench.max_len, obj_bench.max_len) != 0) return -1; auto *obj_worker = (struct obj_worker *)worker->priv; obj_worker->list_move = (struct obj_worker *)calloc(1, sizeof(struct obj_worker)); if (obj_worker->list_move == nullptr) { perror("calloc"); goto free; } size_t i; if (obj_bench.position_mode == POSITION_MODE_RAND) { obj_worker->list_move->fn_positions = random_positions(); if (obj_worker->list_move->fn_positions == nullptr) goto free_list_move; } for (i = 0; i < obj_bench.min_len; i++) { size_t size = obj_bench.alloc_sizes[i]; POBJ_LIST_INSERT_NEW_TAIL(obj_bench.pop, &obj_worker->list_move->head, field, size, nullptr, nullptr); if (TOID_IS_NULL(POBJ_LIST_LAST(&obj_worker->list_move->head, field))) { perror("pmemobj_list_insert_new"); goto free_all; } } return 0; free_all: for (; i > 0; i--) { POBJ_LIST_REMOVE_FREE( obj_bench.pop, &obj_worker->list_move->head, POBJ_LIST_LAST(&obj_worker->list_move->head, field), field); } free(obj_worker->list_move->fn_positions); free_list_move: free(obj_worker->list_move); free: free_worker_list(bench, args, worker); return -1; } /* * obj_init - common part of the benchmark initialization for: obj_insert, * obj_insert_new, obj_remove, obj_remove_free and obj_move used in their init * functions. Parses command line arguments, sets variables and * creates persistent pool. */ static int obj_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); assert(args->opts != nullptr); enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } obj_bench.args = (struct obj_list_args *)args->opts; obj_bench.min_len = obj_bench.args->list_len + 1; obj_bench.max_len = args->n_ops_per_thread + obj_bench.min_len; obj_bench.fn_init = obj_bench.args->queue ? queue_init_list : obj_init_list; /* Decide if use random or state allocation sizes */ size_t obj_size = args->dsize < sizeof(struct item) ? sizeof(struct item) : args->dsize; size_t min_size = obj_bench.args->min_size < sizeof(struct item) ? sizeof(struct item) : obj_bench.args->min_size; obj_bench.alloc_sizes = random_values( min_size, obj_size, obj_bench.max_len, sizeof(struct item)); if (obj_bench.alloc_sizes == nullptr) goto free_random_types; /* Decide where operations will be performed */ obj_bench.position_mode = parse_args(obj_bench.args->position, POSITION_MODE_UNKNOWN, position_names); if (obj_bench.position_mode == POSITION_MODE_UNKNOWN) goto free_all; obj_bench.fn_position = positions[obj_bench.position_mode]; if (!obj_bench.args->queue) { /* Decide what type number will be used */ obj_bench.type_mode = parse_args(obj_bench.args->type_num, TYPE_MODE_UNKNOWN, type_num_names); if (obj_bench.type_mode == TYPE_MODE_UNKNOWN) return -1; obj_bench.fn_type_num = type_num_modes[obj_bench.type_mode]; if (obj_bench.type_mode == TYPE_MODE_RAND) { obj_bench.random_types = random_values( 1, UINT32_MAX, obj_bench.max_len, 0); if (obj_bench.random_types == nullptr) return -1; } /* * Multiplication by FACTOR prevents from out of memory error * as the actual size of the allocated persistent objects * is always larger than requested. */ size_t psize = (args->n_ops_per_thread + obj_bench.min_len + 1) * obj_size * args->n_threads * FACTOR; if (args->is_poolset || type == TYPE_DEVDAX) { if (args->fsize < psize) { fprintf(stderr, "file size too large\n"); goto free_all; } psize = 0; } else if (psize < PMEMOBJ_MIN_POOL) { psize = PMEMOBJ_MIN_POOL; } /* Create pmemobj pool. */ if ((obj_bench.pop = pmemobj_create(args->fname, LAYOUT_NAME, psize, args->fmode)) == nullptr) { perror(pmemobj_errormsg()); goto free_all; } } return 0; free_all: free(obj_bench.alloc_sizes); free_random_types: if (obj_bench.type_mode == TYPE_MODE_RAND) free(obj_bench.random_types); return -1; } /* * obj_exit -- common part for the exit function for: obj_insert, * obj_insert_new, obj_remove, obj_remove_free and obj_move used in their exit * functions. */ static int obj_exit(struct benchmark *bench, struct benchmark_args *args) { if (!obj_bench.args->queue) { pmemobj_close(obj_bench.pop); if (obj_bench.type_mode == TYPE_MODE_RAND) free(obj_bench.random_types); } free(obj_bench.alloc_sizes); return 0; } /* obj_list_clo -- array defining common command line arguments. */ static struct benchmark_clo obj_list_clo[6]; static struct benchmark_info obj_insert; static struct benchmark_info obj_remove; static struct benchmark_info obj_insert_new; static struct benchmark_info obj_remove_free; static struct benchmark_info obj_move; CONSTRUCTOR(pmem_atomic_list_constructor) void pmem_atomic_list_constructor(void) { obj_list_clo[0].opt_short = 'T'; obj_list_clo[0].opt_long = "type-number"; obj_list_clo[0].descr = "Type number mode - one, per-thread, " "rand"; obj_list_clo[0].def = "one"; obj_list_clo[0].off = clo_field_offset(struct obj_list_args, type_num); obj_list_clo[0].type = CLO_TYPE_STR; obj_list_clo[1].opt_short = 'P'; obj_list_clo[1].opt_long = "position"; obj_list_clo[1].descr = "Place where operation will be " "performed - head, tail, rand, middle"; obj_list_clo[1].def = "middle"; obj_list_clo[1].off = clo_field_offset(struct obj_list_args, position); obj_list_clo[1].type = CLO_TYPE_STR; obj_list_clo[2].opt_short = 'l'; obj_list_clo[2].opt_long = "list-len"; obj_list_clo[2].type = CLO_TYPE_UINT; obj_list_clo[2].descr = "Initial list len"; obj_list_clo[2].off = clo_field_offset(struct obj_list_args, list_len); obj_list_clo[2].def = "1"; obj_list_clo[2].type_uint.size = clo_field_size(struct obj_list_args, list_len); obj_list_clo[2].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; obj_list_clo[2].type_uint.min = 1; obj_list_clo[2].type_uint.max = ULONG_MAX; obj_list_clo[3].opt_short = 'm'; obj_list_clo[3].opt_long = "min-size"; obj_list_clo[3].type = CLO_TYPE_UINT; obj_list_clo[3].descr = "Min allocation size"; obj_list_clo[3].off = clo_field_offset(struct obj_list_args, min_size); obj_list_clo[3].def = "0"; obj_list_clo[3].type_uint.size = clo_field_size(struct obj_list_args, min_size); obj_list_clo[3].type_uint.base = CLO_INT_BASE_DEC; obj_list_clo[3].type_uint.min = 0; obj_list_clo[3].type_uint.max = UINT_MAX; obj_list_clo[4].opt_short = 's'; obj_list_clo[4].type_uint.max = INT_MAX; obj_list_clo[4].opt_long = "seed"; obj_list_clo[4].type = CLO_TYPE_UINT; obj_list_clo[4].descr = "Seed value"; obj_list_clo[4].off = clo_field_offset(struct obj_list_args, seed); obj_list_clo[4].def = "0"; obj_list_clo[4].type_uint.size = clo_field_size(struct obj_list_args, seed); obj_list_clo[4].type_uint.base = CLO_INT_BASE_DEC; obj_list_clo[4].type_uint.min = 0; /* * nclos field in benchmark_info structures is decremented to make * queue option available only for obj_isert, obj_remove */ obj_list_clo[5].opt_short = 'q'; obj_list_clo[5].opt_long = "queue"; obj_list_clo[5].descr = "Use circleq from queue.h instead " "pmemobj"; obj_list_clo[5].type = CLO_TYPE_FLAG; obj_list_clo[5].off = clo_field_offset(struct obj_list_args, queue); obj_insert.name = "obj_insert"; obj_insert.brief = "pmemobj_list_insert() benchmark"; obj_insert.init = obj_init; obj_insert.exit = obj_exit; obj_insert.multithread = true; obj_insert.multiops = true; obj_insert.init_worker = obj_insert_init_worker; obj_insert.free_worker = free_worker_items; obj_insert.operation = insert_op; obj_insert.measure_time = true; obj_insert.clos = obj_list_clo; obj_insert.nclos = ARRAY_SIZE(obj_list_clo); obj_insert.opts_size = sizeof(struct obj_list_args); obj_insert.rm_file = true; obj_insert.allow_poolset = true; REGISTER_BENCHMARK(obj_insert); obj_remove.name = "obj_remove"; obj_remove.brief = "pmemobj_list_remove() benchmark " "without freeing element"; obj_remove.init = obj_init; obj_remove.exit = obj_exit; obj_remove.multithread = true; obj_remove.multiops = true; obj_remove.init_worker = obj_remove_init_worker; obj_remove.free_worker = free_worker_items; obj_remove.operation = remove_op; obj_remove.measure_time = true; obj_remove.clos = obj_list_clo; obj_remove.nclos = ARRAY_SIZE(obj_list_clo); obj_remove.opts_size = sizeof(struct obj_list_args); obj_remove.rm_file = true; obj_remove.allow_poolset = true; REGISTER_BENCHMARK(obj_remove); obj_insert_new.name = "obj_insert_new"; obj_insert_new.brief = "pmemobj_list_insert_new() benchmark"; obj_insert_new.init = obj_init; obj_insert_new.exit = obj_exit; obj_insert_new.multithread = true; obj_insert_new.multiops = true; obj_insert_new.init_worker = obj_insert_new_init_worker; obj_insert_new.free_worker = free_worker_list; obj_insert_new.operation = obj_insert_new_op; obj_insert_new.measure_time = true; obj_insert_new.clos = obj_list_clo; obj_insert_new.nclos = ARRAY_SIZE(obj_list_clo) - 1; obj_insert_new.opts_size = sizeof(struct obj_list_args); obj_insert_new.rm_file = true; obj_insert_new.allow_poolset = true; REGISTER_BENCHMARK(obj_insert_new); obj_remove_free.name = "obj_remove_free"; obj_remove_free.brief = "pmemobj_list_remove() benchmark " "with freeing element"; obj_remove_free.init = obj_init; obj_remove_free.exit = obj_exit; obj_remove_free.multithread = true; obj_remove_free.multiops = true; obj_remove_free.init_worker = obj_remove_init_worker; obj_remove_free.free_worker = free_worker_list; obj_remove_free.operation = obj_remove_free_op; obj_remove_free.measure_time = true; obj_remove_free.clos = obj_list_clo; obj_remove_free.nclos = ARRAY_SIZE(obj_list_clo) - 1; obj_remove_free.opts_size = sizeof(struct obj_list_args); obj_remove_free.rm_file = true; obj_remove_free.allow_poolset = true; REGISTER_BENCHMARK(obj_remove_free); obj_move.name = "obj_move"; obj_move.brief = "pmemobj_list_move() benchmark"; obj_move.init = obj_init; obj_move.exit = obj_exit; obj_move.multithread = true; obj_move.multiops = true; obj_move.init_worker = obj_move_init_worker; obj_move.free_worker = obj_move_free_worker; obj_move.operation = obj_move_op; obj_move.measure_time = true; obj_move.clos = obj_list_clo; obj_move.nclos = ARRAY_SIZE(obj_list_clo) - 1; obj_move.opts_size = sizeof(struct obj_list_args); obj_move.rm_file = true; obj_move.allow_poolset = true; REGISTER_BENCHMARK(obj_move); } pmdk-1.8/src/benchmarks/pmembench_obj_persist.cfg0000664000000000000000000000073513615011243020744 0ustar rootroot# # pmembench_obj_persist.cfg -- this is an example config file for pmembench # with scenarios for pmemobj_persist() benchmark # # Global parameters [global] group = obj_persist file = testfile.obj_persist ops-per-thread = 10000 [pmemobj_persist_DS8] bench = pmemobj_persist threads = 1:+1:32 data-size = 8 [pmemobj_persist_DS512] bench = pmemobj_persist threads = 1:+1:32 data-size = 512 [pmemobj_persist_DS1024] bench = pmemobj_persist threads = 1:+1:32 data-size = 1024 pmdk-1.8/src/benchmarks/pmembench.cpp0000664000000000000000000012317213615011243016365 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmembench.cpp -- main source file for benchmark framework */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include "benchmark.hpp" #include "benchmark_worker.hpp" #include "clo.hpp" #include "clo_vec.hpp" #include "config_reader.hpp" #include "file.h" #include "libpmempool.h" #include "mmap.h" #include "os.h" #include "os_thread.h" #include "queue.h" #include "scenario.hpp" #include "set.h" #include "util.h" #ifndef _WIN32 #include "rpmem_common.h" #include "rpmem_ssh.h" #include "rpmem_util.h" #endif /* average time required to get a current time from the system */ unsigned long long Get_time_avg; #define MIN_EXE_TIME_E 0.5 /* * struct pmembench -- main context */ struct pmembench { int argc; char **argv; struct scenario *scenario; struct clo_vec *clovec; bool override_clos; }; /* * struct benchmark -- benchmark's context */ struct benchmark { PMDK_LIST_ENTRY(benchmark) next; struct benchmark_info *info; void *priv; struct benchmark_clo *clos; size_t nclos; size_t args_size; }; /* * struct bench_list -- list of available benchmarks */ struct bench_list { PMDK_LIST_HEAD(benchmarks_head, benchmark) head; bool initialized; }; /* * struct benchmark_opts -- arguments for pmembench */ struct benchmark_opts { bool help; bool version; const char *file_name; }; static struct version_s { unsigned major; unsigned minor; } version = {1, 0}; /* benchmarks list initialization */ static struct bench_list benchmarks; /* common arguments for benchmarks */ static struct benchmark_clo pmembench_clos[13]; /* list of arguments for pmembench */ static struct benchmark_clo pmembench_opts[2]; CONSTRUCTOR(pmembench_constructor) void pmembench_constructor(void) { pmembench_opts[0].opt_short = 'h'; pmembench_opts[0].opt_long = "help"; pmembench_opts[0].descr = "Print help"; pmembench_opts[0].type = CLO_TYPE_FLAG; pmembench_opts[0].off = clo_field_offset(struct benchmark_opts, help); pmembench_opts[0].ignore_in_res = true; pmembench_opts[1].opt_short = 'v'; pmembench_opts[1].opt_long = "version"; pmembench_opts[1].descr = "Print version"; pmembench_opts[1].type = CLO_TYPE_FLAG; pmembench_opts[1].off = clo_field_offset(struct benchmark_opts, version); pmembench_opts[1].ignore_in_res = true; pmembench_clos[0].opt_short = 'h'; pmembench_clos[0].opt_long = "help"; pmembench_clos[0].descr = "Print help for single benchmark"; pmembench_clos[0].type = CLO_TYPE_FLAG; pmembench_clos[0].off = clo_field_offset(struct benchmark_args, help); pmembench_clos[0].ignore_in_res = true; pmembench_clos[1].opt_short = 't'; pmembench_clos[1].opt_long = "threads"; pmembench_clos[1].type = CLO_TYPE_UINT; pmembench_clos[1].descr = "Number of working threads"; pmembench_clos[1].off = clo_field_offset(struct benchmark_args, n_threads); pmembench_clos[1].def = "1"; pmembench_clos[1].type_uint.size = clo_field_size(struct benchmark_args, n_threads); pmembench_clos[1].type_uint.base = CLO_INT_BASE_DEC; pmembench_clos[1].type_uint.min = 1; pmembench_clos[1].type_uint.max = UINT_MAX; pmembench_clos[2].opt_short = 'n'; pmembench_clos[2].opt_long = "ops-per-thread"; pmembench_clos[2].type = CLO_TYPE_UINT; pmembench_clos[2].descr = "Number of operations per thread"; pmembench_clos[2].off = clo_field_offset(struct benchmark_args, n_ops_per_thread); pmembench_clos[2].def = "1"; pmembench_clos[2].type_uint.size = clo_field_size(struct benchmark_args, n_ops_per_thread); pmembench_clos[2].type_uint.base = CLO_INT_BASE_DEC; pmembench_clos[2].type_uint.min = 1; pmembench_clos[2].type_uint.max = ULLONG_MAX; pmembench_clos[3].opt_short = 'd'; pmembench_clos[3].opt_long = "data-size"; pmembench_clos[3].type = CLO_TYPE_UINT; pmembench_clos[3].descr = "IO data size"; pmembench_clos[3].off = clo_field_offset(struct benchmark_args, dsize); pmembench_clos[3].def = "1"; pmembench_clos[3].type_uint.size = clo_field_size(struct benchmark_args, dsize); pmembench_clos[3].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; pmembench_clos[3].type_uint.min = 1; pmembench_clos[3].type_uint.max = ULONG_MAX; pmembench_clos[4].opt_short = 'f'; pmembench_clos[4].opt_long = "file"; pmembench_clos[4].type = CLO_TYPE_STR; pmembench_clos[4].descr = "File name"; pmembench_clos[4].off = clo_field_offset(struct benchmark_args, fname); pmembench_clos[4].def = "/mnt/pmem/testfile"; pmembench_clos[4].ignore_in_res = true; pmembench_clos[5].opt_short = 'm'; pmembench_clos[5].opt_long = "fmode"; pmembench_clos[5].type = CLO_TYPE_UINT; pmembench_clos[5].descr = "File mode"; pmembench_clos[5].off = clo_field_offset(struct benchmark_args, fmode); pmembench_clos[5].def = "0666"; pmembench_clos[5].ignore_in_res = true; pmembench_clos[5].type_uint.size = clo_field_size(struct benchmark_args, fmode); pmembench_clos[5].type_uint.base = CLO_INT_BASE_OCT; pmembench_clos[5].type_uint.min = 0; pmembench_clos[5].type_uint.max = ULONG_MAX; pmembench_clos[6].opt_short = 's'; pmembench_clos[6].opt_long = "seed"; pmembench_clos[6].type = CLO_TYPE_UINT; pmembench_clos[6].descr = "PRNG seed"; pmembench_clos[6].off = clo_field_offset(struct benchmark_args, seed); pmembench_clos[6].def = "0"; pmembench_clos[6].type_uint.size = clo_field_size(struct benchmark_args, seed); pmembench_clos[6].type_uint.base = CLO_INT_BASE_DEC; pmembench_clos[6].type_uint.min = 0; pmembench_clos[6].type_uint.max = ~0; pmembench_clos[7].opt_short = 'r'; pmembench_clos[7].opt_long = "repeats"; pmembench_clos[7].type = CLO_TYPE_UINT; pmembench_clos[7].descr = "Number of repeats of scenario"; pmembench_clos[7].off = clo_field_offset(struct benchmark_args, repeats); pmembench_clos[7].def = "1"; pmembench_clos[7].type_uint.size = clo_field_size(struct benchmark_args, repeats); pmembench_clos[7].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; pmembench_clos[7].type_uint.min = 1; pmembench_clos[7].type_uint.max = ULONG_MAX; pmembench_clos[8].opt_short = 'F'; pmembench_clos[8].opt_long = "thread-affinity"; pmembench_clos[8].descr = "Set worker threads CPU affinity mask"; pmembench_clos[8].type = CLO_TYPE_FLAG; pmembench_clos[8].off = clo_field_offset(struct benchmark_args, thread_affinity); pmembench_clos[8].def = "false"; /* * XXX: add link to blog post about optimal affinity * when it will be done */ pmembench_clos[9].opt_short = 'I'; pmembench_clos[9].opt_long = "affinity-list"; pmembench_clos[9].descr = "Set affinity mask as a list of CPUs separated by semicolon"; pmembench_clos[9].type = CLO_TYPE_STR; pmembench_clos[9].off = clo_field_offset(struct benchmark_args, affinity_list); pmembench_clos[9].def = ""; pmembench_clos[9].ignore_in_res = true; pmembench_clos[10].opt_long = "main-affinity"; pmembench_clos[10].descr = "Set affinity for main thread"; pmembench_clos[10].type = CLO_TYPE_INT; pmembench_clos[10].off = clo_field_offset(struct benchmark_args, main_affinity); pmembench_clos[10].def = "-1"; pmembench_clos[10].ignore_in_res = false; pmembench_clos[10].type_int.size = clo_field_size(struct benchmark_args, main_affinity); pmembench_clos[10].type_int.base = CLO_INT_BASE_DEC; pmembench_clos[10].type_int.min = (-1); pmembench_clos[10].type_int.max = LONG_MAX; pmembench_clos[11].opt_short = 'e'; pmembench_clos[11].opt_long = "min-exe-time"; pmembench_clos[11].type = CLO_TYPE_UINT; pmembench_clos[11].descr = "Minimal execution time in seconds"; pmembench_clos[11].off = clo_field_offset(struct benchmark_args, min_exe_time); pmembench_clos[11].def = "0"; pmembench_clos[11].type_uint.size = clo_field_size(struct benchmark_args, min_exe_time); pmembench_clos[11].type_uint.base = CLO_INT_BASE_DEC; pmembench_clos[11].type_uint.min = 0; pmembench_clos[11].type_uint.max = ULONG_MAX; pmembench_clos[12].opt_short = 'p'; pmembench_clos[12].opt_long = "dynamic-poolset"; pmembench_clos[12].type = CLO_TYPE_FLAG; pmembench_clos[12].descr = "Allow benchmark to create poolset and reuse files"; pmembench_clos[12].off = clo_field_offset(struct benchmark_args, is_dynamic_poolset); pmembench_clos[12].ignore_in_res = true; } /* * pmembench_get_priv -- return private structure of benchmark */ void * pmembench_get_priv(struct benchmark *bench) { return bench->priv; } /* * pmembench_set_priv -- set private structure of benchmark */ void pmembench_set_priv(struct benchmark *bench, void *priv) { bench->priv = priv; } /* * pmembench_register -- register benchmark */ int pmembench_register(struct benchmark_info *bench_info) { assert(bench_info->name && bench_info->brief); struct benchmark *bench = (struct benchmark *)calloc(1, sizeof(*bench)); assert(bench != nullptr); bench->info = bench_info; if (!benchmarks.initialized) { PMDK_LIST_INIT(&benchmarks.head); benchmarks.initialized = true; } PMDK_LIST_INSERT_HEAD(&benchmarks.head, bench, next); return 0; } /* * pmembench_get_info -- return structure with information about benchmark */ struct benchmark_info * pmembench_get_info(struct benchmark *bench) { return bench->info; } /* * pmembench_release_clos -- release CLO structure */ static void pmembench_release_clos(struct benchmark *bench) { free(bench->clos); } /* * pmembench_merge_clos -- merge benchmark's CLOs with common CLOs */ static void pmembench_merge_clos(struct benchmark *bench) { size_t size = sizeof(struct benchmark_args); size_t pb_nclos = ARRAY_SIZE(pmembench_clos); size_t nclos = pb_nclos; size_t i; if (bench->info->clos) { size += bench->info->opts_size; nclos += bench->info->nclos; } auto *clos = (struct benchmark_clo *)calloc( nclos, sizeof(struct benchmark_clo)); assert(clos != nullptr); memcpy(clos, pmembench_clos, pb_nclos * sizeof(struct benchmark_clo)); if (bench->info->clos) { memcpy(&clos[pb_nclos], bench->info->clos, bench->info->nclos * sizeof(struct benchmark_clo)); for (i = 0; i < bench->info->nclos; i++) { clos[pb_nclos + i].off += sizeof(struct benchmark_args); } } bench->clos = clos; bench->nclos = nclos; bench->args_size = size; } /* * pmembench_run_worker -- run worker with benchmark operation */ static int pmembench_run_worker(struct benchmark *bench, struct worker_info *winfo) { benchmark_time_get(&winfo->beg); for (size_t i = 0; i < winfo->nops; i++) { if (bench->info->operation(bench, &winfo->opinfo[i])) return -1; benchmark_time_get(&winfo->opinfo[i].end); } benchmark_time_get(&winfo->end); return 0; } /* * pmembench_print_header -- print header of benchmark's results */ static void pmembench_print_header(struct pmembench *pb, struct benchmark *bench, struct clo_vec *clovec) { if (pb->scenario) { printf("%s: %s [%" PRIu64 "]%s%s%s\n", pb->scenario->name, bench->info->name, clovec->nargs, pb->scenario->group ? " [group: " : "", pb->scenario->group ? pb->scenario->group : "", pb->scenario->group ? "]" : ""); } else { printf("%s [%" PRIu64 "]\n", bench->info->name, clovec->nargs); } printf("total-avg[sec];" "ops-per-second[1/sec];" "total-max[sec];" "total-min[sec];" "total-median[sec];" "total-std-dev[sec];" "latency-avg[nsec];" "latency-min[nsec];" "latency-max[nsec];" "latency-std-dev[nsec];" "latency-pctl-50.0%%[nsec];" "latency-pctl-99.0%%[nsec];" "latency-pctl-99.9%%[nsec]"); size_t i; for (i = 0; i < bench->nclos; i++) { if (!bench->clos[i].ignore_in_res) { printf(";%s", bench->clos[i].opt_long); } } if (bench->info->print_bandwidth) printf(";bandwidth[MiB/s]"); if (bench->info->print_extra_headers) bench->info->print_extra_headers(); printf("\n"); } /* * pmembench_print_results -- print benchmark's results */ static void pmembench_print_results(struct benchmark *bench, struct benchmark_args *args, struct total_results *res) { printf("%f;%f;%f;%f;%f;%f;%" PRIu64 ";%" PRIu64 ";%" PRIu64 ";%f;%" PRIu64 ";%" PRIu64 ";%" PRIu64, res->total.avg, res->nopsps, res->total.max, res->total.min, res->total.med, res->total.std_dev, res->latency.avg, res->latency.min, res->latency.max, res->latency.std_dev, res->latency.pctl50_0p, res->latency.pctl99_0p, res->latency.pctl99_9p); size_t i; for (i = 0; i < bench->nclos; i++) { if (!bench->clos[i].ignore_in_res) printf(";%s", benchmark_clo_str(&bench->clos[i], args, bench->args_size)); } if (bench->info->print_bandwidth) printf(";%f", res->nopsps * args->dsize / 1024 / 1024); if (bench->info->print_extra_values) bench->info->print_extra_values(bench, args, res); printf("\n"); } /* * pmembench_parse_clos -- parse command line arguments for benchmark */ static int pmembench_parse_clo(struct pmembench *pb, struct benchmark *bench, struct clo_vec *clovec) { if (!pb->scenario) { return benchmark_clo_parse(pb->argc, pb->argv, bench->clos, bench->nclos, clovec); } if (pb->override_clos) { /* * Use only ARRAY_SIZE(pmembench_clos) clos - these are the * general clos and are placed at the beginning of the * clos array. */ int ret = benchmark_override_clos_in_scenario( pb->scenario, pb->argc, pb->argv, bench->clos, ARRAY_SIZE(pmembench_clos)); /* reset for the next benchmark in the config file */ optind = 1; if (ret) return ret; } return benchmark_clo_parse_scenario(pb->scenario, bench->clos, bench->nclos, clovec); } /* * pmembench_parse_affinity -- parse affinity list */ static int pmembench_parse_affinity(const char *list, char **saveptr) { char *str = nullptr; char *end; int cpu = 0; if (*saveptr) { str = strtok(nullptr, ";"); if (str == nullptr) { /* end of list - we have to start over */ free(*saveptr); *saveptr = nullptr; } } if (!*saveptr) { *saveptr = strdup(list); if (*saveptr == nullptr) { perror("strdup"); return -1; } str = strtok(*saveptr, ";"); if (str == nullptr) goto err; } if ((str == nullptr) || (*str == '\0')) goto err; cpu = strtol(str, &end, 10); if (*end != '\0') goto err; return cpu; err: errno = EINVAL; perror("pmembench_parse_affinity"); free(*saveptr); *saveptr = nullptr; return -1; } /* * pmembench_init_workers -- init benchmark's workers */ static int pmembench_init_workers(struct benchmark_worker **workers, struct benchmark *bench, struct benchmark_args *args) { unsigned i; int ncpus = 0; char *saveptr = nullptr; int ret = 0; if (args->thread_affinity) { ncpus = sysconf(_SC_NPROCESSORS_ONLN); if (ncpus <= 0) return -1; } for (i = 0; i < args->n_threads; i++) { workers[i] = benchmark_worker_alloc(); if (args->thread_affinity) { int cpu; os_cpu_set_t cpuset; if (args->affinity_list && *args->affinity_list != '\0') { cpu = pmembench_parse_affinity( args->affinity_list, &saveptr); if (cpu == -1) { ret = -1; goto end; } } else { cpu = (int)i; } assert(ncpus > 0); cpu %= ncpus; os_cpu_zero(&cpuset); os_cpu_set(cpu, &cpuset); errno = os_thread_setaffinity_np(&workers[i]->thread, sizeof(os_cpu_set_t), &cpuset); if (errno) { perror("os_thread_setaffinity_np"); ret = -1; goto end; } } workers[i]->info.index = i; workers[i]->info.nops = args->n_ops_per_thread; workers[i]->info.opinfo = (struct operation_info *)calloc( args->n_ops_per_thread, sizeof(struct operation_info)); size_t j; for (j = 0; j < args->n_ops_per_thread; j++) { workers[i]->info.opinfo[j].worker = &workers[i]->info; workers[i]->info.opinfo[j].args = args; workers[i]->info.opinfo[j].index = j; } workers[i]->bench = bench; workers[i]->args = args; workers[i]->func = pmembench_run_worker; workers[i]->init = bench->info->init_worker; workers[i]->exit = bench->info->free_worker; if (benchmark_worker_init(workers[i])) { fprintf(stderr, "thread number %u initialization failed\n", i); ret = -1; goto end; } } end: free(saveptr); return ret; } /* * results_store -- store results of a single repeat */ static void results_store(struct bench_results *res, struct benchmark_worker **workers, unsigned nthreads, size_t nops) { for (unsigned i = 0; i < nthreads; i++) { res->thres[i]->beg = workers[i]->info.beg; res->thres[i]->end = workers[i]->info.end; for (size_t j = 0; j < nops; j++) { res->thres[i]->end_op[j] = workers[i]->info.opinfo[j].end; } } } /* * compare_time -- compare time values */ static int compare_time(const void *p1, const void *p2) { const auto *t1 = (const benchmark_time_t *)p1; const auto *t2 = (const benchmark_time_t *)p2; return benchmark_time_compare(t1, t2); } /* * compare_doubles -- comparing function used for sorting */ static int compare_doubles(const void *a1, const void *b1) { const auto *a = (const double *)a1; const auto *b = (const double *)b1; return (*a > *b) - (*a < *b); } /* * compare_uint64t -- comparing function used for sorting */ static int compare_uint64t(const void *a1, const void *b1) { const auto *a = (const uint64_t *)a1; const auto *b = (const uint64_t *)b1; return (*a > *b) - (*a < *b); } /* * results_alloc -- prepare structure to store all benchmark results */ static struct total_results * results_alloc(struct benchmark_args *args) { struct total_results *total = (struct total_results *)malloc(sizeof(*total)); assert(total != nullptr); total->nrepeats = args->repeats; total->nthreads = args->n_threads; total->nops = args->n_ops_per_thread; total->res = (struct bench_results *)malloc(args->repeats * sizeof(*total->res)); assert(total->res != nullptr); for (size_t i = 0; i < args->repeats; i++) { struct bench_results *res = &total->res[i]; assert(args->n_threads != 0); res->thres = (struct thread_results **)malloc( args->n_threads * sizeof(*res->thres)); assert(res->thres != nullptr); for (size_t j = 0; j < args->n_threads; j++) { res->thres[j] = (struct thread_results *)malloc( sizeof(*res->thres[j]) + args->n_ops_per_thread * sizeof(benchmark_time_t)); assert(res->thres[j] != nullptr); } } return total; } /* * results_free -- release results structure */ static void results_free(struct total_results *total) { for (size_t i = 0; i < total->nrepeats; i++) { for (size_t j = 0; j < total->nthreads; j++) free(total->res[i].thres[j]); free(total->res[i].thres); } free(total->res); free(total); } /* * get_total_results -- return results of all repeats of scenario */ static void get_total_results(struct total_results *tres) { assert(tres->nrepeats != 0); assert(tres->nthreads != 0); assert(tres->nops != 0); /* reset results */ memset(&tres->total, 0, sizeof(tres->total)); memset(&tres->latency, 0, sizeof(tres->latency)); tres->total.min = DBL_MAX; tres->total.max = DBL_MIN; tres->latency.min = UINT64_MAX; tres->latency.max = 0; /* allocate helper arrays */ benchmark_time_t *tbeg = (benchmark_time_t *)malloc(tres->nthreads * sizeof(*tbeg)); assert(tbeg != nullptr); benchmark_time_t *tend = (benchmark_time_t *)malloc(tres->nthreads * sizeof(*tend)); assert(tend != nullptr); auto *totals = (double *)malloc(tres->nrepeats * sizeof(double)); assert(totals != nullptr); /* estimate total penalty of getting time from the system */ benchmark_time_t Tget; unsigned long long nsecs = tres->nops * Get_time_avg; benchmark_time_set(&Tget, nsecs); for (size_t i = 0; i < tres->nrepeats; i++) { struct bench_results *res = &tres->res[i]; /* get start and end timestamps of each worker */ for (size_t j = 0; j < tres->nthreads; j++) { tbeg[j] = res->thres[j]->beg; tend[j] = res->thres[j]->end; } /* sort start and end timestamps */ qsort(tbeg, tres->nthreads, sizeof(benchmark_time_t), compare_time); qsort(tend, tres->nthreads, sizeof(benchmark_time_t), compare_time); /* calculating time interval between start and end time */ benchmark_time_t Tbeg = tbeg[0]; benchmark_time_t Tend = tend[tres->nthreads - 1]; benchmark_time_t Ttot_ove; benchmark_time_diff(&Ttot_ove, &Tbeg, &Tend); /* * subtract time used for getting the current time from the * system */ benchmark_time_t Ttot; benchmark_time_diff(&Ttot, &Tget, &Ttot_ove); double Stot = benchmark_time_get_secs(&Ttot); if (Stot > tres->total.max) tres->total.max = Stot; if (Stot < tres->total.min) tres->total.min = Stot; tres->total.avg += Stot; totals[i] = Stot; } /* median */ qsort(totals, tres->nrepeats, sizeof(double), compare_doubles); if (tres->nrepeats % 2) { tres->total.med = totals[tres->nrepeats / 2]; } else { double m1 = totals[tres->nrepeats / 2]; double m2 = totals[tres->nrepeats / 2 - 1]; tres->total.med = (m1 + m2) / 2.0; } /* total average time */ tres->total.avg /= (double)tres->nrepeats; /* number of operations per second */ tres->nopsps = (double)tres->nops * (double)tres->nthreads / tres->total.avg; /* std deviation of total time */ for (size_t i = 0; i < tres->nrepeats; i++) { double dev = (totals[i] - tres->total.avg); dev *= dev; tres->total.std_dev += dev; } tres->total.std_dev = sqrt(tres->total.std_dev / tres->nrepeats); /* latency */ for (size_t i = 0; i < tres->nrepeats; i++) { struct bench_results *res = &tres->res[i]; for (size_t j = 0; j < tres->nthreads; j++) { struct thread_results *thres = res->thres[j]; benchmark_time_t *beg = &thres->beg; for (size_t o = 0; o < tres->nops; o++) { benchmark_time_t lat; benchmark_time_diff(&lat, beg, &thres->end_op[o]); uint64_t nsecs = benchmark_time_get_nsecs(&lat); /* min, max latency */ if (nsecs > tres->latency.max) tres->latency.max = nsecs; if (nsecs < tres->latency.min) tres->latency.min = nsecs; tres->latency.avg += nsecs; beg = &thres->end_op[o]; } } } /* average latency */ size_t count = tres->nrepeats * tres->nthreads * tres->nops; assert(count > 0); tres->latency.avg /= count; auto *ntotals = (uint64_t *)calloc(count, sizeof(uint64_t)); assert(ntotals != nullptr); count = 0; /* std deviation of latency and percentiles */ for (size_t i = 0; i < tres->nrepeats; i++) { struct bench_results *res = &tres->res[i]; for (size_t j = 0; j < tres->nthreads; j++) { struct thread_results *thres = res->thres[j]; benchmark_time_t *beg = &thres->beg; for (size_t o = 0; o < tres->nops; o++) { benchmark_time_t lat; benchmark_time_diff(&lat, beg, &thres->end_op[o]); uint64_t nsecs = benchmark_time_get_nsecs(&lat); uint64_t dev = (nsecs - tres->latency.avg); dev *= dev; tres->latency.std_dev += dev; beg = &thres->end_op[o]; ntotals[count] = nsecs; ++count; } } } tres->latency.std_dev = sqrt(tres->latency.std_dev / count); /* find 50%, 99.0% and 99.9% percentiles */ qsort(ntotals, count, sizeof(uint64_t), compare_uint64t); uint64_t p50_0 = count * 50 / 100; uint64_t p99_0 = count * 99 / 100; uint64_t p99_9 = count * 999 / 1000; tres->latency.pctl50_0p = ntotals[p50_0]; tres->latency.pctl99_0p = ntotals[p99_0]; tres->latency.pctl99_9p = ntotals[p99_9]; free(ntotals); free(totals); free(tend); free(tbeg); } /* * pmembench_print_args -- print arguments for one benchmark */ static void pmembench_print_args(struct benchmark_clo *clos, size_t nclos) { struct benchmark_clo clo; for (size_t i = 0; i < nclos; i++) { clo = clos[i]; if (clo.opt_short != 0) printf("\t-%c,", clo.opt_short); else printf("\t"); printf("\t--%-15s\t\t%s", clo.opt_long, clo.descr); if (clo.type != CLO_TYPE_FLAG) printf(" [default: %s]", clo.def); if (clo.type == CLO_TYPE_INT) { if (clo.type_int.min != LONG_MIN) printf(" [min: %" PRId64 "]", clo.type_int.min); if (clo.type_int.max != LONG_MAX) printf(" [max: %" PRId64 "]", clo.type_int.max); } else if (clo.type == CLO_TYPE_UINT) { if (clo.type_uint.min != 0) printf(" [min: %" PRIu64 "]", clo.type_uint.min); if (clo.type_uint.max != ULONG_MAX) printf(" [max: %" PRIu64 "]", clo.type_uint.max); } printf("\n"); } } /* * pmembench_print_help_single -- prints help for single benchmark */ static void pmembench_print_help_single(struct benchmark *bench) { struct benchmark_info *info = bench->info; printf("%s\n%s\n", info->name, info->brief); printf("\nArguments:\n"); size_t nclos = sizeof(pmembench_clos) / sizeof(struct benchmark_clo); pmembench_print_args(pmembench_clos, nclos); if (info->clos == nullptr) return; pmembench_print_args(info->clos, info->nclos); } /* * pmembench_print_usage -- print usage of framework */ static void pmembench_print_usage() { printf("Usage: $ pmembench [-h|--help] [-v|--version]" "\t[[]]\n"); printf("\t\t\t\t\t\t[[]]\n"); printf("\t\t\t\t\t\t[[[]]]\n"); } /* * pmembench_print_version -- print version of framework */ static void pmembench_print_version() { printf("Benchmark framework - version %u.%u\n", version.major, version.minor); } /* * pmembench_print_examples() -- print examples of using framework */ static void pmembench_print_examples() { printf("\nExamples:\n"); printf("$ pmembench \n"); printf(" # runs benchmark of name with arguments \n"); printf("or\n"); printf("$ pmembench \n"); printf(" # runs all scenarios from config file\n"); printf("or\n"); printf("$ pmembench [] [-h|--help [-v|--version]\n"); printf(" # prints help\n"); printf("or\n"); printf("$ pmembench \n"); printf(" # runs the specified scenario from config file\n"); printf("$ pmembench " " \n"); printf(" # runs the specified scenarios from config file and overwrites" " the given common_args from the config file\n"); } /* * pmembench_print_help -- print help for framework */ static void pmembench_print_help() { pmembench_print_version(); pmembench_print_usage(); printf("\nCommon arguments:\n"); size_t nclos = sizeof(pmembench_opts) / sizeof(struct benchmark_clo); pmembench_print_args(pmembench_opts, nclos); printf("\nAvaliable benchmarks:\n"); struct benchmark *bench = nullptr; PMDK_LIST_FOREACH(bench, &benchmarks.head, next) printf("\t%-20s\t\t%s\n", bench->info->name, bench->info->brief); printf("\n$ pmembench --help to print detailed information" " about benchmark arguments\n"); pmembench_print_examples(); } /* * pmembench_get_bench -- searching benchmarks by name */ static struct benchmark * pmembench_get_bench(const char *name) { struct benchmark *bench; PMDK_LIST_FOREACH(bench, &benchmarks.head, next) { if (strcmp(name, bench->info->name) == 0) return bench; } return nullptr; } /* * pmembench_parse_opts -- parse arguments for framework */ static int pmembench_parse_opts(struct pmembench *pb) { int ret = 0; int argc = ++pb->argc; char **argv = --pb->argv; struct benchmark_opts *opts = nullptr; struct clo_vec *clovec; size_t size, n_clos; size = sizeof(struct benchmark_opts); n_clos = ARRAY_SIZE(pmembench_opts); clovec = clo_vec_alloc(size); assert(clovec != nullptr); if (benchmark_clo_parse(argc, argv, pmembench_opts, n_clos, clovec)) { ret = -1; goto out; } opts = (struct benchmark_opts *)clo_vec_get_args(clovec, 0); if (opts == nullptr) { ret = -1; goto out; } if (opts->help) pmembench_print_help(); if (opts->version) pmembench_print_version(); out: clo_vec_free(clovec); return ret; } /* * pmembench_remove_file -- remove file or directory if exists */ static int pmembench_remove_file(const char *path) { int ret = 0; os_stat_t status; char *tmp; int exists = util_file_exists(path); if (exists < 0) return -1; if (!exists) return 0; if (os_stat(path, &status) != 0) return 0; if (!(status.st_mode & S_IFDIR)) return pmempool_rm(path, 0); struct dir_handle it; struct file_info info; if (util_file_dir_open(&it, path)) { return -1; } while (util_file_dir_next(&it, &info) == 0) { if (strcmp(info.filename, ".") == 0 || strcmp(info.filename, "..") == 0) continue; tmp = (char *)malloc(strlen(path) + strlen(info.filename) + 2); if (tmp == nullptr) return -1; sprintf(tmp, "%s" OS_DIR_SEP_STR "%s", path, info.filename); ret = info.is_dir ? pmembench_remove_file(tmp) : util_unlink(tmp); free(tmp); if (ret != 0) { util_file_dir_close(&it); return ret; } } util_file_dir_close(&it); return util_file_dir_remove(path); } /* * pmembench_single_repeat -- runs benchmark ones */ static int pmembench_single_repeat(struct benchmark *bench, struct benchmark_args *args, struct bench_results *res) { int ret = 0; if (args->main_affinity != -1) { os_cpu_set_t cpuset; os_cpu_zero(&cpuset); os_thread_t self; os_thread_self(&self); os_cpu_set(args->main_affinity, &cpuset); errno = os_thread_setaffinity_np(&self, sizeof(os_cpu_set_t), &cpuset); if (errno) { perror("os_thread_setaffinity_np"); return -1; } sched_yield(); } if (bench->info->rm_file && !args->is_dynamic_poolset) { ret = pmembench_remove_file(args->fname); if (ret != 0 && errno != ENOENT) { perror("removing file failed"); return ret; } } if (bench->info->init) { if (bench->info->init(bench, args)) { warn("%s: initialization failed", bench->info->name); return -1; } } assert(bench->info->operation != nullptr); assert(args->n_threads != 0); struct benchmark_worker **workers; workers = (struct benchmark_worker **)malloc( args->n_threads * sizeof(struct benchmark_worker *)); assert(workers != nullptr); if ((ret = pmembench_init_workers(workers, bench, args)) != 0) { goto out; } unsigned j; for (j = 0; j < args->n_threads; j++) { benchmark_worker_run(workers[j]); } for (j = 0; j < args->n_threads; j++) { benchmark_worker_join(workers[j]); if (workers[j]->ret != 0) { ret = workers[j]->ret; fprintf(stderr, "thread number %u failed\n", j); } } results_store(res, workers, args->n_threads, args->n_ops_per_thread); for (j = 0; j < args->n_threads; j++) { benchmark_worker_exit(workers[j]); free(workers[j]->info.opinfo); benchmark_worker_free(workers[j]); } out: free(workers); if (bench->info->exit) bench->info->exit(bench, args); return ret; } /* * scale_up_min_exe_time -- scale up the number of operations to obtain an * execution time not smaller than the assumed minimal execution time */ int scale_up_min_exe_time(struct benchmark *bench, struct benchmark_args *args, struct total_results **total_results) { const double min_exe_time = args->min_exe_time; struct total_results *total_res = *total_results; total_res->nrepeats = 1; do { /* * run single benchmark repeat to probe execution time */ int ret = pmembench_single_repeat(bench, args, &total_res->res[0]); if (ret != 0) return 1; get_total_results(total_res); if (min_exe_time < total_res->total.min + MIN_EXE_TIME_E) break; /* * scale up number of operations to get assumed minimal * execution time */ args->n_ops_per_thread = (size_t)( (double)args->n_ops_per_thread * (min_exe_time + MIN_EXE_TIME_E) / total_res->total.min); results_free(total_res); *total_results = results_alloc(args); assert(*total_results != nullptr); total_res = *total_results; total_res->nrepeats = 1; } while (1); total_res->nrepeats = args->repeats; return 0; } /* * is_absolute_path_to_directory -- checks if passed argument is absolute * path to directory */ static bool is_absolute_path_to_directory(const char *path) { os_stat_t sb; return util_is_absolute_path(path) && os_stat(path, &sb) == 0 && S_ISDIR(sb.st_mode); } /* * pmembench_run -- runs one benchmark. Parses arguments and performs * specific functions. */ static int pmembench_run(struct pmembench *pb, struct benchmark *bench) { enum file_type type; char old_wd[PATH_MAX]; int ret = 0; struct benchmark_args *args = nullptr; struct total_results *total_res = nullptr; struct latency *stats = nullptr; double *workers_times = nullptr; struct clo_vec *clovec = nullptr; assert(bench->info != nullptr); pmembench_merge_clos(bench); /* * Check if PMEMBENCH_DIR env var is set and change * the working directory accordingly. */ char *wd = os_getenv("PMEMBENCH_DIR"); if (wd != nullptr) { /* get current dir name */ if (getcwd(old_wd, PATH_MAX) == nullptr) { perror("getcwd"); ret = -1; goto out_release_clos; } os_stat_t stat_buf; if (os_stat(wd, &stat_buf) != 0) { perror("os_stat"); ret = -1; goto out_release_clos; } if (!S_ISDIR(stat_buf.st_mode)) { warn("PMEMBENCH_DIR is not a directory: %s", wd); ret = -1; goto out_release_clos; } if (chdir(wd)) { perror("chdir(wd)"); ret = -1; goto out_release_clos; } } if (bench->info->pre_init) { if (bench->info->pre_init(bench)) { warn("%s: pre-init failed", bench->info->name); ret = -1; goto out_old_wd; } } clovec = clo_vec_alloc(bench->args_size); assert(clovec != nullptr); if (pmembench_parse_clo(pb, bench, clovec)) { warn("%s: parsing command line arguments failed", bench->info->name); ret = -1; goto out_release_args; } args = (struct benchmark_args *)clo_vec_get_args(clovec, 0); if (args == nullptr) { warn("%s: parsing command line arguments failed", bench->info->name); ret = -1; goto out_release_args; } if (args->help) { pmembench_print_help_single(bench); goto out; } if (strlen(args->fname) > PATH_MAX) { warn("Filename too long"); ret = -1; goto out; } type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } pmembench_print_header(pb, bench, clovec); size_t args_i; for (args_i = 0; args_i < clovec->nargs; args_i++) { args = (struct benchmark_args *)clo_vec_get_args(clovec, args_i); if (args == nullptr) { warn("%s: parsing command line arguments failed", bench->info->name); ret = -1; goto out; } args->opts = (void *)((uintptr_t)args + sizeof(struct benchmark_args)); if (args->is_dynamic_poolset) { if (!bench->info->allow_poolset) { fprintf(stderr, "dynamic poolset not supported\n"); goto out; } if (!is_absolute_path_to_directory(args->fname)) { fprintf(stderr, "path must be absolute and point to a directory\n"); goto out; } } else { args->is_poolset = util_is_poolset_file(args->fname) == 1; if (args->is_poolset) { if (!bench->info->allow_poolset) { fprintf(stderr, "poolset files not supported\n"); goto out; } args->fsize = util_poolset_size(args->fname); if (!args->fsize) { fprintf(stderr, "invalid size of poolset\n"); goto out; } } else if (type == TYPE_DEVDAX) { args->fsize = util_file_get_size(args->fname); if (!args->fsize) { fprintf(stderr, "invalid size of device dax\n"); goto out; } } } unsigned n_threads_copy = args->n_threads; args->n_threads = !bench->info->multithread ? 1 : args->n_threads; size_t n_ops_per_thread_copy = args->n_ops_per_thread; args->n_ops_per_thread = !bench->info->multiops ? 1 : args->n_ops_per_thread; stats = (struct latency *)calloc(args->repeats, sizeof(struct latency)); assert(stats != nullptr); workers_times = (double *)calloc( args->n_threads * args->repeats, sizeof(double)); assert(workers_times != nullptr); total_res = results_alloc(args); assert(total_res != nullptr); unsigned i = 0; if (args->min_exe_time != 0 && bench->info->multiops) { ret = scale_up_min_exe_time(bench, args, &total_res); if (ret != 0) goto out; i = 1; } for (; i < args->repeats; i++) { ret = pmembench_single_repeat(bench, args, &total_res->res[i]); if (ret != 0) goto out; } get_total_results(total_res); pmembench_print_results(bench, args, total_res); args->n_ops_per_thread = n_ops_per_thread_copy; args->n_threads = n_threads_copy; results_free(total_res); free(stats); free(workers_times); total_res = nullptr; stats = nullptr; workers_times = nullptr; } out: if (total_res) results_free(total_res); if (stats) free(stats); if (workers_times) free(workers_times); out_release_args: clo_vec_free(clovec); out_old_wd: /* restore the original working directory */ if (wd != nullptr) { /* Only if PMEMBENCH_DIR env var was defined */ if (chdir(old_wd)) { perror("chdir(old_wd)"); ret = -1; } } out_release_clos: pmembench_release_clos(bench); return ret; } /* * pmembench_free_benchmarks -- release all benchmarks */ static void __attribute__((destructor)) pmembench_free_benchmarks(void) { while (!PMDK_LIST_EMPTY(&benchmarks.head)) { struct benchmark *bench = PMDK_LIST_FIRST(&benchmarks.head); PMDK_LIST_REMOVE(bench, next); free(bench); } } /* * pmembench_run_scenario -- run single benchmark's scenario */ static int pmembench_run_scenario(struct pmembench *pb, struct scenario *scenario) { struct benchmark *bench = pmembench_get_bench(scenario->benchmark); if (nullptr == bench) { fprintf(stderr, "unknown benchmark: %s\n", scenario->benchmark); return -1; } pb->scenario = scenario; return pmembench_run(pb, bench); } /* * pmembench_run_scenarios -- run all scenarios */ static int pmembench_run_scenarios(struct pmembench *pb, struct scenarios *ss) { struct scenario *scenario; FOREACH_SCENARIO(scenario, ss) { if (pmembench_run_scenario(pb, scenario) != 0) return -1; } return 0; } /* * pmembench_run_config -- run one or all scenarios from config file */ static int pmembench_run_config(struct pmembench *pb, const char *config) { struct scenarios *ss = nullptr; struct config_reader *cr = config_reader_alloc(); assert(cr != nullptr); int ret = 0; if ((ret = config_reader_read(cr, config))) goto out; if ((ret = config_reader_get_scenarios(cr, &ss))) goto out; assert(ss != nullptr); if (pb->argc == 1) { if ((ret = pmembench_run_scenarios(pb, ss)) != 0) goto out_scenarios; } else { /* Skip the config file name in cmd line params */ int tmp_argc = pb->argc - 1; char **tmp_argv = pb->argv + 1; if (!contains_scenarios(tmp_argc, tmp_argv, ss)) { /* no scenarios in cmd line arguments - parse params */ pb->override_clos = true; if ((ret = pmembench_run_scenarios(pb, ss)) != 0) goto out_scenarios; } else { /* scenarios in cmd line */ struct scenarios *cmd_ss = scenarios_alloc(); assert(cmd_ss != nullptr); int parsed_scenarios = clo_get_scenarios( tmp_argc, tmp_argv, ss, cmd_ss); if (parsed_scenarios < 0) goto out_cmd; /* * If there are any cmd line args left, treat * them as config file params override. */ if (tmp_argc - parsed_scenarios) pb->override_clos = true; /* * Skip the scenarios in the cmd line, * pmembench_run_scenarios does not expect them and will * fail otherwise. */ pb->argc -= parsed_scenarios; pb->argv += parsed_scenarios; ret = pmembench_run_scenarios(pb, cmd_ss); out_cmd: scenarios_free(cmd_ss); } } out_scenarios: scenarios_free(ss); out: config_reader_free(cr); return ret; } int main(int argc, char *argv[]) { util_init(); util_mmap_init(); /* * Parse common command line arguments and * benchmark's specific ones. */ if (argc < 2) { pmembench_print_usage(); exit(EXIT_FAILURE); } int ret = 0; int fexists; struct benchmark *bench; struct pmembench *pb = (struct pmembench *)calloc(1, sizeof(*pb)); assert(pb != nullptr); Get_time_avg = benchmark_get_avg_get_time(); pb->argc = --argc; pb->argv = ++argv; char *bench_name = pb->argv[0]; if (nullptr == bench_name) { ret = -1; goto out; } fexists = os_access(bench_name, R_OK) == 0; bench = pmembench_get_bench(bench_name); if (nullptr != bench) ret = pmembench_run(pb, bench); else if (fexists) ret = pmembench_run_config(pb, bench_name); else if ((ret = pmembench_parse_opts(pb)) != 0) { pmembench_print_usage(); goto out; } out: free(pb); util_mmap_fini(); return ret; } #ifdef _MSC_VER extern "C" { /* * Since libpmemobj is linked statically, * we need to invoke its ctor/dtor. */ MSVC_CONSTR(libpmemobj_init) MSVC_DESTR(libpmemobj_fini) } #endif pmdk-1.8/src/benchmarks/benchmark_time.cpp0000664000000000000000000000752613615011243017403 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * benchmark_time.cpp -- benchmark_time module definitions */ #include "benchmark_time.hpp" #include "os.h" #include #include #include #include #define NSECPSEC 1000000000 /* * benchmark_time_get -- get timestamp from clock source */ void benchmark_time_get(benchmark_time_t *time) { os_clock_gettime(CLOCK_MONOTONIC, time); } /* * benchmark_time_diff -- get time interval */ void benchmark_time_diff(benchmark_time_t *d, benchmark_time_t *t1, benchmark_time_t *t2) { long long nsecs = (t2->tv_sec - t1->tv_sec) * NSECPSEC + t2->tv_nsec - t1->tv_nsec; assert(nsecs >= 0); d->tv_sec = nsecs / NSECPSEC; d->tv_nsec = nsecs % NSECPSEC; } /* * benchmark_time_get_secs -- get total number of seconds */ double benchmark_time_get_secs(benchmark_time_t *t) { return (double)t->tv_sec + (double)t->tv_nsec / NSECPSEC; } /* * benchmark_time_get_nsecs -- get total number of nanoseconds */ unsigned long long benchmark_time_get_nsecs(benchmark_time_t *t) { unsigned long long ret = t->tv_nsec; ret += t->tv_sec * NSECPSEC; return ret; } /* * benchmark_time_compare -- compare two moments in time */ int benchmark_time_compare(const benchmark_time_t *t1, const benchmark_time_t *t2) { if (t1->tv_sec == t2->tv_sec) return (int)((long long)t1->tv_nsec - (long long)t2->tv_nsec); else return (int)((long long)t1->tv_sec - (long long)t2->tv_sec); } /* * benchmark_time_set -- set time using number of nanoseconds */ void benchmark_time_set(benchmark_time_t *time, unsigned long long nsecs) { time->tv_sec = nsecs / NSECPSEC; time->tv_nsec = nsecs % NSECPSEC; } /* * number of samples used to calculate average time required to get a current * time from the system */ #define N_PROBES_GET_TIME 10000000UL /* * benchmark_get_avg_get_time -- calculates average time required to get the * current time from the system in nanoseconds */ unsigned long long benchmark_get_avg_get_time(void) { benchmark_time_t time; benchmark_time_t start; benchmark_time_t stop; benchmark_time_get(&start); for (size_t i = 0; i < N_PROBES_GET_TIME; i++) { benchmark_time_get(&time); } benchmark_time_get(&stop); benchmark_time_diff(&time, &start, &stop); unsigned long long avg = benchmark_time_get_nsecs(&time) / N_PROBES_GET_TIME; return avg; } pmdk-1.8/src/benchmarks/log.cpp0000664000000000000000000004345513615011243015215 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * log.cpp -- pmemlog benchmarks definitions */ #include #include #include #include #include #include #include #include "benchmark.hpp" #include "file.h" #include "libpmemlog.h" #include "os.h" #include "poolset_util.hpp" /* * Size of pool header, pool descriptor * and additional page alignment overhead */ #define POOL_HDR_SIZE (3 * 4096) #define MIN_VEC_SIZE 1 /* * prog_args - benchmark's specific command line arguments */ struct prog_args { unsigned seed; /* seed for pseudo-random generator */ bool rand; /* use random numbers */ int vec_size; /* vector size */ size_t el_size; /* size of single append */ size_t min_size; /* minimum size for random mode */ bool no_warmup; /* don't do warmup */ bool fileio; /* use file io instead of pmemlog */ }; /* * thread_info - thread specific data */ struct log_worker_info { unsigned seed; struct iovec *iov; /* io vector */ char *buf; /* buffer for write/read operations */ size_t buf_size; /* buffer size */ size_t buf_ptr; /* pointer for read operations */ size_t *rand_sizes; size_t *vec_sizes; /* sum of sizes in vector */ }; /* * log_bench - main context of benchmark */ struct log_bench { size_t psize; /* size of pool */ PMEMlogpool *plp; /* pmemlog handle */ struct prog_args *args; /* benchmark specific arguments */ int fd; /* file descriptor for file io mode */ unsigned seed; /* * Pointer to the main benchmark operation. The appropriate function * will be assigned depending on the benchmark specific arguments. */ int (*func_op)(struct benchmark *, struct operation_info *); }; /* * do_warmup -- do warmup by writing the whole pool area */ static int do_warmup(struct log_bench *lb, size_t nops) { int ret = 0; size_t bsize = lb->args->vec_size * lb->args->el_size; auto *buf = (char *)calloc(1, bsize); if (!buf) { perror("calloc"); return -1; } if (!lb->args->fileio) { for (size_t i = 0; i < nops; i++) { if (pmemlog_append(lb->plp, buf, lb->args->el_size) < 0) { ret = -1; perror("pmemlog_append"); goto out; } } pmemlog_rewind(lb->plp); } else { for (size_t i = 0; i < nops; i++) { if (write(lb->fd, buf, (unsigned)lb->args->el_size) != (ssize_t)lb->args->el_size) { ret = -1; perror("write"); os_close(lb->fd); goto out; } } if (os_lseek(lb->fd, 0, SEEK_SET) < 0) { ret = -1; perror("lseek"); os_close(lb->fd); } } out: free(buf); return ret; } /* * log_append -- performs pmemlog_append operation */ static int log_append(struct benchmark *bench, struct operation_info *info) { auto *lb = (struct log_bench *)pmembench_get_priv(bench); assert(lb); auto *worker_info = (struct log_worker_info *)info->worker->priv; assert(worker_info); size_t size = lb->args->rand ? worker_info->rand_sizes[info->index] : lb->args->el_size; if (pmemlog_append(lb->plp, worker_info->buf, size) < 0) { perror("pmemlog_append"); return -1; } return 0; } /* * log_appendv -- performs pmemlog_appendv operation */ static int log_appendv(struct benchmark *bench, struct operation_info *info) { auto *lb = (struct log_bench *)pmembench_get_priv(bench); assert(lb); auto *worker_info = (struct log_worker_info *)info->worker->priv; assert(worker_info); struct iovec *iov = &worker_info->iov[info->index * lb->args->vec_size]; if (pmemlog_appendv(lb->plp, iov, lb->args->vec_size) < 0) { perror("pmemlog_appendv"); return -1; } return 0; } /* * fileio_append -- performs fileio append operation */ static int fileio_append(struct benchmark *bench, struct operation_info *info) { auto *lb = (struct log_bench *)pmembench_get_priv(bench); assert(lb); auto *worker_info = (struct log_worker_info *)info->worker->priv; assert(worker_info); size_t size = lb->args->rand ? worker_info->rand_sizes[info->index] : lb->args->el_size; if (write(lb->fd, worker_info->buf, (unsigned)size) != (ssize_t)size) { perror("write"); return -1; } return 0; } /* * fileio_appendv -- performs fileio appendv operation */ static int fileio_appendv(struct benchmark *bench, struct operation_info *info) { auto *lb = (struct log_bench *)pmembench_get_priv(bench); assert(lb != nullptr); auto *worker_info = (struct log_worker_info *)info->worker->priv; assert(worker_info); struct iovec *iov = &worker_info->iov[info->index * lb->args->vec_size]; size_t vec_size = worker_info->vec_sizes[info->index]; if (os_writev(lb->fd, iov, lb->args->vec_size) != (ssize_t)vec_size) { perror("writev"); return -1; } return 0; } /* * log_process_data -- callback function for pmemlog_walk. */ static int log_process_data(const void *buf, size_t len, void *arg) { auto *worker_info = (struct log_worker_info *)arg; size_t left = worker_info->buf_size - worker_info->buf_ptr; if (len > left) { worker_info->buf_ptr = 0; left = worker_info->buf_size; } len = len < left ? len : left; assert(len <= left); void *buff = &worker_info->buf[worker_info->buf_ptr]; memcpy(buff, buf, len); worker_info->buf_ptr += len; return 1; } /* * fileio_read -- perform single fileio read */ static int fileio_read(int fd, ssize_t len, struct log_worker_info *worker_info) { ssize_t left = worker_info->buf_size - worker_info->buf_ptr; if (len > left) { worker_info->buf_ptr = 0; left = worker_info->buf_size; } len = len < left ? len : left; assert(len <= left); size_t off = worker_info->buf_ptr; void *buff = &worker_info->buf[off]; if ((len = pread(fd, buff, len, off)) < 0) return -1; worker_info->buf_ptr += len; return 1; } /* * log_read_op -- perform read operation */ static int log_read_op(struct benchmark *bench, struct operation_info *info) { auto *lb = (struct log_bench *)pmembench_get_priv(bench); assert(lb); auto *worker_info = (struct log_worker_info *)info->worker->priv; assert(worker_info); worker_info->buf_ptr = 0; size_t chunk_size = lb->args->rand ? worker_info->rand_sizes[info->index] : lb->args->el_size; if (!lb->args->fileio) { pmemlog_walk(lb->plp, chunk_size, log_process_data, worker_info); return 0; } int ret; while ((ret = fileio_read(lb->fd, chunk_size, worker_info)) == 1) ; return ret; } /* * log_init_worker -- init benchmark worker */ static int log_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { int ret = 0; auto *lb = (struct log_bench *)pmembench_get_priv(bench); size_t i_size, n_vectors; assert(lb); auto *worker_info = (struct log_worker_info *)malloc( sizeof(struct log_worker_info)); if (!worker_info) { perror("malloc"); return -1; } /* allocate buffer for append / read */ worker_info->buf_size = lb->args->el_size * lb->args->vec_size; worker_info->buf = (char *)malloc(worker_info->buf_size); if (!worker_info->buf) { perror("malloc"); ret = -1; goto err_free_worker_info; } /* * For random mode, each operation has its own vector with * random sizes. Otherwise there is only one vector with * equal sizes. */ n_vectors = args->n_ops_per_thread; worker_info->iov = (struct iovec *)malloc( n_vectors * lb->args->vec_size * sizeof(struct iovec)); if (!worker_info->iov) { perror("malloc"); ret = -1; goto err_free_buf; } if (lb->args->rand) { /* each thread has random seed */ worker_info->seed = (unsigned)os_rand_r(&lb->seed); /* each vector element has its own random size */ size_t n_sizes = args->n_ops_per_thread * lb->args->vec_size; worker_info->rand_sizes = (size_t *)malloc( n_sizes * sizeof(*worker_info->rand_sizes)); if (!worker_info->rand_sizes) { perror("malloc"); ret = -1; goto err_free_iov; } /* generate append sizes */ for (size_t i = 0; i < n_sizes; i++) { auto hr = (uint32_t)os_rand_r(&worker_info->seed); auto lr = (uint32_t)os_rand_r(&worker_info->seed); uint64_t r64 = (uint64_t)hr << 32 | lr; size_t width = lb->args->el_size - lb->args->min_size; worker_info->rand_sizes[i] = r64 % width + lb->args->min_size; } } else { worker_info->rand_sizes = nullptr; } worker_info->vec_sizes = (size_t *)calloc( args->n_ops_per_thread, sizeof(*worker_info->vec_sizes)); if (!worker_info->vec_sizes) { perror("malloc\n"); ret = -1; goto err_free_rand_sizes; } /* fill up the io vectors */ i_size = 0; for (size_t n = 0; n < args->n_ops_per_thread; n++) { size_t buf_ptr = 0; size_t vec_off = n * lb->args->vec_size; for (int i = 0; i < lb->args->vec_size; ++i) { size_t el_size = lb->args->rand ? worker_info->rand_sizes[i_size++] : lb->args->el_size; worker_info->iov[vec_off + i].iov_base = &worker_info->buf[buf_ptr]; worker_info->iov[vec_off + i].iov_len = el_size; worker_info->vec_sizes[n] += el_size; buf_ptr += el_size; } } worker->priv = worker_info; return 0; err_free_rand_sizes: free(worker_info->rand_sizes); err_free_iov: free(worker_info->iov); err_free_buf: free(worker_info->buf); err_free_worker_info: free(worker_info); return ret; } /* * log_free_worker -- cleanup benchmark worker */ static void log_free_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *worker_info = (struct log_worker_info *)worker->priv; assert(worker_info); free(worker_info->buf); free(worker_info->iov); free(worker_info->rand_sizes); free(worker_info->vec_sizes); free(worker_info); } /* * log_init -- benchmark initialization function */ static int log_init(struct benchmark *bench, struct benchmark_args *args) { int ret = 0; assert(bench); assert(args != nullptr); assert(args->opts != nullptr); struct benchmark_info *bench_info; char path[PATH_MAX]; if (util_safe_strcpy(path, args->fname, sizeof(path)) != 0) return -1; enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } auto *lb = (struct log_bench *)malloc(sizeof(struct log_bench)); if (!lb) { perror("malloc"); return -1; } lb->args = (struct prog_args *)args->opts; lb->args->el_size = args->dsize; if (lb->args->vec_size == 0) lb->args->vec_size = 1; if (lb->args->rand && lb->args->min_size > lb->args->el_size) { errno = EINVAL; ret = -1; goto err_free_lb; } if (lb->args->rand && lb->args->min_size == lb->args->el_size) lb->args->rand = false; /* align pool size to ensure that we have enough usable space */ lb->psize = ALIGN_UP(POOL_HDR_SIZE + args->n_ops_per_thread * args->n_threads * lb->args->vec_size * lb->args->el_size, Mmap_align); /* calculate a required pool size */ if (lb->psize < PMEMLOG_MIN_POOL) lb->psize = PMEMLOG_MIN_POOL; if (args->is_poolset || type == TYPE_DEVDAX) { if (lb->args->fileio) { fprintf(stderr, "fileio not supported on device dax nor poolset\n"); ret = -1; goto err_free_lb; } if (args->fsize < lb->psize) { fprintf(stderr, "file size too large\n"); ret = -1; goto err_free_lb; } lb->psize = 0; } else if (args->is_dynamic_poolset) { if (lb->args->fileio) { fprintf(stderr, "fileio not supported with dynamic poolset\n"); ret = -1; goto err_free_lb; } ret = dynamic_poolset_create(args->fname, lb->psize); if (ret == -1) goto err_free_lb; if (util_safe_strcpy(path, POOLSET_PATH, sizeof(path)) != 0) goto err_free_lb; lb->psize = 0; } bench_info = pmembench_get_info(bench); if (!lb->args->fileio) { if ((lb->plp = pmemlog_create(path, lb->psize, args->fmode)) == nullptr) { perror("pmemlog_create"); ret = -1; goto err_free_lb; } bench_info->operation = (lb->args->vec_size > 1) ? log_appendv : log_append; } else { int flags = O_CREAT | O_RDWR | O_SYNC; /* Create a file if it does not exist. */ if ((lb->fd = os_open(args->fname, flags, args->fmode)) < 0) { perror(args->fname); ret = -1; goto err_free_lb; } /* allocate the pmem */ if ((errno = os_posix_fallocate(lb->fd, 0, lb->psize)) != 0) { perror("posix_fallocate"); ret = -1; goto err_close; } bench_info->operation = (lb->args->vec_size > 1) ? fileio_appendv : fileio_append; } if (!lb->args->no_warmup && type != TYPE_DEVDAX) { size_t warmup_nops = args->n_threads * args->n_ops_per_thread; if (do_warmup(lb, warmup_nops)) { fprintf(stderr, "warmup failed\n"); ret = -1; goto err_close; } } pmembench_set_priv(bench, lb); return 0; err_close: if (lb->args->fileio) os_close(lb->fd); else pmemlog_close(lb->plp); err_free_lb: free(lb); return ret; } /* * log_exit -- cleanup benchmark */ static int log_exit(struct benchmark *bench, struct benchmark_args *args) { auto *lb = (struct log_bench *)pmembench_get_priv(bench); if (!lb->args->fileio) pmemlog_close(lb->plp); else os_close(lb->fd); free(lb); return 0; } /* command line options definition */ static struct benchmark_clo log_clo[6]; /* log_append benchmark info */ static struct benchmark_info log_append_info; /* log_read benchmark info */ static struct benchmark_info log_read_info; CONSTRUCTOR(log_constructor) void log_constructor(void) { log_clo[0].opt_short = 'r'; log_clo[0].opt_long = "random"; log_clo[0].descr = "Use random sizes for append/read"; log_clo[0].off = clo_field_offset(struct prog_args, rand); log_clo[0].type = CLO_TYPE_FLAG; log_clo[1].opt_short = 'S'; log_clo[1].opt_long = "seed"; log_clo[1].descr = "Random mode"; log_clo[1].off = clo_field_offset(struct prog_args, seed); log_clo[1].def = "1"; log_clo[1].type = CLO_TYPE_UINT; log_clo[1].type_uint.size = clo_field_size(struct prog_args, seed); log_clo[1].type_uint.base = CLO_INT_BASE_DEC; log_clo[1].type_uint.min = 1; log_clo[1].type_uint.max = UINT_MAX; log_clo[2].opt_short = 'i'; log_clo[2].opt_long = "file-io"; log_clo[2].descr = "File I/O mode"; log_clo[2].off = clo_field_offset(struct prog_args, fileio); log_clo[2].type = CLO_TYPE_FLAG; log_clo[3].opt_short = 'w'; log_clo[3].opt_long = "no-warmup"; log_clo[3].descr = "Don't do warmup", log_clo[3].type = CLO_TYPE_FLAG; log_clo[3].off = clo_field_offset(struct prog_args, no_warmup); log_clo[4].opt_short = 'm'; log_clo[4].opt_long = "min-size"; log_clo[4].descr = "Minimum size of append/read for " "random mode"; log_clo[4].type = CLO_TYPE_UINT; log_clo[4].off = clo_field_offset(struct prog_args, min_size); log_clo[4].def = "1"; log_clo[4].type_uint.size = clo_field_size(struct prog_args, min_size); log_clo[4].type_uint.base = CLO_INT_BASE_DEC; log_clo[4].type_uint.min = 1; log_clo[4].type_uint.max = UINT64_MAX; /* this one is only for log_append */ log_clo[5].opt_short = 'v'; log_clo[5].opt_long = "vector"; log_clo[5].descr = "Vector size"; log_clo[5].off = clo_field_offset(struct prog_args, vec_size); log_clo[5].def = "1"; log_clo[5].type = CLO_TYPE_INT; log_clo[5].type_int.size = clo_field_size(struct prog_args, vec_size); log_clo[5].type_int.base = CLO_INT_BASE_DEC; log_clo[5].type_int.min = MIN_VEC_SIZE; log_clo[5].type_int.max = INT_MAX; log_append_info.name = "log_append"; log_append_info.brief = "Benchmark for pmemlog_append() " "operation"; log_append_info.init = log_init; log_append_info.exit = log_exit; log_append_info.multithread = true; log_append_info.multiops = true; log_append_info.init_worker = log_init_worker; log_append_info.free_worker = log_free_worker; /* this will be assigned in log_init */ log_append_info.operation = nullptr; log_append_info.measure_time = true; log_append_info.clos = log_clo; log_append_info.nclos = ARRAY_SIZE(log_clo); log_append_info.opts_size = sizeof(struct prog_args); log_append_info.rm_file = true; log_append_info.allow_poolset = true; REGISTER_BENCHMARK(log_append_info); log_read_info.name = "log_read"; log_read_info.brief = "Benchmark for pmemlog_walk() " "operation"; log_read_info.init = log_init; log_read_info.exit = log_exit; log_read_info.multithread = true; log_read_info.multiops = true; log_read_info.init_worker = log_init_worker; log_read_info.free_worker = log_free_worker; log_read_info.operation = log_read_op; log_read_info.measure_time = true; log_read_info.clos = log_clo; /* without vector */ log_read_info.nclos = ARRAY_SIZE(log_clo) - 1; log_read_info.opts_size = sizeof(struct prog_args); log_read_info.rm_file = true; log_read_info.allow_poolset = true; REGISTER_BENCHMARK(log_read_info); }; pmdk-1.8/src/benchmarks/pmembench_obj_gen.cfg0000664000000000000000000000204313615011243020016 0ustar rootroot# This is an example config file for pmembench # Global parameters [global] group = pmemobj file = testfile.obj ops-per-thread = 100 [obj_open_sizes] bench = obj_open data-size = 2:*2:1024 min-size = 1 objects = 10000 type-number = rand [obj_open_objects] bench = obj_open data-size = 1024 objects = 1:*10:100000 type-number = rand # pmemobj_open/close are not yet thread-safe #[obj_open_threads] #bench = obj_open #threads = 1:+1:10 #data-size = 1024 #min-size = 1 #objects = 1000 #type-number = rand [obj_direct_threads_one_pool] bench = obj_direct threads = 1:+1:10 data-size = 1025 min-size = 1024 type-number = rand one-pool = true [obj_direct_threads_many_pools] bench = obj_direct threads = 1:+1:10 data-size = 1025 min-size = 1024 type-number = rand [obj_direct_data_one_pool_one_obj] bench = obj_direct data-size = 2:*2:8192 type-number = rand one-pool = true one-object = true ops-per-thread = 10000 [obj_direct_threads_many_pools_one_obj] bench = obj_direct data-size = 2:*2:8192 type-number = rand one-object = true ops-per-thread = 10000 pmdk-1.8/src/benchmarks/pmemobj_gen.cpp0000664000000000000000000004646613615011243016723 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmemobj_gen.cpp -- benchmark for pmemobj_direct() * and pmemobj_open() functions. */ #include #include #include #include #include #include #include #include #include #include "benchmark.hpp" #include "libpmemobj.h" #define LAYOUT_NAME "benchmark" #define FACTOR 4 #define DIR_MODE 0700 #define FILE_MODE 0666 #define PART_NAME "/part" #define MAX_DIGITS 2 struct pobj_bench; struct pobj_worker; typedef size_t (*fn_type_num_t)(struct pobj_bench *ob, size_t worker_idx, size_t op_idx); typedef size_t (*fn_size_t)(struct pobj_bench *ob, size_t idx); typedef size_t (*fn_num_t)(size_t idx); /* * Enumeration used to determine the mode of the assigning type_number * value to the persistent objects. */ enum type_mode { TYPE_MODE_ONE, TYPE_MODE_PER_THREAD, TYPE_MODE_RAND, MAX_TYPE_MODE, }; /* * pobj_args - Stores command line parsed arguments. * * rand_type : Use random type number for every new allocated object. * Default, there is one type number for all objects. * * range : Use random allocation size. * * min_size : Minimum allocation size. * * n_objs : Number of objects allocated per thread * * one_pool : Use one common pool for all thread * * one_obj : Create and use one object per thread * * obj_size : Size of each allocated object * * n_ops : Number of operations */ struct pobj_args { char *type_num; bool range; unsigned min_size; size_t n_objs; bool one_pool; bool one_obj; size_t obj_size; size_t n_ops; }; /* * pobj_bench - Stores variables used in benchmark, passed within functions. * * pop : Pointer to the persistent pool. * * pa : Stores pobj_args structure. * * sets : Stores files names using to create pool per thread * * random_types : Random type numbers for persistent objects. * * rand_sizes : random values with allocation sizes. * * n_pools : Number of created pools. * * n_objs : Number of object created per thread. * * type_mode : Type_mode enum value * * fn_type_num : Function returning proper type number for each object. * * fn_size : Function returning proper size of allocation. * * pool : Functions returning number of thread if * one pool per thread created or index 0 if not. * * obj : Function returning number of operation if flag set * to false or index 0 if set to true. */ struct pobj_bench { PMEMobjpool **pop; struct pobj_args *args_priv; const char **sets; size_t *random_types; size_t *rand_sizes; size_t n_pools; int type_mode; fn_type_num_t fn_type_num; fn_size_t fn_size; fn_num_t pool; fn_num_t obj; }; /* * pobj_worker - Stores variables used by one thread. */ struct pobj_worker { PMEMoid *oids; }; /* * type_mode_one -- always returns 0, as in the mode TYPE_MODE_ONE * all of the persistent objects have the same type_number value. */ static size_t type_mode_one(struct pobj_bench *bench_priv, size_t worker_idx, size_t op_idx) { return 0; } /* * type_mode_per_thread -- always returns worker index, as in the mode * TYPE_MODE_PER_THREAD all persistent object allocated by the same thread * have the same type_number value. */ static size_t type_mode_per_thread(struct pobj_bench *bench_priv, size_t worker_idx, size_t op_idx) { return worker_idx; } /* * type_mode_rand -- returns the value from the random_types array assigned * for the specific operation in a specific thread. */ static size_t type_mode_rand(struct pobj_bench *bench_priv, size_t worker_idx, size_t op_idx) { return bench_priv->random_types[op_idx]; } /* * range_size -- returns size of object allocation from rand_sizes array. */ static size_t range_size(struct pobj_bench *bench_priv, size_t idx) { return bench_priv->rand_sizes[idx]; } /* * static_size -- returns always the same size of object allocation. */ static size_t static_size(struct pobj_bench *bench_priv, size_t idx) { return bench_priv->args_priv->obj_size; } /* * diff_num -- returns given index */ static size_t diff_num(size_t idx) { return idx; } /* * one_num -- returns always the same index. */ static size_t one_num(size_t idx) { return 0; } static fn_type_num_t type_mode_func[MAX_TYPE_MODE] = { type_mode_one, type_mode_per_thread, type_mode_rand}; const char *type_mode_names[MAX_TYPE_MODE] = {"one", "per-thread", "rand"}; /* * parse_type_mode -- parses command line "--type-number" argument * and returns proper type_mode enum value. */ static enum type_mode parse_type_mode(const char *arg) { enum type_mode i = TYPE_MODE_ONE; for (; i < MAX_TYPE_MODE && strcmp(arg, type_mode_names[i]) != 0; i = (enum type_mode)(i + 1)) ; return i; } /* * rand_sizes -- allocates array and calculates random values as allocation * sizes for each object. Used only when range flag set. */ static size_t * rand_sizes(size_t min, size_t max, size_t n_ops) { assert(n_ops != 0); auto *rand_sizes = (size_t *)malloc(n_ops * sizeof(size_t)); if (rand_sizes == nullptr) { perror("malloc"); return nullptr; } for (size_t i = 0; i < n_ops; i++) { rand_sizes[i] = RRAND(max, min); } return rand_sizes; } /* * random_types -- allocates array and calculates random values to assign * type_number for each object. */ static int random_types(struct pobj_bench *bench_priv, struct benchmark_args *args) { assert(bench_priv->args_priv->n_objs != 0); bench_priv->random_types = (size_t *)malloc( bench_priv->args_priv->n_objs * sizeof(size_t)); if (bench_priv->random_types == nullptr) { perror("malloc"); return -1; } for (size_t i = 0; i < bench_priv->args_priv->n_objs; i++) bench_priv->random_types[i] = rand() % UINT32_MAX; return 0; } /* * pobj_init - common part of the benchmark initialization functions. * Parses command line arguments, set variables and creates persistent pools. */ static int pobj_init(struct benchmark *bench, struct benchmark_args *args) { unsigned i = 0; size_t psize; size_t n_objs; assert(bench != nullptr); assert(args != nullptr); enum file_type type = util_file_get_type(args->fname); if (type == OTHER_ERROR) { fprintf(stderr, "could not check type of file %s\n", args->fname); return -1; } auto *bench_priv = (struct pobj_bench *)malloc(sizeof(struct pobj_bench)); if (bench_priv == nullptr) { perror("malloc"); return -1; } assert(args->opts != nullptr); bench_priv->args_priv = (struct pobj_args *)args->opts; bench_priv->args_priv->obj_size = args->dsize; bench_priv->args_priv->range = bench_priv->args_priv->min_size > 0 ? true : false; bench_priv->n_pools = !bench_priv->args_priv->one_pool ? args->n_threads : 1; bench_priv->pool = bench_priv->n_pools > 1 ? diff_num : one_num; bench_priv->obj = !bench_priv->args_priv->one_obj ? diff_num : one_num; if ((args->is_poolset || type == TYPE_DEVDAX) && bench_priv->n_pools > 1) { fprintf(stderr, "cannot use poolset nor device dax for multiple pools," " please use -P|--one-pool option instead"); goto free_bench_priv; } /* * Multiplication by FACTOR prevents from out of memory error * as the actual size of the allocated persistent objects * is always larger than requested. */ n_objs = bench_priv->args_priv->n_objs; if (bench_priv->n_pools == 1) n_objs *= args->n_threads; psize = PMEMOBJ_MIN_POOL + n_objs * args->dsize * args->n_threads * FACTOR; /* assign type_number determining function */ bench_priv->type_mode = parse_type_mode(bench_priv->args_priv->type_num); switch (bench_priv->type_mode) { case MAX_TYPE_MODE: fprintf(stderr, "unknown type mode"); goto free_bench_priv; case TYPE_MODE_RAND: if (random_types(bench_priv, args)) goto free_bench_priv; break; default: bench_priv->random_types = nullptr; } bench_priv->fn_type_num = type_mode_func[bench_priv->type_mode]; /* assign size determining function */ bench_priv->fn_size = bench_priv->args_priv->range ? range_size : static_size; bench_priv->rand_sizes = nullptr; if (bench_priv->args_priv->range) { if (bench_priv->args_priv->min_size > args->dsize) { fprintf(stderr, "Invalid allocation size"); goto free_random_types; } bench_priv->rand_sizes = rand_sizes(bench_priv->args_priv->min_size, bench_priv->args_priv->obj_size, bench_priv->args_priv->n_objs); if (bench_priv->rand_sizes == nullptr) goto free_random_types; } assert(bench_priv->n_pools > 0); bench_priv->pop = (PMEMobjpool **)calloc(bench_priv->n_pools, sizeof(PMEMobjpool *)); if (bench_priv->pop == nullptr) { perror("calloc"); goto free_random_sizes; } bench_priv->sets = (const char **)calloc(bench_priv->n_pools, sizeof(const char *)); if (bench_priv->sets == nullptr) { perror("calloc"); goto free_pop; } if (bench_priv->n_pools > 1) { assert(!args->is_poolset); if (util_file_mkdir(args->fname, DIR_MODE) != 0) { fprintf(stderr, "cannot create directory\n"); goto free_sets; } size_t path_len = (strlen(PART_NAME) + strlen(args->fname)) + MAX_DIGITS + 1; for (i = 0; i < bench_priv->n_pools; i++) { bench_priv->sets[i] = (char *)malloc(path_len * sizeof(char)); if (bench_priv->sets[i] == nullptr) { perror("malloc"); goto free_sets; } int ret = snprintf((char *)bench_priv->sets[i], path_len, "%s%s%02x", args->fname, PART_NAME, i); if (ret < 0 || ret >= (int)path_len) { perror("snprintf"); goto free_sets; } bench_priv->pop[i] = pmemobj_create(bench_priv->sets[i], LAYOUT_NAME, psize, FILE_MODE); if (bench_priv->pop[i] == nullptr) { perror(pmemobj_errormsg()); goto free_sets; } } } else { if (args->is_poolset || type == TYPE_DEVDAX) { if (args->fsize < psize) { fprintf(stderr, "file size too large\n"); goto free_pools; } psize = 0; } bench_priv->sets[0] = args->fname; bench_priv->pop[0] = pmemobj_create( bench_priv->sets[0], LAYOUT_NAME, psize, FILE_MODE); if (bench_priv->pop[0] == nullptr) { perror(pmemobj_errormsg()); goto free_pools; } } pmembench_set_priv(bench, bench_priv); return 0; free_sets: for (; i > 0; i--) { pmemobj_close(bench_priv->pop[i - 1]); free((char *)bench_priv->sets[i - 1]); } free_pools: free(bench_priv->sets); free_pop: free(bench_priv->pop); free_random_sizes: free(bench_priv->rand_sizes); free_random_types: free(bench_priv->random_types); free_bench_priv: free(bench_priv); return -1; } /* * pobj_direct_init -- special part of pobj_direct benchmark initialization. */ static int pobj_direct_init(struct benchmark *bench, struct benchmark_args *args) { auto *pa = (struct pobj_args *)args->opts; pa->n_objs = pa->one_obj ? 1 : args->n_ops_per_thread; if (pobj_init(bench, args) != 0) return -1; return 0; } /* * pobj_exit -- common part for the benchmarks exit functions */ static int pobj_exit(struct benchmark *bench, struct benchmark_args *args) { size_t i; auto *bench_priv = (struct pobj_bench *)pmembench_get_priv(bench); if (bench_priv->n_pools > 1) { for (i = 0; i < bench_priv->n_pools; i++) { pmemobj_close(bench_priv->pop[i]); free((char *)bench_priv->sets[i]); } } else { pmemobj_close(bench_priv->pop[0]); } free(bench_priv->sets); free(bench_priv->pop); free(bench_priv->rand_sizes); free(bench_priv->random_types); free(bench_priv); return 0; } /* * pobj_init_worker -- worker initialization */ static int pobj_init_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { size_t i, idx = worker->index; auto *bench_priv = (struct pobj_bench *)pmembench_get_priv(bench); auto *pw = (struct pobj_worker *)calloc(1, sizeof(struct pobj_worker)); if (pw == nullptr) { perror("calloc"); return -1; } worker->priv = pw; pw->oids = (PMEMoid *)calloc(bench_priv->args_priv->n_objs, sizeof(PMEMoid)); if (pw->oids == nullptr) { free(pw); perror("calloc"); return -1; } PMEMobjpool *pop = bench_priv->pop[bench_priv->pool(idx)]; for (i = 0; i < bench_priv->args_priv->n_objs; i++) { size_t size = bench_priv->fn_size(bench_priv, i); size_t type = bench_priv->fn_type_num(bench_priv, idx, i); if (pmemobj_alloc(pop, &pw->oids[i], size, type, nullptr, nullptr) != 0) { perror("pmemobj_alloc"); goto out; } } return 0; out: for (; i > 0; i--) pmemobj_free(&pw->oids[i - 1]); free(pw->oids); free(pw); return -1; } /* * pobj_direct_op -- main operations of the obj_direct benchmark. */ static int pobj_direct_op(struct benchmark *bench, struct operation_info *info) { auto *bench_priv = (struct pobj_bench *)pmembench_get_priv(bench); auto *pw = (struct pobj_worker *)info->worker->priv; size_t idx = bench_priv->obj(info->index); /* Query an invalid uuid:off pair to invalidate the cache. */ PMEMoid bad = {1, 1}; #define OBJ_DIRECT_NITER 1024 /* * As we measure a very fast operation, we need a loop inside the * test harness. */ for (int i = 0; i < OBJ_DIRECT_NITER; i++) { if (pmemobj_direct(pw->oids[idx]) == nullptr) return -1; if (pmemobj_direct(bad) != nullptr) return -1; } return 0; #undef OBJ_DIRECT_NITER } /* * pobj_open_op -- main operations of the obj_open benchmark. */ static int pobj_open_op(struct benchmark *bench, struct operation_info *info) { auto *bench_priv = (struct pobj_bench *)pmembench_get_priv(bench); size_t idx = bench_priv->pool(info->worker->index); pmemobj_close(bench_priv->pop[idx]); bench_priv->pop[idx] = pmemobj_open(bench_priv->sets[idx], LAYOUT_NAME); if (bench_priv->pop[idx] == nullptr) return -1; return 0; } /* * pobj_free_worker -- worker exit function */ static void pobj_free_worker(struct benchmark *bench, struct benchmark_args *args, struct worker_info *worker) { auto *pw = (struct pobj_worker *)worker->priv; auto *bench_priv = (struct pobj_bench *)pmembench_get_priv(bench); for (size_t i = 0; i < bench_priv->args_priv->n_objs; i++) pmemobj_free(&pw->oids[i]); free(pw->oids); free(pw); } static struct benchmark_info obj_open; static struct benchmark_info obj_direct; /* Array defining common command line arguments. */ static struct benchmark_clo pobj_direct_clo[4]; static struct benchmark_clo pobj_open_clo[3]; CONSTRUCTOR(pmemobj_gen_constructor) void pmemobj_gen_constructor(void) { pobj_direct_clo[0].opt_short = 'T'; pobj_direct_clo[0].opt_long = "type-number"; pobj_direct_clo[0].descr = "Type number mode - one, per-thread, " "rand"; pobj_direct_clo[0].def = "one"; pobj_direct_clo[0].off = clo_field_offset(struct pobj_args, type_num); pobj_direct_clo[0].type = CLO_TYPE_STR; pobj_direct_clo[1].opt_short = 'm'; pobj_direct_clo[1].opt_long = "min-size"; pobj_direct_clo[1].type = CLO_TYPE_UINT; pobj_direct_clo[1].descr = "Minimum allocation size"; pobj_direct_clo[1].off = clo_field_offset(struct pobj_args, min_size); pobj_direct_clo[1].def = "0"; pobj_direct_clo[1].type_uint.size = clo_field_size(struct pobj_args, min_size); pobj_direct_clo[1].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; pobj_direct_clo[1].type_uint.min = 0; pobj_direct_clo[1].type_uint.max = UINT_MAX; pobj_direct_clo[2].opt_short = 'P'; pobj_direct_clo[2].opt_long = "one-pool"; pobj_direct_clo[2].descr = "Create one pool for all threads"; pobj_direct_clo[2].type = CLO_TYPE_FLAG; pobj_direct_clo[2].off = clo_field_offset(struct pobj_args, one_pool); pobj_direct_clo[3].opt_short = 'O'; pobj_direct_clo[3].opt_long = "one-object"; pobj_direct_clo[3].descr = "Use only one object per thread"; pobj_direct_clo[3].type = CLO_TYPE_FLAG; pobj_direct_clo[3].off = clo_field_offset(struct pobj_args, one_obj); pobj_open_clo[0].opt_short = 'T', pobj_open_clo[0].opt_long = "type-number", pobj_open_clo[0].descr = "Type number mode - one, " "per-thread, rand", pobj_open_clo[0].def = "one", pobj_open_clo[0].off = clo_field_offset(struct pobj_args, type_num), pobj_open_clo[0].type = CLO_TYPE_STR, pobj_open_clo[1].opt_short = 'm', pobj_open_clo[1].opt_long = "min-size", pobj_open_clo[1].type = CLO_TYPE_UINT, pobj_open_clo[1].descr = "Minimum allocation size", pobj_open_clo[1].off = clo_field_offset(struct pobj_args, min_size), pobj_open_clo[1].def = "0", pobj_open_clo[1].type_uint.size = clo_field_size(struct pobj_args, min_size), pobj_open_clo[1].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX, pobj_open_clo[1].type_uint.min = 0, pobj_open_clo[1].type_uint.max = UINT_MAX, pobj_open_clo[2].opt_short = 'o'; pobj_open_clo[2].opt_long = "objects"; pobj_open_clo[2].type = CLO_TYPE_UINT; pobj_open_clo[2].descr = "Number of objects in each pool"; pobj_open_clo[2].off = clo_field_offset(struct pobj_args, n_objs); pobj_open_clo[2].def = "1"; pobj_open_clo[2].type_uint.size = clo_field_size(struct pobj_args, n_objs); pobj_open_clo[2].type_uint.base = CLO_INT_BASE_DEC | CLO_INT_BASE_HEX; pobj_open_clo[2].type_uint.min = 1; pobj_open_clo[2].type_uint.max = UINT_MAX; obj_open.name = "obj_open"; obj_open.brief = "pmemobj_open() benchmark"; obj_open.init = pobj_init; obj_open.exit = pobj_exit; obj_open.multithread = true; obj_open.multiops = true; obj_open.init_worker = pobj_init_worker; obj_open.free_worker = pobj_free_worker; obj_open.operation = pobj_open_op; obj_open.measure_time = true; obj_open.clos = pobj_open_clo; obj_open.nclos = ARRAY_SIZE(pobj_open_clo); obj_open.opts_size = sizeof(struct pobj_args); obj_open.rm_file = true; obj_open.allow_poolset = true; REGISTER_BENCHMARK(obj_open); obj_direct.name = "obj_direct"; obj_direct.brief = "pmemobj_direct() benchmark"; obj_direct.init = pobj_direct_init; obj_direct.exit = pobj_exit; obj_direct.multithread = true; obj_direct.multiops = true; obj_direct.init_worker = pobj_init_worker; obj_direct.free_worker = pobj_free_worker; obj_direct.operation = pobj_direct_op; obj_direct.measure_time = true; obj_direct.clos = pobj_direct_clo; obj_direct.nclos = ARRAY_SIZE(pobj_direct_clo); obj_direct.opts_size = sizeof(struct pobj_args); obj_direct.rm_file = true; obj_direct.allow_poolset = true; REGISTER_BENCHMARK(obj_direct); }; pmdk-1.8/src/benchmarks/Makefile0000664000000000000000000001333213615011243015357 0ustar rootroot# # Copyright 2014-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # src/benchmarks/Makefile -- build all benchmarks # TOP := $(dir $(lastword $(MAKEFILE_LIST)))../.. include $(TOP)/src/common.inc vpath %.c $(TOP)/src/examples/libpmemobj/tree_map vpath %.c $(TOP)/src/examples/libpmemobj/map vpath %.c $(TOP)/src/examples/libpmemobj/hashmap vpath %.c $(TOP)/src/libpmemobj vpath %.c $(TOP)/src/common vpath %.c $(TOP)/src/librpmem vpath %.c $(TOP)/src/rpmem_common vpath %.c $(TOP)/src/libpmempool BENCHMARK = pmembench GLIB:= $(call check_package, glib-2.0) ifeq ($(GLIB),y) TARGET = $(BENCHMARK) else $(info NOTE: Skipping pmembench because glib-2.0 is missing \ -- see src/benchmarks/README for details.) endif all: $(TARGET) SRC=pmembench.cpp\ benchmark_time.cpp\ benchmark_worker.cpp\ clo.cpp\ clo_vec.cpp\ config_reader.cpp\ scenario.cpp\ log.cpp\ blk.cpp\ pmem_memset.cpp\ pmem_memcpy.cpp\ pmem_flush.cpp\ pmemobj_gen.cpp\ pmemobj_persist.cpp\ obj_pmalloc.cpp\ obj_locks.cpp\ obj_lanes.cpp\ map_bench.cpp\ pmemobj_tx.cpp\ pmemobj_atomic_lists.cpp\ poolset_util.cpp # Configuration file without the .cfg extension CONFIGS=pmembench_log\ pmembench_blk\ pmembench_memset\ pmembench_memcpy\ pmembench_flush\ pmembench_obj_pmalloc\ pmembench_obj_persist\ pmembench_obj_gen\ pmembench_obj_locks\ pmembench_obj_lanes\ pmembench_map\ pmembench_tx\ pmembench_atomic_lists OBJS=$(SRC:.cpp=.o) ifneq ($(filter 1 2, $(CSTYLEON)),) TMP_HEADERS := $(addsuffix tmp, $(HEADERS)) endif LDFLAGS = -L$(LIBS_PATH) $(OS_LIBS) LDFLAGS += -L../examples/libpmemobj/map LDFLAGS += $(EXTRA_LDFLAGS) ifeq ($(DEBUG),) LIBS += ../nondebug/libpmemcommon.a else LIBS += ../debug/libpmemcommon.a endif CFLAGS += $(LIBNDCTL_CFLAGS) LIBS += -lpmemobj -lpmemlog -lpmemblk -lpmempool -lpmem -pthread -lm \ $(LIBDL) $(LIBUUID) $(LIBNDCTL_LIBS) ifeq ($(LIBRT_NEEDED), y) LIBS += -lrt endif CXXFLAGS = -Wall CXXFLAGS += -Werror CXXFLAGS += -std=c++11 CXXFLAGS += -Wno-invalid-offsetof CXXFLAGS += -Wpointer-arith CXXFLAGS += -Wunused-macros CXXFLAGS += -pthread CXXFLAGS += -I../include CXXFLAGS += -I../libpmemobj CXXFLAGS += -I../common CXXFLAGS += -I../examples/libpmemobj/map CXXFLAGS += -I../rpmem_common CXXFLAGS += -I../librpmem CXXFLAGS += $(OS_INCS) CXXFLAGS += -DSRCVERSION='"$(SRCVERSION)"' ifeq ($(OS_DIMM),ndctl) CXXFLAGS += -DSDS_ENABLED endif ifeq ($(COVERAGE),1) CXXFLAGS += $(GCOV_CFLAGS) LDFLAGS += $(GCOV_LDFLAGS) LIBS += $(GCOV_LIBS) endif ifneq ($(SANITIZE),) CXXFLAGS += -fsanitize=$(SANITIZE) LDFLAGS += -fsanitize=$(SANITIZE) endif ifeq ($(BUILD_RPMEM),y) SRC += rpmem.cpp LIBS += -lrpmem $(shell pkg-config --libs libfabric) CONFIGS += pmembench_rpmem CXXFLAGS += -DRPMEM_AVAILABLE else $(info NOTE: Skipping librpmem benchmark because $(BUILD_RPMEM_INFO)) endif ifeq ($(GLIB),y) CXXFLAGS += $(shell $(PKG_CONFIG) --cflags glib-2.0) LIBS += $(shell $(PKG_CONFIG) --libs glib-2.0) endif LIBMAP_DIR=../examples/libpmemobj/map LIBMAP=$(LIBMAP_DIR)/libmap.a OBJS += pmemobj.o ifeq ($(DEBUG),) CXXFLAGS += -O3 LIBS_PATH=../nondebug else CXXFLAGS += -ggdb LIBS_PATH=../debug endif GLIB_TEST_PROG="\#include \nint main(){return 0;}" GLIB_SILENCE := $(shell printf $(GLIB_TEST_PROG) |\ $(CXX) $(CXXFLAGS) -x c -o /dev/null - 2>/dev/null && echo n || echo y) ifeq ($(GLIB_SILENCE), y) CXXFLAGS += -Wno-unknown-attributes endif CXXFLAGS += $(EXTRA_CXXFLAGS) objdir=. %.o: %.cpp Makefile $(call check-cstyle, $<) @mkdir -p .deps $(CXX) -MD -c -o $@ $(CXXFLAGS) $(call coverage-path, $<) $(call check-os, $@, $<) $(create-deps) %.htmp: %.h $(call check-cstyle, $<, $@) %.hpptmp: %.hpp $(call check-cstyle, $<, $@) $(BENCHMARK): $(TMP_HEADERS) $(OBJS) $(LIBMAP) $(CXX) -o $@ $(LDFLAGS) $(OBJS) $(LIBMAP) $(LIBS) $(LIBMAP): $(MAKE) -C $(LIBMAP_DIR) map clean: $(RM) $(OBJS) $(TMP_HEADERS) $(RM) *.csv clobber: clean $(RM) $(BENCHMARK) $(RM) *.csv $(RM) -r .deps $(CONFIGS): LD_LIBRARY_PATH=$(LIBS_PATH) ./$(BENCHMARK) $@.cfg > $@.csv run: $(BENCHMARK) $(CONFIGS) sparse: .PHONY: all clean clobber run $(CONFIGS) PMEMOBJ_SYMBOLS=pmalloc pfree lane_hold lane_release pmemobj.o: $(LIBS_PATH)/libpmemobj/libpmemobj_unscoped.o $(OBJCOPY) --localize-hidden $(addprefix -G, $(PMEMOBJ_SYMBOLS)) $< $@ -include .deps/*.P pmdk-1.8/src/benchmarks/poolset_util.cpp0000664000000000000000000000641613615011243017152 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include #include #include #include "os.h" #include "poolset_util.hpp" #include "set.h" #define PART_TEMPLATE "part." #define POOL_PART_SIZE (1UL << 30) /* * dynamic_poolset_clear -- clears header in first part if it exists */ static int dynamic_poolset_clear(const char *dir) { char path[PATH_MAX]; int count = snprintf(path, sizeof(path), "%s" OS_DIR_SEP_STR PART_TEMPLATE "0", dir); assert(count > 0); if ((size_t)count >= sizeof(path)) { fprintf(stderr, "path to a poolset part too long\n"); return -1; } int exists = util_file_exists(path); if (exists < 0) return -1; if (!exists) return 0; return util_file_zero(path, 0, POOL_HDR_SIZE); } /* * dynamic_poolset_create -- clear pool's header and create new poolset */ int dynamic_poolset_create(const char *path, size_t size) { /* buffer for part's path and size */ char buff[PATH_MAX + 20]; int ret; int fd; int count; int curr_part = 0; ret = dynamic_poolset_clear(path); if (ret == -1) return -1; fd = os_open(POOLSET_PATH, O_RDWR | O_CREAT, 0644); if (fd == -1) { perror("open"); return -1; } char header[] = "PMEMPOOLSET\nOPTION SINGLEHDR\n"; ret = util_write_all(fd, header, sizeof(header) - 1); if (ret == -1) goto err; while (curr_part * POOL_PART_SIZE < size + POOL_HDR_SIZE) { count = snprintf(buff, sizeof(buff), "%lu %s" OS_DIR_SEP_STR PART_TEMPLATE "%d\n", POOL_PART_SIZE, path, curr_part); assert(count > 0); if ((size_t)count >= sizeof(buff)) { fprintf(stderr, "path to a poolset part too long\n"); goto err; } ret = util_write_all(fd, buff, count); if (ret == -1) goto err; curr_part++; } close(fd); return 0; err: close(fd); return -1; } pmdk-1.8/src/benchmarks/pmembench_tx.cfg0000664000000000000000000001771513615011243017062 0ustar rootroot# This is an example config file for pmembench # Global parameters [global] file = /dev/shm/testfile.tx group = pmemobj ops-per-thread = 100 # obj_tx_alloc benchmark # variable threads number # one type-number [obj_tx_alloc_thread_one_type_num] bench = obj_tx_alloc threads = 1:+1:5 data-size = 512 # obj_tx_alloc benchmark # variable threads number # one type-number per thread [obj_tx_alloc_thread_per_thread_type_num] bench = obj_tx_alloc threads = 1:+1:5 data-size = 512 type-number = per-thread # obj_tx_alloc benchmark # variable allocation size # abort every nested transaction # rand type-number [obj_tx_alloc_sizes_abort_nested] bench = obj_tx_alloc data-size = 8:*2:16384 operation = abort-nested nestings = 100 type-number = rand # obj_tx_alloc benchmark # variable allocation size # use file io instead pmem # rand type-number [obj_tx_alloc_sizes_file_io] bench = obj_tx_alloc data-size = 8:*2:16384 nestings = 100 lib = dram type-number = rand # obj_tx_alloc benchmark # variable allocation size # pmem without transactions # rand type-number [obj_tx_alloc_sizes_atomic] bench = obj_tx_alloc data-size = 8:*2:16384 nestings = 100 lib = pmem type-number = rand # obj_tx_alloc benchmark # variable number of nested transactions # abort every nested transaction # rand type-number [obj_tx_alloc_nestings_abort_nested] bench = obj_tx_alloc data-size = 512 min-size = 1 operation = abort-nested nestings = 1:*10:10000 type-number = rand # obj_tx_alloc benchmark # variable number of nested transactions # use file io instead pmem # rand type-number [obj_tx_alloc_nestings_file_io] bench = obj_tx_alloc data-size = 512 min-size = 1 nestings = 1:*10:10000 lib = dram type-number = rand # obj_tx_alloc benchmark # variable number of nested transactions # pmem without transactions # rand type-number [obj_tx_alloc_nestings_atomic] bench = obj_tx_alloc data-size = 512 min-size = 1 nestings = 1:*10:10000 lib = pmem type-number = rand # obj_tx_free benchmark # variable allocation size # abort every nested transaction # rand type-number [obj_tx_free_sizes_abort_nested] bench = obj_tx_free data-size = 8:*2:8192 operation = abort-nested nestings = 100 type-number = rand # obj_tx_free benchmark # variable allocation size # use file io instead pmem # rand type-number [obj_tx_free_sizes_file_io] bench = obj_tx_free data-size = 8:*2:8192 nestings = 100 lib = dram type-number = rand # obj_tx_free benchmark # variable allocation size # pmem without transactions # rand type-number [obj_tx_free_sizes_atomic] bench = obj_tx_free data-size = 8:*2:8192 nestings = 100 lib = pmem type-number = rand # obj_tx_free benchmark # variable number of nested transactions # abort every nested transaction # rand type-number [obj_tx_free_nestings_abort_nested] bench = obj_tx_free data-size = 512 min-size = 1 operation = abort-nested nestings = 1:*10:10000 type-number = rand # obj_tx_free benchmark # variable number of nested transactions # use file io instead pmem # rand type-number [obj_tx_free_nestings_file_io] bench = obj_tx_free data-size = 512 min-size = 1 nestings = 1:*10:10000 lib = dram type-number = rand # obj_tx_free benchmark # variable number of nested transactions # pmem without transactions # rand type-number [obj_tx_free_nestings_atomic] bench = obj_tx_free data-size = 512 min-size = 1 nestings = 1:*10:10000 lib = pmem type-number = rand # obj_tx_realloc benchmark # variable allocation size # abort every nested transaction # rand type-number [obj_tx_realloc_data_sizes_abort] bench = obj_tx_realloc data-size = 8:*2:8192 realloc-size = 64 min-rsize = 1 operation = abort nestings = 100 type-number = rand # obj_tx_realloc benchmark # variable allocation size # use file io instead pmem # rand type-number [obj_tx_realloc_data_sizes_file_io] bench = obj_tx_realloc data-size = 8:*2:8192 realloc-size = 64 min-rsize = 1 nestings = 100 lib = dram type-number = rand # obj_tx_realloc benchmark # variable allocation size # pmem without transactions # rand type-number [obj_tx_realloc_data_sizes_atomic] bench = obj_tx_realloc data-size = 8:*2:8192 realloc-size = 64 min-rsize = 1 nestings = 100 lib = pmem type-number = rand # obj_tx_realloc benchmark # variable reallocation size # abort every nested transaction # rand type-number # another type number in realloc than in alloc [obj_tx_realloc_sizes_abort] bench = obj_tx_realloc data-size = 4 min-size = 1 realloc-size = 256:*2:16384 operation = abort changed-type = true type-number = rand # obj_tx_realloc benchmark # variable reallocation size # use file io instead pmem # rand type-number [obj_tx_realloc_sizes_file_io] bench = obj_tx_realloc data-size = 4 min-size = 1 realloc-size = 256:*2:16384 lib = dram type-number = rand # obj_tx_realloc benchmark # variable reallocation size # abort every nested transaction # rand type-number # another type number in realloc than in alloc [obj_tx_realloc_sizes_atomic] bench = obj_tx_realloc data-size = 4 min-size = 1 realloc-size = 256:*2:16384 lib = pmem changed-type = true type-number = rand # obj_tx_add_range benchmark # variable allocation size # allocate different objects # in one transaction # rand type-number [obj_tx_add_sizes_all_obj] bench = obj_tx_add_range data-size = 128:*2:16384 operation = all-obj type-number = rand # obj_tx_add_range benchmark # variable operations number # allocate different objects # in one transaction # rand type-number [obj_tx_add_ops_all_obj] bench = obj_tx_add_range data-size = 512 operation = all-obj ops-per-thread = 1:*5:625 type-number = rand # obj_tx_add_range benchmark # variable allocation size # allocate one object # in one transaction # rand type-number [obj_tx_add_sizes_one_obj] bench = obj_tx_add_range data-size = 128:*2:16384 operation = basic type-number = rand # obj_tx_add_range benchmark # variable operations number # allocate one object # in one transaction # rand type-number [obj_tx_add_ops_one_obj] bench = obj_tx_add_range data-size = 10000 operation = basic ops-per-thread = 1:*5:78125 type-number = rand # obj_tx_add_range benchmark # variable allocation size # allocate parts of one object # in one transaction # rand type-number [obj_tx_add_sizes_range] bench = obj_tx_add_range data-size = 128:*2:16384 operation = range type-number = rand # obj_tx_add_range benchmark # variable operations number # allocate parts of one object # in one transaction # rand type-number [obj_tx_add_ops_range] bench = obj_tx_add_range data-size = 10000 operation = range ops-per-thread = 1:*5:625 type-number = rand # obj_tx_add_range benchmark # variable allocation size # allocate all objects # in many nested transactions # rand type-number [obj_tx_add_sizes_all_obj_nested] bench = obj_tx_add_range data-size = 128:*2:16384 operation = all-obj-nested type-number = rand # obj_tx_add_range benchmark # variable operations number # allocate all objects # in many nested transactions # rand type-number [obj_tx_add_ops_all_obj_nested] bench = obj_tx_add_range data-size = 10000 operation = all-obj-nested ops-per-thread = 1:*5:625 type-number = rand # obj_tx_add_range benchmark # variable allocation size # allocate one object # in many nested transactions # rand type-number [obj_tx_add_sizes_one_obj_nested] bench = obj_tx_add_range data-size = 128:*2:16384 operation = one-obj-nested type-number = rand # obj_tx_add_range benchmark # variable operations number # allocate one object # in many nested transactions # rand type-number [obj_tx_add_ops_one_obj_nested] bench = obj_tx_add_range data-size = 10000 operation = one-obj-nested ops-per-thread = 1:*5:625 type-number = rand # obj_tx_add_range benchmark # variable allocation size # allocate parts of one object # in many nested transactions # rand type-number [obj_tx_add_sizes_range_nested] bench = obj_tx_add_range data-size = 128:*2:16384 operation = range-nested type-number = rand # obj_tx_add_range benchmark # variable operations number # allocate parts of one objects # in many nested transactions # rand type-number [obj_tx_add_ops_range_nested] bench = obj_tx_add_range data-size = 10000 operation = range-nested ops-per-thread = 1:*5:625 type-number = rand pmdk-1.8/src/benchmarks/rpmem.cpp0000664000000000000000000006037513615011243015554 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rpmem.cpp -- rpmem benchmarks definition */ #include #include #include #include #include #include #include #include #include #include #include "benchmark.hpp" #include "libpmem.h" #include "librpmem.h" #include "os.h" #include "set.h" #include "util.h" #define CL_ALIGNMENT 64 #define MAX_OFFSET (CL_ALIGNMENT - 1) #define ALIGN_CL(x) (((x) + CL_ALIGNMENT - 1) & ~(CL_ALIGNMENT - 1)) #define BENCH_RPMEM_FLUSH_NAME "rpmem_flush_drain" #define BENCH_RPMEM_PERSIST_NAME "rpmem_persist" #define BENCH_RPMEM_MIXED_NAME "rpmem_mixed" /* * rpmem_args -- benchmark specific command line options */ struct rpmem_args { char *mode; /* operation mode: stat, seq, rand */ bool no_warmup; /* do not do warmup */ bool no_memset; /* do not call memset before each persist */ size_t chunk_size; /* elementary chunk size */ size_t dest_off; /* destination address offset */ bool relaxed; /* use RPMEM_PERSIST_RELAXED / RPMEM_FLUSH_RELAXED flag */ char *workload; /* workload */ int flushes_per_drain; /* # of flushes between drains */ }; /* * rpmem_bench -- benchmark context */ struct rpmem_bench { struct rpmem_args *pargs; /* benchmark specific arguments */ size_t *offsets; /* random/sequential address offsets */ size_t n_offsets; /* number of random elements */ size_t *offsets_pos; /* position within offsets */ int const_b; /* memset() value */ size_t min_size; /* minimum file size */ void *addrp; /* mapped file address */ void *pool; /* memory pool address */ size_t pool_size; /* size of memory pool */ size_t mapped_len; /* mapped length */ RPMEMpool **rpp; /* rpmem pool pointers */ unsigned *nlanes; /* number of lanes for each remote replica */ unsigned nreplicas; /* number of remote replicas */ size_t csize_align; /* aligned elementary chunk size */ unsigned *flags; /* flags for ops */ size_t workload_len; /* length of the workload */ unsigned n_flushing_ops_per_thread; /* # of operation which require offsets per thread */ }; /* * operation_mode -- mode of operation */ enum operation_mode { OP_MODE_UNKNOWN, OP_MODE_STAT, /* always use the same chunk */ OP_MODE_SEQ, /* use consecutive chunks */ OP_MODE_RAND, /* use random chunks */ OP_MODE_SEQ_WRAP, /* use consecutive chunks, but use file size */ OP_MODE_RAND_WRAP, /* use random chunks, but use file size */ }; /* * parse_op_mode -- parse operation mode from string */ static enum operation_mode parse_op_mode(const char *arg) { if (strcmp(arg, "stat") == 0) return OP_MODE_STAT; else if (strcmp(arg, "seq") == 0) return OP_MODE_SEQ; else if (strcmp(arg, "rand") == 0) return OP_MODE_RAND; else if (strcmp(arg, "seq-wrap") == 0) return OP_MODE_SEQ_WRAP; else if (strcmp(arg, "rand-wrap") == 0) return OP_MODE_RAND_WRAP; else return OP_MODE_UNKNOWN; } /* * get_flushing_op_num -- return # of operations in the workload which require * offsets */ static unsigned get_flushing_op_num(struct benchmark *bench, struct rpmem_bench *mb) { assert(bench); struct benchmark_info *info = pmembench_get_info(bench); assert(info); /* * The rpmem_persist benchmark does one rpmem_persist() per worker op. * The rpmem_flush_drain benchmark does one rpmem_flush() or * rpmem_flush() + rpmem_drain() per worker op. Either way, it * requires one offset per worker op. */ if (strcmp(info->name, BENCH_RPMEM_PERSIST_NAME) == 0 || strcmp(info->name, BENCH_RPMEM_FLUSH_NAME) == 0) return 1; assert(strcmp(info->name, BENCH_RPMEM_MIXED_NAME) == 0); assert(mb); assert(mb->pargs); assert(mb->pargs->workload); assert(mb->workload_len > 0); unsigned num = 0; /* * The rpmem_mixed benchmark performs multiple API calls per worker * op some of them flushes ergo requires its own offset. */ for (size_t i = 0; i < mb->workload_len; ++i) { switch (mb->pargs->workload[i]) { case 'f': /* rpmem_flush */ case 'g': /* rpmem_flush + RPMEM_FLUSH_RELAXED */ case 'p': /* rpmem_persist */ case 'r': /* rpmem_persist + RPMEM_PERSIST_RELAXED */ ++num; break; } } /* * To simplify checks it is assumed each worker op requires at least one * flushing operation even though it doesn't have to use it. */ if (num < 1) num = 1; return num; } /* * init_offsets -- initialize offsets[] array depending on the selected mode */ static int init_offsets(struct benchmark_args *args, struct rpmem_bench *mb, enum operation_mode op_mode) { size_t n_ops_by_size = (mb->pool_size - POOL_HDR_SIZE) / (args->n_threads * mb->csize_align); mb->n_offsets = mb->n_flushing_ops_per_thread * args->n_threads; mb->offsets = (size_t *)malloc(mb->n_offsets * sizeof(*mb->offsets)); if (!mb->offsets) { perror("malloc"); return -1; } mb->offsets_pos = (size_t *)calloc(args->n_threads, sizeof(size_t)); if (!mb->offsets_pos) { perror("calloc"); free(mb->offsets); return -1; } unsigned seed = args->seed; for (size_t i = 0; i < args->n_threads; i++) { for (size_t j = 0; j < mb->n_flushing_ops_per_thread; j++) { size_t off_idx = i * mb->n_flushing_ops_per_thread + j; size_t chunk_idx; switch (op_mode) { case OP_MODE_STAT: chunk_idx = i; break; case OP_MODE_SEQ: chunk_idx = i * mb->n_flushing_ops_per_thread + j; break; case OP_MODE_RAND: chunk_idx = i * mb->n_flushing_ops_per_thread + os_rand_r(&seed) % mb->n_flushing_ops_per_thread; break; case OP_MODE_SEQ_WRAP: chunk_idx = i * n_ops_by_size + j % n_ops_by_size; break; case OP_MODE_RAND_WRAP: chunk_idx = i * n_ops_by_size + os_rand_r(&seed) % n_ops_by_size; break; default: assert(0); return -1; } mb->offsets[off_idx] = POOL_HDR_SIZE + chunk_idx * mb->csize_align + mb->pargs->dest_off; } } return 0; } /* * do_warmup -- does the warmup by writing the whole pool area */ static int do_warmup(struct rpmem_bench *mb) { /* clear the entire pool */ memset((char *)mb->pool + POOL_HDR_SIZE, 0, mb->pool_size - POOL_HDR_SIZE); for (unsigned r = 0; r < mb->nreplicas; ++r) { int ret = rpmem_persist(mb->rpp[r], POOL_HDR_SIZE, mb->pool_size - POOL_HDR_SIZE, 0, RPMEM_PERSIST_RELAXED); if (ret) return ret; } /* if no memset for each operation, do one big memset */ if (mb->pargs->no_memset) { memset((char *)mb->pool + POOL_HDR_SIZE, 0xFF, mb->pool_size - POOL_HDR_SIZE); } return 0; } /* * rpmem_mixed_op_flush -- perform rpmem_flush */ static inline int rpmem_mixed_op_flush(struct rpmem_bench *mb, struct operation_info *info) { size_t *pos = &mb->offsets_pos[info->worker->index]; uint64_t idx = info->worker->index * mb->n_flushing_ops_per_thread + *pos; assert(idx < mb->n_offsets); size_t offset = mb->offsets[idx]; size_t len = mb->pargs->chunk_size; if (!mb->pargs->no_memset) { void *dest = (char *)mb->pool + offset; /* thread id on MS 4 bits and operation id on LS 4 bits */ int c = ((info->worker->index & 0xf) << 4) + ((0xf & info->index)); memset(dest, c, len); } int ret = 0; for (unsigned r = 0; r < mb->nreplicas; ++r) { assert(info->worker->index < mb->nlanes[r]); ret = rpmem_flush(mb->rpp[r], offset, len, info->worker->index, mb->flags[info->worker->index]); if (ret) { fprintf(stderr, "rpmem_flush replica #%u: %s\n", r, rpmem_errormsg()); return ret; } } ++*pos; return 0; } /* * rpmem_mixed_op_drain -- perform rpmem_drain */ static inline int rpmem_mixed_op_drain(struct rpmem_bench *mb, struct operation_info *info) { int ret = 0; for (unsigned r = 0; r < mb->nreplicas; ++r) { ret = rpmem_drain(mb->rpp[r], info->worker->index, 0); if (unlikely(ret)) { fprintf(stderr, "rpmem_drain replica #%u: %s\n", r, rpmem_errormsg()); return ret; } } return 0; } /* * rpmem_flush_drain_op -- actual benchmark operation for the rpmem_flush_drain * benchmark */ static int rpmem_flush_drain_op(struct benchmark *bench, struct operation_info *info) { auto *mb = (struct rpmem_bench *)pmembench_get_priv(bench); int ret = 0; if (mb->pargs->flushes_per_drain != 0) { ret |= rpmem_mixed_op_flush(mb, info); /* no rpmem_drain() required */ if (mb->pargs->flushes_per_drain < 0) return ret; /* more rpmem_flush() required before rpmem_drain() */ if ((info->index + 1) % mb->pargs->flushes_per_drain != 0) return ret; /* rpmem_drain() required */ } ret |= rpmem_mixed_op_drain(mb, info); return ret; } /* * rpmem_persist_op -- actual benchmark operation for the rpmem_persist * benchmark */ static int rpmem_persist_op(struct benchmark *bench, struct operation_info *info) { auto *mb = (struct rpmem_bench *)pmembench_get_priv(bench); size_t *pos = &mb->offsets_pos[info->worker->index]; uint64_t idx = info->worker->index * mb->n_flushing_ops_per_thread + *pos; assert(idx < mb->n_offsets); size_t offset = mb->offsets[idx]; size_t len = mb->pargs->chunk_size; if (!mb->pargs->no_memset) { void *dest = (char *)mb->pool + offset; /* thread id on MS 4 bits and operation id on LS 4 bits */ int c = ((info->worker->index & 0xf) << 4) + ((0xf & info->index)); memset(dest, c, len); } int ret = 0; for (unsigned r = 0; r < mb->nreplicas; ++r) { assert(info->worker->index < mb->nlanes[r]); ret = rpmem_persist(mb->rpp[r], offset, len, info->worker->index, mb->flags[info->worker->index]); if (ret) { fprintf(stderr, "rpmem_persist replica #%u: %s\n", r, rpmem_errormsg()); return ret; } } ++*pos; return 0; } /* * rpmem_mixed_op -- actual benchmark operation for the rpmem_mixed * benchmark */ static int rpmem_mixed_op(struct benchmark *bench, struct operation_info *info) { auto *mb = (struct rpmem_bench *)pmembench_get_priv(bench); assert(mb->workload_len != 0); int ret = 0; for (size_t i = 0; i < mb->workload_len; ++i) { char op = mb->pargs->workload[i]; mb->flags[info->worker->index] = 0; switch (op) { case 'g': mb->flags[info->worker->index] = RPMEM_FLUSH_RELAXED; /* no break here */ case 'f': ret |= rpmem_mixed_op_flush(mb, info); break; case 'd': ret |= rpmem_mixed_op_drain(mb, info); break; case 'r': mb->flags[info->worker->index] = RPMEM_PERSIST_RELAXED; /* no break here */ case 'p': ret |= rpmem_persist_op(bench, info); break; default: fprintf(stderr, "unknown operation: %c", op); return 1; } } return ret; } /* * rpmem_map_file -- map local file */ static int rpmem_map_file(const char *path, struct rpmem_bench *mb, size_t size) { int mode; #ifndef _WIN32 mode = S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH; #else mode = S_IWRITE | S_IREAD; #endif mb->addrp = pmem_map_file(path, size, PMEM_FILE_CREATE, mode, &mb->mapped_len, nullptr); if (!mb->addrp) return -1; return 0; } /* * rpmem_unmap_file -- unmap local file */ static int rpmem_unmap_file(struct rpmem_bench *mb) { return pmem_unmap(mb->addrp, mb->mapped_len); } /* * rpmem_poolset_init -- read poolset file and initialize benchmark accordingly */ static int rpmem_poolset_init(const char *path, struct rpmem_bench *mb, struct benchmark_args *args) { struct pool_set *set; struct pool_replica *rep; struct remote_replica *remote; struct pool_set_part *part; struct rpmem_pool_attr attr; memset(&attr, 0, sizeof(attr)); memcpy(attr.signature, "PMEMBNCH", sizeof(attr.signature)); /* read and validate poolset */ if (util_poolset_read(&set, path)) { fprintf(stderr, "Invalid poolset file '%s'\n", path); return -1; } assert(set); if (set->nreplicas < 2) { fprintf(stderr, "No replicas defined\n"); goto err_poolset_free; } if (set->remote == 0) { fprintf(stderr, "No remote replicas defined\n"); goto err_poolset_free; } for (unsigned i = 1; i < set->nreplicas; ++i) { if (!set->replica[i]->remote) { fprintf(stderr, "Local replicas are not supported\n"); goto err_poolset_free; } } /* read and validate master replica */ rep = set->replica[0]; assert(rep); assert(rep->remote == nullptr); if (rep->nparts != 1) { fprintf(stderr, "Multipart master replicas are not supported\n"); goto err_poolset_free; } if (rep->repsize < mb->min_size) { fprintf(stderr, "A master replica is too small (%zu < %zu)\n", rep->repsize, mb->min_size); goto err_poolset_free; } part = (struct pool_set_part *)&rep->part[0]; if (rpmem_map_file(part->path, mb, rep->repsize)) { perror(part->path); goto err_poolset_free; } mb->pool_size = mb->mapped_len; mb->pool = (void *)((uintptr_t)mb->addrp); /* prepare remote replicas */ mb->nreplicas = set->nreplicas - 1; mb->nlanes = (unsigned *)malloc(mb->nreplicas * sizeof(unsigned)); if (mb->nlanes == nullptr) { perror("malloc"); goto err_unmap_file; } mb->rpp = (RPMEMpool **)malloc(mb->nreplicas * sizeof(RPMEMpool *)); if (mb->rpp == nullptr) { perror("malloc"); goto err_free_lanes; } unsigned r; for (r = 0; r < mb->nreplicas; ++r) { remote = set->replica[r + 1]->remote; assert(remote); mb->nlanes[r] = args->n_threads; /* Temporary WA for librpmem issue */ ++mb->nlanes[r]; mb->rpp[r] = rpmem_create(remote->node_addr, remote->pool_desc, mb->addrp, mb->pool_size, &mb->nlanes[r], &attr); if (!mb->rpp[r]) { perror("rpmem_create"); goto err_rpmem_close; } if (mb->nlanes[r] < args->n_threads) { fprintf(stderr, "Number of threads too large for replica #%u (max: %u)\n", r, mb->nlanes[r]); r++; /* close current replica */ goto err_rpmem_close; } } util_poolset_free(set); return 0; err_rpmem_close: for (unsigned i = 0; i < r; i++) rpmem_close(mb->rpp[i]); free(mb->rpp); err_free_lanes: free(mb->nlanes); err_unmap_file: rpmem_unmap_file(mb); err_poolset_free: util_poolset_free(set); return -1; } /* * rpmem_poolset_fini -- close opened local and remote replicas */ static void rpmem_poolset_fini(struct rpmem_bench *mb) { for (unsigned r = 0; r < mb->nreplicas; ++r) { rpmem_close(mb->rpp[r]); } free(mb->rpp); rpmem_unmap_file(mb); } /* * rpmem_set_min_size -- compute minimal file size based on benchmark arguments */ static void rpmem_set_min_size(struct rpmem_bench *mb, enum operation_mode op_mode, struct benchmark_args *args) { mb->csize_align = ALIGN_CL(mb->pargs->chunk_size); switch (op_mode) { case OP_MODE_STAT: mb->min_size = mb->csize_align * args->n_threads; break; case OP_MODE_SEQ: case OP_MODE_RAND: mb->min_size = mb->csize_align * args->n_ops_per_thread * args->n_threads; break; case OP_MODE_SEQ_WRAP: case OP_MODE_RAND_WRAP: /* * at least one chunk per thread to avoid false sharing */ mb->min_size = mb->csize_align * args->n_threads; break; default: assert(0); } mb->min_size += POOL_HDR_SIZE; } /* * rpmem_flags_init -- initialize flags[] array depending on the selected mode */ static int rpmem_flags_init(struct benchmark *bench, struct benchmark_args *args, struct rpmem_bench *mb) { assert(bench); struct benchmark_info *info = pmembench_get_info(bench); assert(info); mb->flags = (unsigned *)calloc(args->n_threads, sizeof(unsigned)); if (!mb->flags) { perror("calloc"); return -1; } unsigned relaxed_flag = 0; if (strcmp(info->name, BENCH_RPMEM_PERSIST_NAME) == 0) relaxed_flag = RPMEM_PERSIST_RELAXED; else if (strcmp(info->name, BENCH_RPMEM_FLUSH_NAME) == 0) relaxed_flag = RPMEM_FLUSH_RELAXED; /* for rpmem_mixed benchmark flags are set during the benchmark */ /* for rpmem_persist and rpmem_flush_drain benchmark all ops have the * same flags */ if (mb->pargs->relaxed) { for (unsigned i = 0; i < args->n_threads; ++i) mb->flags[i] = relaxed_flag; } return 0; } /* * rpmem_init -- initialization function */ static int rpmem_init(struct benchmark *bench, struct benchmark_args *args) { assert(bench != nullptr); assert(args != nullptr); assert(args->opts != nullptr); auto *mb = (struct rpmem_bench *)malloc(sizeof(struct rpmem_bench)); if (!mb) { perror("malloc"); return -1; } mb->pargs = (struct rpmem_args *)args->opts; mb->pargs->chunk_size = args->dsize; enum operation_mode op_mode = parse_op_mode(mb->pargs->mode); if (op_mode == OP_MODE_UNKNOWN) { fprintf(stderr, "Invalid operation mode argument '%s'\n", mb->pargs->mode); goto err_parse_mode; } if (rpmem_flags_init(bench, args, mb)) goto err_flags_init; mb->workload_len = 0; if (mb->pargs->workload) { mb->workload_len = strlen(mb->pargs->workload); assert(mb->workload_len > 0); } rpmem_set_min_size(mb, op_mode, args); if (rpmem_poolset_init(args->fname, mb, args)) { goto err_poolset_init; } /* initialize offsets[] array depending on benchmark args */ mb->n_flushing_ops_per_thread = get_flushing_op_num(bench, mb) * args->n_ops_per_thread; if (init_offsets(args, mb, op_mode) < 0) { goto err_init_offsets; } if (!mb->pargs->no_warmup) { if (do_warmup(mb) != 0) { fprintf(stderr, "do_warmup() function failed.\n"); goto err_warmup; } } pmembench_set_priv(bench, mb); return 0; err_warmup: free(mb->offsets_pos); free(mb->offsets); err_init_offsets: rpmem_poolset_fini(mb); err_poolset_init: free(mb->flags); err_flags_init: err_parse_mode: free(mb); return -1; } /* * rpmem_exit -- benchmark cleanup function */ static int rpmem_exit(struct benchmark *bench, struct benchmark_args *args) { auto *mb = (struct rpmem_bench *)pmembench_get_priv(bench); rpmem_poolset_fini(mb); free(mb->offsets_pos); free(mb->offsets); free(mb->flags); free(mb); return 0; } static struct benchmark_clo rpmem_flush_clo[6]; static struct benchmark_clo rpmem_persist_clo[5]; static struct benchmark_clo rpmem_mixed_clo[5]; /* Stores information about benchmark. */ static struct benchmark_info rpmem_flush_info; static struct benchmark_info rpmem_persist_info; static struct benchmark_info rpmem_mixed_info; CONSTRUCTOR(rpmem_constructor) void rpmem_constructor(void) { static struct benchmark_clo common_clo[4]; static struct benchmark_info common_info; memset(&common_info, 0, sizeof(common_info)); /* common benchmarks definitions */ common_clo[0].opt_short = 'M'; common_clo[0].opt_long = "mem-mode"; common_clo[0].descr = "Memory writing mode :" " stat, seq[-wrap], rand[-wrap]"; common_clo[0].def = "seq"; common_clo[0].off = clo_field_offset(struct rpmem_args, mode); common_clo[0].type = CLO_TYPE_STR; common_clo[1].opt_short = 'D'; common_clo[1].opt_long = "dest-offset"; common_clo[1].descr = "Destination cache line " "alignment offset"; common_clo[1].def = "0"; common_clo[1].off = clo_field_offset(struct rpmem_args, dest_off); common_clo[1].type = CLO_TYPE_UINT; common_clo[1].type_uint.size = clo_field_size(struct rpmem_args, dest_off); common_clo[1].type_uint.base = CLO_INT_BASE_DEC; common_clo[1].type_uint.min = 0; common_clo[1].type_uint.max = MAX_OFFSET; common_clo[2].opt_short = 'w'; common_clo[2].opt_long = "no-warmup"; common_clo[2].descr = "Don't do warmup"; common_clo[2].def = "false"; common_clo[2].type = CLO_TYPE_FLAG; common_clo[2].off = clo_field_offset(struct rpmem_args, no_warmup); common_clo[3].opt_short = 'T'; common_clo[3].opt_long = "no-memset"; common_clo[3].descr = "Don't call memset for all rpmem_persist"; common_clo[3].def = "false"; common_clo[3].off = clo_field_offset(struct rpmem_args, no_memset); common_clo[3].type = CLO_TYPE_FLAG; common_info.init = rpmem_init; common_info.exit = rpmem_exit; common_info.multithread = true; common_info.multiops = true; common_info.measure_time = true; common_info.opts_size = sizeof(struct rpmem_args); common_info.rm_file = true; common_info.allow_poolset = true; common_info.print_bandwidth = true; /* rpmem_flush_drain benchmark definitions */ assert(sizeof(rpmem_flush_clo) >= sizeof(common_clo)); memcpy(rpmem_flush_clo, common_clo, sizeof(common_clo)); rpmem_flush_clo[4].opt_short = 0; rpmem_flush_clo[4].opt_long = "flushes-per-drain"; rpmem_flush_clo[4].descr = "Number of flushes between drains (-1 means flushes only)"; rpmem_flush_clo[4].def = "-1"; rpmem_flush_clo[4].off = clo_field_offset(struct rpmem_args, flushes_per_drain); rpmem_flush_clo[4].type = CLO_TYPE_INT; rpmem_flush_clo[4].type_int.size = clo_field_size(struct rpmem_args, flushes_per_drain); rpmem_flush_clo[4].type_int.base = CLO_INT_BASE_DEC; rpmem_flush_clo[4].type_int.min = -1; rpmem_flush_clo[4].type_int.max = INT_MAX; rpmem_flush_clo[5].opt_short = 0; rpmem_flush_clo[5].opt_long = "flush-relaxed"; rpmem_flush_clo[5].descr = "Use RPMEM_FLUSH_RELAXED flag"; rpmem_flush_clo[5].def = "false"; rpmem_flush_clo[5].off = clo_field_offset(struct rpmem_args, relaxed); rpmem_flush_clo[5].type = CLO_TYPE_FLAG; memcpy(&rpmem_flush_info, &common_info, sizeof(common_info)); rpmem_flush_info.name = BENCH_RPMEM_FLUSH_NAME; rpmem_flush_info.brief = "Benchmark for rpmem_flush() and rpmem_drain() operations"; rpmem_flush_info.operation = rpmem_flush_drain_op; rpmem_flush_info.clos = rpmem_flush_clo; rpmem_flush_info.nclos = ARRAY_SIZE(rpmem_flush_clo); REGISTER_BENCHMARK(rpmem_flush_info); /* rpmem_persist benchmark definitions */ assert(sizeof(rpmem_persist_clo) >= sizeof(common_clo)); memcpy(rpmem_persist_clo, common_clo, sizeof(common_clo)); rpmem_persist_clo[4].opt_short = 0; rpmem_persist_clo[4].opt_long = "persist-relaxed"; rpmem_persist_clo[4].descr = "Use RPMEM_PERSIST_RELAXED flag"; rpmem_persist_clo[4].def = "false"; rpmem_persist_clo[4].off = clo_field_offset(struct rpmem_args, relaxed); rpmem_persist_clo[4].type = CLO_TYPE_FLAG; memcpy(&rpmem_persist_info, &common_info, sizeof(common_info)); rpmem_persist_info.name = BENCH_RPMEM_PERSIST_NAME; rpmem_persist_info.brief = "Benchmark for rpmem_persist() operation"; rpmem_persist_info.operation = rpmem_persist_op; rpmem_persist_info.clos = rpmem_persist_clo; rpmem_persist_info.nclos = ARRAY_SIZE(rpmem_persist_clo); REGISTER_BENCHMARK(rpmem_persist_info); /* rpmem_mixed benchmark definitions */ assert(sizeof(rpmem_mixed_clo) >= sizeof(common_clo)); memcpy(rpmem_mixed_clo, common_clo, sizeof(common_clo)); rpmem_mixed_clo[4].opt_short = 0; rpmem_mixed_clo[4].opt_long = "workload"; rpmem_mixed_clo[4].descr = "Workload e.g.: 'prfgd' means " "rpmem_persist(), " "rpmem_persist() + RPMEM_PERSIST_RELAXED, " "rpmem_flush()," "rpmem_flush() + RPMEM_FLUSH_RELAXED " "and rpmem_drain()"; rpmem_mixed_clo[4].def = "fd"; rpmem_mixed_clo[4].off = clo_field_offset(struct rpmem_args, workload); rpmem_mixed_clo[4].type = CLO_TYPE_STR; memcpy(&rpmem_mixed_info, &common_info, sizeof(common_info)); rpmem_mixed_info.name = BENCH_RPMEM_MIXED_NAME; rpmem_mixed_info.brief = "Benchmark for mixed rpmem workloads"; rpmem_mixed_info.operation = rpmem_mixed_op; rpmem_mixed_info.clos = rpmem_mixed_clo; rpmem_mixed_info.nclos = ARRAY_SIZE(rpmem_mixed_clo); REGISTER_BENCHMARK(rpmem_mixed_info); }; pmdk-1.8/src/README0000664000000000000000000000117013615011243012457 0ustar rootrootPersistent Memory Development Kit This is src/README. This directory contains the source for the Persistent Memory Development Kit. The subdirectory "include" contains header files that get delivered along with the libraries. Everything else is internal to the libraries and lives in this directory. Two versions of the libraries are built, a debug version and a nondebug version. The object files and the libraries themselves end up in the subdirectories "debug" and "nondebug". See the top-level README for build, test, and installation instructions. The basic "make" and "make test" targets also work from this directory. pmdk-1.8/src/examples/0000775000000000000000000000000013615011243013416 5ustar rootrootpmdk-1.8/src/examples/ex_common.h0000664000000000000000000000523213615011243015555 0ustar rootroot/* * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ex_common.h -- examples utilities */ #ifndef EX_COMMON_H #define EX_COMMON_H #include #define MIN(a, b) (((a) < (b)) ? (a) : (b)) #ifdef __cplusplus extern "C" { #endif #ifndef _WIN32 #include #define CREATE_MODE_RW (S_IWUSR | S_IRUSR) /* * file_exists -- checks if file exists */ static inline int file_exists(char const *file) { return access(file, F_OK); } /* * find_last_set_64 -- returns last set bit position or -1 if set bit not found */ static inline int find_last_set_64(uint64_t val) { return 64 - __builtin_clzll(val) - 1; } #else #include #include #include #define CREATE_MODE_RW (S_IWRITE | S_IREAD) /* * file_exists -- checks if file exists */ static inline int file_exists(char const *file) { return _access(file, 0); } /* * find_last_set_64 -- returns last set bit position or -1 if set bit not found */ static inline int find_last_set_64(uint64_t val) { DWORD lz = 0; if (BitScanReverse64(&lz, val)) return (int)lz; else return -1; } #endif #ifdef __cplusplus } #endif #endif /* ex_common.h */ pmdk-1.8/src/examples/libpmemblk/0000775000000000000000000000000013615011243015534 5ustar rootrootpmdk-1.8/src/examples/libpmemblk/manpage.vcxproj0000664000000000000000000000532213615011243020563 0ustar rootroot Debug x64 Release x64 {8010BBB0-C71B-4EFF-95EB-65C01E5EC197} pmemblk 10.0.16299.0 Application true v140 Application false v140 true ..\..\LongPath.manifest libpmem.lib;libpmemblk.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) {f7c6c6b6-4142-4c82-8699-4a9d8183181b} {9e9e3d25-2139-4a5d-9200-18148ddead45} pmdk-1.8/src/examples/libpmemblk/manpage.vcxproj.filters0000664000000000000000000000125413615011243022232 0ustar rootroot {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {93995380-89BD-4b04-88EB-625FBE52EBFB} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files pmdk-1.8/src/examples/libpmemblk/assetdb/0000775000000000000000000000000013615011243017161 5ustar rootrootpmdk-1.8/src/examples/libpmemblk/assetdb/asset_checkout.c0000664000000000000000000000611513615011243022334 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * asset_checkout -- mark an asset as checked out to someone * * Usage: * asset_checkin /path/to/pm-aware/file asset-ID name */ #include #include #include #include #include #include #include #include #include "asset.h" int main(int argc, char *argv[]) { PMEMblkpool *pbp; struct asset asset; int assetid; if (argc < 4) { fprintf(stderr, "usage: %s assetdb asset-ID name\n", argv[0]); exit(1); } const char *path = argv[1]; assetid = atoi(argv[2]); assert(assetid > 0); /* open an array of atomically writable elements */ if ((pbp = pmemblk_open(path, sizeof(struct asset))) == NULL) { perror("pmemblk_open"); exit(1); } /* read a required element in */ if (pmemblk_read(pbp, &asset, assetid) < 0) { perror("pmemblk_read"); exit(1); } /* check if it contains any data */ if ((asset.state != ASSET_FREE) && (asset.state != ASSET_CHECKED_OUT)) { fprintf(stderr, "Asset ID %d not found", assetid); exit(1); } if (asset.state == ASSET_CHECKED_OUT) { fprintf(stderr, "Asset ID %d already checked out\n", assetid); exit(1); } /* update user name, set checked out state, and take timestamp */ strncpy(asset.user, argv[3], ASSET_USER_NAME_MAX - 1); asset.user[ASSET_USER_NAME_MAX - 1] = '\0'; asset.state = ASSET_CHECKED_OUT; time(&asset.time); /* put it back in the block */ if (pmemblk_write(pbp, &asset, assetid) < 0) { perror("pmemblk_write"); exit(1); } pmemblk_close(pbp); return 0; } pmdk-1.8/src/examples/libpmemblk/assetdb/asset_checkin.vcxproj.filters0000664000000000000000000000144313615011243025052 0ustar rootroot {7b7d2f5a-464e-4d55-894b-3110e15303f1} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {c2e783a4-4f6d-419d-bbf6-1056eddfdd8f} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files Header Files pmdk-1.8/src/examples/libpmemblk/assetdb/asset_list.vcxproj.filters0000664000000000000000000000144013615011243024416 0ustar rootroot {7b7d2f5a-464e-4d55-894b-3110e15303f1} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {c2e783a4-4f6d-419d-bbf6-1056eddfdd8f} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files Header Files pmdk-1.8/src/examples/libpmemblk/assetdb/asset_list.vcxproj0000664000000000000000000000523013615011243022750 0ustar rootroot Debug x64 Release x64 {8008010F-8718-4C5F-86B2-195AEBF73422} pmemblk 10.0.16299.0 {f7c6c6b6-4142-4c82-8699-4a9d8183181b} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemblk.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemblk/assetdb/asset_checkout.vcxproj.filters0000664000000000000000000000144413615011243025254 0ustar rootroot {7b7d2f5a-464e-4d55-894b-3110e15303f1} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {c2e783a4-4f6d-419d-bbf6-1056eddfdd8f} h;hh;hpp;hxx;hm;inl;inc;xsd Header Files Source Files pmdk-1.8/src/examples/libpmemblk/assetdb/README0000664000000000000000000000327313615011243020046 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemblk/assetdb/README. This example illustrates how a very simple database can be built using the atomic arrays provided by libpmemblk. An array of records is created to track a list of assets (like books or computers, for example). Each asset is represented by a fixed-length record in the asset database. Since updates to a single record are atomic, the file remains consistent even if interrupted during an update. To run this example, follow these steps: 0. Build the example with "make". The libraries must be built first (i.e. by running "make" in ../../..). 1. Create the assetdb file. This can be anywhere but the point is it will be a much faster database file if it is created on a pmem-aware file system. For example, if /pmem is the mount point for a pmem-aware file system: $ fallocate -l 1G /pmem/assetdb 2. Load up the assets from a list of assets, which is just a simple text file containing asset description strings, one per line: $ asset_load /pmem/assetdb assetlist 3. Print the assets using: $ asset_list /pmem/assetdb 4. To mark an asset as checked out to someone: $ asset_checkout /pmem/assetdb asset-ID name The asset-ID is the ID printed by asset_list. The name may require quotes if it contains any whitespace or special characters. 5. To mark an asset as no longer checked out: $ asset_checkin /pmem/assetdb asset-ID As this is just an example to illustrate how pmemblk works, it may take some trial-and-error to find the best size for the assetdb -- the file must be large enough to hold about 512 bytes for each asset record, plus about 1% overhead used by libpmemblk to provide the atomicity. pmdk-1.8/src/examples/libpmemblk/assetdb/asset_checkin.vcxproj0000664000000000000000000000523313615011243023404 0ustar rootroot Debug x64 Release x64 {581B3A58-F3F0-4765-91E5-D0C82816A528} pmemblk 10.0.16299.0 {f7c6c6b6-4142-4c82-8699-4a9d8183181b} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemblk.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemblk/assetdb/.gitignore0000664000000000000000000000006313615011243021150 0ustar rootrootasset_load asset_list asset_checkout asset_checkin pmdk-1.8/src/examples/libpmemblk/assetdb/asset_load.vcxproj.filters0000664000000000000000000000144013615011243024362 0ustar rootroot {7b7d2f5a-464e-4d55-894b-3110e15303f1} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {c2e783a4-4f6d-419d-bbf6-1056eddfdd8f} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files Header Files pmdk-1.8/src/examples/libpmemblk/assetdb/asset_checkout.vcxproj0000664000000000000000000000523413615011243023606 0ustar rootroot Debug x64 Release x64 {513C4CFA-BD5B-4470-BA93-F6D43778A754} pmemblk 10.0.16299.0 {f7c6c6b6-4142-4c82-8699-4a9d8183181b} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemblk.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemblk/assetdb/asset_checkin.c0000664000000000000000000000551213615011243022133 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * asset_checkin -- mark an asset as no longer checked out * * Usage: * asset_checkin /path/to/pm-aware/file asset-ID */ #include #include #include #include #include #include #include #include #include "asset.h" int main(int argc, char *argv[]) { PMEMblkpool *pbp; struct asset asset; int assetid; if (argc < 3) { fprintf(stderr, "usage: %s assetdb asset-ID\n", argv[0]); exit(1); } const char *path = argv[1]; assetid = atoi(argv[2]); assert(assetid > 0); /* open an array of atomically writable elements */ if ((pbp = pmemblk_open(path, sizeof(struct asset))) == NULL) { perror("pmemblk_open"); exit(1); } /* read a required element in */ if (pmemblk_read(pbp, &asset, assetid) < 0) { perror("pmemblk_read"); exit(1); } /* check if it contains any data */ if ((asset.state != ASSET_FREE) && (asset.state != ASSET_CHECKED_OUT)) { fprintf(stderr, "Asset ID %d not found\n", assetid); exit(1); } /* change state to free, clear user name and timestamp */ asset.state = ASSET_FREE; asset.user[0] = '\0'; asset.time = 0; if (pmemblk_write(pbp, &asset, assetid) < 0) { perror("pmemblk_write"); exit(1); } pmemblk_close(pbp); return 0; } pmdk-1.8/src/examples/libpmemblk/assetdb/asset.h0000664000000000000000000000342713615011243020457 0ustar rootroot/* * Copyright 2014-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #define ASSET_NAME_MAX 256 #define ASSET_USER_NAME_MAX 64 #define ASSET_CHECKED_OUT 2 #define ASSET_FREE 1 struct asset { char name[ASSET_NAME_MAX]; char user[ASSET_USER_NAME_MAX]; time_t time; int state; }; pmdk-1.8/src/examples/libpmemblk/assetdb/Makefile0000664000000000000000000000363613615011243020631 0ustar rootroot# # Copyright 2014-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemblk/assetdb/Makefile -- build the assetdb example # PROGS = asset_load asset_list asset_checkout asset_checkin LIBS = -lpmemblk -lpmem -pthread include ../../Makefile.inc asset_load: asset_load.o asset_list: asset_list.o asset_checkout: asset_checkout.o asset_checkin: asset_checkin.o asset_load.o asset_list.o asset_checkout.o asset_checkin.o: asset.h pmdk-1.8/src/examples/libpmemblk/assetdb/asset_list.c0000664000000000000000000000556613615011243021513 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * asset_list -- list all assets in an assetdb file * * Usage: * asset_list /path/to/pm-aware/file */ #include #include #include #include #include #include #include #include "asset.h" int main(int argc, char *argv[]) { PMEMblkpool *pbp; int assetid; size_t nelements; struct asset asset; if (argc < 2) { fprintf(stderr, "usage: %s assetdb\n", argv[0]); exit(1); } const char *path = argv[1]; /* open an array of atomically writable elements */ if ((pbp = pmemblk_open(path, sizeof(struct asset))) == NULL) { perror(path); exit(1); } /* how many elements do we have? */ nelements = pmemblk_nblock(pbp); /* print out all the elements that contain assets data */ for (assetid = 0; assetid < nelements; ++assetid) { if (pmemblk_read(pbp, &asset, assetid) < 0) { perror("pmemblk_read"); exit(1); } if ((asset.state != ASSET_FREE) && (asset.state != ASSET_CHECKED_OUT)) { break; } printf("Asset ID: %d\n", assetid); if (asset.state == ASSET_FREE) printf(" State: Free\n"); else { printf(" State: Checked out\n"); printf(" User: %s\n", asset.user); printf(" Time: %s", ctime(&asset.time)); } printf(" Name: %s\n", asset.name); } pmemblk_close(pbp); return 0; } pmdk-1.8/src/examples/libpmemblk/assetdb/asset_load.vcxproj0000664000000000000000000000523013615011243022714 0ustar rootroot Debug x64 Release x64 {C7E42AE1-052F-4024-B8BA-DE5DCE6BBEEC} pmemblk 10.0.16299.0 {f7c6c6b6-4142-4c82-8699-4a9d8183181b} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemblk.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemblk/assetdb/asset_load.c0000664000000000000000000000662413615011243021453 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * asset_load -- given pre-allocated assetdb file, load it up with assets * * Usage: * fallocate -l 1G /path/to/pm-aware/file * asset_load /path/to/pm-aware/file asset-file * * The asset-file should contain the names of the assets, one per line. */ #include #include #include #include #include #include #include #include #include "asset.h" int main(int argc, char *argv[]) { FILE *fp; int len = ASSET_NAME_MAX; PMEMblkpool *pbp; int assetid = 0; size_t nelements; char *line; if (argc < 3) { fprintf(stderr, "usage: %s assetdb assetlist\n", argv[0]); exit(1); } const char *path_pool = argv[1]; const char *path_list = argv[2]; /* create pmemblk pool in existing (but as yet unmodified) file */ pbp = pmemblk_create(path_pool, sizeof(struct asset), 0, CREATE_MODE_RW); if (pbp == NULL) { perror(path_pool); exit(1); } nelements = pmemblk_nblock(pbp); if ((fp = fopen(path_list, "r")) == NULL) { perror(path_list); exit(1); } /* * Read in all the assets from the assetfile and put them in the * array, if a name of the asset is longer than ASSET_NAME_SIZE_MAX, * truncate it. */ line = malloc(len); if (line == NULL) { perror("malloc"); exit(1); } while (fgets(line, len, fp) != NULL) { struct asset asset; if (assetid >= nelements) { fprintf(stderr, "%s: too many assets to fit in %s " "(only %d assets loaded)\n", path_list, path_pool, assetid); exit(1); } memset(&asset, '\0', sizeof(asset)); asset.state = ASSET_FREE; strncpy(asset.name, line, ASSET_NAME_MAX - 1); asset.name[ASSET_NAME_MAX - 1] = '\0'; if (pmemblk_write(pbp, &asset, assetid) < 0) { perror("pmemblk_write"); exit(1); } assetid++; } free(line); fclose(fp); pmemblk_close(pbp); return 0; } pmdk-1.8/src/examples/libpmemblk/manpage.c0000664000000000000000000000557113615011243017320 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * manpage.c -- simple example for the libpmemblk man page */ #include #include #include #include #ifndef _WIN32 #include #endif #include #include /* size of the pmemblk pool -- 1 GB */ #define POOL_SIZE ((size_t)(1 << 30)) /* size of each element in the pmem pool */ #define ELEMENT_SIZE 1024 int main(int argc, char *argv[]) { const char path[] = "/pmem-fs/myfile"; PMEMblkpool *pbp; size_t nelements; char buf[ELEMENT_SIZE]; /* create the pmemblk pool or open it if it already exists */ pbp = pmemblk_create(path, ELEMENT_SIZE, POOL_SIZE, 0666); if (pbp == NULL) pbp = pmemblk_open(path, ELEMENT_SIZE); if (pbp == NULL) { perror(path); exit(1); } /* how many elements fit into the file? */ nelements = pmemblk_nblock(pbp); printf("file holds %zu elements\n", nelements); /* store a block at index 5 */ strcpy(buf, "hello, world"); if (pmemblk_write(pbp, buf, 5) < 0) { perror("pmemblk_write"); exit(1); } /* read the block at index 10 (reads as zeros initially) */ if (pmemblk_read(pbp, buf, 10) < 0) { perror("pmemblk_read"); exit(1); } /* zero out the block at index 5 */ if (pmemblk_set_zero(pbp, 5) < 0) { perror("pmemblk_set_zero"); exit(1); } /* ... */ pmemblk_close(pbp); return 0; } pmdk-1.8/src/examples/libpmemblk/README0000664000000000000000000000133513615011243016416 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemblk/README. This directory contains examples for libpmemblk, the library providing pmem-resident arrays of blocks. Some of these examples are explained in more detail here: https://pmem.io/pmdk/libpmemblk manpage.c is the example used in the libpmemblk man page. assetdb contains a simple database built using libpmemblk. To build these examples: make These examples can be built against an installed system using: make LIBDIR=/usr/lib INCDIR=/usr/include If you're looking for documentation to get you started using PMDK, start here: https://pmem.io/pmdk and follow the links to examples and man pages. Developers new to PMDK are probably looking for libpmemobj. pmdk-1.8/src/examples/libpmemblk/.gitignore0000664000000000000000000000001013615011243017513 0ustar rootrootmanpage pmdk-1.8/src/examples/libpmemblk/Makefile0000664000000000000000000000332413615011243017176 0ustar rootroot# # Copyright 2014-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemblk/Makefile -- build the libpmemblk examples # PROGS = manpage DIRS = assetdb LIBS = -lpmemblk -lpmem -pthread include ../Makefile.inc manpage: manpage.o pmdk-1.8/src/examples/Makefile.inc0000664000000000000000000001135513615011243015633 0ustar rootroot# # Copyright 2015-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/Makefile.inc -- build the Persistent Memory Development Kit examples # TOP_SRC := $(dir $(lastword $(MAKEFILE_LIST))).. TOP := $(TOP_SRC)/.. HEADERS = $(wildcard *.h) $(wildcard *.hpp) INCDIR = $(TOP_SRC)/include LIBDIR = $(TOP_SRC)/debug include $(TOP)/src/common.inc CXXFLAGS = -std=c++11 -ggdb -Wall -Werror CXXFLAGS += $(GLIBC_CXXFLAGS) CXXFLAGS += $(EXTRA_CXXFLAGS) CFLAGS = -std=gnu99 -ggdb -Wall -Werror -Wmissing-prototypes $(EXTRA_CFLAGS) LDFLAGS = -Wl,-rpath=$(LIBDIR) -L$(LIBDIR) $(EXTRA_LDFLAGS) ifneq ($(SANITIZE),) CFLAGS += -fsanitize=$(SANITIZE) CXXFLAGS += -fsanitize=$(SANITIZE) LDFLAGS += -fsanitize=$(SANITIZE) endif INCS = -I$(INCDIR) -I. -I$(TOP_SRC)/examples $(OS_INCS) LIBS += $(OS_LIBS) $(LIBUUID) LINKER=$(CC) ifeq ($(COMPILE_LANG), cpp) LINKER=$(CXX) endif all-dirs: TARGET = all clean-dirs: TARGET = clean clobber-dirs: TARGET = clobber cstyle-dirs: TARGET = cstyle format-dirs: TARGET = format sparse-dirs: TARGET = sparse all: $(if $(DIRS), all-dirs) $(if $(LIBRARIES), all-libraries) $(if $(PROGS), all-progs) clean: $(if $(DIRS), clean-dirs) $(if $(PROGS), clean-progs) $(if $(LIBRARIES), clean-libraries) clobber: $(if $(DIRS), clobber-dirs) $(if $(PROGS), clobber-progs) $(if $(LIBRARIES), clobber-libraries) cstyle: $(if $(DIRS), cstyle-dirs) format: $(if $(DIRS), format-dirs) sparse: $(if $(DIRS), sparse-dirs) $(if $(DIRS), , $(sparse-c)) DYNAMIC_LIBRARIES = $(addprefix lib, $(addsuffix .so, $(LIBRARIES))) STATIC_LIBRARIES = $(addprefix lib, $(addsuffix .a, $(LIBRARIES))) all-dirs clean-dirs clobber-dirs cstyle-dirs format-dirs sparse-dirs: $(DIRS) all-progs: $(PROGS) all-libraries: $(DYNAMIC_LIBRARIES) $(STATIC_LIBRARIES) $(foreach l, $(LIBRARIES), $(eval lib$(l).so: lib$(l).o)) $(foreach l, $(LIBRARIES), $(eval lib$(l).a: lib$(l).o)) $(foreach l, $(LIBRARIES), $(eval lib$(l).o: CFLAGS+=-fPIC)) $(foreach l, $(LIBRARIES), $(eval lib$(l).o: CXXFLAGS+=-fPIC)) $(foreach l, $(LIBRARIES), $(eval $(l): lib$(l).so lib$(l).a)) $(foreach l, $(LIBRARIES), $(eval .PHONY: $(l))) $(DIRS): $(MAKE) -C $@ $(TARGET) clobber-progs: clean-progs clobber-libraries: clean-libraries clobber-progs clobber-libraries: ifneq ($(PROGS),) $(RM) $(PROGS) endif ifneq ($(LIBRARIES),) $(RM) $(DYNAMIC_LIBRARIES) $(STATIC_LIBRARIES) endif clean-progs clean-libraries: $(RM) *.o $(TMP_HEADERS) MAKEFILE_DEPS=Makefile $(TOP)/src/examples/Makefile.inc $(TOP)/src/common.inc ifneq ($(HEADERS),) ifneq ($(filter 1 2, $(CSTYLEON)),) TMP_HEADERS := $(addsuffix tmp, $(HEADERS)) endif endif all: $(TMP_HEADERS) %.o: %.c $(MAKEFILE_DEPS) $(call check-cstyle, $<) $(CC) -c -o $@ $(CFLAGS) $(INCS) $< %.o: %.cpp $(MAKEFILE_DEPS) $(call check-cstyle, $<) $(CXX) -c -o $@ $(CXXFLAGS) $(INCS) $< %.htmp: %.h $(call check-cstyle, $<, $@) %.hpptmp: %.hpp $(call check-cstyle, $<, $@) $(PROGS): | $(TMP_HEADERS) LD_LIBRARY_PATH=$(LIBNDCTL_LD_LIBRARY_PATHS):$(LD_LIBRARY_PATH) $(LINKER) -o $@ $^ $(LDFLAGS) $(LIBS) lib%.o: $(LD) -o $@ -r $^ $(STATIC_LIBRARIES): $(AR) rv $@ $< $(DYNAMIC_LIBRARIES): $(LINKER) -shared -o $@ $(LDFLAGS) -Wl,-shared,-soname=$@ $(LIBS) $< .PHONY: all clean clobber cstyle\ all-dirs clean-dirs clobber-dirs cstyle-dirs\ all-progs clean-progs clobber-progs cstyle-progs\ $(DIRS) pmdk-1.8/src/examples/libpmemobj++/0000775000000000000000000000000013615011243015664 5ustar rootrootpmdk-1.8/src/examples/libpmemobj++/README.md0000664000000000000000000000022013615011243017135 0ustar rootrootThis folder contained examples for libpmemobj C++ bindings. They have been moved to https://github.com/pmem/libpmemobj-cpp/tree/master/examples pmdk-1.8/src/examples/librpmem/0000775000000000000000000000000013615011243015225 5ustar rootrootpmdk-1.8/src/examples/librpmem/basic.c0000664000000000000000000001306513615011243016457 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * basic.c -- basic example for the librpmem */ #include #include #include #include #include #include #include #define POOL_SIZE (32 * 1024 * 1024) #define DATA_OFF 4096 /* pool header size */ #define DATA_SIZE (POOL_SIZE - DATA_OFF) #define NLANES 64 #define SET_POOLSET_UUID 1 #define SET_UUID 2 #define SET_NEXT 3 #define SET_PREV 4 #define SET_FLAGS 5 static void default_attr(struct rpmem_pool_attr *attr) { memset(attr, 0, sizeof(*attr)); attr->major = 1; strncpy(attr->signature, "EXAMPLE", RPMEM_POOL_HDR_SIG_LEN); memset(attr->poolset_uuid, SET_POOLSET_UUID, RPMEM_POOL_HDR_UUID_LEN); memset(attr->uuid, SET_UUID, RPMEM_POOL_HDR_UUID_LEN); memset(attr->next_uuid, SET_NEXT, RPMEM_POOL_HDR_UUID_LEN); memset(attr->prev_uuid, SET_PREV, RPMEM_POOL_HDR_UUID_LEN); memset(attr->user_flags, SET_FLAGS, RPMEM_POOL_USER_FLAGS_LEN); } static int do_create(const char *target, const char *poolset, void *pool) { unsigned nlanes = NLANES; int ret = 0; struct rpmem_pool_attr pool_attr; default_attr(&pool_attr); RPMEMpool *rpp = rpmem_create(target, poolset, pool, POOL_SIZE, &nlanes, &pool_attr); if (!rpp) { fprintf(stderr, "rpmem_create: %s\n", rpmem_errormsg()); return 1; } if (rpmem_persist(rpp, DATA_OFF, DATA_SIZE, 0, 0)) { fprintf(stderr, "rpmem_persist: %s\n", rpmem_errormsg()); ret = 1; } if (rpmem_close(rpp)) { fprintf(stderr, "rpmem_close: %s\n", rpmem_errormsg()); exit(1); } return ret; } static int do_open(const char *target, const char *poolset, void *pool) { unsigned nlanes = NLANES; int ret = 0; struct rpmem_pool_attr def_attr; default_attr(&def_attr); struct rpmem_pool_attr pool_attr; RPMEMpool *rpp = rpmem_open(target, poolset, pool, POOL_SIZE, &nlanes, &pool_attr); if (!rpp) { fprintf(stderr, "rpmem_open: %s\n", rpmem_errormsg()); return 1; } if (memcmp(&def_attr, &pool_attr, sizeof(def_attr))) { fprintf(stderr, "remote pool not consistent\n"); ret = 1; goto end; } if (rpmem_persist(rpp, DATA_OFF, DATA_SIZE, 0, 0)) { fprintf(stderr, "rpmem_persist: %s\n", rpmem_errormsg()); ret = 1; } end: if (rpmem_close(rpp)) { fprintf(stderr, "rpmem_close: %s\n", rpmem_errormsg()); exit(1); } return ret; } static int do_remove(const char *target, const char *poolset) { if (rpmem_remove(target, poolset, 0)) { fprintf(stderr, "removing pool failed: %s\n", rpmem_errormsg()); return 1; } return 0; } enum op_t {op_create, op_open, op_remove, op_max}; static const char *str2op[] = { [op_create] = "create", [op_open] = "open", [op_remove] = "remove" }; static void parse_args(int argc, char *argv[], enum op_t *op, const char **target, const char **poolset) { if (argc < 4) { fprintf(stderr, "usage:\t%s [create|open|remove] " " \n", argv[0]); exit(1); } /* convert string to op */ *op = op_max; const size_t str2op_size = sizeof(str2op) / sizeof(str2op[0]); for (int i = 0; i < str2op_size; ++i) { if (strcmp(str2op[i], argv[1]) == 0) { *op = (enum op_t)i; break; } } if (*op == op_max) { fprintf(stderr, "unsupported operation -- '%s'\n", argv[1]); exit(1); } *target = argv[2]; *poolset = argv[3]; } static void * alloc_memory() { long pagesize = sysconf(_SC_PAGESIZE); if (pagesize < 0) { perror("sysconf"); exit(1); } /* allocate a page size aligned local memory pool */ void *mem; int ret = posix_memalign(&mem, pagesize, POOL_SIZE); if (ret) { fprintf(stderr, "posix_memalign: %s\n", strerror(ret)); exit(1); } assert(mem != NULL); return mem; } int main(int argc, char *argv[]) { enum op_t op; const char *target, *poolset; parse_args(argc, argv, &op, &target, &poolset); void *pool = alloc_memory(); int ret = 0; switch (op) { case op_create: ret = do_create(target, poolset, pool); break; case op_open: ret = do_open(target, poolset, pool); break; case op_remove: ret = do_remove(target, poolset); break; default: assert(0); } free(pool); return ret; } pmdk-1.8/src/examples/librpmem/fibonacci/0000775000000000000000000000000013615011243017142 5ustar rootrootpmdk-1.8/src/examples/librpmem/fibonacci/fibonacci.c0000664000000000000000000002206613615011243021231 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * fibonacci.c -- fibonacci sequence generator for librpmem */ #include #include #include #include #include #include #include #include #include #define POOL_SIGNATURE "FIBO" #define FIBO_SIG_LEN RPMEM_POOL_HDR_SIG_LEN struct fibo_t { char signature[FIBO_SIG_LEN]; unsigned n; /* index */ uint64_t fn; /* F_{n} */ uint64_t fn1; /* F_{n + 1} */ int checksum; }; #define POOL_SIZE (size_t)(32 * 1024 * 1024) #define RPMEM_HDR_SIZE 4096 #define FIBO_OFF RPMEM_HDR_SIZE #define FIBO_SIZE (sizeof(struct fibo_t)) #define RESERVED_SIZE (POOL_SIZE - RPMEM_HDR_SIZE - FIBO_SIZE) struct pool_t { unsigned char pool_hdr[RPMEM_HDR_SIZE]; struct fibo_t fibo; unsigned char reserved[RESERVED_SIZE]; }; #define NLANES 4 #define BROKEN_LOCAL (1 << 0) #define BROKEN_REMOTE (1 << 1) static int fibo_checksum(struct fibo_t *fibo) { return fibo->signature[0] + fibo->fn + fibo->fn1; } static void fibo_init(struct fibo_t *fibo) { printf("initializing...\n"); memset(fibo, 0, FIBO_SIZE); strncpy(fibo->signature, POOL_SIGNATURE, FIBO_SIG_LEN); fibo->n = 0; fibo->fn = 0; fibo->fn1 = 1; fibo->checksum = fibo_checksum(fibo); pmem_persist(fibo, FIBO_SIZE); } static int fibo_is_valid(struct fibo_t *fibo) { if (strncmp(fibo->signature, POOL_SIGNATURE, FIBO_SIG_LEN) != 0) return 0; return fibo->checksum == fibo_checksum(fibo); } static int fibo_is_zeroed(struct fibo_t *fibo) { char *data = (char *)fibo; for (size_t i = 0; i < FIBO_SIZE; ++i) if (data[i] != 0) return 0; return 1; } /* * fibo_validate -- validate local and remote copies of the F sequence */ static struct fibo_t * fibo_validate(struct fibo_t *fibo, struct fibo_t *fibo_r, unsigned *state) { struct fibo_t *valid = NULL; if (fibo_is_valid(fibo)) valid = fibo; else if (!fibo_is_zeroed(fibo)) { fprintf(stderr, "broken local memory pool!\n"); *state |= BROKEN_LOCAL; } if (fibo_is_valid(fibo_r)) { if (!valid) valid = fibo_r; else if (fibo_r->n > valid->n) valid = fibo_r; } else if (!fibo_is_zeroed(fibo_r)) { fprintf(stderr, "broken remote memory pool!\n"); *state |= BROKEN_REMOTE; } if (!valid) fprintf(stderr, "no valid Fibonacci numbers found.\n"); return valid; } /* * fibo_recovery -- based on validation outcome cleanup copies and initialize * if required */ static int fibo_recovery(RPMEMpool *rpp, struct pool_t *pool, struct fibo_t *fibo_r, int *initialized) { struct fibo_t *fibo = &pool->fibo; unsigned state = 0; int ret; struct fibo_t *valid = fibo_validate(fibo, fibo_r, &state); /* store valid fibonacci data in local */ if (valid) { if (valid != fibo) pmem_memcpy_persist(fibo, valid, FIBO_SIZE); *initialized = 0; } else { /* init local */ fibo_init(fibo); *initialized = 1; } /* local cleanup */ if (state & BROKEN_LOCAL) { /* zero reserved parts */ memset(pool->pool_hdr, 0, RPMEM_HDR_SIZE); memset(pool->reserved, 0, RESERVED_SIZE); pmem_persist(pool, POOL_SIZE); } /* remote cleanup */ if (state & BROKEN_REMOTE) { /* replicate data + reserved */ ret = rpmem_persist(rpp, FIBO_OFF, POOL_SIZE - FIBO_OFF, 0, 0); if (ret) { fprintf(stderr, "remote recovery failed: %s\n", rpmem_errormsg()); return ret; } } return 0; } /* * fibo_generate -- generate next Fibonacci number */ static void fibo_generate(struct fibo_t *fibo) { uint64_t fn2 = fibo->fn1 + fibo->fn; if (fn2 < fibo->fn1) { printf("overflow detected!\n"); fibo_init(fibo); return; } fibo->fn = fibo->fn1; fibo->fn1 = fn2; ++fibo->n; fibo->checksum = fibo_checksum(fibo); pmem_persist(fibo, FIBO_SIZE); } static void fibo_print(struct fibo_t *fibo) { if (fibo->n == 0) printf("F[0] = %lu\n", fibo->fn); printf("F[%u] = %lu\n", fibo->n + 1, fibo->fn1); } /* * remote_create_or_open -- create or open the remote replica */ static RPMEMpool * remote_create_or_open(const char *target, const char *poolset, struct pool_t *pool, int *created) { struct rpmem_pool_attr pool_attr; unsigned nlanes = NLANES; *created = 1; RPMEMpool *rpp; /* fill pool_attributes */ memset(&pool_attr, 0, sizeof(pool_attr)); strncpy(pool_attr.signature, POOL_SIGNATURE, RPMEM_POOL_HDR_SIG_LEN); /* create a remote pool */ rpp = rpmem_create(target, poolset, pool, POOL_SIZE, &nlanes, &pool_attr); if (rpp) goto verify; if (errno != EEXIST) { fprintf(stderr, "rpmem_create: %s\n", rpmem_errormsg()); return NULL; } /* the remote pool exists */ *created = 0; /* open a remote pool */ rpp = rpmem_open(target, poolset, pool, POOL_SIZE, &nlanes, &pool_attr); if (!rpp) { fprintf(stderr, "rpmem_open: %s\n", rpmem_errormsg()); return NULL; } verify: /* verify signature */ if (strcmp(pool_attr.signature, POOL_SIGNATURE) != 0) { fprintf(stderr, "invalid signature\n"); goto err; } return rpp; err: /* close the remote pool */ if (rpmem_close(rpp)) { fprintf(stderr, "rpmem_close: %s\n", rpmem_errormsg()); exit(1); } return NULL; } static int remote_write(RPMEMpool *rpp) { printf("storing Fibonacci numbers on the target...\n"); if (rpmem_persist(rpp, FIBO_OFF, FIBO_SIZE, 0, 0)) { printf("store failed: %s\n", rpmem_errormsg()); return 1; } return 0; } static int remote_read(RPMEMpool *rpp, void *buff) { printf("restore Fibonacci numbers from the target...\n"); if (rpmem_read(rpp, buff, FIBO_OFF, FIBO_SIZE, 0)) { printf("restore failed: %s\n", rpmem_errormsg()); return 1; } return 0; } static void parse_args(int argc, char *argv[], const char **target, const char **poolset, const char **path) { if (argc < 4) { fprintf(stderr, "usage:\t%s \n", argv[0]); exit(1); } *target = argv[1]; *poolset = argv[2]; *path = argv[3]; } static struct pool_t * map_pmem(const char *path, size_t *mapped_len) { int is_pmem; struct pool_t *pool = pmem_map_file(path, 0, 0, 0, mapped_len, &is_pmem); if (!pool) return NULL; if (!is_pmem) { fprintf(stderr, "is not persistent memory: %s\n", path); goto err; } if (*mapped_len < POOL_SIZE) { fprintf(stderr, "too small: %ld < %ld\n", *mapped_len, POOL_SIZE); goto err; } return pool; err: pmem_unmap(pool, *mapped_len); exit(1); } int main(int argc, char *argv[]) { const char *target, *poolset, *path; parse_args(argc, argv, &target, &poolset, &path); struct fibo_t fibo_r; /* copy from remote */ int created; /* remote: 1 == created, 0 == opened */ int initialized; /* sequence: 1 == initialized, 0 = con be continued */ int ret; /* map local pool */ size_t mapped_len; struct pool_t *pool = map_pmem(path, &mapped_len); /* open remote pool */ RPMEMpool *rpp = remote_create_or_open(target, poolset, pool, &created); if (!rpp) { ret = 1; goto unmap; } if (created) { /* zero remote copy */ memset(&fibo_r, 0, FIBO_SIZE); } else { /* restore from remote */ ret = remote_read(rpp, &fibo_r); if (ret) { /* invalidate remote copy */ memset(&fibo_r, 1, FIBO_SIZE); } } /* validate copies from local and remote */ ret = fibo_recovery(rpp, pool, &fibo_r, &initialized); if (ret) { fprintf(stderr, "recovery failed.\n"); goto err; } /* generate new number */ if (!initialized) fibo_generate(&pool->fibo); fibo_print(&pool->fibo); /* store to remote */ ret = remote_write(rpp); if (ret) goto err; printf("rerun application to generate next Fibonacci number.\n"); err: /* close the remote pool */ if (rpmem_close(rpp)) { fprintf(stderr, "rpmem_close: %s\n", rpmem_errormsg()); exit(1); } unmap: pmem_unmap(pool, mapped_len); return ret; } pmdk-1.8/src/examples/librpmem/fibonacci/.gitignore0000664000000000000000000000001213615011243021123 0ustar rootrootfibonacci pmdk-1.8/src/examples/librpmem/fibonacci/Makefile0000664000000000000000000000331513615011243020604 0ustar rootroot# # Copyright 2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/librpmem/fibonacci/Makefile -- build the fibonacci example # PROGS = fibonacci LIBS = -lrpmem -lpmem -pthread include ../../Makefile.inc fibonacci: fibonacci.o pmdk-1.8/src/examples/librpmem/manpage.c0000664000000000000000000000667513615011243017017 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * manpage.c -- example from librpmem manpage */ #include #include #include #include #include #include #define POOL_SIGNATURE "MANPAGE" #define POOL_SIZE (32 * 1024 * 1024) #define NLANES 4 #define DATA_OFF 4096 #define DATA_SIZE (POOL_SIZE - DATA_OFF) static void parse_args(int argc, char *argv[], const char **target, const char **poolset) { if (argc < 3) { fprintf(stderr, "usage:\t%s \n", argv[0]); exit(1); } *target = argv[1]; *poolset = argv[2]; } static void * alloc_memory() { long pagesize = sysconf(_SC_PAGESIZE); if (pagesize < 0) { perror("sysconf"); exit(1); } /* allocate a page size aligned local memory pool */ void *mem; int ret = posix_memalign(&mem, pagesize, POOL_SIZE); if (ret) { fprintf(stderr, "posix_memalign: %s\n", strerror(ret)); exit(1); } assert(mem != NULL); return mem; } int main(int argc, char *argv[]) { const char *target, *poolset; parse_args(argc, argv, &target, &poolset); unsigned nlanes = NLANES; void *pool = alloc_memory(); int ret; /* fill pool_attributes */ struct rpmem_pool_attr pool_attr; memset(&pool_attr, 0, sizeof(pool_attr)); strncpy(pool_attr.signature, POOL_SIGNATURE, RPMEM_POOL_HDR_SIG_LEN); /* create a remote pool */ RPMEMpool *rpp = rpmem_create(target, poolset, pool, POOL_SIZE, &nlanes, &pool_attr); if (!rpp) { fprintf(stderr, "rpmem_create: %s\n", rpmem_errormsg()); return 1; } /* store data on local pool */ memset(pool, 0, POOL_SIZE); /* make local data persistent on remote node */ ret = rpmem_persist(rpp, DATA_OFF, DATA_SIZE, 0, 0); if (ret) { fprintf(stderr, "rpmem_persist: %s\n", rpmem_errormsg()); return 1; } /* close the remote pool */ ret = rpmem_close(rpp); if (ret) { fprintf(stderr, "rpmem_close: %s\n", rpmem_errormsg()); return 1; } free(pool); return 0; } pmdk-1.8/src/examples/librpmem/README0000664000000000000000000000146713615011243016115 0ustar rootrootPersistent Memory Development Kit This is src/example/librpmem/README file. This directory contains a basic example application utilizing a librpmem library: usage: ./basic [create|remove|open] This example creates, removes or opens a specified pool set file on a target node. After opening a remote pool, pool attributes are compared with default values which were used to create a pool set. This example works with the rpmemd daemon, which must serve a specified pool set file, for example: PMEMPOOLSET 64M /path/to/file1.dat 64M /path/to/file2.dat The daemon must be invoked in the same directory where the pool set file is stored or must take an argument which points to the directory which contains the pool set file: $ ls pool.set $ ./rpmemd or ls /my/dir pool.set $ ./rpmemd -d /my/dir pmdk-1.8/src/examples/librpmem/.gitignore0000664000000000000000000000002413615011243017211 0ustar rootrootbasic hello manpage pmdk-1.8/src/examples/librpmem/hello.c0000664000000000000000000001274313615011243016503 0ustar rootroot/* * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * hello.c -- hello world for librpmem */ #include #include #include #include #include #include #include #define POOL_SIGNATURE "HELLO" enum lang_t {en, es}; static const char *hello_str[] = { [en] = "Hello world!", [es] = "¡Hola Mundo!" }; #define LANG_NUM (sizeof(hello_str) / sizeof(hello_str[0])) #define STR_SIZE 100 struct hello_t { enum lang_t lang; char str[STR_SIZE]; }; #define POOL_SIZE (32 * 1024 * 1024) #define DATA_OFF 4096 /* rpmem header size */ #define NLANES 4 #define DATA_SIZE (sizeof(struct hello_t)) static inline void write_hello_str(struct hello_t *hello, enum lang_t lang) { hello->lang = lang; strncpy(hello->str, hello_str[hello->lang], STR_SIZE); } static void translate(struct hello_t *hello) { printf("translating...\n"); enum lang_t lang = (enum lang_t)((hello->lang + 1) % LANG_NUM); write_hello_str(hello, lang); } static int remote_write(RPMEMpool *rpp) { printf("write message to the target...\n"); if (rpmem_persist(rpp, DATA_OFF, DATA_SIZE, 0, 0)) { printf("upload failed: %s\n", rpmem_errormsg()); return 1; } return 0; } static int remote_read(RPMEMpool *rpp, void *buff) { printf("read message from the target...\n"); if (rpmem_read(rpp, buff, DATA_OFF, DATA_SIZE, 0)) { printf("download failed: %s\n", rpmem_errormsg()); return 1; } return 0; } static RPMEMpool * remote_open(const char *target, const char *poolset, void *pool, int *created) { struct rpmem_pool_attr pool_attr; unsigned nlanes = NLANES; RPMEMpool *rpp; /* fill pool_attributes */ memset(&pool_attr, 0, sizeof(pool_attr)); strncpy(pool_attr.signature, POOL_SIGNATURE, RPMEM_POOL_HDR_SIG_LEN); /* create a remote pool */ rpp = rpmem_create(target, poolset, pool, POOL_SIZE, &nlanes, &pool_attr); if (rpp) { memset(pool, 0, POOL_SIZE); *created = 1; return rpp; } if (errno != EEXIST) { fprintf(stderr, "rpmem_create: %s\n", rpmem_errormsg()); return NULL; } /* open a remote pool */ rpp = rpmem_open(target, poolset, pool, POOL_SIZE, &nlanes, &pool_attr); if (!rpp) { fprintf(stderr, "rpmem_open: %s\n", rpmem_errormsg()); return NULL; } /* verify signature */ if (strcmp(pool_attr.signature, POOL_SIGNATURE) != 0) { fprintf(stderr, "invalid signature\n"); goto err; } *created = 0; return rpp; err: /* close the remote pool */ if (rpmem_close(rpp)) { fprintf(stderr, "rpmem_close: %s\n", rpmem_errormsg()); exit(1); } return NULL; } static void parse_args(int argc, char *argv[], const char **target, const char **poolset) { if (argc < 3) { fprintf(stderr, "usage:\t%s \n" "\n" "e.g.:\t%s localhost pool.set\n", argv[0], argv[0]); exit(1); } *target = argv[1]; *poolset = argv[2]; } static void * alloc_memory() { long pagesize = sysconf(_SC_PAGESIZE); if (pagesize < 0) { perror("sysconf"); return NULL; } /* allocate a page size aligned local memory pool */ void *mem; int ret = posix_memalign(&mem, pagesize, POOL_SIZE); if (ret) { fprintf(stderr, "posix_memalign: %s\n", strerror(ret)); return NULL; } return mem; } int main(int argc, char *argv[]) { const char *target, *poolset; parse_args(argc, argv, &target, &poolset); void *pool = alloc_memory(); if (!pool) exit(1); struct hello_t *hello = (struct hello_t *)(pool + DATA_OFF); int created; int ret; RPMEMpool *rpp = remote_open(target, poolset, pool, &created); if (!rpp) { ret = 1; goto exit_free; } if (created) { write_hello_str(hello, en); } else { ret = remote_read(rpp, hello); if (ret) goto exit_close; printf("\n%s\n\n", hello->str); translate(hello); } ret = remote_write(rpp); if (ret) goto exit_close; printf("rerun application to read the translation.\n"); exit_close: /* close the remote pool */ if (rpmem_close(rpp)) { fprintf(stderr, "rpmem_close: %s\n", rpmem_errormsg()); exit(1); } exit_free: free(pool); return ret; } pmdk-1.8/src/examples/librpmem/Makefile0000664000000000000000000000361313615011243016670 0ustar rootroot# # Copyright 2016-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/librpmem/Makefile -- build the librpmem examples # TOP := ../../.. include $(TOP)/src/common.inc ifeq ($(BUILD_RPMEM), y) PROGS = basic hello manpage DIRS = fibonacci LIBS = -lrpmem -pthread else $(info NOTE: Skipping librpmem examples because $(BUILD_RPMEM_INFO)) endif include ../Makefile.inc basic: basic.o hello: hello.o manpage: manpage.o pmdk-1.8/src/examples/libpmemlog/0000775000000000000000000000000013615011243015545 5ustar rootrootpmdk-1.8/src/examples/libpmemlog/manpage.vcxproj0000664000000000000000000000532313615011243020575 0ustar rootroot Debug x64 Release x64 {9FF51F3E-AF36-4F45-A797-C5F03A090298} pememlog 10.0.16299.0 Application true v140 Application false v140 true ..\..\LongPath.manifest libpmem.lib;libpmemlog.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) {0b1818eb-bdc8-4865-964f-db8bf05cfd86} {9e9e3d25-2139-4a5d-9200-18148ddead45} pmdk-1.8/src/examples/libpmemlog/manpage.vcxproj.filters0000664000000000000000000000125413615011243022243 0ustar rootroot {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {93995380-89BD-4b04-88EB-625FBE52EBFB} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files pmdk-1.8/src/examples/libpmemlog/manpage.c0000664000000000000000000000563313615011243017330 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * manpage.c -- simple example for the libpmemlog man page */ #include #include #include #include #ifndef _WIN32 #include #endif #include #include /* size of the pmemlog pool -- 1 GB */ #define POOL_SIZE ((size_t)(1 << 30)) /* * printit -- log processing callback for use with pmemlog_walk() */ static int printit(const void *buf, size_t len, void *arg) { fwrite(buf, len, 1, stdout); return 0; } int main(int argc, char *argv[]) { const char path[] = "/pmem-fs/myfile"; PMEMlogpool *plp; size_t nbyte; char *str; /* create the pmemlog pool or open it if it already exists */ plp = pmemlog_create(path, POOL_SIZE, 0666); if (plp == NULL) plp = pmemlog_open(path); if (plp == NULL) { perror(path); exit(1); } /* how many bytes does the log hold? */ nbyte = pmemlog_nbyte(plp); printf("log holds %zu bytes\n", nbyte); /* append to the log... */ str = "This is the first string appended\n"; if (pmemlog_append(plp, str, strlen(str)) < 0) { perror("pmemlog_append"); exit(1); } str = "This is the second string appended\n"; if (pmemlog_append(plp, str, strlen(str)) < 0) { perror("pmemlog_append"); exit(1); } /* print the log contents */ printf("log contains:\n"); pmemlog_walk(plp, 0, printit, NULL); pmemlog_close(plp); return 0; } pmdk-1.8/src/examples/libpmemlog/README0000664000000000000000000000131513615011243016425 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemlog/README. This directory contains examples for libpmemlog, the library providing pmem-resident log files. Some of these examples are explained in more detail here: https://pmem.io/pmdk/libpmemlog manpage.c is the example used in the libpmemlog man page. logfile implements a simple log using libpmemlog. To build these examples: make These examples can be built against an installed system using: make LIBDIR=/usr/lib INCDIR=/usr/include If you're looking for documentation to get you started using PMDK, start here: https://pmem.io/pmdk and follow the links to examples and man pages. Developers new to PMDK are probably looking for libpmemobj. pmdk-1.8/src/examples/libpmemlog/.gitignore0000664000000000000000000000001013615011243017524 0ustar rootrootmanpage pmdk-1.8/src/examples/libpmemlog/logfile/0000775000000000000000000000000013615011243017166 5ustar rootrootpmdk-1.8/src/examples/libpmemlog/logfile/addlog.c0000664000000000000000000000767213615011243020600 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * addlog -- given a log file, append a log entry * * Usage: * fallocate -l 1G /path/to/pm-aware/file * addlog /path/to/pm-aware/file "first line of entry" "second line" */ #include #include #include #include #include #include #include #include #include "logentry.h" int main(int argc, char *argv[]) { PMEMlogpool *plp; struct logentry header; struct iovec *iovp; struct iovec *next_iovp; int iovcnt; if (argc < 3) { fprintf(stderr, "usage: %s filename lines...\n", argv[0]); exit(1); } const char *path = argv[1]; /* create the log in the given file, or open it if already created */ plp = pmemlog_create(path, 0, CREATE_MODE_RW); if (plp == NULL && (plp = pmemlog_open(path)) == NULL) { perror(path); exit(1); } /* fill in the header */ time(&header.timestamp); header.pid = getpid(); /* * Create an iov for pmemlog_appendv(). For each argument given, * allocate two entries (one for the string, one for the newline * appended to the string). Allocate 1 additional entry for the * header that gets prepended to the entry. */ iovcnt = (argc - 2) * 2 + 2; if ((iovp = malloc(sizeof(*iovp) * iovcnt)) == NULL) { perror("malloc"); exit(1); } next_iovp = iovp; /* put the header into iov first */ next_iovp->iov_base = &header; next_iovp->iov_len = sizeof(header); next_iovp++; /* * Now put each arg in, following it with the string "\n". * Calculate a total character count in header.len along the way. */ header.len = 0; for (int arg = 2; arg < argc; arg++) { /* add the string given */ next_iovp->iov_base = argv[arg]; next_iovp->iov_len = strlen(argv[arg]); header.len += next_iovp->iov_len; next_iovp++; /* add the newline */ next_iovp->iov_base = "\n"; next_iovp->iov_len = 1; header.len += 1; next_iovp++; } /* * pad with NULs (at least one) to align next entry to sizeof(long long) * bytes */ int a = sizeof(long long); int len_to_round = 1 + (a - (header.len + 1) % a) % a; char *buf[sizeof(long long)] = {0}; next_iovp->iov_base = buf; next_iovp->iov_len = len_to_round; header.len += len_to_round; next_iovp++; /* atomically add it all to the log */ if (pmemlog_appendv(plp, iovp, iovcnt) < 0) { perror("pmemlog_appendv"); free(iovp); exit(1); } free(iovp); pmemlog_close(plp); return 0; } pmdk-1.8/src/examples/libpmemlog/logfile/printlog.filters0000664000000000000000000000124013615011243022413 0ustar rootroot {7b7d2f5a-464e-4d55-894b-3110e15303f1} {c2e783a4-4f6d-419d-bbf6-1056eddfdd8f} Source Files Header Files pmdk-1.8/src/examples/libpmemlog/logfile/printlog.vcxproj.filters0000664000000000000000000000144413615011243024113 0ustar rootroot {a0af7281-6084-422f-946c-e05d18224229} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {9eebac3f-530b-4b3b-b0c7-68f43650d681} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files Header Files pmdk-1.8/src/examples/libpmemlog/logfile/README0000664000000000000000000000171313615011243020050 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemlog/logfile/README. The example in this directory uses persistent memory to implement a simple log file using libpmemlog. To run this example, follow these steps: 0. Build the example with "make". The libraries must be built first (i.e. by running "make" in ../../..). 1. Create the log file. This can be anywhere but the point is it will be a much faster log file if it is created on a pmem-aware file system. For example, if /pmem is the mount point for a pmem-aware file system: $ fallocate -l 1G /pmem/logfile 2. Append to the log file as many times as you like: $ addlog /pmem/logfile "Hello there." $ addlog /pmem/logfile "First line." "Second line." ... 3. Print the contents of the log any time using: $ printlog /pmem/logfile 4. The printlog command will throw away the current log file contents after printing it if given the -t argument: $ printlog -t /pmem/logfile pmdk-1.8/src/examples/libpmemlog/logfile/addlog.vcxproj.filters0000664000000000000000000000144213615011243023505 0ustar rootroot {d3d73d42-d99e-4314-97df-e9d6289423a5} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx {416df294-73b9-4a41-b319-8d0c1922b01b} h;hh;hpp;hxx;hm;inl;inc;xsd Source Files Header Files pmdk-1.8/src/examples/libpmemlog/logfile/.gitignore0000664000000000000000000000002013615011243021146 0ustar rootrootaddlog printlog pmdk-1.8/src/examples/libpmemlog/logfile/printlog.c0000664000000000000000000000607113615011243021174 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * printlog -- given a log file, print the entries * * Usage: * printlog [-t] /path/to/pm-aware/file * * -t option means truncate the file after printing it. */ #include #include #include #include #include #include #include #include "logentry.h" /* * printlog -- callback function called when walking the log */ static int printlog(const void *buf, size_t len, void *arg) { /* first byte after log contents */ const void *endp = (char *)buf + len; /* for each entry in the log... */ while (buf < endp) { struct logentry *headerp = (struct logentry *)buf; buf = (char *)buf + sizeof(struct logentry); /* print the header */ printf("Entry from pid: %d\n", headerp->pid); printf(" Created: %s", ctime(&headerp->timestamp)); printf(" Contents:\n"); /* print the log data itself, it is NUL-terminated */ printf("%s", (char *)buf); buf = (char *)buf + headerp->len; } return 0; } int main(int argc, char *argv[]) { int ind = 1; int tflag = 0; PMEMlogpool *plp; if (argc > 2) { if (strcmp(argv[1], "-t") == 0) { tflag = 1; ind++; } else { fprintf(stderr, "usage: %s [-t] file\n", argv[0]); exit(1); } } const char *path = argv[ind]; if ((plp = pmemlog_open(path)) == NULL) { perror(path); exit(1); } /* the rest of the work happens in printlog() above */ pmemlog_walk(plp, 0, printlog, NULL); if (tflag) pmemlog_rewind(plp); pmemlog_close(plp); return 0; } pmdk-1.8/src/examples/libpmemlog/logfile/addlog.filters0000664000000000000000000000123613615011243022014 0ustar rootroot {7b7d2f5a-464e-4d55-894b-3110e15303f1} {c2e783a4-4f6d-419d-bbf6-1056eddfdd8f} Source Files Header Files pmdk-1.8/src/examples/libpmemlog/logfile/logentry.h0000664000000000000000000000340313615011243021202 0ustar rootroot/* * Copyright 2014-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * info prepended to each log entry... */ struct logentry { size_t len; /* length of the rest of the log entry */ time_t timestamp; #ifndef _WIN32 pid_t pid; #else int pid; #endif }; pmdk-1.8/src/examples/libpmemlog/logfile/printlog.vcxproj0000664000000000000000000000544613615011243022452 0ustar rootroot Debug x64 Release x64 {C3CEE34C-29E0-4A22-B258-3FBAF662AA19} pmemlog 10.0.16299.0 {0b1818eb-bdc8-4865-964f-db8bf05cfd86} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemlog.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemlog/logfile/Makefile0000664000000000000000000000343113615011243020627 0ustar rootroot# # Copyright 2014-2017, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/logfile/Makefile -- build the Persistent Memory Development Kit examples # PROGS = addlog printlog LIBS = -lpmemlog -lpmem -pthread include ../../Makefile.inc addlog: addlog.o printlog: printlog.o addlog.o printlog.o: logentry.h pmdk-1.8/src/examples/libpmemlog/logfile/addlog.vcxproj0000664000000000000000000000551413615011243022042 0ustar rootroot Debug x64 Release x64 {A7CA7975-CEDB-48E6-9AEB-1209DCBD07F2} pmemlog 10.0.16299.0 {0b1818eb-bdc8-4865-964f-db8bf05cfd86} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 NotSet Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemlog.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemlog/Makefile0000664000000000000000000000332413615011243017207 0ustar rootroot# # Copyright 2014-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemlog/Makefile -- build the libpmemlog examples # PROGS = manpage DIRS = logfile LIBS = -lpmemlog -lpmem -pthread include ../Makefile.inc manpage: manpage.o pmdk-1.8/src/examples/libpmempool/0000775000000000000000000000000013615011243015735 5ustar rootrootpmdk-1.8/src/examples/libpmempool/manpage.c0000664000000000000000000000577613615011243017530 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * manpage.c -- simple example for the libpmempool man page */ #include #include #include #include #include #define PATH "./pmem-fs/myfile" #define CHECK_FLAGS (PMEMPOOL_CHECK_FORMAT_STR|PMEMPOOL_CHECK_REPAIR|\ PMEMPOOL_CHECK_VERBOSE) int main(int argc, char *argv[]) { PMEMpoolcheck *ppc; struct pmempool_check_status *status; enum pmempool_check_result ret; /* arguments for check */ struct pmempool_check_args args = { .path = PATH, .backup_path = NULL, .pool_type = PMEMPOOL_POOL_TYPE_DETECT, .flags = CHECK_FLAGS }; /* initialize check context */ if ((ppc = pmempool_check_init(&args, sizeof(args))) == NULL) { perror("pmempool_check_init"); exit(EXIT_FAILURE); } /* perform check and repair, answer 'yes' for each question */ while ((status = pmempool_check(ppc)) != NULL) { switch (status->type) { case PMEMPOOL_CHECK_MSG_TYPE_ERROR: printf("%s\n", status->str.msg); break; case PMEMPOOL_CHECK_MSG_TYPE_INFO: printf("%s\n", status->str.msg); break; case PMEMPOOL_CHECK_MSG_TYPE_QUESTION: printf("%s\n", status->str.msg); status->str.answer = "yes"; break; default: pmempool_check_end(ppc); exit(EXIT_FAILURE); } } /* finalize the check and get the result */ ret = pmempool_check_end(ppc); switch (ret) { case PMEMPOOL_CHECK_RESULT_CONSISTENT: case PMEMPOOL_CHECK_RESULT_REPAIRED: return 0; default: return 1; } } pmdk-1.8/src/examples/libpmempool/README0000664000000000000000000000130213615011243016611 0ustar rootrootPersistent Memory Development Kit This is examples/libpmempool/README. This directory contains examples for libpmempool, the library providing support for off-line pool management, diagnostics and repair. Some of these examples are explained in more detail here: https://pmem.io/pmdk/libpmempool manpage.c is the example used in the libpmempool man page. To build these examples: make These examples can be built against an installed system using: make LIBDIR=/usr/lib INCDIR=/usr/include If you're looking for documentation to get you started using PMDK, start here: https://pmem.io/pmdk and follow the links to examples and man pages. Developers new to PMDK are probably looking for libpmemobj. pmdk-1.8/src/examples/libpmempool/.gitignore0000664000000000000000000000001013615011243017714 0ustar rootrootmanpage pmdk-1.8/src/examples/libpmempool/Makefile0000664000000000000000000000327513615011243017404 0ustar rootroot# # Copyright 2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmempool/Makefile -- build the libpmempool examples # PROGS = manpage LIBS = -lpmempool -pthread include ../Makefile.inc manpage: manpage.o pmdk-1.8/src/examples/README0000664000000000000000000000303313615011243014275 0ustar rootrootPersistent Memory Development Kit This is examples/README. This directory contains brief educational examples illustrating the use of the PMDK libraries. For many of these examples, the Makefile rules are here just to check that the example compiles, loads against the appropriate library, and passes cstyle. If you're looking for documentation to get you started using PMDK, start here: https://pmem.io/pmdk and follow the links to examples and man pages. Developers new to PMDK are probably looking for libpmemobj. Many of the examples in this directory are described in more detail on the above web site. libpmem(7) -- low-level persistent memory support Example programs are in the libpmem directory. More documentation: https://pmem.io/pmdk/libpmem libpmemblk(7) -- pmem-resident arrays of blocks Example programs are in the libpmemblk directory. More documentation: https://pmem.io/pmdk/libpmemblk libpmemlog(7) -- pmem-resident log files Example programs are in the libpmemlog directory. More documentation: https://pmem.io/pmdk/libpmemlog libpmemobj(7) -- transactional object store Example programs are in the libpmemobj directory. More documentation: https://pmem.io/pmdk/libpmemobj libpmempool(7) -- pool management, diagnostics and repair Example programs are in the libpmempool directory. More documentation: https://pmem.io/pmdk/libpmempool librpmem(7) -- remote access to persistent memory Example programs are in the librpmem directory. More documentation: https://pmem.io/pmdk/librpmem pmdk-1.8/src/examples/.gitignore0000664000000000000000000000000413615011243015400 0ustar rootroot*.o pmdk-1.8/src/examples/libpmem/0000775000000000000000000000000013615011243015043 5ustar rootrootpmdk-1.8/src/examples/libpmem/full_copy.vcxproj.filters0000664000000000000000000000064613615011243022131 0ustar rootroot {0f7bab61-c1ad-4a74-8663-fe40f393b2eb} Source Files pmdk-1.8/src/examples/libpmem/simple_copy.c0000664000000000000000000000563713615011243017545 0ustar rootroot/* * Copyright 2014-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * simple_copy.c -- show how to use pmem_memcpy_persist() * * usage: simple_copy src-file dst-file * * Reads 4k from src-file and writes it to dst-file. */ #include #include #include #include #include #include #ifndef _WIN32 #include #else #include #endif #include #include /* just copying 4k to pmem for this example */ #define BUF_LEN 4096 int main(int argc, char *argv[]) { int srcfd; char buf[BUF_LEN]; char *pmemaddr; size_t mapped_len; int is_pmem; int cc; if (argc != 3) { fprintf(stderr, "usage: %s src-file dst-file\n", argv[0]); exit(1); } /* open src-file */ if ((srcfd = open(argv[1], O_RDONLY)) < 0) { perror(argv[1]); exit(1); } /* create a pmem file and memory map it */ if ((pmemaddr = pmem_map_file(argv[2], BUF_LEN, PMEM_FILE_CREATE|PMEM_FILE_EXCL, 0666, &mapped_len, &is_pmem)) == NULL) { perror("pmem_map_file"); exit(1); } /* read up to BUF_LEN from srcfd */ if ((cc = read(srcfd, buf, BUF_LEN)) < 0) { pmem_unmap(pmemaddr, mapped_len); perror("read"); exit(1); } /* write it to the pmem */ if (is_pmem) { pmem_memcpy_persist(pmemaddr, buf, cc); } else { memcpy(pmemaddr, buf, cc); pmem_msync(pmemaddr, cc); } close(srcfd); pmem_unmap(pmemaddr, mapped_len); exit(0); } pmdk-1.8/src/examples/libpmem/full_copy.vcxproj0000664000000000000000000000453213615011243020460 0ustar rootroot Debug x64 Release x64 {0287C3DC-AE03-4714-AAFF-C52F062ECA6F} pmem 10.0.16299.0 Application true v140 Application false v140 ..\..\LongPath.manifest libpmem.lib;%(AdditionalDependencies) {9e9e3d25-2139-4a5d-9200-18148ddead45} pmdk-1.8/src/examples/libpmem/simple_copy.vcxproj0000664000000000000000000000453413615011243021011 0ustar rootroot Debug x64 Release x64 {D062166F-0EC7-4C13-A772-0C7157EEFE41} pmem 10.0.16299.0 Application true v140 Application false v140 ..\..\LongPath.manifest libpmem.lib;%(AdditionalDependencies) {9e9e3d25-2139-4a5d-9200-18148ddead45} pmdk-1.8/src/examples/libpmem/manpage.vcxproj0000664000000000000000000000453013615011243020072 0ustar rootroot Debug x64 Release x64 {FCD0587A-4504-4F5E-8E9C-468CC03D250A} pmem 10.0.16299.0 Application true v140 Application false v140 ..\..\LongPath.manifest libpmem.lib;%(AdditionalDependencies) {9e9e3d25-2139-4a5d-9200-18148ddead45} pmdk-1.8/src/examples/libpmem/manpage.vcxproj.filters0000664000000000000000000000075513615011243021546 0ustar rootroot {4FC737F1-C7A5-4376-A066-2A32D752A2FF} cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx Source Files pmdk-1.8/src/examples/libpmem/manpage.c0000664000000000000000000000515313615011243016623 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * manpage.c -- simple example for the libpmem man page */ #include #include #include #include #include #include #ifndef _WIN32 #include #else #include #endif #include #include /* using 4k of pmem for this example */ #define PMEM_LEN 4096 #define PATH "/pmem-fs/myfile" int main(int argc, char *argv[]) { char *pmemaddr; size_t mapped_len; int is_pmem; /* create a pmem file and memory map it */ if ((pmemaddr = pmem_map_file(PATH, PMEM_LEN, PMEM_FILE_CREATE, 0666, &mapped_len, &is_pmem)) == NULL) { perror("pmem_map_file"); exit(1); } /* store a string to the persistent memory */ strcpy(pmemaddr, "hello, persistent memory"); /* flush above strcpy to persistence */ if (is_pmem) pmem_persist(pmemaddr, mapped_len); else pmem_msync(pmemaddr, mapped_len); /* * Delete the mappings. The region is also * automatically unmapped when the process is * terminated. */ pmem_unmap(pmemaddr, mapped_len); return 0; } pmdk-1.8/src/examples/libpmem/README0000664000000000000000000000140413615011243015722 0ustar rootrootPersistent Memory Development Kit This is examples/libpmem/README. This directory contains examples for libpmem, the library containing low-level persistent memory support. A detailed explanation of these examples can be found here: https://pmem.io/pmdk/libpmem manpage.c is the example used in the libpmem man page. simple_copy.c is a simple pmem_memcpy() example. full_copy.c shows how to use pmem_memcpy_nodrain(). To build these examples: make These examples can be built against an installed system using: make LIBDIR=/usr/lib INCDIR=/usr/include If you're looking for documentation to get you started using PMDK, start here: https://pmem.io/pmdk and follow the links to examples and man pages. Developers new to PMDK are probably looking for libpmemobj. pmdk-1.8/src/examples/libpmem/.gitignore0000664000000000000000000000003613615011243017032 0ustar rootrootmanpage simple_copy full_copy pmdk-1.8/src/examples/libpmem/full_copy.c0000664000000000000000000000746613615011243017220 0ustar rootroot/* * Copyright 2014-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * full_copy.c -- show how to use pmem_memcpy_nodrain() * * usage: full_copy src-file dst-file * * Copies src-file to dst-file in 4k chunks. */ #include #include #include #include #include #include #ifndef _WIN32 #include #else #include #endif #include #include /* copying 4k at a time to pmem for this example */ #define BUF_LEN 4096 /* * do_copy_to_pmem -- copy to pmem, postponing drain step until the end */ static void do_copy_to_pmem(char *pmemaddr, int srcfd, off_t len) { char buf[BUF_LEN]; int cc; /* copy the file, saving the last flush step to the end */ while ((cc = read(srcfd, buf, BUF_LEN)) > 0) { pmem_memcpy_nodrain(pmemaddr, buf, cc); pmemaddr += cc; } if (cc < 0) { perror("read"); exit(1); } /* perform final flush step */ pmem_drain(); } /* * do_copy_to_non_pmem -- copy to a non-pmem memory mapped file */ static void do_copy_to_non_pmem(char *addr, int srcfd, off_t len) { char *startaddr = addr; char buf[BUF_LEN]; int cc; /* copy the file, saving the last flush step to the end */ while ((cc = read(srcfd, buf, BUF_LEN)) > 0) { memcpy(addr, buf, cc); addr += cc; } if (cc < 0) { perror("read"); exit(1); } /* flush it */ if (pmem_msync(startaddr, len) < 0) { perror("pmem_msync"); exit(1); } } int main(int argc, char *argv[]) { int srcfd; struct stat stbuf; char *pmemaddr; size_t mapped_len; int is_pmem; if (argc != 3) { fprintf(stderr, "usage: %s src-file dst-file\n", argv[0]); exit(1); } /* open src-file */ if ((srcfd = open(argv[1], O_RDONLY)) < 0) { perror(argv[1]); exit(1); } /* find the size of the src-file */ if (fstat(srcfd, &stbuf) < 0) { perror("fstat"); exit(1); } /* create a pmem file and memory map it */ if ((pmemaddr = pmem_map_file(argv[2], stbuf.st_size, PMEM_FILE_CREATE|PMEM_FILE_EXCL, 0666, &mapped_len, &is_pmem)) == NULL) { perror("pmem_map_file"); exit(1); } /* determine if range is true pmem, call appropriate copy routine */ if (is_pmem) do_copy_to_pmem(pmemaddr, srcfd, stbuf.st_size); else do_copy_to_non_pmem(pmemaddr, srcfd, stbuf.st_size); close(srcfd); pmem_unmap(pmemaddr, mapped_len); exit(0); } pmdk-1.8/src/examples/libpmem/simple_copy.vcxproj.filters0000664000000000000000000000065013615011243022453 0ustar rootroot {3bc86b19-55f2-4b90-9ccd-1470361ca84c} Source Files pmdk-1.8/src/examples/libpmem/Makefile0000664000000000000000000000340013615011243016500 0ustar rootroot# # Copyright 2014-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmem/Makefile -- build the libpmem examples # PROGS = manpage simple_copy full_copy LIBS = -lpmem -pthread include ../Makefile.inc manpage: manpage.o simple_copy: simple_copy.o full_copy: full_copy.o pmdk-1.8/src/examples/examples_release.props0000664000000000000000000000301613615011243020021 0ustar rootroot $(Platform)\$(Configuration)\$(TargetName)\ ex_$(RootNamespace)_$(ProjectName) $(SolutionDir)$(Platform)\$(Configuration)\examples\ $(ProjectDir)..\..\x64\$(Configuration)\libs;$(ProjectDir)..\..\..\x64\$(Configuration)\libs;$(VC_LibraryPath_x64);$(WindowsSDK_LibraryPath_x64);$(NETFXKitsDir)Lib\um\x64 .;$(solutionDir)include;$(solutionDir)..\include;$(ProjectDir)..\..\;$(ProjectDir)..\;$(IncludePath);$(WindowsSDK_IncludePath); Level3 MaxSpeed true 4996 true true PMDK_UTF8_API;SDS_ENABLED;NTDDI_VERSION=NTDDI_WIN10_RS1;_MBCS;%(PreprocessorDefinitions) true true true true pmdk-1.8/src/examples/libpmemobj/0000775000000000000000000000000013615011243015536 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/pmemlog/0000775000000000000000000000000013615011243017176 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_simple.vcxproj.filters0000664000000000000000000000065713615011243026135 0ustar rootroot {60f472c8-e000-4a48-b387-d230b91c9e79} Source Files pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_minimal.c0000664000000000000000000001767113615011243023356 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * obj_pmemlog_macros.c -- alternate pmemlog implementation based on pmemobj * * usage: obj_pmemlog_macros [co] file [cmd[:param]...] * * c - create file * o - open file * * The "cmd" arguments match the pmemlog functions: * a - append * v - appendv * r - rewind * w - walk * n - nbyte * t - tell * "a" and "v" require a parameter string(s) separated by a colon */ #include #include #include #include #include #include #include #include "libpmemobj.h" #include "libpmem.h" #include "libpmemlog.h" #define POOL_SIZE ((size_t)(1024 * 1024 * 100)) POBJ_LAYOUT_BEGIN(obj_pmemlog_minimal); POBJ_LAYOUT_TOID(obj_pmemlog_minimal, struct log); POBJ_LAYOUT_END(obj_pmemlog_minimal); /* struct log stores the entire log entry */ struct log { size_t size; char data[]; }; /* structure containing arguments for the alloc constructor */ struct create_args { size_t size; const void *src; }; /* * create_log_entry -- (internal) constructor for the log entry */ static int create_log_entry(PMEMobjpool *pop, void *ptr, void *arg) { struct log *logptr = ptr; struct create_args *carg = arg; logptr->size = carg->size; pmemobj_persist(pop, &logptr->size, sizeof(logptr->size)); pmemobj_memcpy_persist(pop, logptr->data, carg->src, carg->size); return 0; } /* * pmemlog_open -- pool open wrapper */ PMEMlogpool * pmemlog_open(const char *path) { return (PMEMlogpool *)pmemobj_open(path, POBJ_LAYOUT_NAME(obj_pmemlog_minimal)); } /* * pmemlog_create -- pool create wrapper */ PMEMlogpool * pmemlog_create(const char *path, size_t poolsize, mode_t mode) { return (PMEMlogpool *)pmemobj_create(path, POBJ_LAYOUT_NAME(obj_pmemlog_minimal), poolsize, mode); } /* * pool_close -- pool close wrapper */ void pmemlog_close(PMEMlogpool *plp) { pmemobj_close((PMEMobjpool *)plp); } /* * pmemlog_nbyte -- not available in this implementation */ size_t pmemlog_nbyte(PMEMlogpool *plp) { /* N/A */ return 0; } /* * pmemlog_append -- add data to a log memory pool */ int pmemlog_append(PMEMlogpool *plp, const void *buf, size_t count) { PMEMobjpool *pop = (PMEMobjpool *)plp; struct create_args args = { count, buf }; size_t obj_size = sizeof(size_t) + count; /* alloc-construct to an internal list */ PMEMoid obj; pmemobj_alloc(pop, &obj, obj_size, 0, create_log_entry, &args); return 0; } /* * pmemlog_appendv -- add gathered data to a log memory pool */ int pmemlog_appendv(PMEMlogpool *plp, const struct iovec *iov, int iovcnt) { PMEMobjpool *pop = (PMEMobjpool *)plp; /* append the data */ for (int i = 0; i < iovcnt; ++i) { struct create_args args = { iov[i].iov_len, iov[i].iov_base }; size_t obj_size = sizeof(size_t) + args.size; /* alloc-construct to an internal list */ pmemobj_alloc(pop, NULL, obj_size, 0, create_log_entry, &args); } return 0; } /* * pmemlog_tell -- not available in this implementation */ long long pmemlog_tell(PMEMlogpool *plp) { /* N/A */ return 0; } /* * pmemlog_rewind -- discard all data, resetting a log memory pool to empty */ void pmemlog_rewind(PMEMlogpool *plp) { PMEMobjpool *pop = (PMEMobjpool *)plp; PMEMoid iter, next; /* go through each list and remove all entries */ POBJ_FOREACH_SAFE(pop, iter, next) { pmemobj_free(&iter); } } /* * pmemlog_walk -- walk through all data in a log memory pool * * As this implementation holds the size of each entry, the chunksize is ignored * and the process_chunk function gets the actual entry length. */ void pmemlog_walk(PMEMlogpool *plp, size_t chunksize, int (*process_chunk)(const void *buf, size_t len, void *arg), void *arg) { PMEMobjpool *pop = (PMEMobjpool *)plp; PMEMoid iter; /* process each allocated object */ POBJ_FOREACH(pop, iter) { struct log *logptr = pmemobj_direct(iter); (*process_chunk)(logptr->data, logptr->size, arg); } } /* * process_chunk -- (internal) process function for log_walk */ static int process_chunk(const void *buf, size_t len, void *arg) { char *tmp = malloc(len + 1); if (tmp == NULL) { fprintf(stderr, "malloc error\n"); return 0; } memcpy(tmp, buf, len); tmp[len] = '\0'; printf("log contains:\n"); printf("%s\n", tmp); free(tmp); return 1; /* continue */ } /* * count_iovec -- (internal) count the number of iovec items */ static int count_iovec(char *arg) { int count = 1; char *pch = strchr(arg, ':'); while (pch != NULL) { ++count; pch = strchr(++pch, ':'); } return count; } /* * fill_iovec -- (internal) fill out the iovec */ static void fill_iovec(struct iovec *iov, char *arg) { char *pch = strtok(arg, ":"); while (pch != NULL) { iov->iov_base = pch; iov->iov_len = strlen((char *)iov->iov_base); ++iov; pch = strtok(NULL, ":"); } } int main(int argc, char *argv[]) { if (argc < 2) { fprintf(stderr, "usage: %s [o,c] file [val...]\n", argv[0]); return 1; } PMEMlogpool *plp; if (strncmp(argv[1], "c", 1) == 0) { plp = pmemlog_create(argv[2], POOL_SIZE, CREATE_MODE_RW); } else if (strncmp(argv[1], "o", 1) == 0) { plp = pmemlog_open(argv[2]); } else { fprintf(stderr, "usage: %s [o,c] file [val...]\n", argv[0]); return 1; } if (plp == NULL) { perror("pmemlog_create/pmemlog_open"); return 1; } /* process the command line arguments */ for (int i = 3; i < argc; i++) { switch (*argv[i]) { case 'a': { printf("append: %s\n", argv[i] + 2); if (pmemlog_append(plp, argv[i] + 2, strlen(argv[i] + 2))) fprintf(stderr, "pmemlog_append" " error\n"); break; } case 'v': { printf("appendv: %s\n", argv[i] + 2); int count = count_iovec(argv[i] + 2); struct iovec *iov = calloc(count, sizeof(struct iovec)); if (iov == NULL) { fprintf(stderr, "malloc error\n"); return 1; } fill_iovec(iov, argv[i] + 2); if (pmemlog_appendv(plp, iov, count)) fprintf(stderr, "pmemlog_appendv" " error\n"); free(iov); break; } case 'r': { printf("rewind\n"); pmemlog_rewind(plp); break; } case 'w': { printf("walk\n"); pmemlog_walk(plp, 0, process_chunk, NULL); break; } case 'n': { printf("nbytes: %zu\n", pmemlog_nbyte(plp)); break; } case 't': { printf("offset: %lld\n", pmemlog_tell(plp)); break; } default: { fprintf(stderr, "unrecognized command %s\n", argv[i]); break; } }; } /* all done */ pmemlog_close(plp); return 0; } pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_simple.c0000664000000000000000000002574313615011243023220 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * obj_pmemlog_simple.c -- alternate pmemlog implementation based on pmemobj * * usage: obj_pmemlog_simple [co] file [cmd[:param]...] * * c - create file * o - open file * * The "cmd" arguments match the pmemlog functions: * a - append * v - appendv * r - rewind * w - walk * n - nbyte * t - tell * "a", "w" and "v" require a parameter string(s) separated by a colon */ #include #include #include #include #include #include #include #include "libpmemobj.h" #include "libpmem.h" #include "libpmemlog.h" #define USABLE_SIZE (9.0 / 10) #define MAX_POOL_SIZE (((size_t)1024 * 1024 * 1024 * 16)) #define POOL_SIZE ((size_t)(1024 * 1024 * 100)) POBJ_LAYOUT_BEGIN(obj_pmemlog_simple); POBJ_LAYOUT_ROOT(obj_pmemlog_simple, struct base); POBJ_LAYOUT_TOID(obj_pmemlog_simple, struct log); POBJ_LAYOUT_END(obj_pmemlog_simple); /* log entry header */ struct log_hdr { uint64_t write_offset; /* data write offset */ size_t data_size; /* size available for data */ }; /* struct log stores the entire log entry */ struct log { struct log_hdr hdr; char data[]; }; /* struct base has the lock and log OID */ struct base { PMEMrwlock rwlock; /* lock covering entire log */ TOID(struct log) log; }; /* * pmemblk_map -- (internal) read or initialize the log pool */ static int pmemlog_map(PMEMobjpool *pop, size_t fsize) { int retval = 0; TOID(struct base)bp; bp = POBJ_ROOT(pop, struct base); /* log already initialized */ if (!TOID_IS_NULL(D_RO(bp)->log)) return retval; size_t pool_size = (size_t)(fsize * USABLE_SIZE); /* max size of a single allocation is 16GB */ if (pool_size > MAX_POOL_SIZE) { errno = EINVAL; return 1; } TX_BEGIN(pop) { TX_ADD(bp); D_RW(bp)->log = TX_ZALLOC(struct log, pool_size); D_RW(D_RW(bp)->log)->hdr.data_size = pool_size - sizeof(struct log_hdr); } TX_ONABORT { retval = -1; } TX_END return retval; } /* * pmemlog_open -- pool open wrapper */ PMEMlogpool * pmemlog_open(const char *path) { PMEMobjpool *pop = pmemobj_open(path, POBJ_LAYOUT_NAME(obj_pmemlog_simple)); assert(pop != NULL); struct stat buf; if (stat(path, &buf)) { perror("stat"); return NULL; } return pmemlog_map(pop, buf.st_size) ? NULL : (PMEMlogpool *)pop; } /* * pmemlog_create -- pool create wrapper */ PMEMlogpool * pmemlog_create(const char *path, size_t poolsize, mode_t mode) { PMEMobjpool *pop = pmemobj_create(path, POBJ_LAYOUT_NAME(obj_pmemlog_simple), poolsize, mode); assert(pop != NULL); struct stat buf; if (stat(path, &buf)) { perror("stat"); return NULL; } return pmemlog_map(pop, buf.st_size) ? NULL : (PMEMlogpool *)pop; } /* * pool_close -- pool close wrapper */ void pmemlog_close(PMEMlogpool *plp) { pmemobj_close((PMEMobjpool *)plp); } /* * pmemlog_nbyte -- return usable size of a log memory pool */ size_t pmemlog_nbyte(PMEMlogpool *plp) { PMEMobjpool *pop = (PMEMobjpool *)plp; TOID(struct log) logp; logp = D_RO(POBJ_ROOT(pop, struct base))->log; return D_RO(logp)->hdr.data_size; } /* * pmemlog_append -- add data to a log memory pool */ int pmemlog_append(PMEMlogpool *plp, const void *buf, size_t count) { PMEMobjpool *pop = (PMEMobjpool *)plp; int retval = 0; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); TOID(struct log) logp; logp = D_RW(bp)->log; /* check for overrun */ if ((D_RO(logp)->hdr.write_offset + count) > D_RO(logp)->hdr.data_size) { errno = ENOMEM; return 1; } /* begin a transaction, also acquiring the write lock for the log */ TX_BEGIN_PARAM(pop, TX_PARAM_RWLOCK, &D_RW(bp)->rwlock, TX_PARAM_NONE) { char *dst = D_RW(logp)->data + D_RO(logp)->hdr.write_offset; /* add hdr to undo log */ TX_ADD_FIELD(logp, hdr); /* copy and persist data */ pmemobj_memcpy_persist(pop, dst, buf, count); /* set the new offset */ D_RW(logp)->hdr.write_offset += count; } TX_ONABORT { retval = -1; } TX_END return retval; } /* * pmemlog_appendv -- add gathered data to a log memory pool */ int pmemlog_appendv(PMEMlogpool *plp, const struct iovec *iov, int iovcnt) { PMEMobjpool *pop = (PMEMobjpool *)plp; int retval = 0; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); uint64_t total_count = 0; /* calculate required space */ for (int i = 0; i < iovcnt; ++i) total_count += iov[i].iov_len; TOID(struct log) logp; logp = D_RW(bp)->log; /* check for overrun */ if ((D_RO(logp)->hdr.write_offset + total_count) > D_RO(logp)->hdr.data_size) { errno = ENOMEM; return 1; } /* begin a transaction, also acquiring the write lock for the log */ TX_BEGIN_PARAM(pop, TX_PARAM_RWLOCK, &D_RW(bp)->rwlock, TX_PARAM_NONE) { TX_ADD(D_RW(bp)->log); /* append the data */ for (int i = 0; i < iovcnt; ++i) { char *buf = (char *)iov[i].iov_base; size_t count = iov[i].iov_len; char *dst = D_RW(logp)->data + D_RO(logp)->hdr.write_offset; /* copy and persist data */ pmemobj_memcpy_persist(pop, dst, buf, count); /* set the new offset */ D_RW(logp)->hdr.write_offset += count; } } TX_ONABORT { retval = -1; } TX_END return retval; } /* * pmemlog_tell -- return current write point in a log memory pool */ long long pmemlog_tell(PMEMlogpool *plp) { PMEMobjpool *pop = (PMEMobjpool *)plp; TOID(struct log) logp; logp = D_RO(POBJ_ROOT(pop, struct base))->log; return D_RO(logp)->hdr.write_offset; } /* * pmemlog_rewind -- discard all data, resetting a log memory pool to empty */ void pmemlog_rewind(PMEMlogpool *plp) { PMEMobjpool *pop = (PMEMobjpool *)plp; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); /* begin a transaction, also acquiring the write lock for the log */ TX_BEGIN_PARAM(pop, TX_PARAM_RWLOCK, &D_RW(bp)->rwlock, TX_PARAM_NONE) { /* add the hdr to the undo log */ TX_ADD_FIELD(D_RW(bp)->log, hdr); /* reset the write offset */ D_RW(D_RW(bp)->log)->hdr.write_offset = 0; } TX_END } /* * pmemlog_walk -- walk through all data in a log memory pool * * chunksize of 0 means process_chunk gets called once for all data * as a single chunk. */ void pmemlog_walk(PMEMlogpool *plp, size_t chunksize, int (*process_chunk)(const void *buf, size_t len, void *arg), void *arg) { PMEMobjpool *pop = (PMEMobjpool *)plp; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); /* acquire a rdlock here */ int err; if ((err = pmemobj_rwlock_rdlock(pop, &D_RW(bp)->rwlock)) != 0) { errno = err; return; } TOID(struct log) logp; logp = D_RW(bp)->log; size_t read_size = chunksize ? chunksize : D_RO(logp)->hdr.data_size; char *read_ptr = D_RW(logp)->data; const char *write_ptr = (D_RO(logp)->data + D_RO(logp)->hdr.write_offset); while (read_ptr < write_ptr) { read_size = MIN(read_size, (size_t)(write_ptr - read_ptr)); (*process_chunk)(read_ptr, read_size, arg); read_ptr += read_size; } pmemobj_rwlock_unlock(pop, &D_RW(bp)->rwlock); } /* * process_chunk -- (internal) process function for log_walk */ static int process_chunk(const void *buf, size_t len, void *arg) { char *tmp = (char *)malloc(len + 1); if (tmp == NULL) { fprintf(stderr, "malloc error\n"); return 0; } memcpy(tmp, buf, len); tmp[len] = '\0'; printf("log contains:\n"); printf("%s\n", tmp); free(tmp); return 1; /* continue */ } /* * count_iovec -- (internal) count the number of iovec items */ static int count_iovec(char *arg) { int count = 1; char *pch = strchr(arg, ':'); while (pch != NULL) { ++count; pch = strchr(++pch, ':'); } return count; } /* * fill_iovec -- (internal) fill out the iovec */ static void fill_iovec(struct iovec *iov, char *arg) { char *pch = strtok(arg, ":"); while (pch != NULL) { iov->iov_base = pch; iov->iov_len = strlen((char *)iov->iov_base); ++iov; pch = strtok(NULL, ":"); } } int main(int argc, char *argv[]) { if (argc < 2) { fprintf(stderr, "usage: %s [o,c] file [val...]\n", argv[0]); return 1; } PMEMlogpool *plp; if (strncmp(argv[1], "c", 1) == 0) { plp = pmemlog_create(argv[2], POOL_SIZE, CREATE_MODE_RW); } else if (strncmp(argv[1], "o", 1) == 0) { plp = pmemlog_open(argv[2]); } else { fprintf(stderr, "usage: %s [o,c] file [val...]\n", argv[0]); return 1; } if (plp == NULL) { perror("pmemlog_create/pmemlog_open"); return 1; } /* process the command line arguments */ for (int i = 3; i < argc; i++) { switch (*argv[i]) { case 'a': { printf("append: %s\n", argv[i] + 2); if (pmemlog_append(plp, argv[i] + 2, strlen(argv[i] + 2))) fprintf(stderr, "pmemlog_append" " error\n"); break; } case 'v': { printf("appendv: %s\n", argv[i] + 2); int count = count_iovec(argv[i] + 2); struct iovec *iov = (struct iovec *)malloc( count * sizeof(struct iovec)); if (iov == NULL) { fprintf(stderr, "malloc error\n"); return 1; } fill_iovec(iov, argv[i] + 2); if (pmemlog_appendv(plp, iov, count)) fprintf(stderr, "pmemlog_appendv" " error\n"); free(iov); break; } case 'r': { printf("rewind\n"); pmemlog_rewind(plp); break; } case 'w': { printf("walk\n"); unsigned long walksize = strtoul(argv[i] + 2, NULL, 10); pmemlog_walk(plp, walksize, process_chunk, NULL); break; } case 'n': { printf("nbytes: %zu\n", pmemlog_nbyte(plp)); break; } case 't': { printf("offset: %lld\n", pmemlog_tell(plp)); break; } default: { fprintf(stderr, "unrecognized command %s\n", argv[i]); break; } }; } /* all done */ pmemlog_close(plp); return 0; } pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_macros.c0000664000000000000000000002421513615011243023204 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * obj_pmemlog_macros.c -- alternate pmemlog implementation based on pmemobj * * usage: obj_pmemlog_macros [co] file [cmd[:param]...] * * c - create file * o - open file * * The "cmd" arguments match the pmemlog functions: * a - append * v - appendv * r - rewind * w - walk * n - nbyte * t - tell * "a" and "v" require a parameter string(s) separated by a colon */ #include #include #include #include #include #include #include "libpmemobj.h" #include "libpmem.h" #include "libpmemlog.h" #define POOL_SIZE ((size_t)(1024 * 1024 * 100)) POBJ_LAYOUT_BEGIN(obj_pmemlog_macros); POBJ_LAYOUT_ROOT(obj_pmemlog_macros, struct base); POBJ_LAYOUT_TOID(obj_pmemlog_macros, struct log); POBJ_LAYOUT_END(obj_pmemlog_macros); /* log entry header */ struct log_hdr { TOID(struct log) next; /* object ID of the next log buffer */ size_t size; /* size of this log buffer */ }; /* struct log stores the entire log entry */ struct log { struct log_hdr hdr; /* entry header */ char data[]; /* log entry data */ }; /* struct base keeps track of the beginning of the log list */ struct base { TOID(struct log) head; /* object ID of the first log buffer */ TOID(struct log) tail; /* object ID of the last log buffer */ PMEMrwlock rwlock; /* lock covering entire log */ size_t bytes_written; /* number of bytes stored in the pool */ }; /* * pmemlog_open -- pool open wrapper */ PMEMlogpool * pmemlog_open(const char *path) { return (PMEMlogpool *)pmemobj_open(path, POBJ_LAYOUT_NAME(obj_pmemlog_macros)); } /* * pmemlog_create -- pool create wrapper */ PMEMlogpool * pmemlog_create(const char *path, size_t poolsize, mode_t mode) { return (PMEMlogpool *)pmemobj_create(path, POBJ_LAYOUT_NAME(obj_pmemlog_macros), poolsize, mode); } /* * pool_close -- pool close wrapper */ void pmemlog_close(PMEMlogpool *plp) { pmemobj_close((PMEMobjpool *)plp); } /* * pmemlog_nbyte -- not available in this implementation */ size_t pmemlog_nbyte(PMEMlogpool *plp) { /* N/A */ return 0; } /* * pmemlog_append -- add data to a log memory pool */ int pmemlog_append(PMEMlogpool *plp, const void *buf, size_t count) { PMEMobjpool *pop = (PMEMobjpool *)plp; int retval = 0; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); /* begin a transaction, also acquiring the write lock for the log */ TX_BEGIN_PARAM(pop, TX_PARAM_RWLOCK, &D_RW(bp)->rwlock, TX_PARAM_NONE) { /* allocate the new node to be inserted */ TOID(struct log) logp; logp = TX_ALLOC(struct log, count + sizeof(struct log_hdr)); D_RW(logp)->hdr.size = count; memcpy(D_RW(logp)->data, buf, count); D_RW(logp)->hdr.next = TOID_NULL(struct log); /* add the modified root object to the undo log */ TX_ADD(bp); if (TOID_IS_NULL(D_RO(bp)->tail)) { /* update head */ D_RW(bp)->head = logp; } else { /* add the modified tail entry to the undo log */ TX_ADD(D_RW(bp)->tail); D_RW(D_RW(bp)->tail)->hdr.next = logp; } D_RW(bp)->tail = logp; /* update tail */ D_RW(bp)->bytes_written += count; } TX_ONABORT { retval = -1; } TX_END return retval; } /* * pmemlog_appendv -- add gathered data to a log memory pool */ int pmemlog_appendv(PMEMlogpool *plp, const struct iovec *iov, int iovcnt) { PMEMobjpool *pop = (PMEMobjpool *)plp; int retval = 0; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); /* begin a transaction, also acquiring the write lock for the log */ TX_BEGIN_PARAM(pop, TX_PARAM_RWLOCK, &D_RW(bp)->rwlock, TX_PARAM_NONE) { /* add the base object and tail entry to the undo log */ TX_ADD(bp); if (!TOID_IS_NULL(D_RO(bp)->tail)) TX_ADD(D_RW(bp)->tail); /* append the data */ for (int i = 0; i < iovcnt; ++i) { char *buf = (char *)iov[i].iov_base; size_t count = iov[i].iov_len; /* allocate the new node to be inserted */ TOID(struct log) logp; logp = TX_ALLOC(struct log, count + sizeof(struct log_hdr)); D_RW(logp)->hdr.size = count; memcpy(D_RW(logp)->data, buf, count); D_RW(logp)->hdr.next = TOID_NULL(struct log); /* update head or tail accordingly */ if (TOID_IS_NULL(D_RO(bp)->tail)) D_RW(bp)->head = logp; else D_RW(D_RW(bp)->tail)->hdr.next = logp; /* update tail */ D_RW(bp)->tail = logp; D_RW(bp)->bytes_written += count; } } TX_ONABORT { retval = -1; } TX_END return retval; } /* * pmemlog_tell -- returns the current write point for the log */ long long pmemlog_tell(PMEMlogpool *plp) { TOID(struct base) bp; bp = POBJ_ROOT((PMEMobjpool *)plp, struct base); return D_RO(bp)->bytes_written; } /* * pmemlog_rewind -- discard all data, resetting a log memory pool to empty */ void pmemlog_rewind(PMEMlogpool *plp) { PMEMobjpool *pop = (PMEMobjpool *)plp; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); /* begin a transaction, also acquiring the write lock for the log */ TX_BEGIN_PARAM(pop, TX_PARAM_RWLOCK, &D_RW(bp)->rwlock, TX_PARAM_NONE) { /* add the root object to the undo log */ TX_ADD(bp); while (!TOID_IS_NULL(D_RO(bp)->head)) { TOID(struct log) nextp; nextp = D_RW(D_RW(bp)->head)->hdr.next; TX_FREE(D_RW(bp)->head); D_RW(bp)->head = nextp; } D_RW(bp)->head = TOID_NULL(struct log); D_RW(bp)->tail = TOID_NULL(struct log); D_RW(bp)->bytes_written = 0; } TX_END } /* * pmemlog_walk -- walk through all data in a log memory pool * * As this implementation holds the size of each entry, the chunksize is ignored * and the process_chunk function gets the actual entry length. */ void pmemlog_walk(PMEMlogpool *plp, size_t chunksize, int (*process_chunk)(const void *buf, size_t len, void *arg), void *arg) { PMEMobjpool *pop = (PMEMobjpool *)plp; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); /* acquire a read lock */ if (pmemobj_rwlock_rdlock(pop, &D_RW(bp)->rwlock) != 0) return; TOID(struct log) next; next = D_RO(bp)->head; /* process all chunks */ while (!TOID_IS_NULL(next)) { (*process_chunk)(D_RO(next)->data, D_RO(next)->hdr.size, arg); next = D_RO(next)->hdr.next; } pmemobj_rwlock_unlock(pop, &D_RW(bp)->rwlock); } /* * process_chunk -- (internal) process function for log_walk */ static int process_chunk(const void *buf, size_t len, void *arg) { char *tmp = (char *)malloc(len + 1); if (tmp == NULL) { fprintf(stderr, "malloc error\n"); return 0; } memcpy(tmp, buf, len); tmp[len] = '\0'; printf("log contains:\n"); printf("%s\n", tmp); free(tmp); return 1; /* continue */ } /* * count_iovec -- (internal) count the number of iovec items */ static int count_iovec(char *arg) { int count = 1; char *pch = strchr(arg, ':'); while (pch != NULL) { ++count; pch = strchr(++pch, ':'); } return count; } /* * fill_iovec -- (internal) fill out the iovec */ static void fill_iovec(struct iovec *iov, char *arg) { char *pch = strtok(arg, ":"); while (pch != NULL) { iov->iov_base = pch; iov->iov_len = strlen((char *)iov->iov_base); ++iov; pch = strtok(NULL, ":"); } } int main(int argc, char *argv[]) { if (argc < 2) { fprintf(stderr, "usage: %s [o,c] file [val...]\n", argv[0]); return 1; } PMEMlogpool *plp; if (strncmp(argv[1], "c", 1) == 0) { plp = pmemlog_create(argv[2], POOL_SIZE, CREATE_MODE_RW); } else if (strncmp(argv[1], "o", 1) == 0) { plp = pmemlog_open(argv[2]); } else { fprintf(stderr, "usage: %s [o,c] file [val...]\n", argv[0]); return 1; } if (plp == NULL) { perror("pmemlog_create/pmemlog_open"); return 1; } /* process the command line arguments */ for (int i = 3; i < argc; i++) { switch (*argv[i]) { case 'a': { printf("append: %s\n", argv[i] + 2); if (pmemlog_append(plp, argv[i] + 2, strlen(argv[i] + 2))) fprintf(stderr, "pmemlog_append" " error\n"); break; } case 'v': { printf("appendv: %s\n", argv[i] + 2); int count = count_iovec(argv[i] + 2); struct iovec *iov = (struct iovec *)malloc( count * sizeof(struct iovec)); if (iov == NULL) { fprintf(stderr, "malloc error\n"); break; } fill_iovec(iov, argv[i] + 2); if (pmemlog_appendv(plp, iov, count)) fprintf(stderr, "pmemlog_appendv" " error\n"); free(iov); break; } case 'r': { printf("rewind\n"); pmemlog_rewind(plp); break; } case 'w': { printf("walk\n"); pmemlog_walk(plp, 0, process_chunk, NULL); break; } case 'n': { printf("nbytes: %zu\n", pmemlog_nbyte(plp)); break; } case 't': { printf("offset: %lld\n", pmemlog_tell(plp)); break; } default: { fprintf(stderr, "unrecognized command %s\n", argv[i]); break; } }; } /* all done */ pmemlog_close(plp); return 0; } pmdk-1.8/src/examples/libpmemobj/pmemlog/README0000664000000000000000000000220313615011243020053 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemobj/pmemlog/README. This directory contains a set of example implementations of libpmemlog using libpmemobj. All implementations have exactly the same API as libpmemlog, but due to the different approaches to the implementation, some functions might not be available. To launch the examples: [co] file [cmd[:param]...] Where: c - create file o - open file The "cmd" arguments match the pmemlog functions: a - append v - appendv r - rewind w - walk n - nbyte t - tell "a" and "v" require a parameter string(s) separated by a colon. Due to a different implementation obj_pmemlog_simple requires an additional chunksize parameter for the walk function. These examples demonstrate the usage of the very basics of the pmemobj library. That includes the pool management (open/create/close), type-safety macros, transactions, transactional allocations and synchronization. Please take note that this is only an example and does not implement full user input validation, so for example creating the same pool twice yields undefined behavior. pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_minimal.vcxproj.filters0000664000000000000000000000066013615011243026264 0ustar rootroot {d49d6fd2-19d5-443c-be25-5af7594fea7b} Source Files pmdk-1.8/src/examples/libpmemobj/pmemlog/.gitignore0000664000000000000000000000010613615011243021163 0ustar rootrootobj_pmemlog obj_pmemlog_macros obj_pmemlog_minimal obj_pmemlog_simple pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_simple.vcxproj0000664000000000000000000000565313615011243024467 0ustar rootroot Debug x64 Release x64 {5DB2E259-0D19-4A89-B8EC-B2912F39924D} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest 4996;4200 CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_macros.vcxproj.filters0000664000000000000000000000065713615011243026130 0ustar rootroot {43b4e772-5caf-4b85-9dbb-0aa45671fc94} Source Files pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog.c0000664000000000000000000002537113615011243021644 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * obj_pmemlog.c -- alternate pmemlog implementation based on pmemobj * * usage: obj_pmemlog [co] file [cmd[:param]...] * * c - create file * o - open file * * The "cmd" arguments match the pmemlog functions: * a - append * v - appendv * r - rewind * w - walk * n - nbyte * t - tell * "a" and "v" require a parameter string(s) separated by a colon */ #include #include #include #include #include #include #include "libpmemobj.h" #include "libpmem.h" #include "libpmemlog.h" #define LAYOUT_NAME "obj_pmemlog" #define POOL_SIZE ((size_t)(1024 * 1024 * 100)) /* types of allocations */ enum types { LOG_TYPE, LOG_HDR_TYPE, BASE_TYPE, MAX_TYPES }; /* log entry header */ struct log_hdr { PMEMoid next; /* object ID of the next log buffer */ size_t size; /* size of this log buffer */ }; /* struct log stores the entire log entry */ struct log { struct log_hdr hdr; /* entry header */ char data[]; /* log entry data */ }; /* struct base keeps track of the beginning of the log list */ struct base { PMEMoid head; /* object ID of the first log buffer */ PMEMoid tail; /* object ID of the last log buffer */ PMEMrwlock rwlock; /* lock covering entire log */ size_t bytes_written; /* number of bytes stored in the pool */ }; /* * pmemlog_open -- pool open wrapper */ PMEMlogpool * pmemlog_open(const char *path) { return (PMEMlogpool *)pmemobj_open(path, LAYOUT_NAME); } /* * pmemlog_create -- pool create wrapper */ PMEMlogpool * pmemlog_create(const char *path, size_t poolsize, mode_t mode) { return (PMEMlogpool *)pmemobj_create(path, LAYOUT_NAME, poolsize, mode); } /* * pmemlog_close -- pool close wrapper */ void pmemlog_close(PMEMlogpool *plp) { pmemobj_close((PMEMobjpool *)plp); } /* * pmemlog_nbyte -- not available in this implementation */ size_t pmemlog_nbyte(PMEMlogpool *plp) { /* N/A */ return 0; } /* * pmemlog_append -- add data to a log memory pool */ int pmemlog_append(PMEMlogpool *plp, const void *buf, size_t count) { PMEMobjpool *pop = (PMEMobjpool *)plp; PMEMoid baseoid = pmemobj_root(pop, sizeof(struct base)); struct base *bp = pmemobj_direct(baseoid); /* set the return point */ jmp_buf env; if (setjmp(env)) { /* end the transaction */ (void) pmemobj_tx_end(); return 1; } /* begin a transaction, also acquiring the write lock for the log */ if (pmemobj_tx_begin(pop, env, TX_PARAM_RWLOCK, &bp->rwlock, TX_PARAM_NONE)) return -1; /* allocate the new node to be inserted */ PMEMoid log = pmemobj_tx_alloc(count + sizeof(struct log_hdr), LOG_TYPE); struct log *logp = pmemobj_direct(log); logp->hdr.size = count; memcpy(logp->data, buf, count); logp->hdr.next = OID_NULL; /* add the modified root object to the undo log */ pmemobj_tx_add_range(baseoid, 0, sizeof(struct base)); if (bp->tail.off == 0) { /* update head */ bp->head = log; } else { /* add the modified tail entry to the undo log */ pmemobj_tx_add_range(bp->tail, 0, sizeof(struct log)); ((struct log *)pmemobj_direct(bp->tail))->hdr.next = log; } bp->tail = log; /* update tail */ bp->bytes_written += count; pmemobj_tx_commit(); (void) pmemobj_tx_end(); return 0; } /* * pmemlog_appendv -- add gathered data to a log memory pool */ int pmemlog_appendv(PMEMlogpool *plp, const struct iovec *iov, int iovcnt) { PMEMobjpool *pop = (PMEMobjpool *)plp; PMEMoid baseoid = pmemobj_root(pop, sizeof(struct base)); struct base *bp = pmemobj_direct(baseoid); /* set the return point */ jmp_buf env; if (setjmp(env)) { /* end the transaction */ pmemobj_tx_end(); return 1; } /* begin a transaction, also acquiring the write lock for the log */ if (pmemobj_tx_begin(pop, env, TX_PARAM_RWLOCK, &bp->rwlock, TX_PARAM_NONE)) return -1; /* add the base object to the undo log - once for the transaction */ pmemobj_tx_add_range(baseoid, 0, sizeof(struct base)); /* add the tail entry once to the undo log, if it is set */ if (!OID_IS_NULL(bp->tail)) pmemobj_tx_add_range(bp->tail, 0, sizeof(struct log)); /* append the data */ for (int i = 0; i < iovcnt; ++i) { char *buf = iov[i].iov_base; size_t count = iov[i].iov_len; /* allocate the new node to be inserted */ PMEMoid log = pmemobj_tx_alloc(count + sizeof(struct log_hdr), LOG_TYPE); struct log *logp = pmemobj_direct(log); logp->hdr.size = count; memcpy(logp->data, buf, count); logp->hdr.next = OID_NULL; if (bp->tail.off == 0) { bp->head = log; /* update head */ } else { ((struct log *)pmemobj_direct(bp->tail))->hdr.next = log; } bp->tail = log; /* update tail */ bp->bytes_written += count; } pmemobj_tx_commit(); (void) pmemobj_tx_end(); return 0; } /* * pmemlog_tell -- returns the current write point for the log */ long long pmemlog_tell(PMEMlogpool *plp) { PMEMobjpool *pop = (PMEMobjpool *)plp; struct base *bp = pmemobj_direct(pmemobj_root(pop, sizeof(struct base))); if (pmemobj_rwlock_rdlock(pop, &bp->rwlock) != 0) return 0; long long bytes_written = bp->bytes_written; pmemobj_rwlock_unlock(pop, &bp->rwlock); return bytes_written; } /* * pmemlog_rewind -- discard all data, resetting a log memory pool to empty */ void pmemlog_rewind(PMEMlogpool *plp) { PMEMobjpool *pop = (PMEMobjpool *)plp; PMEMoid baseoid = pmemobj_root(pop, sizeof(struct base)); struct base *bp = pmemobj_direct(baseoid); /* set the return point */ jmp_buf env; if (setjmp(env)) { /* end the transaction */ pmemobj_tx_end(); return; } /* begin a transaction, also acquiring the write lock for the log */ if (pmemobj_tx_begin(pop, env, TX_PARAM_RWLOCK, &bp->rwlock, TX_PARAM_NONE)) return; /* add the root object to the undo log */ pmemobj_tx_add_range(baseoid, 0, sizeof(struct base)); /* free all log nodes */ while (bp->head.off != 0) { PMEMoid nextoid = ((struct log *)pmemobj_direct(bp->head))->hdr.next; pmemobj_tx_free(bp->head); bp->head = nextoid; } bp->head = OID_NULL; bp->tail = OID_NULL; bp->bytes_written = 0; pmemobj_tx_commit(); (void) pmemobj_tx_end(); } /* * pmemlog_walk -- walk through all data in a log memory pool * * As this implementation holds the size of each entry, the chunksize is ignored * and the process_chunk function gets the actual entry length. */ void pmemlog_walk(PMEMlogpool *plp, size_t chunksize, int (*process_chunk)(const void *buf, size_t len, void *arg), void *arg) { PMEMobjpool *pop = (PMEMobjpool *)plp; struct base *bp = pmemobj_direct(pmemobj_root(pop, sizeof(struct base))); if (pmemobj_rwlock_rdlock(pop, &bp->rwlock) != 0) return; /* process all chunks */ struct log *next = pmemobj_direct(bp->head); while (next != NULL) { (*process_chunk)(next->data, next->hdr.size, arg); next = pmemobj_direct(next->hdr.next); } pmemobj_rwlock_unlock(pop, &bp->rwlock); } /* * process_chunk -- (internal) process function for log_walk */ static int process_chunk(const void *buf, size_t len, void *arg) { char *tmp = malloc(len + 1); if (tmp == NULL) { fprintf(stderr, "malloc error\n"); return 0; } memcpy(tmp, buf, len); tmp[len] = '\0'; printf("log contains:\n"); printf("%s\n", tmp); free(tmp); return 1; } /* * count_iovec -- (internal) count the number of iovec items */ static int count_iovec(char *arg) { int count = 1; char *pch = strchr(arg, ':'); while (pch != NULL) { ++count; pch = strchr(++pch, ':'); } return count; } /* * fill_iovec -- (internal) fill out the iovec */ static void fill_iovec(struct iovec *iov, char *arg) { char *pch = strtok(arg, ":"); while (pch != NULL) { iov->iov_base = pch; iov->iov_len = strlen((char *)iov->iov_base); ++iov; pch = strtok(NULL, ":"); } } int main(int argc, char *argv[]) { if (argc < 2) { fprintf(stderr, "usage: %s [o,c] file [val...]\n", argv[0]); return 1; } PMEMlogpool *plp; if (strncmp(argv[1], "c", 1) == 0) { plp = pmemlog_create(argv[2], POOL_SIZE, CREATE_MODE_RW); } else if (strncmp(argv[1], "o", 1) == 0) { plp = pmemlog_open(argv[2]); } else { fprintf(stderr, "usage: %s [o,c] file [val...]\n", argv[0]); return 1; } if (plp == NULL) { perror("pmemlog_create/pmemlog_open"); return 1; } /* process the command line arguments */ for (int i = 3; i < argc; i++) { switch (*argv[i]) { case 'a': { printf("append: %s\n", argv[i] + 2); if (pmemlog_append(plp, argv[i] + 2, strlen(argv[i] + 2))) fprintf(stderr, "pmemlog_append" " error\n"); break; } case 'v': { printf("appendv: %s\n", argv[i] + 2); int count = count_iovec(argv[i] + 2); struct iovec *iov = calloc(count, sizeof(struct iovec)); fill_iovec(iov, argv[i] + 2); if (pmemlog_appendv(plp, iov, count)) fprintf(stderr, "pmemlog_appendv" " error\n"); free(iov); break; } case 'r': { printf("rewind\n"); pmemlog_rewind(plp); break; } case 'w': { printf("walk\n"); pmemlog_walk(plp, 0, process_chunk, NULL); break; } case 'n': { printf("nbytes: %zu\n", pmemlog_nbyte(plp)); break; } case 't': { printf("offset: %lld\n", pmemlog_tell(plp)); break; } default: { fprintf(stderr, "unrecognized command %s\n", argv[i]); break; } }; } /* all done */ pmemlog_close(plp); return 0; } pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog.vcxproj0000664000000000000000000000534513615011243023114 0ustar rootroot Debug x64 Release x64 {60206D22-E132-4695-8486-10BECA32C5CC} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog.vcxproj.filters0000664000000000000000000000065013615011243024555 0ustar rootroot {710436f3-9be8-4a0f-9d2b-498ca570c18c} Source Files pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_minimal.vcxproj0000664000000000000000000000535513615011243024623 0ustar rootroot Debug x64 Release x64 {0056B0B6-CB3E-4F0E-B6DC-48D59CB8E235} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/pmemlog/Makefile0000664000000000000000000000363513615011243020645 0ustar rootroot# # Copyright 2015-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/pmemlog/Makefile -- build obj implementations of pmemlog # PROGS = obj_pmemlog obj_pmemlog_macros obj_pmemlog_minimal obj_pmemlog_simple LIBS = -lpmemobj -lpmem -pthread include ../../Makefile.inc obj_pmemlog: obj_pmemlog.o obj_pmemlog_macros: obj_pmemlog_macros.o obj_pmemlog_minimal: obj_pmemlog_minimal.o obj_pmemlog_simple: obj_pmemlog_simple.o pmdk-1.8/src/examples/libpmemobj/pmemlog/obj_pmemlog_macros.vcxproj0000664000000000000000000000565313615011243024462 0ustar rootroot Debug x64 Release x64 {06877FED-15BA-421F-85C9-1A964FB97446} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest 4996;4200 CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/array/0000775000000000000000000000000013615011243016654 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/array/array.vcxproj.filters0000664000000000000000000000064213615011243023060 0ustar rootroot {71d04e36-7996-4ae7-84d3-fbad27441817} Source Files pmdk-1.8/src/examples/libpmemobj/array/README0000664000000000000000000000355113615011243017540 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemobj/array/README. This directory contains an example application implemented using libpmemobj. The array example allows one to perform basic operations like allocation, reallocation and de-allocation on persistent array. User can choose size of new or reallocated array. There are also 3 types of elements which are supported: int, PMEMoid and TOID with structure containing int variable. Persistent pointer can be treated as a pointer to the first element of array. TOID with user's defined structure can be allocated by using POBJ_ALLOC or POBJ_ZALLOC macros (similarly when using PMEMoid there are pmemobj_alloc() and pmemobj_zalloc() functions) with proper number elements multiplied by user-defined structure's size. Access to array's elements can be executed by using pmemobj_direct() function or just using proper macro. For example D_RW(test1)[0] points to first element of test1 array. To allocate new array using application run the following command: $ array alloc Where is file where pool will be created or opened, is user's defined unique name, is number of elements of persistent array and is one of listed: int, PMEMoid, TOID. To reallocate existing array run the following command: $ array realloc To free existing array run the following command: $ array free To print array's elements run the following command: $ array print Example of usage: $ array /mnt/pmem/testfile alloc test1 10 TOID $ array /mnt/pmem/testfile alloc test2 100 int $ array /mnt/pmem/testfile realloc test1 50 $ array /mnt/pmem/testfile realloc test2 5 $ array /mnt/pmem/testfile print test1 $ array /mnt/pmem/testfile free test1 $ array /mnt/pmem/testfile free test2 pmdk-1.8/src/examples/libpmemobj/array/.gitignore0000664000000000000000000000000613615011243020640 0ustar rootrootarray pmdk-1.8/src/examples/libpmemobj/array/array.vcxproj0000664000000000000000000000570613615011243021417 0ustar rootroot Debug x64 Release x64 {7264C8F6-73FB-4830-9306-1558D3EAC71B} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/array/Makefile0000664000000000000000000000327513615011243020323 0ustar rootroot# # Copyright 2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/array/Makefile -- build the array example # PROGS = array LIBS = -lpmemobj -lpmem -pthread include ../../Makefile.inc array: array.o pmdk-1.8/src/examples/libpmemobj/array/array.c0000664000000000000000000003205213615011243020140 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * array.c -- example of arrays usage */ #include #include #include #include #include #include #include #define TOID_ARRAY(x) TOID(x) #define COUNT_OF(x) (sizeof(x) / sizeof(x[0])) #define MAX_BUFFLEN 30 #define MAX_TYPE_NUM 8 POBJ_LAYOUT_BEGIN(array); POBJ_LAYOUT_TOID(array, struct array_elm); POBJ_LAYOUT_TOID(array, int); POBJ_LAYOUT_TOID(array, PMEMoid); POBJ_LAYOUT_TOID(array, TOID(struct array_elm)); POBJ_LAYOUT_TOID(array, struct array_info); POBJ_LAYOUT_END(array); static PMEMobjpool *pop; enum array_types { UNKNOWN_ARRAY_TYPE, INT_ARRAY_TYPE, PMEMOID_ARRAY_TYPE, TOID_ARRAY_TYPE, MAX_ARRAY_TYPE }; struct array_elm { int id; }; struct array_info { char name[MAX_BUFFLEN]; size_t size; enum array_types type; PMEMoid array; }; /* * print_usage -- print general usage */ static void print_usage(void) { printf("usage: ./array " "" " [ []]\n"); } /* * get type -- parse argument given as type of array */ static enum array_types get_type(const char *type_name) { const char *names[MAX_ARRAY_TYPE] = {"", "int", "PMEMoid", "TOID"}; enum array_types type; for (type = (enum array_types)(MAX_ARRAY_TYPE - 1); type > UNKNOWN_ARRAY_TYPE; type = (enum array_types)(type - 1)) { if (strcmp(names[type], type_name) == 0) break; } if (type == UNKNOWN_ARRAY_TYPE) fprintf(stderr, "unknown type: %s\n", type_name); return type; } /* * find_aray -- return info about array with proper name */ static TOID(struct array_info) find_array(const char *name) { TOID(struct array_info) info; POBJ_FOREACH_TYPE(pop, info) { if (strncmp(D_RO(info)->name, name, MAX_BUFFLEN) == 0) return info; } return TOID_NULL(struct array_info); } /* * elm_constructor -- constructor of array_elm type object */ static int elm_constructor(PMEMobjpool *pop, void *ptr, void *arg) { struct array_elm *obj = (struct array_elm *)ptr; int *id = (int *)arg; obj->id = *id; pmemobj_persist(pop, obj, sizeof(*obj)); return 0; } /* * print_int -- print array of int type */ static void print_int(struct array_info *info) { TOID(int) array; TOID_ASSIGN(array, info->array); for (size_t i = 0; i < info->size; i++) printf("%d ", D_RO(array)[i]); } /* * print_pmemoid -- print array of PMEMoid type */ static void print_pmemoid(struct array_info *info) { TOID(PMEMoid) array; TOID(struct array_elm) elm; TOID_ASSIGN(array, info->array); for (size_t i = 0; i < info->size; i++) { TOID_ASSIGN(elm, D_RW(array)[i]); printf("%d ", D_RO(elm)->id); } } /* * print_toid -- print array of TOID(struct array_elm) type */ static void print_toid(struct array_info *info) { TOID_ARRAY(TOID(struct array_elm)) array; TOID_ASSIGN(array, info->array); for (size_t i = 0; i < info->size; i++) printf("%d ", D_RO(D_RO(array)[i])->id); } typedef void (*fn_print)(struct array_info *info); static fn_print print_array[] = {NULL, print_int, print_pmemoid, print_toid}; /* * free_int -- de-allocate array of int type */ static void free_int(struct array_info *info) { TOID(int) array; TOID_ASSIGN(array, info->array); /* * When there is persistent array of simple type allocated, * there is enough to de-allocate persistent pointer */ POBJ_FREE(&array); } /* * free_pmemoid -- de-allocate array of PMEMoid type */ static void free_pmemoid(struct array_info *info) { TOID(PMEMoid) array; TOID_ASSIGN(array, info->array); /* * When there is persistent array of persistent pointer type allocated, * there is necessary to de-allocate each element, if they were * allocated earlier */ for (size_t i = 0; i < info->size; i++) pmemobj_free(&D_RW(array)[i]); POBJ_FREE(&array); } /* * free_toid -- de-allocate array of TOID(struct array_elm) type */ static void free_toid(struct array_info *info) { TOID_ARRAY(TOID(struct array_elm)) array; TOID_ASSIGN(array, info->array); /* * When there is persistent array of persistent pointer type allocated, * there is necessary to de-allocate each element, if they were * allocated earlier */ for (size_t i = 0; i < info->size; i++) POBJ_FREE(&D_RW(array)[i]); POBJ_FREE(&array); } typedef void (*fn_free)(struct array_info *info); static fn_free free_array[] = {NULL, free_int, free_pmemoid, free_toid}; /* * realloc_int -- reallocate array of int type */ static PMEMoid realloc_int(PMEMoid *info, size_t prev_size, size_t size) { TOID(int) array; TOID_ASSIGN(array, *info); POBJ_REALLOC(pop, &array, int, size * sizeof(int)); if (size > prev_size) { for (size_t i = prev_size; i < size; i++) D_RW(array)[i] = (int)i; pmemobj_persist(pop, D_RW(array) + prev_size, (size - prev_size) * sizeof(*D_RW(array))); } return array.oid; } /* * realloc_pmemoid -- reallocate array of PMEMoid type */ static PMEMoid realloc_pmemoid(PMEMoid *info, size_t prev_size, size_t size) { TOID(PMEMoid) array; TOID_ASSIGN(array, *info); pmemobj_zrealloc(pop, &array.oid, sizeof(PMEMoid) * size, TOID_TYPE_NUM(PMEMoid)); for (size_t i = prev_size; i < size; i++) { if (pmemobj_alloc(pop, &D_RW(array)[i], sizeof(struct array_elm), TOID_TYPE_NUM(PMEMoid), elm_constructor, &i)) { fprintf(stderr, "pmemobj_alloc\n"); assert(0); } } return array.oid; } /* * realloc_toid -- reallocate array of TOID(struct array_elm) type */ static PMEMoid realloc_toid(PMEMoid *info, size_t prev_size, size_t size) { TOID_ARRAY(TOID(struct array_elm)) array; TOID_ASSIGN(array, *info); pmemobj_zrealloc(pop, &array.oid, sizeof(TOID(struct array_elm)) * size, TOID_TYPE_NUM_OF(array)); for (size_t i = prev_size; i < size; i++) { POBJ_NEW(pop, &D_RW(array)[i], struct array_elm, elm_constructor, &i); if (TOID_IS_NULL(D_RW(array)[i])) { fprintf(stderr, "POBJ_ALLOC\n"); assert(0); } } return array.oid; } typedef PMEMoid (*fn_realloc)(PMEMoid *info, size_t prev_size, size_t size); static fn_realloc realloc_array[] = {NULL, realloc_int, realloc_pmemoid, realloc_toid}; /* * alloc_int -- allocate array of int type */ static PMEMoid alloc_int(size_t size) { TOID(int) array; /* * To allocate persistent array of simple type is enough to allocate * pointer with size equal to number of elements multiplied by size of * user-defined structure. */ POBJ_ALLOC(pop, &array, int, sizeof(int) * size, NULL, NULL); if (TOID_IS_NULL(array)) { fprintf(stderr, "POBJ_ALLOC\n"); return OID_NULL; } for (size_t i = 0; i < size; i++) D_RW(array)[i] = (int)i; pmemobj_persist(pop, D_RW(array), size * sizeof(*D_RW(array))); return array.oid; } /* * alloc_pmemoid -- allocate array of PMEMoid type */ static PMEMoid alloc_pmemoid(size_t size) { TOID(PMEMoid) array; /* * To allocate persistent array of PMEMoid type is necessary to allocate * pointer with size equal to number of elements multiplied by size of * PMEMoid and to allocate each of elements separately. */ POBJ_ALLOC(pop, &array, PMEMoid, sizeof(PMEMoid) * size, NULL, NULL); if (TOID_IS_NULL(array)) { fprintf(stderr, "POBJ_ALLOC\n"); return OID_NULL; } for (size_t i = 0; i < size; i++) { if (pmemobj_alloc(pop, &D_RW(array)[i], sizeof(struct array_elm), TOID_TYPE_NUM(PMEMoid), elm_constructor, &i)) { fprintf(stderr, "pmemobj_alloc\n"); } } return array.oid; } /* * alloc_toid -- allocate array of TOID(struct array_elm) type */ static PMEMoid alloc_toid(size_t size) { TOID_ARRAY(TOID(struct array_elm)) array; /* * To allocate persistent array of TOID with user-defined structure type * is necessary to allocate pointer with size equal to number of * elements multiplied by size of TOID of proper type and to allocate * each of elements separately. */ POBJ_ALLOC(pop, &array, TOID(struct array_elm), sizeof(TOID(struct array_elm)) * size, NULL, NULL); if (TOID_IS_NULL(array)) { fprintf(stderr, "POBJ_ALLOC\n"); return OID_NULL; } for (size_t i = 0; i < size; i++) { POBJ_NEW(pop, &D_RW(array)[i], struct array_elm, elm_constructor, &i); if (TOID_IS_NULL(D_RW(array)[i])) { fprintf(stderr, "POBJ_ALLOC\n"); assert(0); } } return array.oid; } typedef PMEMoid (*fn_alloc)(size_t size); static fn_alloc alloc_array[] = {NULL, alloc_int, alloc_pmemoid, alloc_toid}; /* * do_print -- print values stored by proper array */ static void do_print(int argc, char *argv[]) { if (argc != 1) { printf("usage: ./array print \n"); return; } TOID(struct array_info) array_info = find_array(argv[0]); if (TOID_IS_NULL(array_info)) { printf("%s doesn't exist\n", argv[0]); return; } printf("%s:\n", argv[0]); print_array[D_RO(array_info)->type](D_RW(array_info)); printf("\n"); } /* * do_free -- de-allocate proper array and proper TOID of array_info type */ static void do_free(int argc, char *argv[]) { if (argc != 1) { printf("usage: ./array free \n"); return; } TOID(struct array_info) array_info = find_array(argv[0]); if (TOID_IS_NULL(array_info)) { printf("%s doesn't exist\n", argv[0]); return; } free_array[D_RO(array_info)->type](D_RW(array_info)); POBJ_FREE(&array_info); } /* * do_realloc -- reallocate proper array to given size and update information * in array_info structure */ static void do_realloc(int argc, char *argv[]) { if (argc != 2) { printf("usage: ./array realloc" " \n"); return; } size_t size = atoi(argv[1]); TOID(struct array_info) array_info = find_array(argv[0]); if (TOID_IS_NULL(array_info)) { printf("%s doesn't exist\n", argv[0]); return; } struct array_info *info = D_RW(array_info); info->array = realloc_array[info->type](&info->array, info->size, size); if (OID_IS_NULL(info->array)) { if (size != 0) printf("POBJ_REALLOC\n"); } info->size = size; pmemobj_persist(pop, info, sizeof(*info)); } /* * do_alloc -- allocate persistent array and TOID of array_info type * and set it with information about new array */ static void do_alloc(int argc, char *argv[]) { if (argc != 3) { printf("usage: ./array alloc " " \n"); return; } enum array_types type = get_type(argv[2]); if (type == UNKNOWN_ARRAY_TYPE) return; size_t size = atoi(argv[1]); TOID(struct array_info) array_info = find_array(argv[0]); if (!TOID_IS_NULL(array_info)) POBJ_FREE(&array_info); POBJ_ZNEW(pop, &array_info, struct array_info); struct array_info *info = D_RW(array_info); strncpy(info->name, argv[0], MAX_BUFFLEN - 1); info->name[MAX_BUFFLEN - 1] = '\0'; info->size = size; info->type = type; info->array = alloc_array[type](size); if (OID_IS_NULL(info->array)) assert(0); pmemobj_persist(pop, info, sizeof(*info)); } typedef void (*fn_op)(int argc, char *argv[]); static fn_op operations[] = {do_alloc, do_realloc, do_free, do_print}; int main(int argc, char *argv[]) { if (argc < 3) { print_usage(); return 1; } const char *path = argv[1]; pop = NULL; if (file_exists(path) != 0) { if ((pop = pmemobj_create(path, POBJ_LAYOUT_NAME(array), PMEMOBJ_MIN_POOL, CREATE_MODE_RW)) == NULL) { printf("failed to create pool\n"); return 1; } } else { if ((pop = pmemobj_open(path, POBJ_LAYOUT_NAME(array))) == NULL) { printf("failed to open pool\n"); return 1; } } const char *option = argv[2]; argv += 3; argc -= 3; const char *names[] = {"alloc", "realloc", "free", "print"}; int i = 0; for (; i < COUNT_OF(names) && strcmp(option, names[i]) != 0; i++); if (i != COUNT_OF(names)) operations[i](argc, argv); else print_usage(); pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/manpage.vcxproj0000664000000000000000000000532213615011243020565 0ustar rootroot Debug x64 Release x64 {EDA88BAB-9FA7-4A2D-8974-EFCFA24B3FEB} pmemobj 10.0.16299.0 Application true v140 Application false v140 true ..\..\LongPath.manifest libpmemobj.lib;libpmem.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} pmdk-1.8/src/examples/libpmemobj/lists.vcxproj0000664000000000000000000000551413615011243020316 0ustar rootroot Debug x64 Release x64 {2CD7408E-2F60-43C3-ACEB-C7D58CDD8462} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\LongPath.manifest CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/manpage.vcxproj.filters0000664000000000000000000000064413615011243022236 0ustar rootroot {72590f7a-670a-487a-8fda-695d23bb172c} Source Files pmdk-1.8/src/examples/libpmemobj/pi.vcxproj.filters0000664000000000000000000000063713615011243021240 0ustar rootroot {d1b32241-e048-49ce-904f-d7f06f71f38c} Source Files pmdk-1.8/src/examples/libpmemobj/pmemblk/0000775000000000000000000000000013615011243017165 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/pmemblk/README0000664000000000000000000000165213615011243020051 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemobj/pmemblk/README. This directory contains an example implementation of libpmemblk using libpmemobj. The implementation has exactly the same API as libpmemblk. To launch the example: usage: obj_pmemblk [co] file blk_size [cmd[:blk_num[:data]]...] Where: c - create file o - open file The "cmd" arguments match the pmemblk functions: w - write to a block r - read a block z - zero a block n - write out number of available blocks e - put a block in error state This example demonstrates the usage of the very basics of the pmemobj library. That includes the pool management (open/create/close), type-safety macros, transactions, transactional allocations and synchronization. Please take note that this is only an example and does not implement full user input validation, so for example creating the same pool twice yields undefined behavior. pmdk-1.8/src/examples/libpmemobj/pmemblk/obj_pmemblk.c0000664000000000000000000002532213615011243021616 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * obj_pmemblk.c -- alternate pmemblk implementation based on pmemobj * * usage: obj_pmemblk [co] file blk_size [cmd[:blk_num[:data]]...] * * c - create file * o - open file * * The "cmd" arguments match the pmemblk functions: * w - write to a block * r - read a block * z - zero a block * n - write out number of available blocks * e - put a block in error state */ #include #include #include #include #include #include #include #include "libpmemobj.h" #include "libpmem.h" #include "libpmemblk.h" #define USABLE_SIZE (7.0 / 10) #define POOL_SIZE ((size_t)(1024 * 1024 * 100)) #define MAX_POOL_SIZE ((size_t)1024 * 1024 * 1024 * 16) #define MAX_THREADS 256 #define BSIZE_MAX ((size_t)(1024 * 1024 * 10)) #define ZERO_MASK (1 << 0) #define ERROR_MASK (1 << 1) POBJ_LAYOUT_BEGIN(obj_pmemblk); POBJ_LAYOUT_ROOT(obj_pmemblk, struct base); POBJ_LAYOUT_TOID(obj_pmemblk, uint8_t); POBJ_LAYOUT_END(obj_pmemblk); /* The root object struct holding all necessary data */ struct base { TOID(uint8_t) data; /* contiguous memory region */ TOID(uint8_t) flags; /* block flags */ size_t bsize; /* block size */ size_t nblocks; /* number of available blocks */ PMEMmutex locks[MAX_THREADS]; /* thread synchronization locks */ }; /* * pmemblk_map -- (internal) read or initialize the blk pool */ static int pmemblk_map(PMEMobjpool *pop, size_t bsize, size_t fsize) { int retval = 0; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); /* read pool descriptor and validate user provided values */ if (D_RO(bp)->bsize) { if (bsize && D_RO(bp)->bsize != bsize) return -1; else return 0; } /* new pool, calculate and store metadata */ TX_BEGIN(pop) { TX_ADD(bp); D_RW(bp)->bsize = bsize; size_t pool_size = (size_t)(fsize * USABLE_SIZE); D_RW(bp)->nblocks = pool_size / bsize; D_RW(bp)->data = TX_ZALLOC(uint8_t, pool_size); D_RW(bp)->flags = TX_ZALLOC(uint8_t, sizeof(uint8_t) * D_RO(bp)->nblocks); } TX_ONABORT { retval = -1; } TX_END return retval; } /* * pmemblk_open -- open a block memory pool */ PMEMblkpool * pmemblk_open(const char *path, size_t bsize) { PMEMobjpool *pop = pmemobj_open(path, POBJ_LAYOUT_NAME(obj_pmemblk)); if (pop == NULL) return NULL; struct stat buf; if (stat(path, &buf)) { perror("stat"); return NULL; } return pmemblk_map(pop, bsize, buf.st_size) ? NULL : (PMEMblkpool *)pop; } /* * pmemblk_create -- create a block memory pool */ PMEMblkpool * pmemblk_create(const char *path, size_t bsize, size_t poolsize, mode_t mode) { /* max size of a single allocation is 16GB */ if (poolsize > MAX_POOL_SIZE) { errno = EINVAL; return NULL; } PMEMobjpool *pop = pmemobj_create(path, POBJ_LAYOUT_NAME(obj_pmemblk), poolsize, mode); if (pop == NULL) return NULL; return pmemblk_map(pop, bsize, poolsize) ? NULL : (PMEMblkpool *)pop; } /* * pmemblk_close -- close a block memory pool */ void pmemblk_close(PMEMblkpool *pbp) { pmemobj_close((PMEMobjpool *)pbp); } /* * pmemblk_check -- block memory pool consistency check */ int pmemblk_check(const char *path, size_t bsize) { int ret = pmemobj_check(path, POBJ_LAYOUT_NAME(obj_pmemblk)); if (ret) return ret; /* open just to validate block size */ PMEMblkpool *pop = pmemblk_open(path, bsize); if (!pop) return -1; pmemblk_close(pop); return 0; } /* * pmemblk_set_error -- not available in this implementation */ int pmemblk_set_error(PMEMblkpool *pbp, long long blockno) { PMEMobjpool *pop = (PMEMobjpool *)pbp; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); int retval = 0; if (blockno >= (long long)D_RO(bp)->nblocks) return 1; TX_BEGIN_PARAM(pop, TX_PARAM_MUTEX, &D_RW(bp)->locks[blockno % MAX_THREADS], TX_PARAM_NONE) { uint8_t *flags = D_RW(D_RW(bp)->flags) + blockno; /* add the modified flags to the undo log */ pmemobj_tx_add_range_direct(flags, sizeof(*flags)); *flags |= ERROR_MASK; } TX_ONABORT { retval = 1; } TX_END return retval; } /* * pmemblk_nblock -- return number of usable blocks in a block memory pool */ size_t pmemblk_nblock(PMEMblkpool *pbp) { PMEMobjpool *pop = (PMEMobjpool *)pbp; return ((struct base *)pmemobj_direct(pmemobj_root(pop, sizeof(struct base))))->nblocks; } /* * pmemblk_read -- read a block in a block memory pool */ int pmemblk_read(PMEMblkpool *pbp, void *buf, long long blockno) { PMEMobjpool *pop = (PMEMobjpool *)pbp; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); if (blockno >= (long long)D_RO(bp)->nblocks) return 1; pmemobj_mutex_lock(pop, &D_RW(bp)->locks[blockno % MAX_THREADS]); /* check the error mask */ uint8_t *flags = D_RW(D_RW(bp)->flags) + blockno; if ((*flags & ERROR_MASK) != 0) { pmemobj_mutex_unlock(pop, &D_RW(bp)->locks[blockno % MAX_THREADS]); errno = EIO; return 1; } /* the block is zeroed, reverse zeroing logic */ if ((*flags & ZERO_MASK) == 0) { memset(buf, 0, D_RO(bp)->bsize); } else { size_t block_off = blockno * D_RO(bp)->bsize; uint8_t *src = D_RW(D_RW(bp)->data) + block_off; memcpy(buf, src, D_RO(bp)->bsize); } pmemobj_mutex_unlock(pop, &D_RW(bp)->locks[blockno % MAX_THREADS]); return 0; } /* * pmemblk_write -- write a block (atomically) in a block memory pool */ int pmemblk_write(PMEMblkpool *pbp, const void *buf, long long blockno) { PMEMobjpool *pop = (PMEMobjpool *)pbp; int retval = 0; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); if (blockno >= (long long)D_RO(bp)->nblocks) return 1; TX_BEGIN_PARAM(pop, TX_PARAM_MUTEX, &D_RW(bp)->locks[blockno % MAX_THREADS], TX_PARAM_NONE) { size_t block_off = blockno * D_RO(bp)->bsize; uint8_t *dst = D_RW(D_RW(bp)->data) + block_off; /* add the modified block to the undo log */ pmemobj_tx_add_range_direct(dst, D_RO(bp)->bsize); memcpy(dst, buf, D_RO(bp)->bsize); /* clear the error flag and set the zero flag */ uint8_t *flags = D_RW(D_RW(bp)->flags) + blockno; /* add the modified flags to the undo log */ pmemobj_tx_add_range_direct(flags, sizeof(*flags)); *flags &= ~ERROR_MASK; /* use reverse logic for zero mask */ *flags |= ZERO_MASK; } TX_ONABORT { retval = 1; } TX_END return retval; } /* * pmemblk_set_zero -- zero a block in a block memory pool */ int pmemblk_set_zero(PMEMblkpool *pbp, long long blockno) { PMEMobjpool *pop = (PMEMobjpool *)pbp; int retval = 0; TOID(struct base) bp; bp = POBJ_ROOT(pop, struct base); if (blockno >= (long long)D_RO(bp)->nblocks) return 1; TX_BEGIN_PARAM(pop, TX_PARAM_MUTEX, &D_RW(bp)->locks[blockno % MAX_THREADS], TX_PARAM_NONE) { uint8_t *flags = D_RW(D_RW(bp)->flags) + blockno; /* add the modified flags to the undo log */ pmemobj_tx_add_range_direct(flags, sizeof(*flags)); /* use reverse logic for zero mask */ *flags &= ~ZERO_MASK; } TX_ONABORT { retval = 1; } TX_END return retval; } int main(int argc, char *argv[]) { if (argc < 4) { fprintf(stderr, "usage: %s [co] file blk_size"\ " [cmd[:blk_num[:data]]...]\n", argv[0]); return 1; } unsigned long bsize = strtoul(argv[3], NULL, 10); assert(bsize <= BSIZE_MAX); if (bsize == 0) { perror("blk_size cannot be 0"); return 1; } PMEMblkpool *pbp; if (strncmp(argv[1], "c", 1) == 0) { pbp = pmemblk_create(argv[2], bsize, POOL_SIZE, CREATE_MODE_RW); } else if (strncmp(argv[1], "o", 1) == 0) { pbp = pmemblk_open(argv[2], bsize); } else { fprintf(stderr, "usage: %s [co] file blk_size" " [cmd[:blk_num[:data]]...]\n", argv[0]); return 1; } if (pbp == NULL) { perror("pmemblk_create/pmemblk_open"); return 1; } /* process the command line arguments */ for (int i = 4; i < argc; i++) { switch (*argv[i]) { case 'w': { printf("write: %s\n", argv[i] + 2); const char *block_str = strtok(argv[i] + 2, ":"); const char *data = strtok(NULL, ":"); assert(block_str != NULL); assert(data != NULL); unsigned long block = strtoul(block_str, NULL, 10); if (pmemblk_write(pbp, data, block)) perror("pmemblk_write failed"); break; } case 'r': { printf("read: %s\n", argv[i] + 2); char *buf = (char *)malloc(bsize); assert(buf != NULL); const char *block_str = strtok(argv[i] + 2, ":"); assert(block_str != NULL); if (pmemblk_read(pbp, buf, strtoul(block_str, NULL, 10))) { perror("pmemblk_read failed"); free(buf); break; } buf[bsize - 1] = '\0'; printf("%s\n", buf); free(buf); break; } case 'z': { printf("zero: %s\n", argv[i] + 2); const char *block_str = strtok(argv[i] + 2, ":"); assert(block_str != NULL); if (pmemblk_set_zero(pbp, strtoul(block_str, NULL, 10))) perror("pmemblk_set_zero failed"); break; } case 'e': { printf("error: %s\n", argv[i] + 2); const char *block_str = strtok(argv[i] + 2, ":"); assert(block_str != NULL); if (pmemblk_set_error(pbp, strtoul(block_str, NULL, 10))) perror("pmemblk_set_error failed"); break; } case 'n': { printf("nblocks: "); printf("%zu\n", pmemblk_nblock(pbp)); break; } default: { fprintf(stderr, "unrecognized command %s\n", argv[i]); break; } }; } /* all done */ pmemblk_close(pbp); return 0; } pmdk-1.8/src/examples/libpmemobj/pmemblk/.gitignore0000664000000000000000000000001413615011243021150 0ustar rootrootobj_pmemblk pmdk-1.8/src/examples/libpmemobj/pmemblk/obj_pmemblk.vcxproj0000664000000000000000000000554113615011243023070 0ustar rootroot Debug x64 Release x64 {8C42CA7C-1543-4F1B-A55F-28CD419C7D35} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/pmemblk/Makefile0000664000000000000000000000334313615011243020630 0ustar rootroot# # Copyright 2015-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/pmemblk/Makefile -- build obj implementations of pmemblk # PROGS = obj_pmemblk LIBS = -lpmemobj -lpmem -pthread include ../../Makefile.inc obj_pmemblk: obj_pmemblk.o pmdk-1.8/src/examples/libpmemobj/pmemblk/obj_pmemblk.vcxproj.filters0000664000000000000000000000065013615011243024533 0ustar rootroot {f66bb1c3-362d-4a6e-8e49-848a97b48588} Source Files pmdk-1.8/src/examples/libpmemobj/libart/0000775000000000000000000000000013615011243017013 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/libart/art.c0000664000000000000000000011262613615011243017755 0ustar rootroot/* * Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH * Copyright 2012, Armon Dadgar. All rights reserved. * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * =========================================================================== * * Filename: art.c * * Description: implement ART tree using libpmemobj based on libart * * Author: Andreas Bluemle, Dieter Kasper * Andreas.Bluemle.external@ts.fujitsu.com * dieter.kasper@ts.fujitsu.com * * Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH * ============================================================================ */ /* * based on https://github.com/armon/libart/src/art.c */ #include #include #include #include #include #include #include #include #include #include #include #include #include "libpmemobj.h" #include "art.h" TOID(var_string) null_var_string; TOID(art_leaf) null_art_leaf; TOID(art_node_u) null_art_node_u; int art_tree_init(PMEMobjpool *pop, int *newpool); TOID(art_node_u) make_leaf(PMEMobjpool *pop, const unsigned char *key, int key_len, void *value, int val_len); int fill_leaf(PMEMobjpool *pop, TOID(art_leaf) al, const unsigned char *key, int key_len, void *value, int val_len); TOID(art_node_u) alloc_node(PMEMobjpool *pop, art_node_type node_type); TOID(var_string) art_insert(PMEMobjpool *pop, const unsigned char *key, int key_len, void *value, int val_len); TOID(var_string) art_delete(PMEMobjpool *pop, const unsigned char *key, int key_len); static TOID(var_string) recursive_insert(PMEMobjpool *pop, TOID(art_node_u) n, TOID(art_node_u) *ref, const unsigned char *key, int key_len, void *value, int val_len, int depth, int *old_val); static TOID(art_leaf) recursive_delete(PMEMobjpool *pop, TOID(art_node_u) n, TOID(art_node_u) *ref, const unsigned char *key, int key_len, int depth); static int leaf_matches(TOID(art_leaf) n, const unsigned char *key, int key_len, int depth); static int longest_common_prefix(TOID(art_leaf) l1, TOID(art_leaf) l2, int depth); static int prefix_mismatch(TOID(art_node_u) n, unsigned char *key, int key_len, int depth); #ifdef LIBART_ITER_PREFIX static int leaf_prefix_matches(TOID(art_leaf) n, const unsigned char *prefix, int prefix_len); #endif static TOID(art_leaf) minimum(TOID(art_node_u) n_u); static TOID(art_leaf) maximum(TOID(art_node_u) n_u); static void copy_header(art_node *dest, art_node *src); static void add_child(PMEMobjpool *pop, TOID(art_node_u) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child); static void add_child4(PMEMobjpool *pop, TOID(art_node4) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child); static void add_child16(PMEMobjpool *pop, TOID(art_node16) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child); static void add_child48(PMEMobjpool *pop, TOID(art_node48) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child); static void add_child256(PMEMobjpool *pop, TOID(art_node256) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child); static void remove_child(PMEMobjpool *pop, TOID(art_node_u) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) *l); static void remove_child4(PMEMobjpool *pop, TOID(art_node4) n, TOID(art_node_u) *ref, TOID(art_node_u) *l); static void remove_child16(PMEMobjpool *pop, TOID(art_node16) n, TOID(art_node_u) *ref, TOID(art_node_u) *l); static void remove_child48(PMEMobjpool *pop, TOID(art_node48) n, TOID(art_node_u) *ref, unsigned char c); static void remove_child256(PMEMobjpool *pop, TOID(art_node256) n, TOID(art_node_u) *ref, unsigned char c); static TOID(art_node_u)* find_child(TOID(art_node_u) n, unsigned char c); static int check_prefix(const art_node *n, const unsigned char *key, int key_len, int depth); static int leaf_matches(TOID(art_leaf) n, const unsigned char *key, int key_len, int depth); TOID(art_leaf) art_minimum(TOID(struct art_tree_root) t); TOID(art_leaf) art_maximum(TOID(struct art_tree_root) t); #if 0 static void destroy_node(TOID(art_node_u) n_u); #endif int art_iter(PMEMobjpool *pop, art_callback cb, void *data); static void PMEMOIDcopy(PMEMoid *dest, const PMEMoid *src, const int n); static void PMEMOIDmove(PMEMoid *dest, PMEMoid *src, const int n); static void PMEMOIDcopy(PMEMoid *dest, const PMEMoid *src, const int n) { int i; for (i = 0; i < n; i++) { dest[i] = src[i]; } } static void PMEMOIDmove(PMEMoid *dest, PMEMoid *src, const int n) { int i; if (dest > src) { for (i = n - 1; i >= 0; i--) { dest[i] = src[i]; } } else { for (i = 0; i < n; i++) { dest[i] = src[i]; } } } TOID(art_node_u) alloc_node(PMEMobjpool *pop, art_node_type node_type) { TOID(art_node_u) node; TOID(art_node4) an4; TOID(art_node16) an16; TOID(art_node48) an48; TOID(art_node256) an256; TOID(art_leaf) al; node = TX_ZNEW(art_node_u); D_RW(node)->art_node_type = (uint8_t)node_type; switch (node_type) { case NODE4: an4 = TX_ZNEW(art_node4); D_RW(node)->u.an4 = an4; break; case NODE16: an16 = TX_ZNEW(art_node16); D_RW(node)->u.an16 = an16; break; case NODE48: an48 = TX_ZNEW(art_node48); D_RW(node)->u.an48 = an48; break; case NODE256: an256 = TX_ZNEW(art_node256); D_RW(node)->u.an256 = an256; break; case art_leaf_t: al = TX_ZNEW(art_leaf); D_RW(node)->u.al = al; break; default: /* invalid node type */ D_RW(node)->art_node_type = (uint8_t)art_node_types; break; } return node; } int art_tree_init(PMEMobjpool *pop, int *newpool) { int errors = 0; TOID(struct art_tree_root) root; if (pop == NULL) { errors++; } null_var_string.oid = OID_NULL; null_art_leaf.oid = OID_NULL; null_art_node_u.oid = OID_NULL; if (!errors) { TX_BEGIN(pop) { root = POBJ_ROOT(pop, struct art_tree_root); if (*newpool) { TX_ADD(root); D_RW(root)->root.oid = OID_NULL; D_RW(root)->size = 0; *newpool = 0; } } TX_END } return errors; } #if 0 // Recursively destroys the tree static void destroy_node(TOID(art_node_u) n_u) { // Break if null if (TOID_IS_NULL(n_u)) return; // Special case leafs if (IS_LEAF(D_RO(n_u))) { TX_FREE(n_u); return; } // Handle each node type int i; TOID(art_node4) an4; TOID(art_node16) an16; TOID(art_node48) an48; TOID(art_node256) an256; switch (D_RO(n_u)->art_node_type) { case NODE4: an4 = D_RO(n_u)->u.an4; for (i = 0; i < D_RO(an4)->n.num_children; i++) { destroy_node(D_RW(an4)->children[i]); } break; case NODE16: an16 = D_RO(n_u)->u.an16; for (i = 0; i < D_RO(an16)->n.num_children; i++) { destroy_node(D_RW(an16)->children[i]); } break; case NODE48: an48 = D_RO(n_u)->u.an48; for (i = 0; i < D_RO(an48)->n.num_children; i++) { destroy_node(D_RW(an48)->children[i]); } break; case NODE256: an256 = D_RO(n_u)->u.an256; for (i = 0; i < D_RO(an256)->n.num_children; i++) { if (!(TOID_IS_NULL(D_RO(an256)->children[i]))) { destroy_node(D_RW(an256)->children[i]); } } break; default: abort(); } // Free ourself on the way up TX_FREE(n_u); } /* * Destroys an ART tree * @return 0 on success. */ static int art_tree_destroy(TOID(struct art_tree_root) t) { destroy_node(D_RO(t)->root); return 0; } #endif static TOID(art_node_u)* find_child(TOID(art_node_u) n, unsigned char c) { int i; int mask; int bitfield; TOID(art_node4) an4; TOID(art_node16) an16; TOID(art_node48) an48; TOID(art_node256) an256; switch (D_RO(n)->art_node_type) { case NODE4: an4 = D_RO(n)->u.an4; for (i = 0; i < D_RO(an4)->n.num_children; i++) { if (D_RO(an4)->keys[i] == c) { return &(D_RW(an4)->children[i]); } } break; case NODE16: { __m128i cmp; an16 = D_RO(n)->u.an16; // Compare the key to all 16 stored keys cmp = _mm_cmpeq_epi8(_mm_set1_epi8(c), _mm_loadu_si128((__m128i *)D_RO(an16)->keys)); // Use a mask to ignore children that don't exist mask = (1 << D_RO(an16)->n.num_children) - 1; bitfield = _mm_movemask_epi8(cmp) & mask; /* * If we have a match (any bit set) then we can * return the pointer match using ctz to get the index. */ if (bitfield) { return &(D_RW(an16)->children[__builtin_ctz(bitfield)]); } break; } case NODE48: an48 = D_RO(n)->u.an48; i = D_RO(an48)->keys[c]; if (i) { return &(D_RW(an48)->children[i - 1]); } break; case NODE256: an256 = D_RO(n)->u.an256; if (!TOID_IS_NULL(D_RO(an256)->children[c])) { return &(D_RW(an256)->children[c]); } break; default: abort(); } return &null_art_node_u; } static inline int min(int a, int b) { return (a < b) ? a : b; } /* * Returns the number of prefix characters shared between * the key and node. */ static int check_prefix(const art_node *n, const unsigned char *key, int key_len, int depth) { int max_cmp = min(min(n->partial_len, MAX_PREFIX_LEN), key_len - depth); int idx; for (idx = 0; idx < max_cmp; idx++) { if (n->partial[idx] != key[depth + idx]) return idx; } return idx; } /* * Checks if a leaf matches * @return 0 on success. */ static int leaf_matches(TOID(art_leaf) n, const unsigned char *key, int key_len, int depth) { (void) depth; // Fail if the key lengths are different if (D_RO(D_RO(n)->key)->len != (uint32_t)key_len) return 1; // Compare the keys starting at the depth return memcmp(D_RO(D_RO(n)->key)->s, key, key_len); } /* * Searches for a value in the ART tree * @arg t The tree * @arg key The key * @arg key_len The length of the key * @return NULL if the item was not found, otherwise * the value pointer is returned. */ TOID(var_string) art_search(PMEMobjpool *pop, const unsigned char *key, int key_len) { TOID(struct art_tree_root)t = POBJ_ROOT(pop, struct art_tree_root); TOID(art_node_u) *child; TOID(art_node_u) n = D_RO(t)->root; const art_node *n_an; int prefix_len; int depth = 0; while (!TOID_IS_NULL(n)) { // Might be a leaf if (IS_LEAF(D_RO(n))) { // n = LEAF_RAW(n); // Check if the expanded path matches if (!leaf_matches(D_RO(n)->u.al, key, key_len, depth)) { return (D_RO(D_RO(n)->u.al))->value; } return null_var_string; } switch (D_RO(n)->art_node_type) { case NODE4: n_an = &(D_RO(D_RO(n)->u.an4)->n); break; case NODE16: n_an = &(D_RO(D_RO(n)->u.an16)->n); break; case NODE48: n_an = &(D_RO(D_RO(n)->u.an48)->n); break; case NODE256: n_an = &(D_RO(D_RO(n)->u.an256)->n); break; default: return null_var_string; } // Bail if the prefix does not match if (n_an->partial_len) { prefix_len = check_prefix(n_an, key, key_len, depth); if (prefix_len != min(MAX_PREFIX_LEN, n_an->partial_len)) return null_var_string; depth = depth + n_an->partial_len; } // Recursively search child = find_child(n, key[depth]); if (TOID_IS_NULL(*child)) { n.oid = OID_NULL; } else { n = *child; } depth++; } return null_var_string; } // Find the minimum leaf under a node static TOID(art_leaf) minimum(TOID(art_node_u) n_u) { TOID(art_node4) an4; TOID(art_node16) an16; TOID(art_node48) an48; TOID(art_node256) an256; // Handle base cases if (TOID_IS_NULL(n_u)) return null_art_leaf; if (IS_LEAF(D_RO(n_u))) return D_RO(n_u)->u.al; int idx; switch (D_RO(n_u)->art_node_type) { case NODE4: an4 = D_RO(n_u)->u.an4; return minimum(D_RO(an4)->children[0]); case NODE16: an16 = D_RO(n_u)->u.an16; return minimum(D_RO(an16)->children[0]); case NODE48: an48 = D_RO(n_u)->u.an48; idx = 0; while (!(D_RO(an48)->keys[idx])) idx++; idx = D_RO(an48)->keys[idx] - 1; assert(idx < 48); return minimum(D_RO(an48)->children[idx]); case NODE256: an256 = D_RO(n_u)->u.an256; idx = 0; while (!(TOID_IS_NULL(D_RO(an256)->children[idx]))) idx++; return minimum(D_RO(an256)->children[idx]); default: abort(); } } // Find the maximum leaf under a node static TOID(art_leaf) maximum(TOID(art_node_u) n_u) { TOID(art_node4) an4; TOID(art_node16) an16; TOID(art_node48) an48; TOID(art_node256) an256; const art_node *n_an; // Handle base cases if (TOID_IS_NULL(n_u)) return null_art_leaf; if (IS_LEAF(D_RO(n_u))) return D_RO(n_u)->u.al; int idx; switch (D_RO(n_u)->art_node_type) { case NODE4: an4 = D_RO(n_u)->u.an4; n_an = &(D_RO(an4)->n); return maximum(D_RO(an4)->children[n_an->num_children - 1]); case NODE16: an16 = D_RO(n_u)->u.an16; n_an = &(D_RO(an16)->n); return maximum(D_RO(an16)->children[n_an->num_children - 1]); case NODE48: an48 = D_RO(n_u)->u.an48; idx = 255; while (!(D_RO(an48)->keys[idx])) idx--; idx = D_RO(an48)->keys[idx] - 1; assert((idx >= 0) && (idx < 48)); return maximum(D_RO(an48)->children[idx]); case NODE256: an256 = D_RO(n_u)->u.an256; idx = 255; while (!(TOID_IS_NULL(D_RO(an256)->children[idx]))) idx--; return maximum(D_RO(an256)->children[idx]); default: abort(); } } /* * Returns the minimum valued leaf */ TOID(art_leaf) art_minimum(TOID(struct art_tree_root) t) { return minimum(D_RO(t)->root); } /* * Returns the maximum valued leaf */ TOID(art_leaf) art_maximum(TOID(struct art_tree_root) t) { return maximum(D_RO(t)->root); } TOID(art_node_u) make_leaf(PMEMobjpool *pop, const unsigned char *key, int key_len, void *value, int val_len) { TOID(art_node_u)newleaf; newleaf = alloc_node(pop, art_leaf_t); fill_leaf(pop, D_RW(newleaf)->u.al, key, key_len, value, val_len); return newleaf; } static int longest_common_prefix(TOID(art_leaf) l1, TOID(art_leaf) l2, int depth) { TOID(var_string) l1_key = D_RO(l1)->key; TOID(var_string) l2_key = D_RO(l2)->key; int max_cmp; int idx; max_cmp = min(D_RO(l1_key)->len, D_RO(l2_key)->len) - depth; for (idx = 0; idx < max_cmp; idx++) { if (D_RO(l1_key)->s[depth + idx] != D_RO(l2_key)->s[depth + idx]) return idx; } return idx; } static void copy_header(art_node *dest, art_node *src) { dest->num_children = src->num_children; dest->partial_len = src->partial_len; memcpy(dest->partial, src->partial, min(MAX_PREFIX_LEN, src->partial_len)); } static void add_child256(PMEMobjpool *pop, TOID(art_node256) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child) { art_node *n_an; (void) ref; TX_ADD(n); n_an = &(D_RW(n)->n); n_an->num_children++; D_RW(n)->children[c] = child; } static void add_child48(PMEMobjpool *pop, TOID(art_node48) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child) { art_node *n_an; n_an = &(D_RW(n)->n); if (n_an->num_children < 48) { int pos = 0; TX_ADD(n); while (!(TOID_IS_NULL(D_RO(n)->children[pos]))) pos++; D_RW(n)->children[pos] = child; D_RW(n)->keys[c] = pos + 1; n_an->num_children++; } else { TOID(art_node_u) newnode_u = alloc_node(pop, NODE256); TOID(art_node256) newnode = D_RO(newnode_u)->u.an256; pmemobj_tx_add_range_direct(ref, sizeof(TOID(art_node_u))); for (int i = 0; i < 256; i++) { if (D_RO(n)->keys[i]) { D_RW(newnode)->children[i] = D_RO(n)->children[D_RO(n)->keys[i] - 1]; } } copy_header(&(D_RW(newnode)->n), n_an); *ref = newnode_u; TX_FREE(n); add_child256(pop, newnode, ref, c, child); } } static void add_child16(PMEMobjpool *pop, TOID(art_node16) n, TOID(art_node_u)*ref, unsigned char c, TOID(art_node_u) child) { art_node *n_an; n_an = &(D_RW(n)->n); if (n_an->num_children < 16) { __m128i cmp; TX_ADD(n); // Compare the key to all 16 stored keys cmp = _mm_cmplt_epi8(_mm_set1_epi8(c), _mm_loadu_si128((__m128i *)(D_RO(n)->keys))); // Use a mask to ignore children that don't exist unsigned mask = (1 << n_an->num_children) - 1; unsigned bitfield = _mm_movemask_epi8(cmp) & mask; // Check if less than any unsigned idx; if (bitfield) { idx = __builtin_ctz(bitfield); memmove(&(D_RW(n)->keys[idx + 1]), &(D_RO(n)->keys[idx]), n_an->num_children - idx); PMEMOIDmove(&(D_RW(n)->children[idx + 1].oid), &(D_RW(n)->children[idx].oid), n_an->num_children - idx); } else { idx = n_an->num_children; } // Set the child D_RW(n)->keys[idx] = c; D_RW(n)->children[idx] = child; n_an->num_children++; } else { TOID(art_node_u) newnode_u = alloc_node(pop, NODE48); TOID(art_node48) newnode = D_RO(newnode_u)->u.an48; // Copy the child pointers and populate the key map PMEMOIDcopy(&(D_RW(newnode)->children[0].oid), &(D_RO(n)->children[0].oid), n_an->num_children); for (int i = 0; i < n_an->num_children; i++) { D_RW(newnode)->keys[D_RO(n)->keys[i]] = i + 1; } copy_header(&(D_RW(newnode))->n, n_an); *ref = newnode_u; TX_FREE(n); add_child48(pop, newnode, ref, c, child); } } static void add_child4(PMEMobjpool *pop, TOID(art_node4) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child) { art_node *n_an; n_an = &(D_RW(n)->n); if (n_an->num_children < 4) { int idx; TX_ADD(n); for (idx = 0; idx < n_an->num_children; idx++) { if (c < D_RO(n)->keys[idx]) break; } // Shift to make room memmove(D_RW(n)->keys + idx + 1, D_RO(n)->keys + idx, n_an->num_children - idx); assert((idx + 1) < 4); PMEMOIDmove(&(D_RW(n)->children[idx + 1].oid), &(D_RW(n)->children[idx].oid), n_an->num_children - idx); // Insert element D_RW(n)->keys[idx] = c; D_RW(n)->children[idx] = child; n_an->num_children++; } else { TOID(art_node_u) newnode_u = alloc_node(pop, NODE16); TOID(art_node16) newnode = D_RO(newnode_u)->u.an16; pmemobj_tx_add_range_direct(ref, sizeof(TOID(art_node_u))); // Copy the child pointers and the key map PMEMOIDcopy(&(D_RW(newnode)->children[0].oid), &(D_RO(n)->children[0].oid), n_an->num_children); memcpy(D_RW(newnode)->keys, D_RO(n)->keys, n_an->num_children); copy_header(&(D_RW(newnode)->n), n_an); *ref = newnode_u; TX_FREE(n); add_child16(pop, newnode, ref, c, child); } } static void add_child(PMEMobjpool *pop, TOID(art_node_u) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) child) { switch (D_RO(n)->art_node_type) { case NODE4: add_child4(pop, D_RO(n)->u.an4, ref, c, child); break; case NODE16: add_child16(pop, D_RO(n)->u.an16, ref, c, child); break; case NODE48: add_child48(pop, D_RO(n)->u.an48, ref, c, child); break; case NODE256: add_child256(pop, D_RO(n)->u.an256, ref, c, child); break; default: abort(); } } static int prefix_mismatch(TOID(art_node_u) n, unsigned char *key, int key_len, int depth) { const art_node *n_an; int max_cmp; int idx; switch (D_RO(n)->art_node_type) { case NODE4: n_an = &(D_RO(D_RO(n)->u.an4)->n); break; case NODE16: n_an = &(D_RO(D_RO(n)->u.an16)->n); break; case NODE48: n_an = &(D_RO(D_RO(n)->u.an48)->n); break; case NODE256: n_an = &(D_RO(D_RO(n)->u.an256)->n); break; default: return 0; } max_cmp = min(min(MAX_PREFIX_LEN, n_an->partial_len), key_len - depth); for (idx = 0; idx < max_cmp; idx++) { if (n_an->partial[idx] != key[depth + idx]) return idx; } // If the prefix is short we can avoid finding a leaf if (n_an->partial_len > MAX_PREFIX_LEN) { // Prefix is longer than what we've checked, find a leaf TOID(art_leaf) l = minimum(n); max_cmp = min(D_RO(D_RO(l)->key)->len, key_len) - depth; for (; idx < max_cmp; idx++) { if (D_RO(D_RO(l)->key)->s[idx + depth] != key[depth + idx]) return idx; } } return idx; } static TOID(var_string) recursive_insert(PMEMobjpool *pop, TOID(art_node_u) n, TOID(art_node_u) *ref, const unsigned char *key, int key_len, void *value, int val_len, int depth, int *old) { art_node *n_an; TOID(var_string) retval; // If we are at a NULL node, inject a leaf if (TOID_IS_NULL(n)) { *ref = make_leaf(pop, key, key_len, value, val_len); TX_ADD(*ref); SET_LEAF(D_RW(*ref)); retval = null_var_string; return retval; } // If we are at a leaf, we need to replace it with a node if (IS_LEAF(D_RO(n))) { TOID(art_leaf)l = D_RO(n)->u.al; // Check if we are updating an existing value if (!leaf_matches(l, key, key_len, depth)) { *old = 1; retval = D_RO(l)->value; TX_ADD(D_RW(l)->value); COPY_BLOB(D_RW(l)->value, value, val_len); return retval; } // New value, we must split the leaf into a node4 pmemobj_tx_add_range_direct(ref, sizeof(TOID(art_node_u))); TOID(art_node_u) newnode_u = alloc_node(pop, NODE4); TOID(art_node4) newnode = D_RO(newnode_u)->u.an4; art_node *newnode_n = &(D_RW(newnode)->n); // Create a new leaf TOID(art_node_u) l2_u = make_leaf(pop, key, key_len, value, val_len); TOID(art_leaf) l2 = D_RO(l2_u)->u.al; // Determine longest prefix int longest_prefix = longest_common_prefix(l, l2, depth); newnode_n->partial_len = longest_prefix; memcpy(newnode_n->partial, key + depth, min(MAX_PREFIX_LEN, longest_prefix)); // Add the leafs to the newnode node4 *ref = newnode_u; add_child4(pop, newnode, ref, D_RO(D_RO(l)->key)->s[depth + longest_prefix], n); add_child4(pop, newnode, ref, D_RO(D_RO(l2)->key)->s[depth + longest_prefix], l2_u); return null_var_string; } // Check if given node has a prefix switch (D_RO(n)->art_node_type) { case NODE4: n_an = &(D_RW(D_RW(n)->u.an4)->n); break; case NODE16: n_an = &(D_RW(D_RW(n)->u.an16)->n); break; case NODE48: n_an = &(D_RW(D_RW(n)->u.an48)->n); break; case NODE256: n_an = &(D_RW(D_RW(n)->u.an256)->n); break; default: abort(); } if (n_an->partial_len) { // Determine if the prefixes differ, since we need to split int prefix_diff = prefix_mismatch(n, (unsigned char *)key, key_len, depth); if ((uint32_t)prefix_diff >= n_an->partial_len) { depth += n_an->partial_len; goto RECURSE_SEARCH; } // Create a new node pmemobj_tx_add_range_direct(ref, sizeof(TOID(art_node_u))); pmemobj_tx_add_range_direct(n_an, sizeof(art_node)); TOID(art_node_u) newnode_u = alloc_node(pop, NODE4); TOID(art_node4) newnode = D_RO(newnode_u)->u.an4; art_node *newnode_n = &(D_RW(newnode)->n); *ref = newnode_u; newnode_n->partial_len = prefix_diff; memcpy(newnode_n->partial, n_an->partial, min(MAX_PREFIX_LEN, prefix_diff)); // Adjust the prefix of the old node if (n_an->partial_len <= MAX_PREFIX_LEN) { add_child4(pop, newnode, ref, n_an->partial[prefix_diff], n); n_an->partial_len -= (prefix_diff + 1); memmove(n_an->partial, n_an->partial + prefix_diff + 1, min(MAX_PREFIX_LEN, n_an->partial_len)); } else { unsigned char *dst; const unsigned char *src; size_t len; n_an->partial_len -= (prefix_diff + 1); TOID(art_leaf) l = minimum(n); add_child4(pop, newnode, ref, D_RO(D_RO(l)->key)->s[depth + prefix_diff], n); dst = n_an->partial; src = &(D_RO(D_RO(l)->key)->s[depth + prefix_diff + 1 ]); len = min(MAX_PREFIX_LEN, n_an->partial_len); memcpy(dst, src, len); } // Insert the new leaf TOID(art_node_u) l = make_leaf(pop, key, key_len, value, val_len); SET_LEAF(D_RW(l)); add_child4(pop, newnode, ref, key[depth + prefix_diff], l); return null_var_string; } RECURSE_SEARCH:; // Find a child to recurse to TOID(art_node_u) *child = find_child(n, key[depth]); if (!TOID_IS_NULL(*child)) { return recursive_insert(pop, *child, child, key, key_len, value, val_len, depth + 1, old); } // No child, node goes within us TOID(art_node_u) l = make_leaf(pop, key, key_len, value, val_len); SET_LEAF(D_RW(l)); add_child(pop, n, ref, key[depth], l); retval = null_var_string; return retval; } /* * Returns the size of the ART tree */ uint64_t art_size(PMEMobjpool *pop) { TOID(struct art_tree_root) root; root = POBJ_ROOT(pop, struct art_tree_root); return D_RO(root)->size; } /* * Inserts a new value into the ART tree * @arg t The tree * @arg key The key * @arg key_len The length of the key * @arg value Opaque value. * @return NULL if the item was newly inserted, otherwise * the old value pointer is returned. */ TOID(var_string) art_insert(PMEMobjpool *pop, const unsigned char *key, int key_len, void *value, int val_len) { int old_val = 0; TOID(var_string) old; TOID(struct art_tree_root) root; TX_BEGIN(pop) { root = POBJ_ROOT(pop, struct art_tree_root); TX_ADD(root); old = recursive_insert(pop, D_RO(root)->root, &(D_RW(root)->root), (const unsigned char *)key, key_len, value, val_len, 0, &old_val); if (!old_val) D_RW(root)->size++; } TX_ONABORT { abort(); } TX_END return old; } static void remove_child256(PMEMobjpool *pop, TOID(art_node256) n, TOID(art_node_u) *ref, unsigned char c) { art_node *n_an = &(D_RW(n)->n); TX_ADD(n); D_RW(n)->children[c].oid = OID_NULL; n_an->num_children--; // Resize to a node48 on underflow, not immediately to prevent // trashing if we sit on the 48/49 boundary if (n_an->num_children == 37) { TOID(art_node_u) newnode_u = alloc_node(pop, NODE48); TOID(art_node48) newnode_an48 = D_RO(newnode_u)->u.an48; pmemobj_tx_add_range_direct(ref, sizeof(TOID(art_node_u))); *ref = newnode_u; copy_header(&(D_RW(newnode_an48)->n), n_an); int pos = 0; for (int i = 0; i < 256; i++) { if (!TOID_IS_NULL(D_RO(n)->children[i])) { assert(pos < 48); D_RW(newnode_an48)->children[pos] = D_RO(n)->children[i]; D_RW(newnode_an48)->keys[i] = pos + 1; pos++; } } TX_FREE(n); } } static void remove_child48(PMEMobjpool *pop, TOID(art_node48) n, TOID(art_node_u) *ref, unsigned char c) { int pos = D_RO(n)->keys[c]; art_node *n_an = &(D_RW(n)->n); TX_ADD(n); D_RW(n)->keys[c] = 0; D_RW(n)->children[pos - 1].oid = OID_NULL; n_an->num_children--; if (n_an->num_children == 12) { TOID(art_node_u) newnode_u = alloc_node(pop, NODE16); TOID(art_node16) newnode_an16 = D_RO(newnode_u)->u.an16; pmemobj_tx_add_range_direct(ref, sizeof(TOID(art_node_u))); *ref = newnode_u; copy_header(&(D_RW(newnode_an16)->n), n_an); int child = 0; for (int i = 0; i < 256; i++) { pos = D_RO(n)->keys[i]; if (pos) { assert(child < 16); D_RW(newnode_an16)->keys[child] = i; D_RW(newnode_an16)->children[child] = D_RO(n)->children[pos - 1]; child++; } } TX_FREE(n); } } static void remove_child16(PMEMobjpool *pop, TOID(art_node16) n, TOID(art_node_u) *ref, TOID(art_node_u) *l) { int pos = l - &(D_RO(n)->children[0]); uint8_t num_children = ((D_RW(n)->n).num_children); TX_ADD(n); memmove(D_RW(n)->keys + pos, D_RO(n)->keys + pos + 1, num_children - 1 - pos); memmove(D_RW(n)->children + pos, D_RO(n)->children + pos + 1, (num_children - 1 - pos) * sizeof(void *)); ((D_RW(n)->n).num_children)--; if (--num_children == 3) { TOID(art_node_u) newnode_u = alloc_node(pop, NODE4); TOID(art_node4) newnode_an4 = D_RO(newnode_u)->u.an4; pmemobj_tx_add_range_direct(ref, sizeof(TOID(art_node_u))); *ref = newnode_u; copy_header(&(D_RW(newnode_an4)->n), &(D_RW(n)->n)); memcpy(D_RW(newnode_an4)->keys, D_RO(n)->keys, 4); memcpy(D_RW(newnode_an4)->children, D_RO(n)->children, 4 * sizeof(TOID(art_node_u))); TX_FREE(n); } } static void remove_child4(PMEMobjpool *pop, TOID(art_node4) n, TOID(art_node_u) *ref, TOID(art_node_u) *l) { int pos = l - &(D_RO(n)->children[0]); uint8_t *num_children = &((D_RW(n)->n).num_children); TX_ADD(n); memmove(D_RW(n)->keys + pos, D_RO(n)->keys + pos + 1, *num_children - 1 - pos); memmove(D_RW(n)->children + pos, D_RO(n)->children + pos + 1, (*num_children - 1 - pos) * sizeof(void *)); (*num_children)--; // Remove nodes with only a single child if (*num_children == 1) { TOID(art_node_u) child_u = D_RO(n)->children[0]; art_node *child = &(D_RW(D_RW(child_u)->u.an4)->n); pmemobj_tx_add_range_direct(ref, sizeof(TOID(art_node_u))); if (!IS_LEAF(D_RO(child_u))) { // Concatenate the prefixes int prefix = (D_RW(n)->n).partial_len; if (prefix < MAX_PREFIX_LEN) { (D_RW(n)->n).partial[prefix] = D_RO(n)->keys[0]; prefix++; } if (prefix < MAX_PREFIX_LEN) { int sub_prefix = min(child->partial_len, MAX_PREFIX_LEN - prefix); memcpy((D_RW(n)->n).partial + prefix, child->partial, sub_prefix); prefix += sub_prefix; } // Store the prefix in the child memcpy(child->partial, (D_RO(n)->n).partial, min(prefix, MAX_PREFIX_LEN)); child->partial_len += (D_RO(n)->n).partial_len + 1; } *ref = child_u; TX_FREE(n); } } static void remove_child(PMEMobjpool *pop, TOID(art_node_u) n, TOID(art_node_u) *ref, unsigned char c, TOID(art_node_u) *l) { switch (D_RO(n)->art_node_type) { case NODE4: return remove_child4(pop, D_RO(n)->u.an4, ref, l); case NODE16: return remove_child16(pop, D_RO(n)->u.an16, ref, l); case NODE48: return remove_child48(pop, D_RO(n)->u.an48, ref, c); case NODE256: return remove_child256(pop, D_RO(n)->u.an256, ref, c); default: abort(); } } static TOID(art_leaf) recursive_delete(PMEMobjpool *pop, TOID(art_node_u) n, TOID(art_node_u) *ref, const unsigned char *key, int key_len, int depth) { const art_node *n_an; // Search terminated if (TOID_IS_NULL(n)) return null_art_leaf; // Handle hitting a leaf node if (IS_LEAF(D_RO(n))) { TOID(art_leaf) l = D_RO(n)->u.al; if (!leaf_matches(l, key, key_len, depth)) { *ref = null_art_node_u; return l; } return null_art_leaf; } // get art_node component switch (D_RO(n)->art_node_type) { case NODE4: n_an = &(D_RO(D_RO(n)->u.an4)->n); break; case NODE16: n_an = &(D_RO(D_RO(n)->u.an16)->n); break; case NODE48: n_an = &(D_RO(D_RO(n)->u.an48)->n); break; case NODE256: n_an = &(D_RO(D_RO(n)->u.an256)->n); break; default: abort(); } // Bail if the prefix does not match if (n_an->partial_len) { int prefix_len = check_prefix(n_an, key, key_len, depth); if (prefix_len != min(MAX_PREFIX_LEN, n_an->partial_len)) { return null_art_leaf; } depth = depth + n_an->partial_len; } // Find child node TOID(art_node_u) *child = find_child(n, key[depth]); if (TOID_IS_NULL(*child)) return null_art_leaf; // If the child is leaf, delete from this node if (IS_LEAF(D_RO(*child))) { TOID(art_leaf)l = D_RO(*child)->u.al; if (!leaf_matches(l, key, key_len, depth)) { remove_child(pop, n, ref, key[depth], child); return l; } return null_art_leaf; } else { // Recurse return recursive_delete(pop, *child, child, (const unsigned char *)key, key_len, depth + 1); } } /* * Deletes a value from the ART tree * @arg t The tree * @arg key The key * @arg key_len The length of the key * @return NULL if the item was not found, otherwise * the value pointer is returned. */ TOID(var_string) art_delete(PMEMobjpool *pop, const unsigned char *key, int key_len) { TOID(struct art_tree_root)root = POBJ_ROOT(pop, struct art_tree_root); TOID(art_leaf) l; TOID(var_string) retval; retval = null_var_string; TX_BEGIN(pop) { TX_ADD(root); l = recursive_delete(pop, D_RO(root)->root, &D_RW(root)->root, key, key_len, 0); if (!TOID_IS_NULL(l)) { D_RW(root)->size--; TOID(var_string)old = D_RO(l)->value; TX_FREE(l); retval = old; } } TX_ONABORT { abort(); } TX_END return retval; } // Recursively iterates over the tree static int recursive_iter(TOID(art_node_u)n, art_callback cb, void *data) { const art_node *n_an; TOID(art_node4) an4; TOID(art_node16) an16; TOID(art_node48) an48; TOID(art_node256) an256; TOID(art_leaf) l; TOID(var_string) key; TOID(var_string) value; cb_data cbd; // Handle base cases if (TOID_IS_NULL(n)) { return 0; } cbd.node = n; cbd.child_idx = -1; if (IS_LEAF(D_RO(n))) { l = D_RO(n)->u.al; key = D_RO(l)->key; value = D_RO(l)->value; return cb(&cbd, D_RO(key)->s, D_RO(key)->len, D_RO(value)->s, D_RO(value)->len); } int idx, res; switch (D_RO(n)->art_node_type) { case NODE4: an4 = D_RO(n)->u.an4; n_an = &(D_RO(an4)->n); for (int i = 0; i < n_an->num_children; i++) { cbd.child_idx = i; cb(&cbd, NULL, 0, NULL, 0); res = recursive_iter(D_RO(an4)->children[i], cb, data); if (res) return res; } break; case NODE16: an16 = D_RO(n)->u.an16; n_an = &(D_RO(an16)->n); for (int i = 0; i < n_an->num_children; i++) { cbd.child_idx = i; cb(&cbd, NULL, 0, NULL, 0); res = recursive_iter(D_RO(an16)->children[i], cb, data); if (res) return res; } break; case NODE48: an48 = D_RO(n)->u.an48; for (int i = 0; i < 256; i++) { idx = D_RO(an48)->keys[i]; if (!idx) continue; cbd.child_idx = idx - 1; cb(&cbd, NULL, 0, NULL, 0); res = recursive_iter(D_RO(an48)->children[idx - 1], cb, data); if (res) return res; } break; case NODE256: an256 = D_RO(n)->u.an256; for (int i = 0; i < 256; i++) { if (TOID_IS_NULL(D_RO(an256)->children[i])) continue; cbd.child_idx = i; cb(&cbd, NULL, 0, NULL, 0); res = recursive_iter(D_RO(an256)->children[i], cb, data); if (res) return res; } break; default: abort(); } return 0; } /* * Iterates through the entries pairs in the map, * invoking a callback for each. The call back gets a * key, value for each and returns an integer stop value. * If the callback returns non-zero, then the iteration stops. * @arg t The tree to iterate over * @arg cb The callback function to invoke * @arg data Opaque handle passed to the callback * @return 0 on success, or the return of the callback. */ int art_iter(PMEMobjpool *pop, art_callback cb, void *data) { TOID(struct art_tree_root) t = POBJ_ROOT(pop, struct art_tree_root); return recursive_iter(D_RO(t)->root, cb, data); } #ifdef LIBART_ITER_PREFIX /* { */ /* * Checks if a leaf prefix matches * @return 0 on success. */ static int leaf_prefix_matches(TOID(art_leaf) n, const unsigned char *prefix, int prefix_len) { // Fail if the key length is too short if (D_RO(D_RO(n)->key)->len < (uint32_t)prefix_len) return 1; // Compare the keys return memcmp(D_RO(D_RO(n)->key)->s, prefix, prefix_len); } /* * Iterates through the entries pairs in the map, * invoking a callback for each that matches a given prefix. * The call back gets a key, value for each and returns an integer stop value. * If the callback returns non-zero, then the iteration stops. * @arg t The tree to iterate over * @arg prefix The prefix of keys to read * @arg prefix_len The length of the prefix * @arg cb The callback function to invoke * @arg data Opaque handle passed to the callback * @return 0 on success, or the return of the callback. */ int art_iter_prefix(art_tree *t, const unsigned char *key, int key_len, art_callback cb, void *data) { art_node **child; art_node *n = t->root; int prefix_len, depth = 0; while (n) { // Might be a leaf if (IS_LEAF(n)) { n = LEAF_RAW(n); // Check if the expanded path matches if (!leaf_prefix_matches((art_leaf *)n, key, key_len)) { art_leaf *l = (art_leaf *)n; return cb(data, (const unsigned char *)l->key, l->key_len, l->value); } return 0; } // If the depth matches the prefix, we need to handle this node if (depth == key_len) { art_leaf *l = minimum(n); if (!leaf_prefix_matches(l, key, key_len)) return recursive_iter(n, cb, data); return 0; } // Bail if the prefix does not match if (n->partial_len) { prefix_len = prefix_mismatch(n, key, key_len, depth); // If there is no match, search is terminated if (!prefix_len) return 0; // If we've matched the prefix, iterate on this node else if (depth + prefix_len == key_len) { return recursive_iter(n, cb, data); } // if there is a full match, go deeper depth = depth + n->partial_len; } // Recursively search child = find_child(n, key[depth]); n = (child) ? *child : NULL; depth++; } return 0; } #endif /* } LIBART_ITER_PREFIX */ int fill_leaf(PMEMobjpool *pop, TOID(art_leaf) al, const unsigned char *key, int key_len, void *value, int val_len) { int retval = 0; size_t l_key; size_t l_val; TOID(var_string) Tkey; TOID(var_string) Tval; l_key = (sizeof(var_string) + key_len); l_val = (sizeof(var_string) + val_len); Tkey = TX_ALLOC(var_string, l_key); Tval = TX_ALLOC(var_string, l_val); COPY_BLOB(Tkey, key, key_len); COPY_BLOB(Tval, value, val_len); D_RW(al)->key = Tkey; D_RW(al)->value = Tval; return retval; } pmdk-1.8/src/examples/libpmemobj/libart/art.h0000664000000000000000000001350213615011243017753 0ustar rootroot/* * Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH * Copyright 2012, Armon Dadgar. All rights reserved. * Copyright 2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * =========================================================================== * * Filename: art.h * * Description: header file for art tree on pmem implementation * * Author: Andreas Bluemle, Dieter Kasper * Andreas.Bluemle.external@ts.fujitsu.com * dieter.kasper@ts.fujitsu.com * * Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH * * =========================================================================== */ /* * based on https://github.com/armon/libart/src/art.h */ #ifndef _ART_H #define _ART_H #ifdef __cplusplus extern "C" { #endif #define MAX_PREFIX_LEN 10 typedef enum { NODE4 = 0, NODE16 = 1, NODE48 = 2, NODE256 = 3, art_leaf_t = 4, art_node_types = 5 /* number of different art_nodes */ } art_node_type; char *art_node_names[] = { "art_node4", "art_node16", "art_node48", "art_node256", "art_leaf" }; /* * forward declarations; these are required when typedef shall be * used instead of struct */ struct _art_node_u; typedef struct _art_node_u art_node_u; struct _art_node; typedef struct _art_node art_node; struct _art_node4; typedef struct _art_node4 art_node4; struct _art_node16; typedef struct _art_node16 art_node16; struct _art_node48; typedef struct _art_node48 art_node48; struct _art_node256; typedef struct _art_node256 art_node256; struct _art_leaf; typedef struct _art_leaf art_leaf; struct _var_string; typedef struct _var_string var_string; POBJ_LAYOUT_BEGIN(arttree_tx); POBJ_LAYOUT_ROOT(arttree_tx, struct art_tree_root); POBJ_LAYOUT_TOID(arttree_tx, art_node_u); POBJ_LAYOUT_TOID(arttree_tx, art_node4); POBJ_LAYOUT_TOID(arttree_tx, art_node16); POBJ_LAYOUT_TOID(arttree_tx, art_node48); POBJ_LAYOUT_TOID(arttree_tx, art_node256); POBJ_LAYOUT_TOID(arttree_tx, art_leaf); POBJ_LAYOUT_TOID(arttree_tx, var_string); POBJ_LAYOUT_END(arttree_tx); struct _var_string { size_t len; unsigned char s[]; }; /* * This struct is included as part of all the various node sizes */ struct _art_node { uint8_t num_children; uint32_t partial_len; unsigned char partial[MAX_PREFIX_LEN]; }; /* * Small node with only 4 children */ struct _art_node4 { art_node n; unsigned char keys[4]; TOID(art_node_u) children[4]; }; /* * Node with 16 children */ struct _art_node16 { art_node n; unsigned char keys[16]; TOID(art_node_u) children[16]; }; /* * Node with 48 children, but a full 256 byte field. */ struct _art_node48 { art_node n; unsigned char keys[256]; TOID(art_node_u) children[48]; }; /* * Full node with 256 children */ struct _art_node256 { art_node n; TOID(art_node_u) children[256]; }; /* * Represents a leaf. These are of arbitrary size, as they include the key. */ struct _art_leaf { TOID(var_string) value; TOID(var_string) key; }; struct _art_node_u { uint8_t art_node_type; uint8_t art_node_tag; union { TOID(art_node4) an4; /* starts with art_node */ TOID(art_node16) an16; /* starts with art_node */ TOID(art_node48) an48; /* starts with art_node */ TOID(art_node256) an256; /* starts with art_node */ TOID(art_leaf) al; } u; }; struct art_tree_root { int size; TOID(art_node_u) root; }; typedef struct _cb_data { TOID(art_node_u) node; int child_idx; } cb_data; /* * Macros to manipulate art_node tags */ #define IS_LEAF(x) (((x)->art_node_type == art_leaf_t)) #define SET_LEAF(x) (((x)->art_node_tag = art_leaf_t)) #define COPY_BLOB(_obj, _blob, _len) \ D_RW(_obj)->len = _len; \ TX_MEMCPY(D_RW(_obj)->s, _blob, _len); \ D_RW(_obj)->s[(_len) - 1] = '\0'; typedef int(*art_callback)(void *data, const unsigned char *key, uint32_t key_len, const unsigned char *value, uint32_t val_len); extern int art_tree_init(PMEMobjpool *pop, int *newpool); extern uint64_t art_size(PMEMobjpool *pop); extern int art_iter(PMEMobjpool *pop, art_callback cb, void *data); extern TOID(var_string) art_insert(PMEMobjpool *pop, const unsigned char *key, int key_len, void *value, int val_len); extern TOID(var_string) art_search(PMEMobjpool *pop, const unsigned char *key, int key_len); extern TOID(var_string) art_delete(PMEMobjpool *pop, const unsigned char *key, int key_len); #ifdef __cplusplus } #endif #endif /* _ART_H */ pmdk-1.8/src/examples/libpmemobj/libart/arttree_structures.c0000664000000000000000000003460713615011243023142 0ustar rootroot/* * Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * =========================================================================== * * Filename: arttree_structures.c * * Description: Examine pmem structures; structures and unions taken from * the preprocessor output of a libpmemobj compatible program. * * Author: Andreas Bluemle, Dieter Kasper * Andreas.Bluemle.external@ts.fujitsu.com * dieter.kasper@ts.fujitsu.com * * Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH * * =========================================================================== */ #ifdef __FreeBSD__ #define _WITH_GETLINE #endif #include #include #include #include #include #include #include #include #include #include #include #include #include "arttree_structures.h" #include #define APPNAME "examine_arttree" #define SRCVERSION "0.2" size_t art_node_sizes[art_node_types] = { sizeof(art_node4), sizeof(art_node16), sizeof(art_node48), sizeof(art_node256), sizeof(art_leaf), sizeof(art_node_u), sizeof(art_node), sizeof(art_tree_root), sizeof(var_string), }; char *art_node_names[art_node_types] = { "art_node4", "art_node16", "art_node48", "art_node256", "art_leaf", "art_node_u", "art_node", "art_tree_root", "var_string" }; /* * long_options -- command line arguments */ static const struct option long_options[] = { {"help", no_argument, NULL, 'h'}, {NULL, 0, NULL, 0 }, }; /* * command -- struct for commands definition */ struct command { const char *name; const char *brief; int (*func)(char *, struct pmem_context *, int, char *[]); void (*help)(char *); }; /* * number of arttree_structures commands */ #define COMMANDS_NUMBER (sizeof(commands) / sizeof(commands[0])) static void print_help(char *appname); static void print_usage(char *appname); static void print_version(char *appname); static int quit_func(char *appname, struct pmem_context *ctx, int argc, char *argv[]); static void quit_help(char *appname); static int set_root_func(char *appname, struct pmem_context *ctx, int argc, char *argv[]); static void set_root_help(char *appname); static int help_func(char *appname, struct pmem_context *ctx, int argc, char *argv[]); static void help_help(char *appname); static struct command *get_command(char *cmd_str); static int ctx_init(struct pmem_context *ctx, char *filename); static int arttree_structures_func(char *appname, struct pmem_context *ctx, int ac, char *av[]); static void arttree_structures_help(char *appname); static int arttree_info_func(char *appname, struct pmem_context *ctx, int ac, char *av[]); static void arttree_info_help(char *appname); extern int arttree_examine_func(); extern void arttree_examine_help(); extern int arttree_search_func(); extern void arttree_search_help(); void outv_err(const char *fmt, ...); void outv_err_vargs(const char *fmt, va_list ap); static struct command commands[] = { { .name = "structures", .brief = "print information about ART structures", .func = arttree_structures_func, .help = arttree_structures_help, }, { .name = "info", .brief = "print information and statistics" " about an ART tree pool", .func = arttree_info_func, .help = arttree_info_help, }, { .name = "examine", .brief = "examine data structures from an ART tree", .func = arttree_examine_func, .help = arttree_examine_help, }, { .name = "search", .brief = "search for a key in an ART tree", .func = arttree_search_func, .help = arttree_search_help, }, { .name = "set_root", .brief = "define offset of root of an ART tree", .func = set_root_func, .help = set_root_help, }, { .name = "help", .brief = "print help text about a command", .func = help_func, .help = help_help, }, { .name = "quit", .brief = "quit ART tree structure examiner", .func = quit_func, .help = quit_help, }, }; static struct pmem_context ctx; /* * outv_err -- print error message */ void outv_err(const char *fmt, ...) { va_list ap; va_start(ap, fmt); outv_err_vargs(fmt, ap); va_end(ap); } /* * outv_err_vargs -- print error message */ void outv_err_vargs(const char *fmt, va_list ap) { fprintf(stderr, "error: "); vfprintf(stderr, fmt, ap); if (!strchr(fmt, '\n')) fprintf(stderr, "\n"); } /* * print_usage -- prints usage message */ static void print_usage(char *appname) { printf("usage: %s [--help] []\n", appname); } /* * print_version -- prints version message */ static void print_version(char *appname) { printf("%s %s\n", appname, SRCVERSION); } /* * print_help -- prints help message */ static void print_help(char *appname) { print_usage(appname); print_version(appname); printf("\n"); printf("Options:\n"); printf(" -h, --help display this help and exit\n"); printf("\n"); printf("The available commands are:\n"); int i; for (i = 0; i < COMMANDS_NUMBER; i++) printf("%s\t- %s\n", commands[i].name, commands[i].brief); printf("\n"); } /* * set_root_help -- prints help message for set root command */ static void set_root_help(char *appname) { printf("Usage: set_root \n"); printf(" define the offset of the art tree root\n"); } /* * set_root_func -- set_root define the offset of the art tree root */ static int set_root_func(char *appname, struct pmem_context *ctx, int argc, char *argv[]) { int retval = 0; uint64_t root_offset; if (argc == 2) { root_offset = strtol(argv[1], NULL, 0); ctx->art_tree_root_offset = root_offset; } else { set_root_help(appname); retval = 1; } return retval; } /* * quit_help -- prints help message for quit command */ static void quit_help(char *appname) { printf("Usage: quit\n"); printf(" terminate arttree structure examiner\n"); } /* * quit_func -- quit arttree structure examiner */ static int quit_func(char *appname, struct pmem_context *ctx, int argc, char *argv[]) { printf("\n"); exit(0); return 0; } /* * help_help -- prints help message for help command */ static void help_help(char *appname) { printf("Usage: %s help \n", appname); } /* * help_func -- prints help message for specified command */ static int help_func(char *appname, struct pmem_context *ctx, int argc, char *argv[]) { if (argc > 1) { char *cmd_str = argv[1]; struct command *cmdp = get_command(cmd_str); if (cmdp && cmdp->help) { cmdp->help(appname); return 0; } else { outv_err("No help text for '%s' command\n", cmd_str); return -1; } } else { print_help(appname); return -1; } } static const char *arttree_structures_help_str = "Show information about known ART tree structures\n" ; static void arttree_structures_help(char *appname) { printf("%s %s\n", appname, arttree_structures_help_str); } static int arttree_structures_func(char *appname, struct pmem_context *ctx, int ac, char *av[]) { (void) appname; (void) ac; (void) av; printf( "typedef struct pmemoid {\n" " uint64_t pool_uuid_lo;\n" " uint64_t off;\n" "} PMEMoid;\n"); printf("sizeof(PMEMoid) = %zu\n\n\n", sizeof(PMEMoid)); printf( "struct _art_node_u; typedef struct _art_node_u art_node_u;\n" "struct _art_node_u { \n" " uint8_t art_node_type; \n" " uint8_t art_node_tag; \n" "};\n"); printf("sizeof(art_node_u) = %zu\n\n\n", sizeof(art_node_u)); printf( "struct _art_node; typedef struct _art_node art_node;\n" "struct _art_node {\n" " uint8_t type;\n" " uint8_t num_children;\n" " uint32_t partial_len;\n" " unsigned char partial[10];\n" "};\n"); printf("sizeof(art_node) = %zu\n\n\n", sizeof(art_node)); printf( "typedef uint8_t _toid_art_node_toid_type_num[8];\n"); printf("sizeof(_toid_art_node_toid_type_num[8]) = %zu\n\n\n", sizeof(_toid_art_node_toid_type_num[8])); printf( "union _toid_art_node_u_toid {\n" " PMEMoid oid;\n" " art_node_u *_type;\n" " _toid_art_node_u_toid_type_num *_type_num;\n" "};\n"); printf("sizeof(union _toid_art_node_u_toid) = %zu\n\n\n", sizeof(union _toid_art_node_u_toid)); printf( "typedef uint8_t _toid_art_node_toid_type_num[8];\n"); printf("sizeof(_toid_art_node_toid_type_num[8]) = %zu\n\n\n", sizeof(_toid_art_node_toid_type_num[8])); printf( "union _toid_art_node_toid {\n" " PMEMoid oid; \n" " art_node *_type; \n" " _toid_art_node_toid_type_num *_type_num;\n" "};\n"); printf("sizeof(union _toid_art_node_toid) = %zu\n\n\n", sizeof(union _toid_art_node_toid)); printf( "struct _art_node4; typedef struct _art_node4 art_node4;\n" "struct _art_node4 {\n" " art_node n;\n" " unsigned char keys[4];\n" " union _toid_art_node_u_toid children[4];\n" "};\n"); printf("sizeof(art_node4) = %zu\n\n\n", sizeof(art_node4)); printf( "struct _art_node16; typedef struct _art_node16 art_node16;\n" "struct _art_node16 {\n" " art_node n;\n" " unsigned char keys[16];\n" " union _toid_art_node_u_toid children[16];\n" "};\n"); printf("sizeof(art_node16) = %zu\n\n\n", sizeof(art_node16)); printf( "struct _art_node48; typedef struct _art_node48 art_node48;\n" "struct _art_node48 {\n" " art_node n;\n" " unsigned char keys[256];\n" " union _toid_art_node_u_toid children[48];\n" "};\n"); printf("sizeof(art_node48) = %zu\n\n\n", sizeof(art_node48)); printf( "struct _art_node256; typedef struct _art_node256 art_node256;\n" "struct _art_node256 {\n" " art_ndoe n;\n" " union _toid_art_node_u_toid children[256];\n" "};\n"); printf("sizeof(art_node256) = %zu\n\n\n", sizeof(art_node256)); printf( "struct _art_leaf; typedef struct _art_leaf art_leaf;\n" "struct _art_leaf {\n" " union _toid_var_string_toid value;\n" " union _toid_var_string_toid key;\n" "};\n"); printf("sizeof(art_leaf) = %zu\n\n\n", sizeof(art_leaf)); return 0; } static const char *arttree_info_help_str = "Show information about known ART tree structures\n" ; static void arttree_info_help(char *appname) { printf("%s %s\n", appname, arttree_info_help_str); } static int arttree_info_func(char *appname, struct pmem_context *ctx, int ac, char *av[]) { printf("%s: %s not yet implemented\n", appname, __FUNCTION__); return 0; } /* * get_command -- returns command for specified command name */ static struct command * get_command(char *cmd_str) { int i; if (cmd_str == NULL) { return NULL; } for (i = 0; i < COMMANDS_NUMBER; i++) { if (strcmp(cmd_str, commands[i].name) == 0) return &commands[i]; } return NULL; } static int ctx_init(struct pmem_context *ctx, char *filename) { int errors = 0; if (filename == NULL) errors++; if (ctx == NULL) errors++; if (errors) return errors; ctx->filename = strdup(filename); assert(ctx->filename != NULL); ctx->fd = -1; ctx->addr = NULL; ctx->art_tree_root_offset = 0; if (access(ctx->filename, F_OK) != 0) return 1; if ((ctx->fd = open(ctx->filename, O_RDONLY)) == -1) return 1; struct stat stbuf; if (fstat(ctx->fd, &stbuf) < 0) return 1; ctx->psize = stbuf.st_size; if ((ctx->addr = mmap(NULL, ctx->psize, PROT_READ, MAP_SHARED, ctx->fd, 0)) == MAP_FAILED) return 1; return 0; } static void ctx_fini(struct pmem_context *ctx) { munmap(ctx->addr, ctx->psize); close(ctx->fd); free(ctx->filename); } int main(int ac, char *av[]) { int opt; int option_index; int ret = 0; size_t len; ssize_t read; char *cmd_str; char *args[20]; int nargs; char *line; struct command *cmdp = NULL; while ((opt = getopt_long(ac, av, "h", long_options, &option_index)) != -1) { switch (opt) { case 'h': print_help(APPNAME); return 0; default: print_usage(APPNAME); return -1; } } if (optind >= ac) { fprintf(stderr, "ERROR: missing arguments\n"); print_usage(APPNAME); return -1; } ctx_init(&ctx, av[optind]); if (optind + 1 < ac) { /* execute command as given on command line */ cmd_str = av[optind + 1]; cmdp = get_command(cmd_str); if (cmdp != NULL) { ret = cmdp->func(APPNAME, &ctx, ac - 2, av + 2); } } else { /* interactive mode: read commands and execute them */ line = NULL; printf("\n> "); while ((read = getline(&line, &len, stdin)) != -1) { if (line[read - 1] == '\n') { line[read - 1] = '\0'; } args[0] = strtok(line, " "); cmdp = get_command(args[0]); if (cmdp == NULL) { printf("[%s]: command not supported\n", args[0] ? args[0] : "NULL"); printf("\n> "); continue; } nargs = 1; while (1) { args[nargs] = strtok(NULL, " "); if (args[nargs] == NULL) { break; } nargs++; } ret = cmdp->func(APPNAME, &ctx, nargs, args); printf("\n> "); } if (line != NULL) { free(line); } } ctx_fini(&ctx); return ret; } pmdk-1.8/src/examples/libpmemobj/libart/arttree.c0000664000000000000000000004000113615011243020620 0ustar rootroot/* * Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * =========================================================================== * * Filename: arttree.c * * Description: implement ART tree using libpmemobj based on libart * * Author: Andreas Bluemle, Dieter Kasper * Andreas.Bluemle.external@ts.fujitsu.com * dieter.kasper@ts.fujitsu.com * * Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH * * =========================================================================== */ #include #include #include #include #include #ifdef __FreeBSD__ #define _WITH_GETLINE #endif #include #include #include #include #include #include #include #include #include #include "libpmemobj.h" #include "arttree.h" /* * dummy structure so far; this should correspond to the datastore * structure as defined in examples/libpmemobj/tree_map/datastore */ struct datastore { void *priv; }; /* * context - main context of datastore */ struct ds_context { char *filename; /* name of pool file */ int mode; /* operation mode */ int insertions; /* number of insert operations to perform */ int newpool; /* complete new memory pool */ size_t psize; /* size of pool */ PMEMobjpool *pop; /* pmemobj handle */ bool fileio; unsigned fmode; int fd; /* file descriptor for file io mode */ char *addr; /* base mapping address for file io mode */ unsigned char *key; /* for SEARCH, INSERT and REMOVE */ uint32_t key_len; unsigned char *value; /* for INSERT */ uint32_t val_len; }; #define FILL (1 << 1) #define DUMP (1 << 2) #define GRAPH (1 << 3) #define INSERT (1 << 4) #define SEARCH (1 << 5) #define REMOVE (1 << 6) struct ds_context my_context; extern TOID(var_string) null_var_string; extern TOID(art_leaf) null_art_leaf; extern TOID(art_node_u) null_art_node_u; #define read_key(p) read_line(p) #define read_value(p) read_line(p) int initialize_context(struct ds_context *ctx, int ac, char *av[]); int initialize_pool(struct ds_context *ctx); int add_elements(struct ds_context *ctx); int insert_element(struct ds_context *ctx); int search_element(struct ds_context *ctx); int delete_element(struct ds_context *ctx); ssize_t read_line(unsigned char **line); void exit_handler(struct ds_context *ctx); int art_tree_map_init(struct datastore *ds, struct ds_context *ctx); void pmemobj_ds_set_priv(struct datastore *ds, void *priv); static int dump_art_leaf_callback(void *data, const unsigned char *key, uint32_t key_len, const unsigned char *val, uint32_t val_len); static int dump_art_node_callback(void *data, const unsigned char *key, uint32_t key_len, const unsigned char *val, uint32_t val_len); static void print_node_info(char *nodetype, uint64_t off, const art_node *an); static int parse_keyval(struct ds_context *ctx, char *arg, int mode); int initialize_context(struct ds_context *ctx, int ac, char *av[]) { int errors = 0; int opt; char mode; if ((ctx == NULL) || (ac < 2)) { errors++; } if (!errors) { ctx->filename = NULL; ctx->psize = PMEMOBJ_MIN_POOL; ctx->newpool = 0; ctx->pop = NULL; ctx->fileio = false; ctx->fmode = 0666; ctx->mode = 0; ctx->fd = -1; } if (!errors) { while ((opt = getopt(ac, av, "s:m:n:")) != -1) { switch (opt) { case 'm': mode = optarg[0]; if (mode == 'f') { ctx->mode |= FILL; } else if (mode == 'd') { ctx->mode |= DUMP; } else if (mode == 'g') { ctx->mode |= GRAPH; } else if (mode == 'i') { ctx->mode |= INSERT; parse_keyval(ctx, av[optind], INSERT); optind++; } else if (mode == 's') { ctx->mode |= SEARCH; parse_keyval(ctx, av[optind], SEARCH); optind++; } else if (mode == 'r') { ctx->mode |= REMOVE; parse_keyval(ctx, av[optind], REMOVE); optind++; } else { errors++; } break; case 'n': { long insertions; insertions = strtol(optarg, NULL, 0); if (insertions > 0 && insertions < LONG_MAX) { ctx->insertions = insertions; } break; } case 's': { long poolsize; poolsize = strtol(optarg, NULL, 0); if (poolsize >= PMEMOBJ_MIN_POOL) { ctx->psize = poolsize; } break; } default: errors++; break; } } } if (!errors) { ctx->filename = strdup(av[optind]); } return errors; } static int parse_keyval(struct ds_context *ctx, char *arg, int mode) { int errors = 0; char *p; p = strtok(arg, ":"); if (p == NULL) { errors++; } if (!errors) { if (ctx->mode & (SEARCH|REMOVE|INSERT)) { ctx->key = (unsigned char *)strdup(p); assert(ctx->key != NULL); ctx->key_len = strlen(p) + 1; } if (ctx->mode & INSERT) { p = strtok(NULL, ":"); assert(p != NULL); ctx->value = (unsigned char *)strdup(p); assert(ctx->value != NULL); ctx->val_len = strlen(p) + 1; } } return errors; } void exit_handler(struct ds_context *ctx) { if (!ctx->fileio) { if (ctx->pop) { pmemobj_close(ctx->pop); } } else { if (ctx->fd > (-1)) { close(ctx->fd); } } } int art_tree_map_init(struct datastore *ds, struct ds_context *ctx) { int errors = 0; char *error_string; /* calculate a required pool size */ if (ctx->psize < PMEMOBJ_MIN_POOL) ctx->psize = PMEMOBJ_MIN_POOL; if (!ctx->fileio) { if (access(ctx->filename, F_OK) != 0) { error_string = "pmemobj_create"; ctx->pop = pmemobj_create(ctx->filename, POBJ_LAYOUT_NAME(arttree_tx), ctx->psize, ctx->fmode); ctx->newpool = 1; } else { error_string = "pmemobj_open"; ctx->pop = pmemobj_open(ctx->filename, POBJ_LAYOUT_NAME(arttree_tx)); } if (ctx->pop == NULL) { perror(error_string); errors++; } } else { int flags = O_CREAT | O_RDWR | O_SYNC; /* Create a file if it does not exist. */ if ((ctx->fd = open(ctx->filename, flags, ctx->fmode)) < 0) { perror(ctx->filename); errors++; } /* allocate the pmem */ if ((errno = posix_fallocate(ctx->fd, 0, ctx->psize)) != 0) { perror("posix_fallocate"); errors++; } /* map file to memory */ if ((ctx->addr = mmap(NULL, ctx->psize, PROT_READ, MAP_SHARED, ctx->fd, 0)) == MAP_FAILED) { perror("mmap"); errors++; } } if (!errors) { pmemobj_ds_set_priv(ds, ctx); } else { if (ctx->fileio) { if (ctx->addr != NULL) { munmap(ctx->addr, ctx->psize); } if (ctx->fd >= 0) { close(ctx->fd); } } else { if (ctx->pop) { pmemobj_close(ctx->pop); } } } return errors; } /* * pmemobj_ds_set_priv -- set private structure of datastore */ void pmemobj_ds_set_priv(struct datastore *ds, void *priv) { ds->priv = priv; } struct datastore myds; static void usage(char *progname) { printf("usage: %s -m [f|d|g] file\n", progname); printf(" -m mode known modes are\n"); printf(" f fill create and fill art tree\n"); printf(" i insert insert an element into the art tree\n"); printf(" s search search for a key in the art tree\n"); printf(" r remove remove an element from the art tree\n"); printf(" d dump dump art tree\n"); printf(" g graph dump art tree as a graphviz dot graph\n"); printf(" -n number of key-value pairs to insert" " into the art tree\n"); printf(" -s size in bytes of the memory pool" " (minimum and default: 8 MB)"); printf("\nfilling an art tree is done by reading key-value pairs\n" "from standard input.\n" "Both keys and values are single line only.\n"); } int main(int argc, char *argv[]) { if (initialize_context(&my_context, argc, argv) != 0) { usage(argv[0]); return 1; } if (art_tree_map_init(&myds, &my_context) != 0) { fprintf(stderr, "failed to initialize memory pool file\n"); return 1; } if (my_context.pop == NULL) { perror("pool initialization"); return 1; } if (art_tree_init(my_context.pop, &my_context.newpool)) { perror("pool setup"); return 1; } if ((my_context.mode & FILL)) { if (add_elements(&my_context)) { perror("add elements"); return 1; } } if ((my_context.mode & INSERT)) { if (insert_element(&my_context)) { perror("insert elements"); return 1; } } if ((my_context.mode & SEARCH)) { if (search_element(&my_context)) { perror("search elements"); return 1; } } if ((my_context.mode & REMOVE)) { if (delete_element(&my_context)) { perror("delete elements"); return 1; } } if (my_context.mode & DUMP) { art_iter(my_context.pop, dump_art_leaf_callback, NULL); } if (my_context.mode & GRAPH) { printf("digraph g {\nrankdir=LR;\n"); art_iter(my_context.pop, dump_art_node_callback, NULL); printf("}"); } exit_handler(&my_context); return 0; } int add_elements(struct ds_context *ctx) { PMEMobjpool *pop; int errors = 0; int i; int key_len; int val_len; unsigned char *key; unsigned char *value; if (ctx == NULL) { errors++; } else if (ctx->pop == NULL) { errors++; } if (!errors) { pop = ctx->pop; for (i = 0; i < ctx->insertions; i++) { key = NULL; value = NULL; key_len = read_key(&key); val_len = read_value(&value); art_insert(pop, key, key_len, value, val_len); if (key != NULL) free(key); if (value != NULL) free(value); } } return errors; } int insert_element(struct ds_context *ctx) { PMEMobjpool *pop; int errors = 0; if (ctx == NULL) { errors++; } else if (ctx->pop == NULL) { errors++; } if (!errors) { pop = ctx->pop; art_insert(pop, ctx->key, ctx->key_len, ctx->value, ctx->val_len); } return errors; } int search_element(struct ds_context *ctx) { PMEMobjpool *pop; TOID(var_string) value; int errors = 0; if (ctx == NULL) { errors++; } else if (ctx->pop == NULL) { errors++; } if (!errors) { pop = ctx->pop; printf("search key [%s]: ", (char *)ctx->key); value = art_search(pop, ctx->key, ctx->key_len); if (TOID_IS_NULL(value)) { printf("not found\n"); } else { printf("value [%s]\n", D_RO(value)->s); } } return errors; } int delete_element(struct ds_context *ctx) { PMEMobjpool *pop; int errors = 0; if (ctx == NULL) { errors++; } else if (ctx->pop == NULL) { errors++; } if (!errors) { pop = ctx->pop; art_delete(pop, ctx->key, ctx->key_len); } return errors; } ssize_t read_line(unsigned char **line) { size_t len = -1; ssize_t read = -1; *line = NULL; if ((read = getline((char **)line, &len, stdin)) > 0) { (*line)[read - 1] = '\0'; } return read; } static int dump_art_leaf_callback(void *data, const unsigned char *key, uint32_t key_len, const unsigned char *val, uint32_t val_len) { cb_data *cbd; if (data != NULL) { cbd = (cb_data *)data; printf("node type %d ", D_RO(cbd->node)->art_node_type); if (D_RO(cbd->node)->art_node_type == art_leaf_t) { printf("key len %" PRIu32 " = [%s], value len %" PRIu32 " = [%s]", key_len, key != NULL ? (char *)key : (char *)"NULL", val_len, val != NULL ? (char *)val : (char *)"NULL"); } printf("\n"); } else { printf("key len %" PRIu32 " = [%s], value len %" PRIu32 " = [%s]\n", key_len, key != NULL ? (char *)key : (char *)"NULL", val_len, val != NULL ? (char *)val : (char *)"NULL"); } return 0; } static void print_node_info(char *nodetype, uint64_t off, const art_node *an) { int p_len, i; p_len = an->partial_len; printf("N%" PRIx64 " [label=\"%s at\\n0x%" PRIx64 "\\n%d children", off, nodetype, off, an->num_children); if (p_len != 0) { printf("\\nlen %d", p_len); printf(": "); for (i = 0; i < p_len; i++) { printf("%c", an->partial[i]); } } printf("\"];\n"); } static int dump_art_node_callback(void *data, const unsigned char *key, uint32_t key_len, const unsigned char *val, uint32_t val_len) { cb_data *cbd; const art_node *an; TOID(art_node4) an4; TOID(art_node16) an16; TOID(art_node48) an48; TOID(art_node256) an256; TOID(art_leaf) al; TOID(art_node_u) child; TOID(var_string) oid_key; TOID(var_string) oid_value; if (data != NULL) { cbd = (cb_data *)data; switch (D_RO(cbd->node)->art_node_type) { case NODE4: an4 = D_RO(cbd->node)->u.an4; an = &(D_RO(an4)->n); child = D_RO(an4)->children[cbd->child_idx]; if (!TOID_IS_NULL(child)) { print_node_info("node4", cbd->node.oid.off, an); printf("N%" PRIx64 " -> N%" PRIx64 " [label=\"%c\"];\n", cbd->node.oid.off, child.oid.off, D_RO(an4)->keys[cbd->child_idx]); } break; case NODE16: an16 = D_RO(cbd->node)->u.an16; an = &(D_RO(an16)->n); child = D_RO(an16)->children[cbd->child_idx]; if (!TOID_IS_NULL(child)) { print_node_info("node16", cbd->node.oid.off, an); printf("N%" PRIx64 " -> N%" PRIx64 " [label=\"%c\"];\n", cbd->node.oid.off, child.oid.off, D_RO(an16)->keys[cbd->child_idx]); } break; case NODE48: an48 = D_RO(cbd->node)->u.an48; an = &(D_RO(an48)->n); child = D_RO(an48)->children[cbd->child_idx]; if (!TOID_IS_NULL(child)) { print_node_info("node48", cbd->node.oid.off, an); printf("N%" PRIx64 " -> N%" PRIx64 " [label=\"%c\"];\n", cbd->node.oid.off, child.oid.off, D_RO(an48)->keys[cbd->child_idx]); } break; case NODE256: an256 = D_RO(cbd->node)->u.an256; an = &(D_RO(an256)->n); child = D_RO(an256)->children[cbd->child_idx]; if (!TOID_IS_NULL(child)) { print_node_info("node256", cbd->node.oid.off, an); printf("N%" PRIx64 " -> N%" PRIx64 " [label=\"0x%x\"];\n", cbd->node.oid.off, child.oid.off, (char)((cbd->child_idx) & 0xff)); } break; case art_leaf_t: al = D_RO(cbd->node)->u.al; oid_key = D_RO(al)->key; oid_value = D_RO(al)->value; printf("N%" PRIx64 " [shape=box," "label=\"leaf at\\n0x%" PRIx64 "\"];\n", cbd->node.oid.off, cbd->node.oid.off); printf("N%" PRIx64 " [shape=box," "label=\"key at 0x%" PRIx64 ": %s\"];\n", oid_key.oid.off, oid_key.oid.off, D_RO(oid_key)->s); printf("N%" PRIx64 " [shape=box," "label=\"value at 0x%" PRIx64 ": %s\"];\n", oid_value.oid.off, oid_value.oid.off, D_RO(oid_value)->s); printf("N%" PRIx64 " -> N%" PRIx64 ";\n", cbd->node.oid.off, oid_key.oid.off); printf("N%" PRIx64 " -> N%" PRIx64 ";\n", cbd->node.oid.off, oid_value.oid.off); break; default: break; } } else { printf("leaf: key len %" PRIu32 " = [%s], value len %" PRIu32 " = [%s]\n", key_len, key, val_len, val); } return 0; } pmdk-1.8/src/examples/libpmemobj/libart/.gitignore0000664000000000000000000000003013615011243020774 0ustar rootrootarttree examine_arttree pmdk-1.8/src/examples/libpmemobj/libart/arttree_search.c0000664000000000000000000002572113615011243022161 0ustar rootroot/* * Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * =========================================================================== * * Filename: arttree_search.c * * Description: implementation of search function for ART tree * * Author: Andreas Bluemle, Dieter Kasper * Andreas.Bluemle.external@ts.fujitsu.com * dieter.kasper@ts.fujitsu.com * * Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH * * =========================================================================== */ #include #include #include #include #include #include #include #include #include #include #include #include "arttree_structures.h" /* * search context */ struct search_ctx { struct pmem_context *pmem_ctx; unsigned char *search_key; int32_t hexdump; }; static struct search_ctx *s_ctx = NULL; struct search { const char *name; const char *brief; char *(*func)(char *, struct search_ctx *); void (*help)(char *); }; /* local functions */ static int search_parse_args(char *appname, int ac, char *av[], struct search_ctx *s_ctx); static struct search *get_search(char *type_name); static void print_usage(char *appname); static void dump_PMEMoid(char *prefix, PMEMoid *oid); static char *search_key(char *appname, struct search_ctx *ctx); static int leaf_matches(struct search_ctx *ctx, art_leaf *n, unsigned char *key, int key_len, int depth); static int check_prefix(art_node *an, unsigned char *key, int key_len, int depth); static uint64_t find_child(art_node *n, int node_type, unsigned char key); static void *get_node(struct search_ctx *ctx, int node_type, uint64_t off); static uint64_t get_offset_an(art_node_u *au); static void dump_PMEMoid(char *prefix, PMEMoid *oid); static void dump_art_tree_root(char *prefix, uint64_t off, void *p); /* global visible interface */ void arttree_search_help(char *appname); int arttree_search_func(char *appname, struct pmem_context *ctx, int ac, char *av[]); static const char *arttree_search_help_str = "Search for key in ART tree\n" "Arguments: \n" " key\n" ; static const struct option long_options[] = { {"hexdump", no_argument, NULL, 'x'}, {NULL, 0, NULL, 0 }, }; static struct search s_funcs[] = { { .name = "key", .brief = "search for key", .func = search_key, .help = NULL, } }; /* Simple inlined function */ static inline int min(int a, int b) { return (a < b) ? b : a; } /* * number of arttree examine commands */ #define COMMANDS_NUMBER (sizeof(s_funcs) / sizeof(s_funcs[0])) void arttree_search_help(char *appname) { printf("%s %s\n", appname, arttree_search_help_str); } int arttree_search_func(char *appname, struct pmem_context *ctx, int ac, char *av[]) { int errors = 0; struct search *s; char *value; value = NULL; if (ctx == NULL) { return -1; } if (s_ctx == NULL) { s_ctx = (struct search_ctx *)malloc(sizeof(struct search_ctx)); if (s_ctx == NULL) { return -1; } memset(s_ctx, 0, sizeof(struct search_ctx)); } if (ctx->art_tree_root_offset == 0) { fprintf(stderr, "search functions require knowledge" " about the art_tree_root.\n"); fprintf(stderr, "Use \"set_root \"" " to define where the \nart_tree_root object" " resides in the pmem file.\n"); errors++; } s_ctx->pmem_ctx = ctx; if (search_parse_args(appname, ac, av, s_ctx) != 0) { fprintf(stderr, "%s::%s: error parsing arguments\n", appname, __FUNCTION__); errors++; } if (!errors) { s = get_search("key"); if (s != NULL) { value = s->func(appname, s_ctx); } if (value != NULL) { printf("key [%s] found, value [%s]\n", s_ctx->search_key, value); } else { printf("key [%s] not found\n", s_ctx->search_key); } } if (s_ctx->search_key != NULL) { free(s_ctx->search_key); } free(s_ctx); return errors; } static int search_parse_args(char *appname, int ac, char *av[], struct search_ctx *s_ctx) { int ret = 0; int opt; optind = 0; while ((opt = getopt_long(ac, av, "x", long_options, NULL)) != -1) { switch (opt) { case 'x': s_ctx->hexdump = 1; break; default: print_usage(appname); ret = 1; } } if (ret == 0) { s_ctx->search_key = (unsigned char *)strdup(av[optind + 0]); } return ret; } static void print_usage(char *appname) { printf("%s: search \n", appname); } /* * get_search -- returns command for specified command name */ static struct search * get_search(char *type_name) { int i; if (type_name == NULL) { return NULL; } for (i = 0; i < COMMANDS_NUMBER; i++) { if (strcmp(type_name, s_funcs[i].name) == 0) return &s_funcs[i]; } return NULL; } static void * get_node(struct search_ctx *ctx, int node_type, uint64_t off) { if (!VALID_NODE_TYPE(node_type)) return NULL; printf("%s at off 0x%" PRIx64 "\n", art_node_names[node_type], off); return ctx->pmem_ctx->addr + off; } static int leaf_matches(struct search_ctx *ctx, art_leaf *n, unsigned char *key, int key_len, int depth) { var_string *n_key; (void) depth; n_key = (var_string *)get_node(ctx, VAR_STRING, n->key.oid.off); if (n_key == NULL) return 1; // HACK for stupid null-terminated strings.... // else if (n_key->len != key_len) // ret = 1; if (n_key->len != key_len + 1) return 1; return memcmp(n_key->s, key, key_len); } static int check_prefix(art_node *n, unsigned char *key, int key_len, int depth) { int max_cmp = min(min(n->partial_len, MAX_PREFIX_LEN), key_len - depth); int idx; for (idx = 0; idx < max_cmp; idx++) { if (n->partial[idx] != key[depth + idx]) return idx; } return idx; } static uint64_t find_child(art_node *n, int node_type, unsigned char c) { int i; union { art_node4 *p1; art_node16 *p2; art_node48 *p3; art_node256 *p4; } p; printf("[%s] children %d search key %c [", art_node_names[node_type], n->num_children, c); switch (node_type) { case ART_NODE4: p.p1 = (art_node4 *)n; for (i = 0; i < n->num_children; i++) { printf("%c ", p.p1->keys[i]); if (p.p1->keys[i] == c) { printf("]\n"); return p.p1->children[i].oid.off; } } break; case ART_NODE16: p.p2 = (art_node16 *)n; for (i = 0; i < n->num_children; i++) { printf("%c ", p.p2->keys[i]); if (p.p2->keys[i] == c) { printf("]\n"); return p.p2->children[i].oid.off; } } break; case ART_NODE48: p.p3 = (art_node48 *)n; i = p.p3->keys[c]; printf("%d ", p.p3->keys[c]); if (i) { printf("]\n"); return p.p3->children[i - 1].oid.off; } break; case ART_NODE256: p.p4 = (art_node256 *)n; printf("0x%" PRIx64, p.p4->children[c].oid.off); if (p.p4->children[c].oid.off != 0) { printf("]\n"); return p.p4->children[c].oid.off; } break; default: abort(); } printf("]\n"); return 0; } static uint64_t get_offset_an(art_node_u *au) { uint64_t offset = 0; switch (au->art_node_type) { case ART_NODE4: offset = au->u.an4.oid.off; break; case ART_NODE16: offset = au->u.an16.oid.off; break; case ART_NODE48: offset = au->u.an48.oid.off; break; case ART_NODE256: offset = au->u.an256.oid.off; break; case ART_LEAF: offset = au->u.al.oid.off; break; default: break; } return offset; } static char * search_key(char *appname, struct search_ctx *ctx) { int errors = 0; void *p; /* something */ off_t p_off; art_node_u *p_au; /* art_node_u */ off_t p_au_off; void *p_an; /* specific art node from art_node_u */ off_t p_an_off; art_node *an; /* art node */ var_string *n_value; char *value; int prefix_len; int depth = 0; int key_len; uint64_t child_off; key_len = strlen((char *)(ctx->search_key)); value = NULL; p_off = ctx->pmem_ctx->art_tree_root_offset; p = get_node(ctx, ART_TREE_ROOT, p_off); assert(p != NULL); dump_art_tree_root("art_tree_root", p_off, p); p_au_off = ((art_tree_root *)p)->root.oid.off; p_au = (art_node_u *)get_node(ctx, ART_NODE_U, p_au_off); if (p_au == NULL) errors++; if (!errors) { while (p_au) { p_an_off = get_offset_an(p_au); p_an = get_node(ctx, p_au->art_node_type, p_an_off); assert(p_an != NULL); if (p_au->art_node_type == ART_LEAF) { if (!leaf_matches(ctx, (art_leaf *)p_an, ctx->search_key, key_len, depth)) { n_value = (var_string *) get_node(ctx, VAR_STRING, ((art_leaf *)p_an)->value.oid.off); return (char *)(n_value->s); } } an = (art_node *)p_an; if (an->partial_len) { prefix_len = check_prefix(an, ctx->search_key, key_len, depth); if (prefix_len != min(MAX_PREFIX_LEN, an->partial_len)) { return NULL; } depth = depth + an->partial_len; } child_off = find_child(an, p_au->art_node_type, ctx->search_key[depth]); if (child_off != 0) { p_au_off = child_off; p_au = get_node(ctx, ART_NODE_U, p_au_off); } else { p_au = NULL; } depth++; } } if (errors) { return NULL; } else { return value; } } static void dump_art_tree_root(char *prefix, uint64_t off, void *p) { art_tree_root *tree_root; tree_root = (art_tree_root *)p; printf("at offset 0x%" PRIx64 ", art_tree_root {\n", off); printf(" size %d\n", tree_root->size); dump_PMEMoid(" art_node_u", (PMEMoid *)&(tree_root->root)); printf("\n};\n"); } static void dump_PMEMoid(char *prefix, PMEMoid *oid) { printf("%s { PMEMoid pool_uuid_lo %" PRIx64 " off 0x%" PRIx64 " = %" PRId64 " }\n", prefix, oid->pool_uuid_lo, oid->off, oid->off); } pmdk-1.8/src/examples/libpmemobj/libart/arttree_structures.h0000664000000000000000000001336713615011243023147 0ustar rootroot/* * Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * =========================================================================== * * Filename: arttree_structures.h * * Description: known structures of the ART tree * * Author: Andreas Bluemle, Dieter Kasper * Andreas.Bluemle.external@ts.fujitsu.com * dieter.kasper@ts.fujitsu.com * * Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH * * =========================================================================== */ #ifndef _ARTTREE_STRUCTURES_H #define _ARTTREE_STRUCTURES_H #define MAX_PREFIX_LEN 10 /* * pmem_context -- structure for pmempool file */ struct pmem_context { char *filename; size_t psize; int fd; char *addr; uint64_t art_tree_root_offset; }; struct _art_node_u; typedef struct _art_node_u art_node_u; struct _art_node; typedef struct _art_node art_node; struct _art_node4; typedef struct _art_node4 art_node4; struct _art_node16; typedef struct _art_node16 art_node16; struct _art_node48; typedef struct _art_node48 art_node48; struct _art_node256; typedef struct _art_node256 art_node256; struct _var_string; typedef struct _var_string var_string; struct _art_leaf; typedef struct _art_leaf art_leaf; struct _art_tree_root; typedef struct _art_tree_root art_tree_root; typedef uint8_t art_tree_root_toid_type_num[65535]; typedef uint8_t _toid_art_node_u_toid_type_num[2]; typedef uint8_t _toid_art_node_toid_type_num[3]; typedef uint8_t _toid_art_node4_toid_type_num[4]; typedef uint8_t _toid_art_node16_toid_type_num[5]; typedef uint8_t _toid_art_node48_toid_type_num[6]; typedef uint8_t _toid_art_node256_toid_type_num[7]; typedef uint8_t _toid_art_leaf_toid_type_num[8]; typedef uint8_t _toid_var_string_toid_type_num[9]; typedef struct pmemoid { uint64_t pool_uuid_lo; uint64_t off; } PMEMoid; union _toid_art_node_u_toid { PMEMoid oid; art_node_u *_type; _toid_art_node_u_toid_type_num *_type_num; }; union art_tree_root_toid { PMEMoid oid; struct art_tree_root *_type; art_tree_root_toid_type_num *_type_num; }; union _toid_art_node_toid { PMEMoid oid; art_node *_type; _toid_art_node_toid_type_num *_type_num; }; union _toid_art_node4_toid { PMEMoid oid; art_node4 *_type; _toid_art_node4_toid_type_num *_type_num; }; union _toid_art_node16_toid { PMEMoid oid; art_node16 *_type; _toid_art_node16_toid_type_num *_type_num; }; union _toid_art_node48_toid { PMEMoid oid; art_node48 *_type; _toid_art_node48_toid_type_num *_type_num; }; union _toid_art_node256_toid { PMEMoid oid; art_node256 *_type; _toid_art_node256_toid_type_num *_type_num; }; union _toid_var_string_toid { PMEMoid oid; var_string *_type; _toid_var_string_toid_type_num *_type_num; }; union _toid_art_leaf_toid { PMEMoid oid; art_leaf *_type; _toid_art_leaf_toid_type_num *_type_num; }; struct _art_tree_root { int size; union _toid_art_node_u_toid root; }; struct _art_node { uint8_t num_children; uint32_t partial_len; unsigned char partial[MAX_PREFIX_LEN]; }; struct _art_node4 { art_node n; unsigned char keys[4]; union _toid_art_node_u_toid children[4]; }; struct _art_node16 { art_node n; unsigned char keys[16]; union _toid_art_node_u_toid children[16]; }; struct _art_node48 { art_node n; unsigned char keys[256]; union _toid_art_node_u_toid children[48]; }; struct _art_node256 { art_node n; union _toid_art_node_u_toid children[256]; }; struct _var_string { size_t len; unsigned char s[]; }; struct _art_leaf { union _toid_var_string_toid value; union _toid_var_string_toid key; }; struct _art_node_u { uint8_t art_node_type; uint8_t art_node_tag; union { union _toid_art_node4_toid an4; union _toid_art_node16_toid an16; union _toid_art_node48_toid an48; union _toid_art_node256_toid an256; union _toid_art_leaf_toid al; } u; }; typedef enum { ART_NODE4 = 0, ART_NODE16 = 1, ART_NODE48 = 2, ART_NODE256 = 3, ART_LEAF = 4, ART_NODE_U = 5, ART_NODE = 6, ART_TREE_ROOT = 7, VAR_STRING = 8, art_node_types = 9 /* number of different art_nodes */ } art_node_type; #define VALID_NODE_TYPE(n) (((n) >= 0) && ((n) < art_node_types)) extern size_t art_node_sizes[]; extern char *art_node_names[]; #endif /* _ARTTREE_STRUCTURES_H */ pmdk-1.8/src/examples/libpmemobj/libart/arttree_examine.c0000664000000000000000000003026313615011243022337 0ustar rootroot/* * Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * =========================================================================== * * Filename: arttree_examine.c * * Description: implementation of examine function for ART tree structures * * Author: Andreas Bluemle, Dieter Kasper * Andreas.Bluemle.external@ts.fujitsu.com * dieter.kasper@ts.fujitsu.com * * Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH * * =========================================================================== */ #include #include #include #include #include #include #include #include #include #include "arttree_structures.h" /* * examine context */ struct examine_ctx { struct pmem_context *pmem_ctx; char *offset_string; uint64_t offset; char *type_name; int32_t type; int32_t hexdump; }; static struct examine_ctx *ex_ctx = NULL; struct examine { const char *name; const char *brief; int (*func)(char *, struct examine_ctx *, off_t); void (*help)(char *); }; /* local functions */ static int examine_parse_args(char *appname, int ac, char *av[], struct examine_ctx *ex_ctx); static struct examine *get_examine(char *type_name); static void print_usage(char *appname); static void dump_PMEMoid(char *prefix, PMEMoid *oid); static int examine_PMEMoid(char *appname, struct examine_ctx *ctx, off_t off); static int examine_art_tree_root(char *appname, struct examine_ctx *ctx, off_t off); static int examine_art_node_u(char *appname, struct examine_ctx *ctx, off_t off); static int examine_art_node4(char *appname, struct examine_ctx *ctx, off_t off); static int examine_art_node16(char *appname, struct examine_ctx *ctx, off_t off); static int examine_art_node48(char *appname, struct examine_ctx *ctx, off_t off); static int examine_art_node256(char *appname, struct examine_ctx *ctx, off_t off); #if 0 /* XXX */ static int examine_art_node(char *appname, struct examine_ctx *ctx, off_t off); #else static int examine_art_node(art_node *an); #endif static int examine_art_leaf(char *appname, struct examine_ctx *ctx, off_t off); static int examine_var_string(char *appname, struct examine_ctx *ctx, off_t off); /* global visible interface */ void arttree_examine_help(char *appname); int arttree_examine_func(char *appname, struct pmem_context *ctx, int ac, char *av[]); static const char *arttree_examine_help_str = "Examine data structures (objects) of ART tree\n" "Arguments: \n" " offset of object in pmem file\n" " one of art_tree_root, art_node_u, art_node," " art_node4, art_node16, art_node48, art_node256, art_leaf\n" ; static const struct option long_options[] = { {"hexdump", no_argument, NULL, 'x'}, {NULL, 0, NULL, 0 }, }; static struct examine ex_funcs[] = { { .name = "PMEMobj", .brief = "examine PMEMoid structure", .func = examine_PMEMoid, .help = NULL, }, { .name = "art_tree_root", .brief = "examine art_tree_root structure", .func = examine_art_tree_root, .help = NULL, }, { .name = "art_node_u", .brief = "examine art_node_u structure", .func = examine_art_node_u, .help = NULL, }, { .name = "art_node4", .brief = "examine art_node4 structure", .func = examine_art_node4, .help = NULL, }, { .name = "art_node16", .brief = "examine art_node16 structure", .func = examine_art_node16, .help = NULL, }, { .name = "art_node48", .brief = "examine art_node48 structure", .func = examine_art_node48, .help = NULL, }, { .name = "art_node256", .brief = "examine art_node256 structure", .func = examine_art_node256, .help = NULL, }, { .name = "art_leaf", .brief = "examine art_leaf structure", .func = examine_art_leaf, .help = NULL, }, { .name = "var_string", .brief = "examine var_string structure", .func = examine_var_string, .help = NULL, }, }; /* * number of arttree examine commands */ #define COMMANDS_NUMBER (sizeof(ex_funcs) / sizeof(ex_funcs[0])) void arttree_examine_help(char *appname) { printf("%s %s\n", appname, arttree_examine_help_str); } int arttree_examine_func(char *appname, struct pmem_context *ctx, int ac, char *av[]) { int errors = 0; off_t offset; struct examine *ex; if (ctx == NULL) { return -1; } if (ex_ctx == NULL) { ex_ctx = (struct examine_ctx *) calloc(1, sizeof(struct examine_ctx)); if (ex_ctx == NULL) { return -1; } } ex_ctx->pmem_ctx = ctx; if (examine_parse_args(appname, ac, av, ex_ctx) != 0) { fprintf(stderr, "%s::%s: error parsing arguments\n", appname, __FUNCTION__); errors++; } if (!errors) { offset = (off_t)strtol(ex_ctx->offset_string, NULL, 0); ex = get_examine(ex_ctx->type_name); if (ex != NULL) { ex->func(appname, ex_ctx, offset); } } return errors; } static int examine_parse_args(char *appname, int ac, char *av[], struct examine_ctx *ex_ctx) { int ret = 0; int opt; optind = 0; while ((opt = getopt_long(ac, av, "x", long_options, NULL)) != -1) { switch (opt) { case 'x': ex_ctx->hexdump = 1; break; default: print_usage(appname); ret = 1; } } if (ret == 0) { ex_ctx->offset_string = strdup(av[optind + 0]); ex_ctx->type_name = strdup(av[optind + 1]); } return ret; } static void print_usage(char *appname) { printf("%s: examine \n", appname); } /* * get_command -- returns command for specified command name */ static struct examine * get_examine(char *type_name) { int i; if (type_name == NULL) { return NULL; } for (i = 0; i < COMMANDS_NUMBER; i++) { if (strcmp(type_name, ex_funcs[i].name) == 0) return &ex_funcs[i]; } return NULL; } static void dump_PMEMoid(char *prefix, PMEMoid *oid) { printf("%s { PMEMoid pool_uuid_lo %" PRIx64 " off 0x%" PRIx64 " = %" PRId64 " }\n", prefix, oid->pool_uuid_lo, oid->off, oid->off); } static int examine_PMEMoid(char *appname, struct examine_ctx *ctx, off_t off) { void *p = (void *)(ctx->pmem_ctx->addr + off); dump_PMEMoid("PMEMoid", p); return 0; } static int examine_art_tree_root(char *appname, struct examine_ctx *ctx, off_t off) { art_tree_root *tree_root = (art_tree_root *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, art_tree_root {\n", (long long)off); printf(" size %d\n", tree_root->size); dump_PMEMoid(" art_node_u", (PMEMoid *)&(tree_root->root)); printf("\n};\n"); return 0; } static int examine_art_node_u(char *appname, struct examine_ctx *ctx, off_t off) { art_node_u *node_u = (art_node_u *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, art_node_u {\n", (long long)off); printf(" type %d [%s]\n", node_u->art_node_type, art_node_names[node_u->art_node_type]); printf(" tag %d\n", node_u->art_node_tag); switch (node_u->art_node_type) { case ART_NODE4: dump_PMEMoid(" art_node4 oid", &(node_u->u.an4.oid)); break; case ART_NODE16: dump_PMEMoid(" art_node16 oid", &(node_u->u.an16.oid)); break; case ART_NODE48: dump_PMEMoid(" art_node48 oid", &(node_u->u.an48.oid)); break; case ART_NODE256: dump_PMEMoid(" art_node256 oid", &(node_u->u.an256.oid)); break; case ART_LEAF: dump_PMEMoid(" art_leaf oid", &(node_u->u.al.oid)); break; default: printf("ERROR: unknown node type\n"); break; } printf("\n};\n"); return 0; } static int examine_art_node4(char *appname, struct examine_ctx *ctx, off_t off) { art_node4 *an4 = (art_node4 *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, art_node4 {\n", (long long)off); examine_art_node(&(an4->n)); printf("keys ["); for (int i = 0; i < 4; i++) { printf("%c ", an4->keys[i]); } printf("]\nnodes [\n"); for (int i = 0; i < 4; i++) { dump_PMEMoid(" art_node_u oid", &(an4->children[i].oid)); } printf("\n]"); printf("\n};\n"); return 0; } static int examine_art_node16(char *appname, struct examine_ctx *ctx, off_t off) { art_node16 *an16 = (art_node16 *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, art_node16 {\n", (long long)off); examine_art_node(&(an16->n)); printf("keys ["); for (int i = 0; i < 16; i++) { printf("%c ", an16->keys[i]); } printf("]\nnodes [\n"); for (int i = 0; i < 16; i++) { dump_PMEMoid(" art_node_u oid", &(an16->children[i].oid)); } printf("\n]"); printf("\n};\n"); return 0; } static int examine_art_node48(char *appname, struct examine_ctx *ctx, off_t off) { art_node48 *an48 = (art_node48 *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, art_node48 {\n", (long long)off); examine_art_node(&(an48->n)); printf("keys ["); for (int i = 0; i < 256; i++) { printf("%c ", an48->keys[i]); } printf("]\nnodes [\n"); for (int i = 0; i < 48; i++) { dump_PMEMoid(" art_node_u oid", &(an48->children[i].oid)); } printf("\n]"); printf("\n};\n"); return 0; } static int examine_art_node256(char *appname, struct examine_ctx *ctx, off_t off) { art_node256 *an256 = (art_node256 *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, art_node256 {\n", (long long)off); examine_art_node(&(an256->n)); printf("nodes [\n"); for (int i = 0; i < 256; i++) { dump_PMEMoid(" art_node_u oid", &(an256->children[i].oid)); } printf("\n]"); printf("\n};\n"); return 0; } #if 0 /* XXX */ static int examine_art_node(char *appname, struct examine_ctx *ctx, off_t off) { art_node *an = (art_node *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, art_node {\n", (long long)off); printf(" num_children %d\n", an->num_children); printf(" partial_len %d\n", an->partial_len); printf(" partial ["); for (int i = 0; i < 10; i++) { printf("%c ", an->partial[i]); } printf("\n]"); printf("\n};\n"); return 0; } #else static int examine_art_node(art_node *an) { printf("art_node {\n"); printf(" num_children %d\n", an->num_children); printf(" partial_len %" PRIu32 "\n", an->partial_len); printf(" partial ["); for (int i = 0; i < 10; i++) { printf("%c ", an->partial[i]); } printf("\n]"); printf("\n};\n"); return 0; } #endif static int examine_art_leaf(char *appname, struct examine_ctx *ctx, off_t off) { art_leaf *al = (art_leaf *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, art_leaf {\n", (long long)off); dump_PMEMoid(" var_string key oid ", &(al->key.oid)); dump_PMEMoid(" var_string value oid", &(al->value.oid)); printf("\n};\n"); return 0; } static int examine_var_string(char *appname, struct examine_ctx *ctx, off_t off) { var_string *vs = (var_string *)(ctx->pmem_ctx->addr + off); printf("at offset 0x%llx, var_string {\n", (long long)off); printf(" len %zu s [%s]", vs->len, vs->s); printf("\n};\n"); return 0; } pmdk-1.8/src/examples/libpmemobj/libart/Makefile0000664000000000000000000000461513615011243020461 0ustar rootroot# # Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH # Copyright 2016-2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # =========================================================================== # # Filename: Makefile # # Description: build libart port to libpmemobj # # Author: Andreas Bluemle, Dieter Kasper # Andreas.Bluemle.external@ts.fujitsu.com # dieter.kasper@ts.fujitsu.com # # Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH # # =========================================================================== # include ../../../common.inc ifeq ($(ARCH), x86_64) # libart uses x86 intrinsics PROGS = arttree examine_arttree LIBRARIES = art endif LIBS = -lpmemobj include ../../Makefile.inc $(PROGS): | $(DYNAMIC_LIBRARIES) $(PROGS): LDFLAGS += -Wl,-rpath=. -L. -lart libart.o: art.o arttree: arttree.o examine_arttree: arttree_structures.o arttree_examine.o arttree_search.o pmdk-1.8/src/examples/libpmemobj/libart/arttree.h0000664000000000000000000000432013615011243020631 0ustar rootroot/* * Copyright 2016, FUJITSU TECHNOLOGY SOLUTIONS GMBH * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * =========================================================================== * * Filename: arttree.h * * Description: header file for art tree on pmem implementation * * Author: Andreas Bluemle, Dieter Kasper * Andreas.Bluemle.external@ts.fujitsu.com * dieter.kasper@ts.fujitsu.com * * Organization: FUJITSU TECHNOLOGY SOLUTIONS GMBH * * =========================================================================== */ #ifndef _ARTTREE_H #define _ARTTREE_H #ifdef __cplusplus extern "C" { #endif #include "art.h" #ifdef __cplusplus } #endif #endif /* _ARTTREE_H */ pmdk-1.8/src/examples/libpmemobj/map/0000775000000000000000000000000013615011243016313 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/map/map_hashmap_rp.h0000664000000000000000000000355713615011243021455 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_hashmap_rp.h -- common interface for maps */ #ifndef MAP_HASHMAP_RP_H #define MAP_HASHMAP_RP_H #include "map.h" #ifdef __cplusplus extern "C" { #endif extern struct map_ops hashmap_rp_ops; #define MAP_HASHMAP_RP (&hashmap_rp_ops) #ifdef __cplusplus } #endif #endif /* MAP_HASHMAP_RP_H */ pmdk-1.8/src/examples/libpmemobj/map/mapcli.vcxproj0000664000000000000000000000600113615011243021172 0ustar rootroot Debug x64 Release x64 {BB248BAC-6E1B-433C-A254-75140A273AB5} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) {49a7cc5a-d5e7-4a07-917f-c6918b982be8} pmdk-1.8/src/examples/libpmemobj/map/map_skiplist.h0000664000000000000000000000344413615011243021170 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_skiplist.h -- common interface for maps */ #ifndef MAP_SKIPLIST_H #define MAP_SKIPLIST_H #include "map.h" extern struct map_ops skiplist_map_ops; #define MAP_SKIPLIST (&skiplist_map_ops) #endif /* MAP_SKIPLIST_H */ pmdk-1.8/src/examples/libpmemobj/map/libmap.vcxproj.filters0000664000000000000000000000403213615011243022642 0ustar rootroot {b263d2a8-4e00-4310-ae46-f24fc6aa8f4f} {f7a2eae5-3cc9-452a-801d-73ca2d879931} Header Files Header Files Header Files Header Files Header Files Header Files Header Files Header Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files Source Files pmdk-1.8/src/examples/libpmemobj/map/map_ctree.c0000664000000000000000000001274613615011243020430 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_ctree.c -- common interface for maps */ #include #include #include "map_ctree.h" /* * map_ctree_check -- wrapper for ctree_map_check */ static int map_ctree_check(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_check(pop, ctree_map); } /* * map_ctree_create -- wrapper for ctree_map_create */ static int map_ctree_create(PMEMobjpool *pop, TOID(struct map) *map, void *arg) { TOID(struct ctree_map) *ctree_map = (TOID(struct ctree_map) *)map; return ctree_map_create(pop, ctree_map, arg); } /* * map_ctree_destroy -- wrapper for ctree_map_destroy */ static int map_ctree_destroy(PMEMobjpool *pop, TOID(struct map) *map) { TOID(struct ctree_map) *ctree_map = (TOID(struct ctree_map) *)map; return ctree_map_destroy(pop, ctree_map); } /* * map_ctree_insert -- wrapper for ctree_map_insert */ static int map_ctree_insert(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_insert(pop, ctree_map, key, value); } /* * map_ctree_insert_new -- wrapper for ctree_map_insert_new */ static int map_ctree_insert_new(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_insert_new(pop, ctree_map, key, size, type_num, constructor, arg); } /* * map_ctree_remove -- wrapper for ctree_map_remove */ static PMEMoid map_ctree_remove(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_remove(pop, ctree_map, key); } /* * map_ctree_remove_free -- wrapper for ctree_map_remove_free */ static int map_ctree_remove_free(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_remove_free(pop, ctree_map, key); } /* * map_ctree_clear -- wrapper for ctree_map_clear */ static int map_ctree_clear(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_clear(pop, ctree_map); } /* * map_ctree_get -- wrapper for ctree_map_get */ static PMEMoid map_ctree_get(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_get(pop, ctree_map, key); } /* * map_ctree_lookup -- wrapper for ctree_map_lookup */ static int map_ctree_lookup(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_lookup(pop, ctree_map, key); } /* * map_ctree_foreach -- wrapper for ctree_map_foreach */ static int map_ctree_foreach(PMEMobjpool *pop, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_foreach(pop, ctree_map, cb, arg); } /* * map_ctree_is_empty -- wrapper for ctree_map_is_empty */ static int map_ctree_is_empty(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct ctree_map) ctree_map; TOID_ASSIGN(ctree_map, map.oid); return ctree_map_is_empty(pop, ctree_map); } struct map_ops ctree_map_ops = { /* .check = */ map_ctree_check, /* .create = */ map_ctree_create, /* .destroy = */ map_ctree_destroy, /* .init = */ NULL, /* .insert = */ map_ctree_insert, /* .insert_new = */ map_ctree_insert_new, /* .remove = */ map_ctree_remove, /* .remove_free = */ map_ctree_remove_free, /* .clear = */ map_ctree_clear, /* .get = */ map_ctree_get, /* .lookup = */ map_ctree_lookup, /* .foreach = */ map_ctree_foreach, /* .is_empty = */ map_ctree_is_empty, /* .count = */ NULL, /* .cmd = */ NULL, }; pmdk-1.8/src/examples/libpmemobj/map/README0000664000000000000000000000303013615011243017167 0ustar rootrootThis directory contains an example application implemented using libpmemobj. The *mapcli* application is a simple CLI application which uses: * three implementations of hashmap: ** hashmap_atomic - hashmap using atomic API of libpmemobj ** hashmap_tx - hashmap using tx API of libpmemobj ** hashmap_rp - hashmap using action API of libpmemobj * four implementations of tree maps: ** ctree - Crit-Bit using tx API of libpmemobj ** btree - B-tree using tx API of libpmemobj ** rtree - Radix-tree using tx API of libpmemobj ** rbtree - red-black tree using tx API of libpmemobj Usage: $ ./mapcli ctree|btree|rtree|rbtree|hashmap_atomic|hashmap_tx|hashmap_rp [] The first argument specifies which map should be used. The file will either be created if it doesn't exist or opened if it contains a valid pool. The third argument specifies seed for RNG - the seed is utilized by three of hashmaps implementations. The application expects one of the below commands on standard input: h - help i $value - insert $value r $value - remove $value c $value - check $value, returns 0/1 n $value - insert $value random values p - print all values d - print debug info b - rebuild q - quit ** NOTE: ** Please note that some of functions may not be implemented by all types of map. In such case the application will abort with proper message. ** DEPENDENCIES: ** In order to build kv_server you need to install libuv development package. rpm-based systems : libuv-devel dpkg-based systems: libuvX-dev (where X is the API/ABI version) pmdk-1.8/src/examples/libpmemobj/map/map_rtree.c0000664000000000000000000001410713615011243020440 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_rtree.c -- common interface for maps */ #include #include "map_rtree.h" /* * map_rtree_check -- wrapper for rtree_map_check */ static int map_rtree_check(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_check(pop, rtree_map); } /* * map_rtree_create -- wrapper for rtree_map_new */ static int map_rtree_create(PMEMobjpool *pop, TOID(struct map) *map, void *arg) { TOID(struct rtree_map) *rtree_map = (TOID(struct rtree_map) *)map; return rtree_map_create(pop, rtree_map, arg); } /* * map_rtree_destroy -- wrapper for rtree_map_delete */ static int map_rtree_destroy(PMEMobjpool *pop, TOID(struct map) *map) { TOID(struct rtree_map) *rtree_map = (TOID(struct rtree_map) *)map; return rtree_map_destroy(pop, rtree_map); } /* * map_rtree_insert -- wrapper for rtree_map_insert */ static int map_rtree_insert(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_insert(pop, rtree_map, (unsigned char *)&key, sizeof(key), value); } /* * map_rtree_insert_new -- wrapper for rtree_map_insert_new */ static int map_rtree_insert_new(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_insert_new(pop, rtree_map, (unsigned char *)&key, sizeof(key), size, type_num, constructor, arg); } /* * map_rtree_remove -- wrapper for rtree_map_remove */ static PMEMoid map_rtree_remove(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_remove(pop, rtree_map, (unsigned char *)&key, sizeof(key)); } /* * map_rtree_remove_free -- wrapper for rtree_map_remove_free */ static int map_rtree_remove_free(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_remove_free(pop, rtree_map, (unsigned char *)&key, sizeof(key)); } /* * map_rtree_clear -- wrapper for rtree_map_clear */ static int map_rtree_clear(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_clear(pop, rtree_map); } /* * map_rtree_get -- wrapper for rtree_map_get */ static PMEMoid map_rtree_get(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_get(pop, rtree_map, (unsigned char *)&key, sizeof(key)); } /* * map_rtree_lookup -- wrapper for rtree_map_lookup */ static int map_rtree_lookup(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_lookup(pop, rtree_map, (unsigned char *)&key, sizeof(key)); } struct cb_arg2 { int (*cb)(uint64_t key, PMEMoid value, void *arg); void *arg; }; /* * map_rtree_foreach_cb -- wrapper for callback */ static int map_rtree_foreach_cb(const unsigned char *key, uint64_t key_size, PMEMoid value, void *arg2) { const struct cb_arg2 *const a2 = (const struct cb_arg2 *)arg2; const uint64_t *const k2 = (uint64_t *)key; return a2->cb(*k2, value, a2->arg); } /* * map_rtree_foreach -- wrapper for rtree_map_foreach */ static int map_rtree_foreach(PMEMobjpool *pop, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { struct cb_arg2 arg2 = {cb, arg}; TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_foreach(pop, rtree_map, map_rtree_foreach_cb, &arg2); } /* * map_rtree_is_empty -- wrapper for rtree_map_is_empty */ static int map_rtree_is_empty(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct rtree_map) rtree_map; TOID_ASSIGN(rtree_map, map.oid); return rtree_map_is_empty(pop, rtree_map); } struct map_ops rtree_map_ops = { /* .check = */map_rtree_check, /* .create = */map_rtree_create, /* .destroy = */map_rtree_destroy, /* .init = */NULL, /* .insert = */map_rtree_insert, /* .insert_new = */map_rtree_insert_new, /* .remove = */map_rtree_remove, /* .remove_free = */map_rtree_remove_free, /* .clear = */map_rtree_clear, /* .get = */map_rtree_get, /* .lookup = */map_rtree_lookup, /* .foreach = */map_rtree_foreach, /* .is_empty = */map_rtree_is_empty, /* .count = */NULL, /* .cmd = */NULL, }; pmdk-1.8/src/examples/libpmemobj/map/map_hashmap_tx.c0000664000000000000000000001134213615011243021451 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_hashmap_tx.c -- common interface for maps */ #include #include #include "map_hashmap_tx.h" /* * map_hm_tx_check -- wrapper for hm_tx_check */ static int map_hm_tx_check(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_check(pop, hashmap_tx); } /* * map_hm_tx_count -- wrapper for hm_tx_count */ static size_t map_hm_tx_count(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_count(pop, hashmap_tx); } /* * map_hm_tx_init -- wrapper for hm_tx_init */ static int map_hm_tx_init(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_init(pop, hashmap_tx); } /* * map_hm_tx_create -- wrapper for hm_tx_create */ static int map_hm_tx_create(PMEMobjpool *pop, TOID(struct map) *map, void *arg) { TOID(struct hashmap_tx) *hashmap_tx = (TOID(struct hashmap_tx) *)map; return hm_tx_create(pop, hashmap_tx, arg); } /* * map_hm_tx_insert -- wrapper for hm_tx_insert */ static int map_hm_tx_insert(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_insert(pop, hashmap_tx, key, value); } /* * map_hm_tx_remove -- wrapper for hm_tx_remove */ static PMEMoid map_hm_tx_remove(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_remove(pop, hashmap_tx, key); } /* * map_hm_tx_get -- wrapper for hm_tx_get */ static PMEMoid map_hm_tx_get(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_get(pop, hashmap_tx, key); } /* * map_hm_tx_lookup -- wrapper for hm_tx_lookup */ static int map_hm_tx_lookup(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_lookup(pop, hashmap_tx, key); } /* * map_hm_tx_foreach -- wrapper for hm_tx_foreach */ static int map_hm_tx_foreach(PMEMobjpool *pop, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_foreach(pop, hashmap_tx, cb, arg); } /* * map_hm_tx_cmd -- wrapper for hm_tx_cmd */ static int map_hm_tx_cmd(PMEMobjpool *pop, TOID(struct map) map, unsigned cmd, uint64_t arg) { TOID(struct hashmap_tx) hashmap_tx; TOID_ASSIGN(hashmap_tx, map.oid); return hm_tx_cmd(pop, hashmap_tx, cmd, arg); } struct map_ops hashmap_tx_ops = { /* .check = */ map_hm_tx_check, /* .create = */ map_hm_tx_create, /* .delete = */ NULL, /* .init = */ map_hm_tx_init, /* .insert = */ map_hm_tx_insert, /* .insert_new = */ NULL, /* .remove = */ map_hm_tx_remove, /* .remove_free = */ NULL, /* .clear = */ NULL, /* .get = */ map_hm_tx_get, /* .lookup = */ map_hm_tx_lookup, /* .foreach = */ map_hm_tx_foreach, /* .is_empty = */ NULL, /* .count = */ map_hm_tx_count, /* .cmd = */ map_hm_tx_cmd, }; pmdk-1.8/src/examples/libpmemobj/map/map_hashmap_tx.h0000664000000000000000000000356413615011243021465 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_hashmap_tx.h -- common interface for maps */ #ifndef MAP_HASHMAP_TX_H #define MAP_HASHMAP_TX_H #include "map.h" #ifdef __cplusplus extern "C" { #endif extern struct map_ops hashmap_tx_ops; #define MAP_HASHMAP_TX (&hashmap_tx_ops) #ifdef __cplusplus } #endif #endif /* MAP_HASHMAP_TX_H */ pmdk-1.8/src/examples/libpmemobj/map/kv_server.c0000664000000000000000000002713313615011243020473 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * kv_server.c -- persistent tcp key-value store server */ #include #include #include #include #include #include #include "libpmemobj.h" #include "map.h" #include "map_ctree.h" #include "map_btree.h" #include "map_rtree.h" #include "map_rbtree.h" #include "map_hashmap_atomic.h" #include "map_hashmap_tx.h" #include "map_hashmap_rp.h" #include "map_skiplist.h" #include "kv_protocol.h" #define COUNT_OF(x) (sizeof(x) / sizeof(0[x])) #define COMPILE_ERROR_ON(cond) ((void)sizeof(char[(cond) ? -1 : 1])) POBJ_LAYOUT_BEGIN(kv_server); POBJ_LAYOUT_ROOT(kv_server, struct root); POBJ_LAYOUT_TOID(kv_server, struct map_value); POBJ_LAYOUT_TOID(kv_server, uint64_t); POBJ_LAYOUT_END(kv_server); struct map_value { uint64_t len; char buf[]; }; struct root { TOID(struct map) map; }; static struct map_ctx *mapc; static PMEMobjpool *pop; static TOID(struct map) map; static uv_tcp_t server; static uv_loop_t *loop; typedef int (*msg_handler)(uv_stream_t *client, const char *msg, size_t len); struct write_req { uv_write_t req; uv_buf_t buf; }; struct client_data { char *buf; /* current message, always NULL terminated */ size_t buf_len; /* sizeof(buf) */ size_t len; /* actual length of the message (while parsing) */ }; /* * djb2_hash -- string hashing function by Dan Bernstein */ static uint32_t djb2_hash(const char *str) { uint32_t hash = 5381; int c; while ((c = *str++)) hash = ((hash << 5) + hash) + c; return hash; } /* * write_done_cb -- callback after message write completes */ static void write_done_cb(uv_write_t *req, int status) { struct write_req *wr = (struct write_req *)req; free(wr); if (status == -1) { printf("response failed"); } } /* * client_close_cb -- callback after client tcp connection closes */ static void client_close_cb(uv_handle_t *handle) { struct client_data *d = handle->data; free(d->buf); free(handle->data); free(handle); } /* * response_write -- response writing helper */ static void response_write(uv_stream_t *client, char *resp, size_t len) { struct write_req *wr = malloc(sizeof(struct write_req)); assert(wr != NULL); wr->buf = uv_buf_init(resp, len); uv_write(&wr->req, client, &wr->buf, 1, write_done_cb); } /* * response_msg -- predefined message writing helper */ static void response_msg(uv_stream_t *client, enum resp_messages msg) { response_write(client, (char *)resp_msg[msg], strlen(resp_msg[msg])); } /* * cmsg_insert_handler -- handler of INSERT client message */ static int cmsg_insert_handler(uv_stream_t *client, const char *msg, size_t len) { int result = 0; TX_BEGIN(pop) { /* * For simplicity sake the length of the value buffer is just * a length of the message. */ TOID(struct map_value) val = TX_ZALLOC(struct map_value, sizeof(struct map_value) + len); char key[MAX_KEY_LEN]; int ret = sscanf(msg, "INSERT %254s %s\n", key, D_RW(val)->buf); assert(ret == 2); D_RW(val)->len = len; /* properly terminate the value */ D_RW(val)->buf[strlen(D_RO(val)->buf)] = '\n'; map_insert(mapc, map, djb2_hash(key), val.oid); } TX_ONABORT { result = 1; } TX_END response_msg(client, result); return 0; } /* * cmsg_remove_handler -- handler of REMOVE client message */ static int cmsg_remove_handler(uv_stream_t *client, const char *msg, size_t len) { char key[MAX_KEY_LEN] = {0}; /* check if the constant used in sscanf() below has the correct value */ COMPILE_ERROR_ON(MAX_KEY_LEN - 1 != 254); int ret = sscanf(msg, "REMOVE %254s\n", key); assert(ret == 1); int result = map_remove_free(mapc, map, djb2_hash(key)); response_msg(client, result); return 0; } /* * cmsg_get_handler -- handler of GET client message */ static int cmsg_get_handler(uv_stream_t *client, const char *msg, size_t len) { char key[MAX_KEY_LEN]; /* check if the constant used in sscanf() below has the correct value */ COMPILE_ERROR_ON(MAX_KEY_LEN - 1 != 254); int ret = sscanf(msg, "GET %254s\n", key); assert(ret == 1); TOID(struct map_value) value; TOID_ASSIGN(value, map_get(mapc, map, djb2_hash(key))); if (TOID_IS_NULL(value)) { response_msg(client, RESP_MSG_NULL); } else { response_write(client, D_RW(value)->buf, D_RO(value)->len); } return 0; } /* * cmsg_bye_handler -- handler of BYE client message */ static int cmsg_bye_handler(uv_stream_t *client, const char *msg, size_t len) { uv_close((uv_handle_t *)client, client_close_cb); return 0; } /* * cmsg_bye_handler -- handler of KILL client message */ static int cmsg_kill_handler(uv_stream_t *client, const char *msg, size_t len) { uv_close((uv_handle_t *)client, client_close_cb); uv_close((uv_handle_t *)&server, NULL); return 0; } /* kv protocol implementation */ static msg_handler protocol_impl[MAX_CMSG] = { cmsg_insert_handler, cmsg_remove_handler, cmsg_get_handler, cmsg_bye_handler, cmsg_kill_handler }; /* * cmsg_handle -- handles current client message */ static int cmsg_handle(uv_stream_t *client, struct client_data *data) { int ret = 0; int i; for (i = 0; i < MAX_CMSG; ++i) if (strncmp(kv_cmsg_token[i], data->buf, strlen(kv_cmsg_token[i])) == 0) break; if (i == MAX_CMSG) { response_msg(client, RESP_MSG_UNKNOWN); } else { ret = protocol_impl[i](client, data->buf, data->len); } data->len = 0; /* reset the message length */ return ret; } /* * cmsg_handle_stream -- handle incoming tcp stream from clients */ static int cmsg_handle_stream(uv_stream_t *client, struct client_data *data, const char *buf, ssize_t nread) { char *last; int ret; size_t len; /* * A single read operation can contain zero or more operations, so this * has to be handled appropriately. Client messages are terminated by * newline character. */ while ((last = memchr(buf, '\n', nread)) != NULL) { len = last - buf + 1; nread -= len; assert(data->len + len <= data->buf_len); memcpy(data->buf + data->len, buf, len); data->len += len; if ((ret = cmsg_handle(client, data)) != 0) return ret; buf = last + 1; } if (nread != 0) { memcpy(data->buf + data->len, buf, nread); data->len += nread; } return 0; } static uv_buf_t msg_buf = {0}; /* * get_read_buf_cb -- returns buffer for incoming client message */ static void get_read_buf_cb(uv_handle_t *handle, size_t size, uv_buf_t *buf) { buf->base = msg_buf.base; buf->len = msg_buf.len; } /* * read_cb -- async tcp read from clients */ static void read_cb(uv_stream_t *client, ssize_t nread, const uv_buf_t *buf) { if (nread <= 0) { printf("client connection closed\n"); uv_close((uv_handle_t *)client, client_close_cb); return; } struct client_data *d = client->data; if (d->buf_len < (d->len + nread + 1)) { char *cbuf = realloc(d->buf, d->buf_len + nread + 1); assert(cbuf != NULL); /* zero only the new memory */ memset(cbuf + d->buf_len, 0, nread + 1); d->buf_len += nread + 1; d->buf = cbuf; } if (cmsg_handle_stream(client, client->data, buf->base, nread)) { printf("client disconnect\n"); uv_close((uv_handle_t *)client, client_close_cb); } } /* * connection_cb -- async incoming client request */ static void connection_cb(uv_stream_t *server, int status) { if (status != 0) { printf("client connect error\n"); return; } printf("new client\n"); uv_tcp_t *client = malloc(sizeof(uv_tcp_t)); assert(client != NULL); client->data = calloc(1, sizeof(struct client_data)); assert(client->data != NULL); uv_tcp_init(loop, client); if (uv_accept(server, (uv_stream_t *)client) == 0) { uv_read_start((uv_stream_t *)client, get_read_buf_cb, read_cb); } else { uv_close((uv_handle_t *)client, client_close_cb); } } static const struct { struct map_ops *ops; const char *name; } maps[] = { {MAP_HASHMAP_TX, "hashmap_tx"}, {MAP_HASHMAP_ATOMIC, "hashmap_atomic"}, {MAP_HASHMAP_RP, "hashmap_rp"}, {MAP_CTREE, "ctree"}, {MAP_BTREE, "btree"}, {MAP_RTREE, "rtree"}, {MAP_RBTREE, "rbtree"}, {MAP_SKIPLIST, "skiplist"} }; /* * get_map_ops_by_string -- parse the type string and return the associated ops */ static const struct map_ops * get_map_ops_by_string(const char *type) { for (int i = 0; i < COUNT_OF(maps); ++i) if (strcmp(maps[i].name, type) == 0) return maps[i].ops; return NULL; } #define KV_SIZE (PMEMOBJ_MIN_POOL) #define MAX_READ_LEN (64 * 1024) /* 64 kilobytes */ int main(int argc, char *argv[]) { if (argc < 4) { printf("usage: %s hashmap_tx|hashmap_atomic|hashmap_rp|" "ctree|btree|rtree|rbtree|skiplist file-name port\n", argv[0]); return 1; } const char *path = argv[2]; const char *type = argv[1]; int port = atoi(argv[3]); /* use only a single buffer for all incoming data */ void *read_buf = malloc(MAX_READ_LEN); assert(read_buf != NULL); msg_buf = uv_buf_init(read_buf, MAX_READ_LEN); if (access(path, F_OK) != 0) { pop = pmemobj_create(path, POBJ_LAYOUT_NAME(kv_server), KV_SIZE, 0666); if (pop == NULL) { fprintf(stderr, "failed to create pool: %s\n", pmemobj_errormsg()); return 1; } } else { pop = pmemobj_open(path, POBJ_LAYOUT_NAME(kv_server)); if (pop == NULL) { fprintf(stderr, "failed to open pool: %s\n", pmemobj_errormsg()); return 1; } } /* map context initialization */ mapc = map_ctx_init(get_map_ops_by_string(type), pop); if (!mapc) { pmemobj_close(pop); fprintf(stderr, "map_ctx_init failed (wrong type?)\n"); return 1; } /* initialize the actual map */ TOID(struct root) root = POBJ_ROOT(pop, struct root); if (TOID_IS_NULL(D_RO(root)->map)) { /* create new if it doesn't exist (a fresh pool) */ map_create(mapc, &D_RW(root)->map, NULL); } map = D_RO(root)->map; loop = uv_default_loop(); /* tcp server initialization */ uv_tcp_init(loop, &server); struct sockaddr_in bind_addr; uv_ip4_addr("0.0.0.0", port, &bind_addr); int ret = uv_tcp_bind(&server, (const struct sockaddr *)&bind_addr, 0); assert(ret == 0); ret = uv_listen((uv_stream_t *)&server, SOMAXCONN, connection_cb); assert(ret == 0); ret = uv_run(loop, UV_RUN_DEFAULT); assert(ret == 0); /* no more events in the loop, release resources and quit */ uv_loop_delete(loop); map_ctx_free(mapc); pmemobj_close(pop); free(read_buf); return 0; } pmdk-1.8/src/examples/libpmemobj/map/data_store.vcxproj.filters0000664000000000000000000000064713615011243023533 0ustar rootroot {5b0d30df-c510-411d-8529-c3695816137d} Source Files pmdk-1.8/src/examples/libpmemobj/map/map_skiplist.c0000664000000000000000000001356313615011243021166 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_skiplist.c -- common interface for maps */ #include #include #include "map_skiplist.h" /* * map_skiplist_check -- wrapper for skiplist_map_check */ static int map_skiplist_check(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_check(pop, skiplist_map); } /* * map_skiplist_create -- wrapper for skiplist_map_new */ static int map_skiplist_create(PMEMobjpool *pop, TOID(struct map) *map, void *arg) { TOID(struct skiplist_map_node) *skiplist_map = (TOID(struct skiplist_map_node) *)map; return skiplist_map_create(pop, skiplist_map, arg); } /* * map_skiplist_destroy -- wrapper for skiplist_map_delete */ static int map_skiplist_destroy(PMEMobjpool *pop, TOID(struct map) *map) { TOID(struct skiplist_map_node) *skiplist_map = (TOID(struct skiplist_map_node) *)map; return skiplist_map_destroy(pop, skiplist_map); } /* * map_skiplist_insert -- wrapper for skiplist_map_insert */ static int map_skiplist_insert(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_insert(pop, skiplist_map, key, value); } /* * map_skiplist_insert_new -- wrapper for skiplist_map_insert_new */ static int map_skiplist_insert_new(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_insert_new(pop, skiplist_map, key, size, type_num, constructor, arg); } /* * map_skiplist_remove -- wrapper for skiplist_map_remove */ static PMEMoid map_skiplist_remove(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_remove(pop, skiplist_map, key); } /* * map_skiplist_remove_free -- wrapper for skiplist_map_remove_free */ static int map_skiplist_remove_free(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_remove_free(pop, skiplist_map, key); } /* * map_skiplist_clear -- wrapper for skiplist_map_clear */ static int map_skiplist_clear(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_clear(pop, skiplist_map); } /* * map_skiplist_get -- wrapper for skiplist_map_get */ static PMEMoid map_skiplist_get(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_get(pop, skiplist_map, key); } /* * map_skiplist_lookup -- wrapper for skiplist_map_lookup */ static int map_skiplist_lookup(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_lookup(pop, skiplist_map, key); } /* * map_skiplist_foreach -- wrapper for skiplist_map_foreach */ static int map_skiplist_foreach(PMEMobjpool *pop, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_foreach(pop, skiplist_map, cb, arg); } /* * map_skiplist_is_empty -- wrapper for skiplist_map_is_empty */ static int map_skiplist_is_empty(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct skiplist_map_node) skiplist_map; TOID_ASSIGN(skiplist_map, map.oid); return skiplist_map_is_empty(pop, skiplist_map); } struct map_ops skiplist_map_ops = { /* .check = */ map_skiplist_check, /* .create = */ map_skiplist_create, /* .destroy = */ map_skiplist_destroy, /* .init = */ NULL, /* .insert = */ map_skiplist_insert, /* .insert_new = */ map_skiplist_insert_new, /* .remove = */ map_skiplist_remove, /* .remove_free = */ map_skiplist_remove_free, /* .clear = */ map_skiplist_clear, /* .get = */ map_skiplist_get, /* .lookup = */ map_skiplist_lookup, /* .foreach = */ map_skiplist_foreach, /* .is_empty = */ map_skiplist_is_empty, /* .count = */ NULL, /* .cmd = */ NULL, }; pmdk-1.8/src/examples/libpmemobj/map/.gitignore0000664000000000000000000000003413615011243020300 0ustar rootrootmapcli data_store kv_server pmdk-1.8/src/examples/libpmemobj/map/data_store.vcxproj0000664000000000000000000000600513615011243022056 0ustar rootroot Debug x64 Release x64 {5B2B9C0D-1B6D-4357-8307-6DE1EE0A41A3} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} {49a7cc5a-d5e7-4a07-917f-c6918b982be8} Application true v140 Application false v140 true ..\..\..\LongPath.manifest CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/map/map_hashmap_rp.c0000664000000000000000000001133613615011243021442 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_hashmap_rp.c -- common interface for maps */ #include #include #include "map_hashmap_rp.h" /* * map_hm_rp_check -- wrapper for hm_rp_check */ static int map_hm_rp_check(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_check(pop, hashmap_rp); } /* * map_hm_rp_count -- wrapper for hm_rp_count */ static size_t map_hm_rp_count(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_count(pop, hashmap_rp); } /* * map_hm_rp_init -- wrapper for hm_rp_init */ static int map_hm_rp_init(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_init(pop, hashmap_rp); } /* * map_hm_rp_create -- wrapper for hm_rp_create */ static int map_hm_rp_create(PMEMobjpool *pop, TOID(struct map) *map, void *arg) { TOID(struct hashmap_rp) *hashmap_rp = (TOID(struct hashmap_rp) *)map; return hm_rp_create(pop, hashmap_rp, arg); } /* * map_hm_rp_insert -- wrapper for hm_rp_insert */ static int map_hm_rp_insert(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_insert(pop, hashmap_rp, key, value); } /* * map_hm_rp_remove -- wrapper for hm_rp_remove */ static PMEMoid map_hm_rp_remove(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_remove(pop, hashmap_rp, key); } /* * map_hm_rp_get -- wrapper for hm_rp_get */ static PMEMoid map_hm_rp_get(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_get(pop, hashmap_rp, key); } /* * map_hm_rp_lookup -- wrapper for hm_rp_lookup */ static int map_hm_rp_lookup(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_lookup(pop, hashmap_rp, key); } /* * map_hm_rp_foreach -- wrapper for hm_rp_foreach */ static int map_hm_rp_foreach(PMEMobjpool *pop, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_foreach(pop, hashmap_rp, cb, arg); } /* * map_hm_rp_cmd -- wrapper for hm_rp_cmd */ static int map_hm_rp_cmd(PMEMobjpool *pop, TOID(struct map) map, unsigned cmd, uint64_t arg) { TOID(struct hashmap_rp) hashmap_rp; TOID_ASSIGN(hashmap_rp, map.oid); return hm_rp_cmd(pop, hashmap_rp, cmd, arg); } struct map_ops hashmap_rp_ops = { /* .check = */ map_hm_rp_check, /* .create = */ map_hm_rp_create, /* .destroy = */ NULL, /* .init = */ map_hm_rp_init, /* .insert = */ map_hm_rp_insert, /* .insert_new = */ NULL, /* .remove = */ map_hm_rp_remove, /* .remove_free = */ NULL, /* .clear = */ NULL, /* .get = */ map_hm_rp_get, /* .lookup = */ map_hm_rp_lookup, /* .foreach = */ map_hm_rp_foreach, /* .is_empty = */ NULL, /* .count = */ map_hm_rp_count, /* .cmd = */ map_hm_rp_cmd, }; pmdk-1.8/src/examples/libpmemobj/map/map.c0000664000000000000000000001312313615011243017234 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map.c -- common interface for maps */ #include #include #include #include "map.h" #define ABORT_NOT_IMPLEMENTED(mapc, func)\ if ((mapc)->ops->func == NULL) {\ fprintf(stderr, "error: '%s'"\ " function not implemented\n", #func);\ exit(1);\ } /* * map_ctx_init -- initialize map context */ struct map_ctx * map_ctx_init(const struct map_ops *ops, PMEMobjpool *pop) { if (!ops) return NULL; struct map_ctx *mapc = (struct map_ctx *)calloc(1, sizeof(*mapc)); if (!mapc) return NULL; mapc->ops = ops; mapc->pop = pop; return mapc; } /* * map_ctx_free -- free map context */ void map_ctx_free(struct map_ctx *mapc) { free(mapc); } /* * map_create -- create new map */ int map_create(struct map_ctx *mapc, TOID(struct map) *map, void *arg) { ABORT_NOT_IMPLEMENTED(mapc, create); return mapc->ops->create(mapc->pop, map, arg); } /* * map_destroy -- free the map */ int map_destroy(struct map_ctx *mapc, TOID(struct map) *map) { ABORT_NOT_IMPLEMENTED(mapc, destroy); return mapc->ops->destroy(mapc->pop, map); } /* * map_init -- initialize map */ int map_init(struct map_ctx *mapc, TOID(struct map) map) { ABORT_NOT_IMPLEMENTED(mapc, init); return mapc->ops->init(mapc->pop, map); } /* * map_check -- check if persistent object is a valid map object */ int map_check(struct map_ctx *mapc, TOID(struct map) map) { ABORT_NOT_IMPLEMENTED(mapc, check); return mapc->ops->check(mapc->pop, map); } /* * map_insert -- insert key value pair */ int map_insert(struct map_ctx *mapc, TOID(struct map) map, uint64_t key, PMEMoid value) { ABORT_NOT_IMPLEMENTED(mapc, insert); return mapc->ops->insert(mapc->pop, map, key, value); } /* * map_insert_new -- allocate and insert key value pair */ int map_insert_new(struct map_ctx *mapc, TOID(struct map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { ABORT_NOT_IMPLEMENTED(mapc, insert_new); return mapc->ops->insert_new(mapc->pop, map, key, size, type_num, constructor, arg); } /* * map_remove -- remove key value pair */ PMEMoid map_remove(struct map_ctx *mapc, TOID(struct map) map, uint64_t key) { ABORT_NOT_IMPLEMENTED(mapc, remove); return mapc->ops->remove(mapc->pop, map, key); } /* * map_remove_free -- remove and free key value pair */ int map_remove_free(struct map_ctx *mapc, TOID(struct map) map, uint64_t key) { ABORT_NOT_IMPLEMENTED(mapc, remove_free); return mapc->ops->remove_free(mapc->pop, map, key); } /* * map_clear -- remove all key value pairs from map */ int map_clear(struct map_ctx *mapc, TOID(struct map) map) { ABORT_NOT_IMPLEMENTED(mapc, clear); return mapc->ops->clear(mapc->pop, map); } /* * map_get -- get value of specified key */ PMEMoid map_get(struct map_ctx *mapc, TOID(struct map) map, uint64_t key) { ABORT_NOT_IMPLEMENTED(mapc, get); return mapc->ops->get(mapc->pop, map, key); } /* * map_lookup -- check if specified key exists in map */ int map_lookup(struct map_ctx *mapc, TOID(struct map) map, uint64_t key) { ABORT_NOT_IMPLEMENTED(mapc, lookup); return mapc->ops->lookup(mapc->pop, map, key); } /* * map_foreach -- iterate through all key value pairs in a map */ int map_foreach(struct map_ctx *mapc, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { ABORT_NOT_IMPLEMENTED(mapc, foreach); return mapc->ops->foreach(mapc->pop, map, cb, arg); } /* * map_is_empty -- check if map is empty */ int map_is_empty(struct map_ctx *mapc, TOID(struct map) map) { ABORT_NOT_IMPLEMENTED(mapc, is_empty); return mapc->ops->is_empty(mapc->pop, map); } /* * map_count -- get number of key value pairs in map */ size_t map_count(struct map_ctx *mapc, TOID(struct map) map) { ABORT_NOT_IMPLEMENTED(mapc, count); return mapc->ops->count(mapc->pop, map); } /* * map_cmd -- execute command specific for map type */ int map_cmd(struct map_ctx *mapc, TOID(struct map) map, unsigned cmd, uint64_t arg) { ABORT_NOT_IMPLEMENTED(mapc, cmd); return mapc->ops->cmd(mapc->pop, map, cmd, arg); } pmdk-1.8/src/examples/libpmemobj/map/map_rbtree.h0000664000000000000000000000354013615011243020606 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_rbtree.h -- common interface for maps */ #ifndef MAP_RBTREE_H #define MAP_RBTREE_H #include "map.h" #ifdef __cplusplus extern "C" { #endif extern struct map_ops rbtree_map_ops; #define MAP_RBTREE (&rbtree_map_ops) #ifdef __cplusplus } #endif #endif /* MAP_RBTREE_H */ pmdk-1.8/src/examples/libpmemobj/map/kv_server_test.sh0000775000000000000000000000375113615011243021725 0ustar rootroot#!/usr/bin/env bash # # Copyright 2015-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # set -euo pipefail MAP=ctree PORT=9100 POOL=$1 # start a new server instance ./kv_server $MAP $POOL $PORT & # wait for the server to properly start sleep 1 # insert a new key value pair and disconnect RESP=`echo -e "INSERT foo bar\nGET foo\nBYE" | nc 127.0.0.1 $PORT` echo $RESP # remove previously inserted key value pair and shutdown the server RESP=`echo -e "GET foo\nREMOVE foo\nGET foo\nKILL" | nc 127.0.0.1 $PORT` echo $RESP pmdk-1.8/src/examples/libpmemobj/map/libmap.vcxproj0000664000000000000000000001347513615011243021206 0ustar rootroot Debug x64 Release x64 {49A7CC5A-D5E7-4A07-917F-C6918B982BE8} $(ProjectName) pmemobj 10.0.16299.0 ..\..\..\LongPath.manifest {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} {f5e2f6c4-19ba-497a-b754-232e469be647} {F5E2F6C4-19BA-497A-B754-232E4666E647} {d93a2683-6d99-4f18-b378-91195d23e007} {3799ba67-3c4f-4ae0-85dc-5baaea01a180} {79d37ffe-ff76-44b3-bb27-3dcaeff2ebe9} {be18f227-a9f0-4b38-b689-4e2f9f09ca5f} {17a4b817-68b1-4719-a9ef-bd8fab747de6} {3ed56e55-84a6-422c-a8d4-a8439fb8f245} StaticLibrary true v140 StaticLibrary false v140 true $(solutionDir)include;$(solutionDir)..\include;$(ProjectDir)..\..\;$(ProjectDir)..\list_map;$(ProjectDir)..\hashmap;$(ProjectDir)..\tree_map;$(IncludePath);$(WindowsSDK_IncludePath);$(SolutionDir)..\$(Platform)\$(Configuration);$(ProjectDir) CompileAsCpp {f5e2f6c4-19ba-497a-b754-232e469be647} {F5E2F6C4-19BA-497A-B754-232E4666E647} {d93a2683-6d99-4f18-b378-91195d23e007} {3799ba67-3c4f-4ae0-85dc-5baaea01a180} {79d37ffe-ff76-44b3-bb27-3dcaeff2ebe9} {be18f227-a9f0-4b38-b689-4e2f9f09ca5f} {17a4b817-68b1-4719-a9ef-bd8fab747de6} {3ed56e55-84a6-422c-a8d4-a8439fb8f245} pmdk-1.8/src/examples/libpmemobj/map/kv_protocol.h0000664000000000000000000000701513615011243021030 0ustar rootroot/* * Copyright 2015-2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * kv_protocol.h -- kv store text protocol */ #ifndef KV_PROTOCOL_H #define KV_PROTOCOL_H #include #define MAX_KEY_LEN 255 /* * All client messages must start with a valid message token and be terminated * by a newline character ('\n'). The message parser is case-sensitive. * * Server responds with newline terminated string literals. * If invalid message token is received RESP_MSG_UNKNOWN is sent. */ enum kv_cmsg { /* * INSERT client message * Syntax: INSERT [key] [value]\n * * The key is limited to 255 characters, the size of a value is limited * by the pmemobj maximum allocation size (~16 gigabytes). * * Operation adds a new key value pair to the map. * Returns RESP_MSG_SUCCESS if successful or RESP_MSG_FAIL otherwise. */ CMSG_INSERT, /* * REMOVE client message * Syntax: REMOVE [key]\n * * Operation removes a key value pair from the map. * Returns RESP_MSG_SUCCESS if successful or RESP_MSG_FAIL otherwise. */ CMSG_REMOVE, /* * GET client message * Syntax: GET [key]\n * * Operation retrieves a key value pair from the map. * Returns the value if found or RESP_MSG_NULL otherwise. */ CMSG_GET, /* * BYE client message * Syntax: BYE\n * * Operation terminates the client connection. * No return value. */ CMSG_BYE, /* * KILL client message * Syntax: KILL\n * * Operation terminates the client connection and gracefully shutdowns * the server. * No return value. */ CMSG_KILL, MAX_CMSG }; enum resp_messages { RESP_MSG_SUCCESS, RESP_MSG_FAIL, RESP_MSG_NULL, RESP_MSG_UNKNOWN, MAX_RESP_MSG }; static const char *resp_msg[MAX_RESP_MSG] = { [RESP_MSG_SUCCESS] = "SUCCESS\n", [RESP_MSG_FAIL] = "FAIL\n", [RESP_MSG_NULL] = "NULL\n", [RESP_MSG_UNKNOWN] = "UNKNOWN\n" }; static const char *kv_cmsg_token[MAX_CMSG] = { [CMSG_INSERT] = "INSERT", [CMSG_REMOVE] = "REMOVE", [CMSG_GET] = "GET", [CMSG_BYE] = "BYE", [CMSG_KILL] = "KILL" }; #endif /* KV_PROTOCOL_H */ pmdk-1.8/src/examples/libpmemobj/map/mapcli.c0000664000000000000000000001715213615011243017732 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #include #include #include #include #include #include #include #include #include "map.h" #include "map_ctree.h" #include "map_btree.h" #include "map_rtree.h" #include "map_rbtree.h" #include "map_hashmap_atomic.h" #include "map_hashmap_tx.h" #include "map_hashmap_rp.h" #include "map_skiplist.h" #include "hashmap/hashmap.h" #define PM_HASHSET_POOL_SIZE (160 * 1024 * 1024) POBJ_LAYOUT_BEGIN(map); POBJ_LAYOUT_ROOT(map, struct root); POBJ_LAYOUT_END(map); struct root { TOID(struct map) map; }; static PMEMobjpool *pop; static struct map_ctx *mapc; static TOID(struct root) root; static TOID(struct map) map; /* * str_insert -- hs_insert wrapper which works on strings */ static void str_insert(const char *str) { uint64_t key; if (sscanf(str, "%" PRIu64, &key) > 0) map_insert(mapc, map, key, OID_NULL); else fprintf(stderr, "insert: invalid syntax\n"); } /* * str_remove -- hs_remove wrapper which works on strings */ static void str_remove(const char *str) { uint64_t key; if (sscanf(str, "%" PRIu64, &key) > 0) { int l = map_lookup(mapc, map, key); if (l) map_remove(mapc, map, key); else fprintf(stderr, "no such value\n"); } else fprintf(stderr, "remove: invalid syntax\n"); } /* * str_check -- hs_check wrapper which works on strings */ static void str_check(const char *str) { uint64_t key; if (sscanf(str, "%" PRIu64, &key) > 0) { int r = map_lookup(mapc, map, key); printf("%d\n", r); } else { fprintf(stderr, "check: invalid syntax\n"); } } /* * str_insert_random -- inserts specified (as string) number of random numbers */ static void str_insert_random(const char *str) { uint64_t val; if (sscanf(str, "%" PRIu64, &val) > 0) for (uint64_t i = 0; i < val; ) { uint64_t r = ((uint64_t)rand()) << 32 | rand(); int ret = map_insert(mapc, map, r, OID_NULL); if (ret < 0) break; if (ret == 0) i += 1; } else fprintf(stderr, "random insert: invalid syntax\n"); } /* * rebuild -- rebuilds hashmap and measures execution time */ static void rebuild(void) { printf("rebuild "); fflush(stdout); time_t t1 = time(NULL); map_cmd(mapc, map, HASHMAP_CMD_REBUILD, 0); printf("%" PRIu64"s\n", (uint64_t)(time(NULL) - t1)); } /* * str_rebuild -- hs_rebuild wrapper which executes specified number of times */ static void str_rebuild(const char *str) { uint64_t val; if (sscanf(str, "%" PRIu64, &val) > 0) { for (uint64_t i = 0; i < val; ++i) { printf("%2" PRIu64 " ", i); rebuild(); } } else { rebuild(); } } static void help(void) { printf("h - help\n"); printf("i $value - insert $value\n"); printf("r $value - remove $value\n"); printf("c $value - check $value, returns 0/1\n"); printf("n $value - insert $value random values\n"); printf("p - print all values\n"); printf("d - print debug info\n"); printf("b [$value] - rebuild $value (default: 1) times\n"); printf("q - quit\n"); } static void unknown_command(const char *str) { fprintf(stderr, "unknown command '%c', use 'h' for help\n", str[0]); } static int hashmap_print(uint64_t key, PMEMoid value, void *arg) { printf("%" PRIu64 " ", key); return 0; } static void print_all(void) { if (mapc->ops->count) printf("count: %zu\n", map_count(mapc, map)); map_foreach(mapc, map, hashmap_print, NULL); printf("\n"); } #define INPUT_BUF_LEN 1000 int main(int argc, char *argv[]) { if (argc < 3 || argc > 4) { printf("usage: %s " "hashmap_tx|hashmap_atomic|hashmap_rp|" "ctree|btree|rtree|rbtree|skiplist" " file-name []\n", argv[0]); return 1; } const struct map_ops *ops = NULL; const char *path = argv[2]; const char *type = argv[1]; if (strcmp(type, "hashmap_tx") == 0) { ops = MAP_HASHMAP_TX; } else if (strcmp(type, "hashmap_atomic") == 0) { ops = MAP_HASHMAP_ATOMIC; } else if (strcmp(type, "hashmap_rp") == 0) { ops = MAP_HASHMAP_RP; } else if (strcmp(type, "ctree") == 0) { ops = MAP_CTREE; } else if (strcmp(type, "btree") == 0) { ops = MAP_BTREE; } else if (strcmp(type, "rtree") == 0) { ops = MAP_RTREE; } else if (strcmp(type, "rbtree") == 0) { ops = MAP_RBTREE; } else if (strcmp(type, "skiplist") == 0) { ops = MAP_SKIPLIST; } else { fprintf(stderr, "invalid container type -- '%s'\n", type); return 1; } if (file_exists(path) != 0) { pop = pmemobj_create(path, POBJ_LAYOUT_NAME(map), PM_HASHSET_POOL_SIZE, CREATE_MODE_RW); if (pop == NULL) { fprintf(stderr, "failed to create pool: %s\n", pmemobj_errormsg()); return 1; } struct hashmap_args args; if (argc > 3) args.seed = atoi(argv[3]); else args.seed = (uint32_t)time(NULL); srand(args.seed); mapc = map_ctx_init(ops, pop); if (!mapc) { pmemobj_close(pop); perror("map_ctx_init"); return 1; } root = POBJ_ROOT(pop, struct root); printf("seed: %u\n", args.seed); map_create(mapc, &D_RW(root)->map, &args); map = D_RO(root)->map; } else { pop = pmemobj_open(path, POBJ_LAYOUT_NAME(map)); if (pop == NULL) { fprintf(stderr, "failed to open pool: %s\n", pmemobj_errormsg()); return 1; } mapc = map_ctx_init(ops, pop); if (!mapc) { pmemobj_close(pop); perror("map_ctx_init"); return 1; } root = POBJ_ROOT(pop, struct root); map = D_RO(root)->map; } char buf[INPUT_BUF_LEN]; if (isatty(fileno(stdout))) printf("Type 'h' for help\n$ "); while (fgets(buf, sizeof(buf), stdin)) { if (buf[0] == 0 || buf[0] == '\n') continue; switch (buf[0]) { case 'i': str_insert(buf + 1); break; case 'r': str_remove(buf + 1); break; case 'c': str_check(buf + 1); break; case 'n': str_insert_random(buf + 1); break; case 'p': print_all(); break; case 'd': map_cmd(mapc, map, HASHMAP_CMD_DEBUG, (uint64_t)stdout); break; case 'b': str_rebuild(buf + 1); break; case 'q': fclose(stdin); break; case 'h': help(); break; default: unknown_command(buf); break; } if (isatty(fileno(stdout))) printf("$ "); } map_ctx_free(mapc); pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/map/map.h0000664000000000000000000001065513615011243017250 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map.h -- common interface for maps */ #ifndef MAP_H #define MAP_H #include #ifdef __cplusplus extern "C" { #endif #ifndef MAP_TYPE_OFFSET #define MAP_TYPE_OFFSET 1000 #endif TOID_DECLARE(struct map, MAP_TYPE_OFFSET + 0); struct map; struct map_ctx; struct map_ops { int(*check)(PMEMobjpool *pop, TOID(struct map) map); int(*create)(PMEMobjpool *pop, TOID(struct map) *map, void *arg); int(*destroy)(PMEMobjpool *pop, TOID(struct map) *map); int(*init)(PMEMobjpool *pop, TOID(struct map) map); int(*insert)(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value); int(*insert_new)(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, size_t size, unsigned type_num, void(*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg); PMEMoid(*remove)(PMEMobjpool *pop, TOID(struct map) map, uint64_t key); int(*remove_free)(PMEMobjpool *pop, TOID(struct map) map, uint64_t key); int(*clear)(PMEMobjpool *pop, TOID(struct map) map); PMEMoid(*get)(PMEMobjpool *pop, TOID(struct map) map, uint64_t key); int(*lookup)(PMEMobjpool *pop, TOID(struct map) map, uint64_t key); int(*foreach)(PMEMobjpool *pop, TOID(struct map) map, int(*cb)(uint64_t key, PMEMoid value, void *arg), void *arg); int(*is_empty)(PMEMobjpool *pop, TOID(struct map) map); size_t(*count)(PMEMobjpool *pop, TOID(struct map) map); int(*cmd)(PMEMobjpool *pop, TOID(struct map) map, unsigned cmd, uint64_t arg); }; struct map_ctx { PMEMobjpool *pop; const struct map_ops *ops; }; struct map_ctx *map_ctx_init(const struct map_ops *ops, PMEMobjpool *pop); void map_ctx_free(struct map_ctx *mapc); int map_check(struct map_ctx *mapc, TOID(struct map) map); int map_create(struct map_ctx *mapc, TOID(struct map) *map, void *arg); int map_destroy(struct map_ctx *mapc, TOID(struct map) *map); int map_init(struct map_ctx *mapc, TOID(struct map) map); int map_insert(struct map_ctx *mapc, TOID(struct map) map, uint64_t key, PMEMoid value); int map_insert_new(struct map_ctx *mapc, TOID(struct map) map, uint64_t key, size_t size, unsigned type_num, void(*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg); PMEMoid map_remove(struct map_ctx *mapc, TOID(struct map) map, uint64_t key); int map_remove_free(struct map_ctx *mapc, TOID(struct map) map, uint64_t key); int map_clear(struct map_ctx *mapc, TOID(struct map) map); PMEMoid map_get(struct map_ctx *mapc, TOID(struct map) map, uint64_t key); int map_lookup(struct map_ctx *mapc, TOID(struct map) map, uint64_t key); int map_foreach(struct map_ctx *mapc, TOID(struct map) map, int(*cb)(uint64_t key, PMEMoid value, void *arg), void *arg); int map_is_empty(struct map_ctx *mapc, TOID(struct map) map); size_t map_count(struct map_ctx *mapc, TOID(struct map) map); int map_cmd(struct map_ctx *mapc, TOID(struct map) map, unsigned cmd, uint64_t arg); #ifdef __cplusplus } #endif #endif /* MAP_H */ pmdk-1.8/src/examples/libpmemobj/map/map_rtree.h0000664000000000000000000000353113615011243020444 0ustar rootroot/* * Copyright 2016-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_rtree.h -- common interface for maps */ #ifndef MAP_RTREE_H #define MAP_RTREE_H #include "map.h" #ifdef __cplusplus extern "C" { #endif extern struct map_ops rtree_map_ops; #define MAP_RTREE (&rtree_map_ops) #ifdef __cplusplus } #endif #endif /* MAP_RTREE_H */ pmdk-1.8/src/examples/libpmemobj/map/map_rbtree.c0000664000000000000000000001312213615011243020576 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_rbtree.c -- common interface for maps */ #include #include #include "map_rbtree.h" /* * map_rbtree_check -- wrapper for rbtree_map_check */ static int map_rbtree_check(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_check(pop, rbtree_map); } /* * map_rbtree_create -- wrapper for rbtree_map_new */ static int map_rbtree_create(PMEMobjpool *pop, TOID(struct map) *map, void *arg) { TOID(struct rbtree_map) *rbtree_map = (TOID(struct rbtree_map) *)map; return rbtree_map_create(pop, rbtree_map, arg); } /* * map_rbtree_destroy -- wrapper for rbtree_map_delete */ static int map_rbtree_destroy(PMEMobjpool *pop, TOID(struct map) *map) { TOID(struct rbtree_map) *rbtree_map = (TOID(struct rbtree_map) *)map; return rbtree_map_destroy(pop, rbtree_map); } /* * map_rbtree_insert -- wrapper for rbtree_map_insert */ static int map_rbtree_insert(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_insert(pop, rbtree_map, key, value); } /* * map_rbtree_insert_new -- wrapper for rbtree_map_insert_new */ static int map_rbtree_insert_new(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_insert_new(pop, rbtree_map, key, size, type_num, constructor, arg); } /* * map_rbtree_remove -- wrapper for rbtree_map_remove */ static PMEMoid map_rbtree_remove(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_remove(pop, rbtree_map, key); } /* * map_rbtree_remove_free -- wrapper for rbtree_map_remove_free */ static int map_rbtree_remove_free(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_remove_free(pop, rbtree_map, key); } /* * map_rbtree_clear -- wrapper for rbtree_map_clear */ static int map_rbtree_clear(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_clear(pop, rbtree_map); } /* * map_rbtree_get -- wrapper for rbtree_map_get */ static PMEMoid map_rbtree_get(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_get(pop, rbtree_map, key); } /* * map_rbtree_lookup -- wrapper for rbtree_map_lookup */ static int map_rbtree_lookup(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_lookup(pop, rbtree_map, key); } /* * map_rbtree_foreach -- wrapper for rbtree_map_foreach */ static int map_rbtree_foreach(PMEMobjpool *pop, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_foreach(pop, rbtree_map, cb, arg); } /* * map_rbtree_is_empty -- wrapper for rbtree_map_is_empty */ static int map_rbtree_is_empty(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct rbtree_map) rbtree_map; TOID_ASSIGN(rbtree_map, map.oid); return rbtree_map_is_empty(pop, rbtree_map); } struct map_ops rbtree_map_ops = { /* .check = */ map_rbtree_check, /* .create = */ map_rbtree_create, /* .destroy = */ map_rbtree_destroy, /* .init = */ NULL, /* .insert = */ map_rbtree_insert, /* .insert_new = */ map_rbtree_insert_new, /* .remove = */ map_rbtree_remove, /* .remove_free = */ map_rbtree_remove_free, /* .clear = */ map_rbtree_clear, /* .get = */ map_rbtree_get, /* .lookup = */ map_rbtree_lookup, /* .foreach = */ map_rbtree_foreach, /* .is_empty = */ map_rbtree_is_empty, /* .count = */ NULL, /* .cmd = */ NULL, }; pmdk-1.8/src/examples/libpmemobj/map/mapcli.vcxproj.filters0000664000000000000000000000064313615011243022647 0ustar rootroot {eec7f2ae-38c2-47d2-90b9-3ef11235f61e} Source Files pmdk-1.8/src/examples/libpmemobj/map/map_btree.c0000664000000000000000000001274613615011243020427 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_btree.c -- common interface for maps */ #include #include #include "map_btree.h" /* * map_btree_check -- wrapper for btree_map_check */ static int map_btree_check(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_check(pop, btree_map); } /* * map_btree_create -- wrapper for btree_map_create */ static int map_btree_create(PMEMobjpool *pop, TOID(struct map) *map, void *arg) { TOID(struct btree_map) *btree_map = (TOID(struct btree_map) *)map; return btree_map_create(pop, btree_map, arg); } /* * map_btree_destroy -- wrapper for btree_map_destroy */ static int map_btree_destroy(PMEMobjpool *pop, TOID(struct map) *map) { TOID(struct btree_map) *btree_map = (TOID(struct btree_map) *)map; return btree_map_destroy(pop, btree_map); } /* * map_btree_insert -- wrapper for btree_map_insert */ static int map_btree_insert(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_insert(pop, btree_map, key, value); } /* * map_btree_insert_new -- wrapper for btree_map_insert_new */ static int map_btree_insert_new(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_insert_new(pop, btree_map, key, size, type_num, constructor, arg); } /* * map_btree_remove -- wrapper for btree_map_remove */ static PMEMoid map_btree_remove(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_remove(pop, btree_map, key); } /* * map_btree_remove_free -- wrapper for btree_map_remove_free */ static int map_btree_remove_free(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_remove_free(pop, btree_map, key); } /* * map_btree_clear -- wrapper for btree_map_clear */ static int map_btree_clear(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_clear(pop, btree_map); } /* * map_btree_get -- wrapper for btree_map_get */ static PMEMoid map_btree_get(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_get(pop, btree_map, key); } /* * map_btree_lookup -- wrapper for btree_map_lookup */ static int map_btree_lookup(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_lookup(pop, btree_map, key); } /* * map_btree_foreach -- wrapper for btree_map_foreach */ static int map_btree_foreach(PMEMobjpool *pop, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_foreach(pop, btree_map, cb, arg); } /* * map_btree_is_empty -- wrapper for btree_map_is_empty */ static int map_btree_is_empty(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct btree_map) btree_map; TOID_ASSIGN(btree_map, map.oid); return btree_map_is_empty(pop, btree_map); } struct map_ops btree_map_ops = { /* .check = */ map_btree_check, /* .create = */ map_btree_create, /* .destroy = */ map_btree_destroy, /* .init = */ NULL, /* .insert = */ map_btree_insert, /* .insert_new = */ map_btree_insert_new, /* .remove = */ map_btree_remove, /* .remove_free = */ map_btree_remove_free, /* .clear = */ map_btree_clear, /* .get = */ map_btree_get, /* .lookup = */ map_btree_lookup, /* .foreach = */ map_btree_foreach, /* .is_empty = */ map_btree_is_empty, /* .count = */ NULL, /* .cmd = */ NULL, }; pmdk-1.8/src/examples/libpmemobj/map/map_hashmap_atomic.h0000664000000000000000000000362013615011243022277 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_hashmap_atomic.h -- common interface for maps */ #ifndef MAP_HASHMAP_ATOMIC_H #define MAP_HASHMAP_ATOMIC_H #include "map.h" #ifdef __cplusplus extern "C" { #endif extern struct map_ops hashmap_atomic_ops; #define MAP_HASHMAP_ATOMIC (&hashmap_atomic_ops) #ifdef __cplusplus } #endif #endif /* MAP_HASHMAP_ATOMIC_H */ pmdk-1.8/src/examples/libpmemobj/map/Makefile0000664000000000000000000000740113615011243017755 0ustar rootroot# # Copyright 2015-2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/map/Makefile -- build the map example # TOP := $(dir $(lastword $(MAKEFILE_LIST)))../../../../ include $(TOP)/src/common.inc PROGS = mapcli data_store LIBRARIES = map_ctree map_btree map_rbtree map_skiplist\ map_hashmap_atomic map_hashmap_tx map_hashmap_rp\ map_rtree map LIBUV := $(call check_package, libuv --atleast-version 1.0) ifeq ($(LIBUV),y) PROGS += kv_server else $(info NOTE: Skipping kv_server because libuv is missing \ -- see src/examples/libpmemobj/map/README for details.) endif LIBS = -lpmemobj -pthread ifeq ($(LIBUV),y) LIBS += $(shell $(PKG_CONFIG) --libs libuv) endif include ../../Makefile.inc CFLAGS += -I../ CFLAGS += -I../hashmap CFLAGS += -I../tree_map CFLAGS += -I../list_map mapcli: mapcli.o libmap.a data_store: data_store.o libmap.a kv_server: kv_server.o libmap.a libmap_ctree.o: map_ctree.o map.o ../tree_map/libctree_map.a libmap_btree.o: map_btree.o map.o ../tree_map/libbtree_map.a libmap_rtree.o: map_rtree.o map.o ../tree_map/librtree_map.a libmap_rbtree.o: map_rbtree.o map.o ../tree_map/librbtree_map.a libmap_hashmap_atomic.o: map_hashmap_atomic.o map.o ../hashmap/libhashmap_atomic.a libmap_hashmap_tx.o: map_hashmap_tx.o map.o ../hashmap/libhashmap_tx.a libmap_hashmap_rp.o: map_hashmap_rp.o map.o ../hashmap/libhashmap_rp.a libmap_skiplist.o: map_skiplist.o map.o ../list_map/libskiplist_map.a libmap.o: map.o map_ctree.o map_btree.o map_rtree.o map_rbtree.o map_skiplist.o\ map_hashmap_atomic.o map_hashmap_tx.o map_hashmap_rp.o\ ../tree_map/libctree_map.a\ ../tree_map/libbtree_map.a\ ../tree_map/librtree_map.a\ ../tree_map/librbtree_map.a\ ../list_map/libskiplist_map.a\ ../hashmap/libhashmap_atomic.a\ ../hashmap/libhashmap_tx.a\ ../hashmap/libhashmap_rp.a ../tree_map/libctree_map.a: $(MAKE) -C ../tree_map ctree_map ../tree_map/libbtree_map.a: $(MAKE) -C ../tree_map btree_map ../tree_map/librtree_map.a: $(MAKE) -C ../tree_map rtree_map ../tree_map/librbtree_map.a: $(MAKE) -C ../tree_map rbtree_map ../list_map/libskiplist_map.a: $(MAKE) -C ../list_map skiplist_map ../hashmap/libhashmap_atomic.a: $(MAKE) -C ../hashmap hashmap_atomic ../hashmap/libhashmap_tx.a: $(MAKE) -C ../hashmap hashmap_tx ../hashmap/libhashmap_rp.a: $(MAKE) -C ../hashmap hashmap_rp pmdk-1.8/src/examples/libpmemobj/map/map_ctree.h0000664000000000000000000000353113615011243020425 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_ctree.h -- common interface for maps */ #ifndef MAP_CTREE_H #define MAP_CTREE_H #include "map.h" #ifdef __cplusplus extern "C" { #endif extern struct map_ops ctree_map_ops; #define MAP_CTREE (&ctree_map_ops) #ifdef __cplusplus } #endif #endif /* MAP_CTREE_H */ pmdk-1.8/src/examples/libpmemobj/map/map_btree.h0000664000000000000000000000353113615011243020424 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_ctree.h -- common interface for maps */ #ifndef MAP_BTREE_H #define MAP_BTREE_H #include "map.h" #ifdef __cplusplus extern "C" { #endif extern struct map_ops btree_map_ops; #define MAP_BTREE (&btree_map_ops) #ifdef __cplusplus } #endif #endif /* MAP_BTREE_H */ pmdk-1.8/src/examples/libpmemobj/map/map_hashmap_atomic.c0000664000000000000000000001213013615011243022266 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * map_hashmap_atomic.c -- common interface for maps */ #include #include #include "map_hashmap_atomic.h" /* * map_hm_atomic_check -- wrapper for hm_atomic_check */ static int map_hm_atomic_check(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_check(pop, hashmap_atomic); } /* * map_hm_atomic_count -- wrapper for hm_atomic_count */ static size_t map_hm_atomic_count(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_count(pop, hashmap_atomic); } /* * map_hm_atomic_init -- wrapper for hm_atomic_init */ static int map_hm_atomic_init(PMEMobjpool *pop, TOID(struct map) map) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_init(pop, hashmap_atomic); } /* * map_hm_atomic_new -- wrapper for hm_atomic_create */ static int map_hm_atomic_create(PMEMobjpool *pop, TOID(struct map) *map, void *arg) { TOID(struct hashmap_atomic) *hashmap_atomic = (TOID(struct hashmap_atomic) *)map; return hm_atomic_create(pop, hashmap_atomic, arg); } /* * map_hm_atomic_insert -- wrapper for hm_atomic_insert */ static int map_hm_atomic_insert(PMEMobjpool *pop, TOID(struct map) map, uint64_t key, PMEMoid value) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_insert(pop, hashmap_atomic, key, value); } /* * map_hm_atomic_remove -- wrapper for hm_atomic_remove */ static PMEMoid map_hm_atomic_remove(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_remove(pop, hashmap_atomic, key); } /* * map_hm_atomic_get -- wrapper for hm_atomic_get */ static PMEMoid map_hm_atomic_get(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_get(pop, hashmap_atomic, key); } /* * map_hm_atomic_lookup -- wrapper for hm_atomic_lookup */ static int map_hm_atomic_lookup(PMEMobjpool *pop, TOID(struct map) map, uint64_t key) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_lookup(pop, hashmap_atomic, key); } /* * map_hm_atomic_foreach -- wrapper for hm_atomic_foreach */ static int map_hm_atomic_foreach(PMEMobjpool *pop, TOID(struct map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_foreach(pop, hashmap_atomic, cb, arg); } /* * map_hm_atomic_cmd -- wrapper for hm_atomic_cmd */ static int map_hm_atomic_cmd(PMEMobjpool *pop, TOID(struct map) map, unsigned cmd, uint64_t arg) { TOID(struct hashmap_atomic) hashmap_atomic; TOID_ASSIGN(hashmap_atomic, map.oid); return hm_atomic_cmd(pop, hashmap_atomic, cmd, arg); } struct map_ops hashmap_atomic_ops = { /* .check = */ map_hm_atomic_check, /* .create = */ map_hm_atomic_create, /* .destroy = */ NULL, /* .init = */ map_hm_atomic_init, /* .insert = */ map_hm_atomic_insert, /* .insert_new = */ NULL, /* .remove = */ map_hm_atomic_remove, /* .remove_free = */ NULL, /* .clear = */ NULL, /* .get = */ map_hm_atomic_get, /* .lookup = */ map_hm_atomic_lookup, /* .foreach = */ map_hm_atomic_foreach, /* .is_empty = */ NULL, /* .count = */ map_hm_atomic_count, /* .cmd = */ map_hm_atomic_cmd, }; pmdk-1.8/src/examples/libpmemobj/map/data_store.c0000664000000000000000000001312113615011243020602 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * data_store.c -- tree_map example usage */ #include #include #include #include #include #include #include #include "map.h" #include "map_ctree.h" #include "map_btree.h" #include "map_rbtree.h" #include "map_hashmap_atomic.h" #include "map_hashmap_tx.h" #include "map_hashmap_rp.h" #include "map_skiplist.h" POBJ_LAYOUT_BEGIN(data_store); POBJ_LAYOUT_ROOT(data_store, struct store_root); POBJ_LAYOUT_TOID(data_store, struct store_item); POBJ_LAYOUT_END(data_store); #define MAX_INSERTS 500 static uint64_t nkeys; static uint64_t keys[MAX_INSERTS]; struct store_item { uint64_t item_data; }; struct store_root { TOID(struct map) map; }; /* * new_store_item -- transactionally creates and initializes new item */ static TOID(struct store_item) new_store_item(void) { TOID(struct store_item) item = TX_NEW(struct store_item); D_RW(item)->item_data = rand(); return item; } /* * get_keys -- inserts the keys of the items by key order (sorted, descending) */ static int get_keys(uint64_t key, PMEMoid value, void *arg) { keys[nkeys++] = key; return 0; } /* * dec_keys -- decrements the keys count for every item */ static int dec_keys(uint64_t key, PMEMoid value, void *arg) { nkeys--; return 0; } /* * parse_map_type -- parse type of map */ static const struct map_ops * parse_map_type(const char *type) { if (strcmp(type, "ctree") == 0) return MAP_CTREE; else if (strcmp(type, "btree") == 0) return MAP_BTREE; else if (strcmp(type, "rbtree") == 0) return MAP_RBTREE; else if (strcmp(type, "hashmap_atomic") == 0) return MAP_HASHMAP_ATOMIC; else if (strcmp(type, "hashmap_tx") == 0) return MAP_HASHMAP_TX; else if (strcmp(type, "hashmap_rp") == 0) return MAP_HASHMAP_RP; else if (strcmp(type, "skiplist") == 0) return MAP_SKIPLIST; return NULL; } int main(int argc, const char *argv[]) { if (argc < 3) { printf("usage: %s " " file-name [nops]\n", argv[0]); return 1; } const char *type = argv[1]; const char *path = argv[2]; const struct map_ops *map_ops = parse_map_type(type); if (!map_ops) { fprintf(stderr, "invalid container type -- '%s'\n", type); return 1; } int nops = MAX_INSERTS; if (argc > 3) { nops = atoi(argv[3]); if (nops <= 0 || nops > MAX_INSERTS) { fprintf(stderr, "number of operations must be " "in range 1..%d\n", MAX_INSERTS); return 1; } } PMEMobjpool *pop; srand((unsigned)time(NULL)); if (file_exists(path) != 0) { if ((pop = pmemobj_create(path, POBJ_LAYOUT_NAME(data_store), PMEMOBJ_MIN_POOL, 0666)) == NULL) { perror("failed to create pool\n"); return 1; } } else { if ((pop = pmemobj_open(path, POBJ_LAYOUT_NAME(data_store))) == NULL) { perror("failed to open pool\n"); return 1; } } TOID(struct store_root) root = POBJ_ROOT(pop, struct store_root); struct map_ctx *mapc = map_ctx_init(map_ops, pop); if (!mapc) { perror("cannot allocate map context\n"); return 1; } /* delete the map if it exists */ if (!map_check(mapc, D_RW(root)->map)) map_destroy(mapc, &D_RW(root)->map); /* insert random items in a transaction */ int aborted = 0; TX_BEGIN(pop) { map_create(mapc, &D_RW(root)->map, NULL); for (int i = 0; i < nops; ++i) { /* new_store_item is transactional! */ map_insert(mapc, D_RW(root)->map, rand(), new_store_item().oid); } } TX_ONABORT { perror("transaction aborted\n"); map_ctx_free(mapc); aborted = 1; } TX_END if (aborted) return -1; /* count the items */ map_foreach(mapc, D_RW(root)->map, get_keys, NULL); /* remove the items without outer transaction */ for (int i = 0; i < nkeys; ++i) { PMEMoid item = map_remove(mapc, D_RW(root)->map, keys[i]); assert(!OID_IS_NULL(item)); assert(OID_INSTANCEOF(item, struct store_item)); } uint64_t old_nkeys = nkeys; /* tree should be empty */ map_foreach(mapc, D_RW(root)->map, dec_keys, NULL); assert(old_nkeys == nkeys); map_ctx_free(mapc); pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/lists.c0000664000000000000000000001221213615011243017036 0ustar rootroot/* * Copyright 2016-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * lists.c -- example usage of atomic lists API */ #include #include #include #include #include #include #include POBJ_LAYOUT_BEGIN(two_lists); POBJ_LAYOUT_ROOT(two_lists, struct my_root); POBJ_LAYOUT_TOID(two_lists, struct foo_el); POBJ_LAYOUT_TOID(two_lists, struct bar_el); POBJ_LAYOUT_END(two_lists); #define MAX_LISTS 10 struct foo_el { POBJ_LIST_ENTRY(struct foo_el) entries; int value; }; struct bar_el { POBJ_LIST_ENTRY(struct bar_el) entries; int value; }; struct listbase { POBJ_LIST_HEAD(foo, struct foo_el) foo_list; POBJ_LIST_HEAD(bar, struct bar_el) bar_list; }; struct my_root { struct listbase lists[MAX_LISTS]; }; enum list_type { LIST_INVALID, LIST_FOO, LIST_BAR, MAX_LIST_TYPES }; struct list_constr_args { enum list_type type; int value; }; /* * list_print -- prints the chosen list content to standard output */ static void list_print(PMEMobjpool *pop, struct listbase *base, enum list_type type) { switch (type) { case LIST_FOO: { TOID(struct foo_el) iter; POBJ_LIST_FOREACH(iter, &base->foo_list, entries) { printf("%d\n", D_RO(iter)->value); } } break; case LIST_BAR: { TOID(struct bar_el) iter; POBJ_LIST_FOREACH(iter, &base->bar_list, entries) { printf("%d\n", D_RO(iter)->value); } } break; default: assert(0); } } /* * list_element_constr -- constructor of the list element */ static int list_element_constr(PMEMobjpool *pop, void *ptr, void *arg) { struct list_constr_args *args = (struct list_constr_args *)arg; switch (args->type) { case LIST_FOO: { struct foo_el *e = (struct foo_el *)ptr; e->value = args->value; pmemobj_persist(pop, &e->value, sizeof(e->value)); } break; case LIST_BAR: { struct bar_el *e = (struct bar_el *)ptr; e->value = args->value; pmemobj_persist(pop, &e->value, sizeof(e->value)); } break; default: assert(0); } return 0; } /* * list_insert -- inserts a new element into the chosen list */ static void list_insert(PMEMobjpool *pop, struct listbase *base, enum list_type type, int value) { struct list_constr_args args = {type, value}; switch (type) { case LIST_FOO: POBJ_LIST_INSERT_NEW_HEAD(pop, &base->foo_list, entries, sizeof(struct foo_el), list_element_constr, &args); break; case LIST_BAR: POBJ_LIST_INSERT_NEW_HEAD(pop, &base->bar_list, entries, sizeof(struct bar_el), list_element_constr, &args); break; default: assert(0); } } int main(int argc, char *argv[]) { if (argc != 5) { printf("usage: %s file-name list_id foo|bar print|val\n", argv[0]); return 1; } const char *path = argv[1]; PMEMobjpool *pop; if (file_exists(path) != 0) { if ((pop = pmemobj_create(path, POBJ_LAYOUT_NAME(two_lists), PMEMOBJ_MIN_POOL, 0666)) == NULL) { perror("failed to create pool\n"); return 1; } } else { if ((pop = pmemobj_open(path, POBJ_LAYOUT_NAME(two_lists))) == NULL) { perror("failed to open pool\n"); return 1; } } int id = atoi(argv[2]); if (id < 0 || id >= MAX_LISTS) { fprintf(stderr, "list index out of scope\n"); return 1; } enum list_type type = LIST_INVALID; if (strcmp(argv[3], "foo") == 0) { type = LIST_FOO; } else if (strcmp(argv[3], "bar") == 0) { type = LIST_BAR; } if (type == LIST_INVALID) { fprintf(stderr, "invalid list type\n"); return 1; } TOID(struct my_root) r = POBJ_ROOT(pop, struct my_root); if (strcmp(argv[4], "print") == 0) { list_print(pop, &D_RW(r)->lists[id], type); } else { int val = atoi(argv[4]); list_insert(pop, &D_RW(r)->lists[id], type, val); } pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/lists.vcxproj.filters0000664000000000000000000000064213615011243021762 0ustar rootroot {53c898ee-aed1-4bcf-b254-9a475a1a0d80} Source Files pmdk-1.8/src/examples/libpmemobj/string_store/0000775000000000000000000000000013615011243020260 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/string_store/layout.h0000664000000000000000000000335113615011243021750 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * layout.h -- example from introduction part 1 */ #define LAYOUT_NAME "intro_1" #define MAX_BUF_LEN 10 struct my_root { size_t len; char buf[MAX_BUF_LEN]; }; pmdk-1.8/src/examples/libpmemobj/string_store/reader.vcxproj.filters0000664000000000000000000000123413615011243024606 0ustar rootroot {1e16629a-714b-4f1f-a688-24b50a3e7937} {c8b38ed3-5b33-4f48-ad3f-51a95613669d} Header Files Source Files pmdk-1.8/src/examples/libpmemobj/string_store/.gitignore0000664000000000000000000000001613615011243022245 0ustar rootrootwriter reader pmdk-1.8/src/examples/libpmemobj/string_store/reader.c0000664000000000000000000000421213615011243021665 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * reader.c -- example from introduction part 1 */ #include #include #include #include "layout.h" int main(int argc, char *argv[]) { if (argc != 2) { printf("usage: %s file-name\n", argv[0]); return 1; } PMEMobjpool *pop = pmemobj_open(argv[1], LAYOUT_NAME); if (pop == NULL) { perror("pmemobj_open"); return 1; } PMEMoid root = pmemobj_root(pop, sizeof(struct my_root)); struct my_root *rootp = pmemobj_direct(root); if (rootp->len == strlen(rootp->buf)) printf("%s\n", rootp->buf); pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/string_store/writer.vcxproj.filters0000664000000000000000000000123413615011243024660 0ustar rootroot {48ecfa7f-5d45-4014-970b-153e338a9a2f} {d54e716e-8b7f-4b37-b2a6-63c43209d4a6} Header Files Source Files pmdk-1.8/src/examples/libpmemobj/string_store/reader.vcxproj0000664000000000000000000000544213615011243023144 0ustar rootroot Debug x64 Release x64 {0BFD78AA-FD94-4DB1-8495-8F5CC06D8F03} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/string_store/Makefile0000664000000000000000000000334613615011243021726 0ustar rootroot# # Copyright 2015-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/string_store/Makefile -- build the intro 1 example # PROGS = writer reader LIBS = -lpmemobj -lpmem -pthread include ../../Makefile.inc writer: writer.o reader: reader.o pmdk-1.8/src/examples/libpmemobj/string_store/writer.vcxproj0000664000000000000000000000544213615011243023216 0ustar rootroot Debug x64 Release x64 {F5D850C9-D353-4B84-99BC-E336C231018C} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/string_store/writer.c0000664000000000000000000000454013615011243021743 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * writer.c -- example from introduction part 1 */ #include #include #include #include "layout.h" int main(int argc, char *argv[]) { if (argc != 2) { printf("usage: %s file-name\n", argv[0]); return 1; } PMEMobjpool *pop = pmemobj_create(argv[1], LAYOUT_NAME, PMEMOBJ_MIN_POOL, 0666); if (pop == NULL) { perror("pmemobj_create"); return 1; } PMEMoid root = pmemobj_root(pop, sizeof(struct my_root)); struct my_root *rootp = pmemobj_direct(root); char buf[MAX_BUF_LEN] = {0}; if (scanf("%9s", buf) == EOF) { fprintf(stderr, "EOF\n"); return 1; } rootp->len = strlen(buf); pmemobj_persist(pop, &rootp->len, sizeof(rootp->len)); pmemobj_memcpy_persist(pop, rootp->buf, buf, rootp->len); pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/manpage.c0000664000000000000000000000452013615011243017313 0ustar rootroot/* * Copyright 2014-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * manpage.c -- simple example for the libpmemobj man page */ #include #include #include #include #ifndef _WIN32 #include #endif #include #include #include /* size of the pmemobj pool -- 1 GB */ #define POOL_SIZE ((size_t)(1 << 30)) /* name of our layout in the pool */ #define LAYOUT_NAME "example_layout" int main(int argc, char *argv[]) { const char path[] = "/pmem-fs/myfile"; PMEMobjpool *pop; /* create the pmemobj pool or open it if it already exists */ pop = pmemobj_create(path, LAYOUT_NAME, POOL_SIZE, 0666); if (pop == NULL) pop = pmemobj_open(path, LAYOUT_NAME); if (pop == NULL) { perror(path); exit(1); } /* ... */ pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/README0000664000000000000000000000117213615011243016417 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemobj/README. This directory contains examples for libpmemobj, the library providing a transactional object store for pmem. Some of these examples are explained in more detail here: https://pmem.io/pmdk/libpmemobj manpage.c and setjmp.c are the examples used in the libpmemobj man page. To build these examples: make These examples can be built against an installed system using: make LIBDIR=/usr/lib INCDIR=/usr/include If you're looking for documentation to get you started using PMDK, start here: https://pmem.io/pmdk and follow the links to examples and man pages. pmdk-1.8/src/examples/libpmemobj/btree.vcxproj.filters0000664000000000000000000000064213615011243021725 0ustar rootroot {e8e0c541-b98b-40f3-851f-5460fc7af49e} Source Files pmdk-1.8/src/examples/libpmemobj/string_store_tx/0000775000000000000000000000000013615011243020773 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/string_store_tx/layout.h0000664000000000000000000000333413615011243022464 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * layout.h -- example from introduction part 2 */ #define LAYOUT_NAME "intro_2" #define MAX_BUF_LEN 10 struct my_root { char buf[MAX_BUF_LEN]; }; pmdk-1.8/src/examples/libpmemobj/string_store_tx/reader.vcxproj.filters0000664000000000000000000000123413615011243025321 0ustar rootroot {1e16629a-714b-4f1f-a688-24b50a3e7937} {c8b38ed3-5b33-4f48-ad3f-51a95613669d} Header Files Source Files pmdk-1.8/src/examples/libpmemobj/string_store_tx/.gitignore0000664000000000000000000000001613615011243022760 0ustar rootrootwriter reader pmdk-1.8/src/examples/libpmemobj/string_store_tx/reader.c0000664000000000000000000000414213615011243022402 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * reader.c -- example from introduction part 2 */ #include #include #include #include "layout.h" int main(int argc, char *argv[]) { if (argc != 2) { printf("usage: %s file-name\n", argv[0]); return 1; } PMEMobjpool *pop = pmemobj_open(argv[1], LAYOUT_NAME); if (pop == NULL) { perror("pmemobj_open"); return 1; } PMEMoid root = pmemobj_root(pop, sizeof(struct my_root)); struct my_root *rootp = pmemobj_direct(root); printf("%s\n", rootp->buf); pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/string_store_tx/writer.vcxproj.filters0000664000000000000000000000123413615011243025373 0ustar rootroot {48ecfa7f-5d45-4014-970b-153e338a9a2f} {d54e716e-8b7f-4b37-b2a6-63c43209d4a6} Header Files Source Files pmdk-1.8/src/examples/libpmemobj/string_store_tx/reader.vcxproj0000664000000000000000000000544513615011243023662 0ustar rootroot Debug x64 Release x64 {59D7A9CD-9912-40E4-96E1-8A873F777F62} pmemobj_tx 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/string_store_tx/Makefile0000664000000000000000000000335113615011243022435 0ustar rootroot# # Copyright 2015-2016, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/string_store_tx/Makefile -- build the intro 2 example # PROGS = writer reader LIBS = -lpmemobj -lpmem -pthread include ../../Makefile.inc writer: writer.o reader: reader.o pmdk-1.8/src/examples/libpmemobj/string_store_tx/writer.vcxproj0000664000000000000000000000544513615011243023734 0ustar rootroot Debug x64 Release x64 {7337E34A-97B0-44FC-988B-7E6AE7E0FBBF} pmemobj_tx 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/string_store_tx/writer.c0000664000000000000000000000451513615011243022460 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * writer.c -- example from introduction part 2 */ #include #include #include #include "layout.h" int main(int argc, char *argv[]) { if (argc != 2) { printf("usage: %s file-name\n", argv[0]); return 1; } PMEMobjpool *pop = pmemobj_create(argv[1], LAYOUT_NAME, PMEMOBJ_MIN_POOL, 0666); if (pop == NULL) { perror("pmemobj_create"); return 1; } PMEMoid root = pmemobj_root(pop, sizeof(struct my_root)); struct my_root *rootp = pmemobj_direct(root); char buf[MAX_BUF_LEN] = {0}; if (scanf("%9s", buf) == EOF) { fprintf(stderr, "EOF\n"); return 1; } TX_BEGIN(pop) { pmemobj_tx_add_range(root, 0, sizeof(struct my_root)); memcpy(rootp->buf, buf, strlen(buf)); } TX_END pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/btree.c0000664000000000000000000001225613615011243017011 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * btree.c -- implementation of persistent binary search tree */ #include #include #include #include #include #include #include POBJ_LAYOUT_BEGIN(btree); POBJ_LAYOUT_ROOT(btree, struct btree); POBJ_LAYOUT_TOID(btree, struct btree_node); POBJ_LAYOUT_END(btree); struct btree_node { int64_t key; TOID(struct btree_node) slots[2]; char value[]; }; struct btree { TOID(struct btree_node) root; }; struct btree_node_arg { size_t size; int64_t key; const char *value; }; /* * btree_node_construct -- constructor of btree node */ static int btree_node_construct(PMEMobjpool *pop, void *ptr, void *arg) { struct btree_node *node = (struct btree_node *)ptr; struct btree_node_arg *a = (struct btree_node_arg *)arg; node->key = a->key; strcpy(node->value, a->value); node->slots[0] = TOID_NULL(struct btree_node); node->slots[1] = TOID_NULL(struct btree_node); pmemobj_persist(pop, node, a->size); return 0; } /* * btree_insert -- inserts new element into the tree */ static void btree_insert(PMEMobjpool *pop, int64_t key, const char *value) { TOID(struct btree) btree = POBJ_ROOT(pop, struct btree); TOID(struct btree_node) *dst = &D_RW(btree)->root; while (!TOID_IS_NULL(*dst)) { dst = &D_RW(*dst)->slots[key > D_RO(*dst)->key]; } struct btree_node_arg args; args.size = sizeof(struct btree_node) + strlen(value) + 1; args.key = key; args.value = value; POBJ_ALLOC(pop, dst, struct btree_node, args.size, btree_node_construct, &args); } /* * btree_find -- searches for key in the tree */ static const char * btree_find(PMEMobjpool *pop, int64_t key) { TOID(struct btree) btree = POBJ_ROOT(pop, struct btree); TOID(struct btree_node) node = D_RO(btree)->root; while (!TOID_IS_NULL(node)) { if (D_RO(node)->key == key) return D_RO(node)->value; else node = D_RO(node)->slots[key > D_RO(node)->key]; } return NULL; } /* * btree_node_print -- prints content of the btree node */ static void btree_node_print(const TOID(struct btree_node) node) { printf("%" PRIu64 " %s\n", D_RO(node)->key, D_RO(node)->value); } /* * btree_foreach -- invoke callback for every node */ static void btree_foreach(PMEMobjpool *pop, const TOID(struct btree_node) node, void(*cb)(const TOID(struct btree_node) node)) { if (TOID_IS_NULL(node)) return; btree_foreach(pop, D_RO(node)->slots[0], cb); cb(node); btree_foreach(pop, D_RO(node)->slots[1], cb); } /* * btree_print -- initiates foreach node print */ static void btree_print(PMEMobjpool *pop) { TOID(struct btree) btree = POBJ_ROOT(pop, struct btree); btree_foreach(pop, D_RO(btree)->root, btree_node_print); } int main(int argc, char *argv[]) { if (argc < 3) { printf("usage: %s file-name [p|i|f] [key] [value] \n", argv[0]); return 1; } const char *path = argv[1]; PMEMobjpool *pop; if (file_exists(path) != 0) { if ((pop = pmemobj_create(path, POBJ_LAYOUT_NAME(btree), PMEMOBJ_MIN_POOL, 0666)) == NULL) { perror("failed to create pool\n"); return 1; } } else { if ((pop = pmemobj_open(path, POBJ_LAYOUT_NAME(btree))) == NULL) { perror("failed to open pool\n"); return 1; } } const char op = argv[2][0]; int64_t key; const char *value; switch (op) { case 'p': btree_print(pop); break; case 'i': key = atoll(argv[3]); value = argv[4]; btree_insert(pop, key, value); break; case 'f': key = atoll(argv[3]); if ((value = btree_find(pop, key)) != NULL) printf("%s\n", value); else printf("not found\n"); break; default: printf("invalid operation\n"); break; } pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/pi.c0000664000000000000000000001573713615011243016327 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pi.c -- example usage of user lists * * Calculates pi number with multiple threads using Leibniz formula. */ #include #include #include #include #include #include #include #include #ifndef _WIN32 #include #endif /* * Layout definition */ POBJ_LAYOUT_BEGIN(pi); POBJ_LAYOUT_ROOT(pi, struct pi); POBJ_LAYOUT_TOID(pi, struct pi_task); POBJ_LAYOUT_END(pi); static PMEMobjpool *pop; struct pi_task_proto { uint64_t start; uint64_t stop; long double result; }; struct pi_task { struct pi_task_proto proto; POBJ_LIST_ENTRY(struct pi_task) todo; POBJ_LIST_ENTRY(struct pi_task) done; }; struct pi { POBJ_LIST_HEAD(todo, struct pi_task) todo; POBJ_LIST_HEAD(done, struct pi_task) done; }; /* * pi_task_construct -- task constructor */ static int pi_task_construct(PMEMobjpool *pop, void *ptr, void *arg) { struct pi_task *t = (struct pi_task *)ptr; struct pi_task_proto *p = (struct pi_task_proto *)arg; t->proto = *p; pmemobj_persist(pop, t, sizeof(*t)); return 0; } /* * calc_pi -- worker for pi calculation */ #ifndef _WIN32 static void * calc_pi(void *arg) #else static DWORD WINAPI calc_pi(LPVOID arg) #endif { TOID(struct pi) pi = POBJ_ROOT(pop, struct pi); TOID(struct pi_task) task = *((TOID(struct pi_task) *)arg); long double result = 0; for (uint64_t i = D_RO(task)->proto.start; i < D_RO(task)->proto.stop; ++i) { result += (pow(-1, (double)i) / (2 * i + 1)); } D_RW(task)->proto.result = result; pmemobj_persist(pop, &D_RW(task)->proto.result, sizeof(result)); POBJ_LIST_MOVE_ELEMENT_HEAD(pop, &D_RW(pi)->todo, &D_RW(pi)->done, task, todo, done); return NULL; } /* * calc_pi_mt -- calculate all the pending to-do tasks */ static void calc_pi_mt(void) { TOID(struct pi) pi = POBJ_ROOT(pop, struct pi); int pending = 0; TOID(struct pi_task) iter; POBJ_LIST_FOREACH(iter, &D_RO(pi)->todo, todo) pending++; if (pending == 0) return; int i = 0; TOID(struct pi_task) *tasks = (TOID(struct pi_task) *)malloc( sizeof(TOID(struct pi_task)) * pending); if (tasks == NULL) { fprintf(stderr, "failed to allocate tasks\n"); return; } POBJ_LIST_FOREACH(iter, &D_RO(pi)->todo, todo) tasks[i++] = iter; #ifndef _WIN32 pthread_t workers[pending]; for (i = 0; i < pending; ++i) if (pthread_create(&workers[i], NULL, calc_pi, &tasks[i]) != 0) break; for (i = i - 1; i >= 0; --i) pthread_join(workers[i], NULL); #else HANDLE *workers = (HANDLE *) malloc(sizeof(HANDLE) * pending); for (i = 0; i < pending; ++i) { workers[i] = CreateThread(NULL, 0, calc_pi, &tasks[i], 0, NULL); if (workers[i] == NULL) break; } WaitForMultipleObjects(i, workers, TRUE, INFINITE); for (i = i - 1; i >= 0; --i) CloseHandle(workers[i]); free(workers); #endif free(tasks); } /* * prep_todo_list -- create tasks to be done */ static int prep_todo_list(int threads, int ops) { TOID(struct pi) pi = POBJ_ROOT(pop, struct pi); if (!POBJ_LIST_EMPTY(&D_RO(pi)->todo)) return -1; int ops_per_thread = ops / threads; uint64_t last = 0; /* last calculated denominator */ TOID(struct pi_task) iter; POBJ_LIST_FOREACH(iter, &D_RO(pi)->done, done) { if (last < D_RO(iter)->proto.stop) last = D_RO(iter)->proto.stop; } int i; for (i = 0; i < threads; ++i) { uint64_t start = last + (i * ops_per_thread); struct pi_task_proto proto; proto.start = start; proto.stop = start + ops_per_thread; proto.result = 0; POBJ_LIST_INSERT_NEW_HEAD(pop, &D_RW(pi)->todo, todo, sizeof(struct pi_task), pi_task_construct, &proto); } return 0; } int main(int argc, char *argv[]) { if (argc < 3) { printf("usage: %s file-name " "[print|done|todo|finish|calc <# of threads> ]\n", argv[0]); return 1; } const char *path = argv[1]; pop = NULL; if (file_exists(path) != 0) { if ((pop = pmemobj_create(path, POBJ_LAYOUT_NAME(pi), PMEMOBJ_MIN_POOL, CREATE_MODE_RW)) == NULL) { printf("failed to create pool\n"); return 1; } } else { if ((pop = pmemobj_open(path, POBJ_LAYOUT_NAME(pi))) == NULL) { printf("failed to open pool\n"); return 1; } } TOID(struct pi) pi = POBJ_ROOT(pop, struct pi); char op = argv[2][0]; switch (op) { case 'p': { /* print pi */ long double pi_val = 0; TOID(struct pi_task) iter; POBJ_LIST_FOREACH(iter, &D_RO(pi)->done, done) { pi_val += D_RO(iter)->proto.result; } printf("pi: %Lf\n", pi_val * 4); } break; case 'd': { /* print done list */ TOID(struct pi_task) iter; POBJ_LIST_FOREACH(iter, &D_RO(pi)->done, done) { printf("(%" PRIu64 " - %" PRIu64 ") = %Lf\n", D_RO(iter)->proto.start, D_RO(iter)->proto.stop, D_RO(iter)->proto.result); } } break; case 't': { /* print to-do list */ TOID(struct pi_task) iter; POBJ_LIST_FOREACH(iter, &D_RO(pi)->todo, todo) { printf("(%" PRIu64 " - %" PRIu64 ") = %Lf\n", D_RO(iter)->proto.start, D_RO(iter)->proto.stop, D_RO(iter)->proto.result); } } break; case 'c': { /* calculate pi */ if (argc < 5) { printf("usage: %s file-name " "calc <# of threads> \n", argv[0]); return 1; } int threads = atoi(argv[3]); int ops = atoi(argv[4]); assert((threads > 0) && (ops > 0)); if (prep_todo_list(threads, ops) == -1) printf("pending todo tasks\n"); else calc_pi_mt(); } break; case 'f': { /* finish to-do tasks */ calc_pi_mt(); } break; } pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/.gitignore0000664000000000000000000000004713615011243017527 0ustar rootroot*~ pi manpage btree rtree lists setjmp pmdk-1.8/src/examples/libpmemobj/tree_map/0000775000000000000000000000000013615011243017332 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/tree_map/ctree_map.vcxproj0000664000000000000000000000523013615011243022706 0ustar rootroot Debug x64 Release x64 {BE18F227-A9F0-4B38-B689-4E2F9F09CA5F} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} StaticLibrary true v140 StaticLibrary false v140 true ..\..\..\LongPath.manifest CompileAsCpp pmdk-1.8/src/examples/libpmemobj/tree_map/btree_map.vcxproj.filters0000664000000000000000000000124213615011243024353 0ustar rootroot {e00bdf1b-1168-4521-8034-629bf8717652} {e34e9a85-44de-435d-815d-fd07b599fadd} Header Files Source Files pmdk-1.8/src/examples/libpmemobj/tree_map/ctree_map.c0000664000000000000000000002252313615011243021441 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctree_map.c -- Crit-bit trie implementation */ #include #include #include #include #include "ctree_map.h" #define BIT_IS_SET(n, i) (!!((n) & (1ULL << (i)))) TOID_DECLARE(struct tree_map_node, CTREE_MAP_TYPE_OFFSET + 1); struct tree_map_entry { uint64_t key; PMEMoid slot; }; struct tree_map_node { int diff; /* most significant differing bit */ struct tree_map_entry entries[2]; }; struct ctree_map { struct tree_map_entry root; }; /* * find_crit_bit -- (internal) finds the most significant differing bit */ static int find_crit_bit(uint64_t lhs, uint64_t rhs) { return find_last_set_64(lhs ^ rhs); } /* * ctree_map_create -- allocates a new crit-bit tree instance */ int ctree_map_create(PMEMobjpool *pop, TOID(struct ctree_map) *map, void *arg) { int ret = 0; TX_BEGIN(pop) { pmemobj_tx_add_range_direct(map, sizeof(*map)); *map = TX_ZNEW(struct ctree_map); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * ctree_map_clear_node -- (internal) clears this node and its children */ static void ctree_map_clear_node(PMEMoid p) { if (OID_IS_NULL(p)) return; if (OID_INSTANCEOF(p, struct tree_map_node)) { TOID(struct tree_map_node) node; TOID_ASSIGN(node, p); ctree_map_clear_node(D_RW(node)->entries[0].slot); ctree_map_clear_node(D_RW(node)->entries[1].slot); } pmemobj_tx_free(p); } /* * ctree_map_clear -- removes all elements from the map */ int ctree_map_clear(PMEMobjpool *pop, TOID(struct ctree_map) map) { TX_BEGIN(pop) { ctree_map_clear_node(D_RW(map)->root.slot); TX_ADD_FIELD(map, root); D_RW(map)->root.slot = OID_NULL; } TX_END return 0; } /* * ctree_map_destroy -- cleanups and frees crit-bit tree instance */ int ctree_map_destroy(PMEMobjpool *pop, TOID(struct ctree_map) *map) { int ret = 0; TX_BEGIN(pop) { ctree_map_clear(pop, *map); pmemobj_tx_add_range_direct(map, sizeof(*map)); TX_FREE(*map); *map = TOID_NULL(struct ctree_map); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * ctree_map_insert_leaf -- (internal) inserts a new leaf at the position */ static void ctree_map_insert_leaf(struct tree_map_entry *p, struct tree_map_entry e, int diff) { TOID(struct tree_map_node) new_node = TX_NEW(struct tree_map_node); D_RW(new_node)->diff = diff; int d = BIT_IS_SET(e.key, D_RO(new_node)->diff); /* insert the leaf at the direction based on the critical bit */ D_RW(new_node)->entries[d] = e; /* find the appropriate position in the tree to insert the node */ TOID(struct tree_map_node) node; while (OID_INSTANCEOF(p->slot, struct tree_map_node)) { TOID_ASSIGN(node, p->slot); /* the critical bits have to be sorted */ if (D_RO(node)->diff < D_RO(new_node)->diff) break; p = &D_RW(node)->entries[BIT_IS_SET(e.key, D_RO(node)->diff)]; } /* insert the found destination in the other slot */ D_RW(new_node)->entries[!d] = *p; pmemobj_tx_add_range_direct(p, sizeof(*p)); p->key = 0; p->slot = new_node.oid; } /* * ctree_map_insert_new -- allocates a new object and inserts it into the tree */ int ctree_map_insert_new(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { int ret = 0; TX_BEGIN(pop) { PMEMoid n = pmemobj_tx_alloc(size, type_num); constructor(pop, pmemobj_direct(n), arg); ctree_map_insert(pop, map, key, n); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * ctree_map_insert -- inserts a new key-value pair into the map */ int ctree_map_insert(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key, PMEMoid value) { struct tree_map_entry *p = &D_RW(map)->root; int ret = 0; /* descend the path until a best matching key is found */ TOID(struct tree_map_node) node; while (!OID_IS_NULL(p->slot) && OID_INSTANCEOF(p->slot, struct tree_map_node)) { TOID_ASSIGN(node, p->slot); p = &D_RW(node)->entries[BIT_IS_SET(key, D_RW(node)->diff)]; } struct tree_map_entry e = {key, value}; TX_BEGIN(pop) { if (p->key == 0 || p->key == key) { pmemobj_tx_add_range_direct(p, sizeof(*p)); *p = e; } else { ctree_map_insert_leaf(&D_RW(map)->root, e, find_crit_bit(p->key, key)); } } TX_ONABORT { ret = 1; } TX_END return ret; } /* * ctree_map_get_leaf -- (internal) searches for a leaf of the key */ static struct tree_map_entry * ctree_map_get_leaf(TOID(struct ctree_map) map, uint64_t key, struct tree_map_entry **parent) { struct tree_map_entry *n = &D_RW(map)->root; struct tree_map_entry *p = NULL; TOID(struct tree_map_node) node; while (!OID_IS_NULL(n->slot) && OID_INSTANCEOF(n->slot, struct tree_map_node)) { TOID_ASSIGN(node, n->slot); p = n; n = &D_RW(node)->entries[BIT_IS_SET(key, D_RW(node)->diff)]; } if (n->key == key) { if (parent) *parent = p; return n; } return NULL; } /* * ctree_map_remove_free -- removes and frees an object from the tree */ int ctree_map_remove_free(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key) { int ret = 0; TX_BEGIN(pop) { PMEMoid val = ctree_map_remove(pop, map, key); pmemobj_tx_free(val); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * ctree_map_remove -- removes key-value pair from the map */ PMEMoid ctree_map_remove(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key) { struct tree_map_entry *parent = NULL; struct tree_map_entry *leaf = ctree_map_get_leaf(map, key, &parent); if (leaf == NULL) return OID_NULL; PMEMoid ret = leaf->slot; if (parent == NULL) { /* root */ TX_BEGIN(pop) { pmemobj_tx_add_range_direct(leaf, sizeof(*leaf)); leaf->key = 0; leaf->slot = OID_NULL; } TX_END } else { /* * In this situation: * parent * / \ * LEFT RIGHT * there's no point in leaving the parent internal node * so it's swapped with the remaining node and then also freed. */ TX_BEGIN(pop) { struct tree_map_entry *dest = parent; TOID(struct tree_map_node) node; TOID_ASSIGN(node, parent->slot); pmemobj_tx_add_range_direct(dest, sizeof(*dest)); *dest = D_RW(node)->entries[ D_RO(node)->entries[0].key == leaf->key]; TX_FREE(node); } TX_END } return ret; } /* * ctree_map_get -- searches for a value of the key */ PMEMoid ctree_map_get(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key) { struct tree_map_entry *entry = ctree_map_get_leaf(map, key, NULL); return entry ? entry->slot : OID_NULL; } /* * ctree_map_lookup -- searches if a key exists */ int ctree_map_lookup(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key) { struct tree_map_entry *entry = ctree_map_get_leaf(map, key, NULL); return entry != NULL; } /* * ctree_map_foreach_node -- (internal) recursively traverses tree */ static int ctree_map_foreach_node(struct tree_map_entry e, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { int ret = 0; if (OID_INSTANCEOF(e.slot, struct tree_map_node)) { TOID(struct tree_map_node) node; TOID_ASSIGN(node, e.slot); if (ctree_map_foreach_node(D_RO(node)->entries[0], cb, arg) == 0) ctree_map_foreach_node(D_RO(node)->entries[1], cb, arg); } else { /* leaf */ ret = cb(e.key, e.slot, arg); } return ret; } /* * ctree_map_foreach -- initiates recursive traversal */ int ctree_map_foreach(PMEMobjpool *pop, TOID(struct ctree_map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { if (OID_IS_NULL(D_RO(map)->root.slot)) return 0; return ctree_map_foreach_node(D_RO(map)->root, cb, arg); } /* * ctree_map_is_empty -- checks whether the tree map is empty */ int ctree_map_is_empty(PMEMobjpool *pop, TOID(struct ctree_map) map) { return D_RO(map)->root.key == 0; } /* * ctree_map_check -- check if given persistent object is a tree map */ int ctree_map_check(PMEMobjpool *pop, TOID(struct ctree_map) map) { return TOID_IS_NULL(map) || !TOID_VALID(map); } pmdk-1.8/src/examples/libpmemobj/tree_map/btree_map.c0000664000000000000000000004403013615011243021435 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * btree_map.c -- textbook implementation of btree /w preemptive splitting */ #include #include #include #include "btree_map.h" TOID_DECLARE(struct tree_map_node, BTREE_MAP_TYPE_OFFSET + 1); #define BTREE_ORDER 8 /* can't be odd */ #define BTREE_MIN ((BTREE_ORDER / 2) - 1) /* min number of keys per node */ struct tree_map_node_item { uint64_t key; PMEMoid value; }; struct tree_map_node { int n; /* number of occupied slots */ struct tree_map_node_item items[BTREE_ORDER - 1]; TOID(struct tree_map_node) slots[BTREE_ORDER]; }; struct btree_map { TOID(struct tree_map_node) root; }; /* * set_empty_item -- (internal) sets null to the item */ static void set_empty_item(struct tree_map_node_item *item) { item->key = 0; item->value = OID_NULL; } /* * btree_map_create -- allocates a new btree instance */ int btree_map_create(PMEMobjpool *pop, TOID(struct btree_map) *map, void *arg) { int ret = 0; TX_BEGIN(pop) { pmemobj_tx_add_range_direct(map, sizeof(*map)); *map = TX_ZNEW(struct btree_map); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * btree_map_clear_node -- (internal) removes all elements from the node */ static void btree_map_clear_node(TOID(struct tree_map_node) node) { for (int i = 0; i < D_RO(node)->n; ++i) { btree_map_clear_node(D_RO(node)->slots[i]); } TX_FREE(node); } /* * btree_map_clear -- removes all elements from the map */ int btree_map_clear(PMEMobjpool *pop, TOID(struct btree_map) map) { int ret = 0; TX_BEGIN(pop) { btree_map_clear_node(D_RO(map)->root); TX_ADD_FIELD(map, root); D_RW(map)->root = TOID_NULL(struct tree_map_node); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * btree_map_destroy -- cleanups and frees btree instance */ int btree_map_destroy(PMEMobjpool *pop, TOID(struct btree_map) *map) { int ret = 0; TX_BEGIN(pop) { btree_map_clear(pop, *map); pmemobj_tx_add_range_direct(map, sizeof(*map)); TX_FREE(*map); *map = TOID_NULL(struct btree_map); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * btree_map_insert_item_at -- (internal) inserts an item at position */ static void btree_map_insert_item_at(TOID(struct tree_map_node) node, int pos, struct tree_map_node_item item) { D_RW(node)->items[pos] = item; D_RW(node)->n += 1; } /* * btree_map_insert_empty -- (internal) inserts an item into an empty node */ static void btree_map_insert_empty(TOID(struct btree_map) map, struct tree_map_node_item item) { TX_ADD_FIELD(map, root); D_RW(map)->root = TX_ZNEW(struct tree_map_node); btree_map_insert_item_at(D_RO(map)->root, 0, item); } /* * btree_map_insert_node -- (internal) inserts and makes space for new node */ static void btree_map_insert_node(TOID(struct tree_map_node) node, int p, struct tree_map_node_item item, TOID(struct tree_map_node) left, TOID(struct tree_map_node) right) { TX_ADD(node); if (D_RO(node)->items[p].key != 0) { /* move all existing data */ memmove(&D_RW(node)->items[p + 1], &D_RW(node)->items[p], sizeof(struct tree_map_node_item) * ((BTREE_ORDER - 2 - p))); memmove(&D_RW(node)->slots[p + 1], &D_RW(node)->slots[p], sizeof(TOID(struct tree_map_node)) * ((BTREE_ORDER - 1 - p))); } D_RW(node)->slots[p] = left; D_RW(node)->slots[p + 1] = right; btree_map_insert_item_at(node, p, item); } /* * btree_map_create_split_node -- (internal) splits a node into two */ static TOID(struct tree_map_node) btree_map_create_split_node(TOID(struct tree_map_node) node, struct tree_map_node_item *m) { TOID(struct tree_map_node) right = TX_ZNEW(struct tree_map_node); int c = (BTREE_ORDER / 2); *m = D_RO(node)->items[c - 1]; /* select median item */ TX_ADD(node); set_empty_item(&D_RW(node)->items[c - 1]); /* move everything right side of median to the new node */ for (int i = c; i < BTREE_ORDER; ++i) { if (i != BTREE_ORDER - 1) { D_RW(right)->items[D_RW(right)->n++] = D_RO(node)->items[i]; set_empty_item(&D_RW(node)->items[i]); } D_RW(right)->slots[i - c] = D_RO(node)->slots[i]; D_RW(node)->slots[i] = TOID_NULL(struct tree_map_node); } D_RW(node)->n = c - 1; return right; } /* * btree_map_find_dest_node -- (internal) finds a place to insert the new key at */ static TOID(struct tree_map_node) btree_map_find_dest_node(TOID(struct btree_map) map, TOID(struct tree_map_node) n, TOID(struct tree_map_node) parent, uint64_t key, int *p) { if (D_RO(n)->n == BTREE_ORDER - 1) { /* node is full, perform a split */ struct tree_map_node_item m; TOID(struct tree_map_node) right = btree_map_create_split_node(n, &m); if (!TOID_IS_NULL(parent)) { btree_map_insert_node(parent, *p, m, n, right); if (key > m.key) /* select node to continue search */ n = right; } else { /* replacing root node, the tree grows in height */ TOID(struct tree_map_node) up = TX_ZNEW(struct tree_map_node); D_RW(up)->n = 1; D_RW(up)->items[0] = m; D_RW(up)->slots[0] = n; D_RW(up)->slots[1] = right; TX_ADD_FIELD(map, root); D_RW(map)->root = up; n = up; } } int i; for (i = 0; i < BTREE_ORDER - 1; ++i) { *p = i; /* * The key either fits somewhere in the middle or at the * right edge of the node. */ if (D_RO(n)->n == i || D_RO(n)->items[i].key > key) { return TOID_IS_NULL(D_RO(n)->slots[i]) ? n : btree_map_find_dest_node(map, D_RO(n)->slots[i], n, key, p); } } /* * The key is bigger than the last node element, go one level deeper * in the rightmost child. */ return btree_map_find_dest_node(map, D_RO(n)->slots[i], n, key, p); } /* * btree_map_insert_item -- (internal) inserts and makes space for new item */ static void btree_map_insert_item(TOID(struct tree_map_node) node, int p, struct tree_map_node_item item) { TX_ADD(node); if (D_RO(node)->items[p].key != 0) { memmove(&D_RW(node)->items[p + 1], &D_RW(node)->items[p], sizeof(struct tree_map_node_item) * ((BTREE_ORDER - 2 - p))); } btree_map_insert_item_at(node, p, item); } /* * btree_map_is_empty -- checks whether the tree map is empty */ int btree_map_is_empty(PMEMobjpool *pop, TOID(struct btree_map) map) { return TOID_IS_NULL(D_RO(map)->root) || D_RO(D_RO(map)->root)->n == 0; } /* * btree_map_insert -- inserts a new key-value pair into the map */ int btree_map_insert(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key, PMEMoid value) { struct tree_map_node_item item = {key, value}; TX_BEGIN(pop) { if (btree_map_is_empty(pop, map)) { btree_map_insert_empty(map, item); } else { int p; /* position at the dest node to insert */ TOID(struct tree_map_node) parent = TOID_NULL(struct tree_map_node); TOID(struct tree_map_node) dest = btree_map_find_dest_node(map, D_RW(map)->root, parent, key, &p); btree_map_insert_item(dest, p, item); } } TX_END return 0; } /* * btree_map_rotate_right -- (internal) takes one element from right sibling */ static void btree_map_rotate_right(TOID(struct tree_map_node) rsb, TOID(struct tree_map_node) node, TOID(struct tree_map_node) parent, int p) { /* move the separator from parent to the deficient node */ struct tree_map_node_item sep = D_RO(parent)->items[p]; btree_map_insert_item(node, D_RO(node)->n, sep); /* the first element of the right sibling is the new separator */ TX_ADD_FIELD(parent, items[p]); D_RW(parent)->items[p] = D_RO(rsb)->items[0]; /* the nodes are not necessarily leafs, so copy also the slot */ TX_ADD_FIELD(node, slots[D_RO(node)->n]); D_RW(node)->slots[D_RO(node)->n] = D_RO(rsb)->slots[0]; TX_ADD(rsb); D_RW(rsb)->n -= 1; /* it loses one element, but still > min */ /* move all existing elements back by one array slot */ memmove(D_RW(rsb)->items, D_RO(rsb)->items + 1, sizeof(struct tree_map_node_item) * (D_RO(rsb)->n)); memmove(D_RW(rsb)->slots, D_RO(rsb)->slots + 1, sizeof(TOID(struct tree_map_node)) * (D_RO(rsb)->n + 1)); } /* * btree_map_rotate_left -- (internal) takes one element from left sibling */ static void btree_map_rotate_left(TOID(struct tree_map_node) lsb, TOID(struct tree_map_node) node, TOID(struct tree_map_node) parent, int p) { /* move the separator from parent to the deficient node */ struct tree_map_node_item sep = D_RO(parent)->items[p - 1]; btree_map_insert_item(node, 0, sep); /* the last element of the left sibling is the new separator */ TX_ADD_FIELD(parent, items[p - 1]); D_RW(parent)->items[p - 1] = D_RO(lsb)->items[D_RO(lsb)->n - 1]; /* rotate the node children */ memmove(D_RW(node)->slots + 1, D_RO(node)->slots, sizeof(TOID(struct tree_map_node)) * (D_RO(node)->n)); /* the nodes are not necessarily leafs, so copy also the slot */ D_RW(node)->slots[0] = D_RO(lsb)->slots[D_RO(lsb)->n]; TX_ADD_FIELD(lsb, n); D_RW(lsb)->n -= 1; /* it loses one element, but still > min */ } /* * btree_map_merge -- (internal) merges node and right sibling */ static void btree_map_merge(TOID(struct btree_map) map, TOID(struct tree_map_node) rn, TOID(struct tree_map_node) node, TOID(struct tree_map_node) parent, int p) { struct tree_map_node_item sep = D_RO(parent)->items[p]; TX_ADD(node); /* add separator to the deficient node */ D_RW(node)->items[D_RW(node)->n++] = sep; /* copy right sibling data to node */ memcpy(&D_RW(node)->items[D_RO(node)->n], D_RO(rn)->items, sizeof(struct tree_map_node_item) * D_RO(rn)->n); memcpy(&D_RW(node)->slots[D_RO(node)->n], D_RO(rn)->slots, sizeof(TOID(struct tree_map_node)) * (D_RO(rn)->n + 1)); D_RW(node)->n += D_RO(rn)->n; TX_FREE(rn); /* right node is now empty */ TX_ADD(parent); D_RW(parent)->n -= 1; /* move everything to the right of the separator by one array slot */ memmove(D_RW(parent)->items + p, D_RW(parent)->items + p + 1, sizeof(struct tree_map_node_item) * (D_RO(parent)->n - p)); memmove(D_RW(parent)->slots + p + 1, D_RW(parent)->slots + p + 2, sizeof(TOID(struct tree_map_node)) * (D_RO(parent)->n - p + 1)); /* if the parent is empty then the tree shrinks in height */ if (D_RO(parent)->n == 0 && TOID_EQUALS(parent, D_RO(map)->root)) { TX_ADD(map); TX_FREE(D_RO(map)->root); D_RW(map)->root = node; } } /* * btree_map_rebalance -- (internal) performs tree rebalance */ static void btree_map_rebalance(TOID(struct btree_map) map, TOID(struct tree_map_node) node, TOID(struct tree_map_node) parent, int p) { TOID(struct tree_map_node) rsb = p >= D_RO(parent)->n ? TOID_NULL(struct tree_map_node) : D_RO(parent)->slots[p + 1]; TOID(struct tree_map_node) lsb = p == 0 ? TOID_NULL(struct tree_map_node) : D_RO(parent)->slots[p - 1]; if (!TOID_IS_NULL(rsb) && D_RO(rsb)->n > BTREE_MIN) btree_map_rotate_right(rsb, node, parent, p); else if (!TOID_IS_NULL(lsb) && D_RO(lsb)->n > BTREE_MIN) btree_map_rotate_left(lsb, node, parent, p); else if (TOID_IS_NULL(rsb)) /* always merge with rightmost node */ btree_map_merge(map, node, lsb, parent, p - 1); else btree_map_merge(map, rsb, node, parent, p); } /* * btree_map_get_leftmost_leaf -- (internal) searches for the successor */ static TOID(struct tree_map_node) btree_map_get_leftmost_leaf(TOID(struct btree_map) map, TOID(struct tree_map_node) n, TOID(struct tree_map_node) *p) { if (TOID_IS_NULL(D_RO(n)->slots[0])) return n; *p = n; return btree_map_get_leftmost_leaf(map, D_RO(n)->slots[0], p); } /* * btree_map_remove_from_node -- (internal) removes element from node */ static void btree_map_remove_from_node(TOID(struct btree_map) map, TOID(struct tree_map_node) node, TOID(struct tree_map_node) parent, int p) { if (TOID_IS_NULL(D_RO(node)->slots[0])) { /* leaf */ TX_ADD(node); if (D_RO(node)->n == 1 || p == BTREE_ORDER - 2) { set_empty_item(&D_RW(node)->items[p]); } else if (D_RO(node)->n != 1) { memmove(&D_RW(node)->items[p], &D_RW(node)->items[p + 1], sizeof(struct tree_map_node_item) * (D_RO(node)->n - p)); } D_RW(node)->n -= 1; return; } /* can't delete from non-leaf nodes, remove successor */ TOID(struct tree_map_node) rchild = D_RW(node)->slots[p + 1]; TOID(struct tree_map_node) lp = node; TOID(struct tree_map_node) lm = btree_map_get_leftmost_leaf(map, rchild, &lp); TX_ADD_FIELD(node, items[p]); D_RW(node)->items[p] = D_RO(lm)->items[0]; btree_map_remove_from_node(map, lm, lp, 0); if (D_RO(lm)->n < BTREE_MIN) /* right child can be deficient now */ btree_map_rebalance(map, lm, lp, TOID_EQUALS(lp, node) ? p + 1 : 0); } #define NODE_CONTAINS_ITEM(_n, _i, _k)\ ((_i) != D_RO(_n)->n && D_RO(_n)->items[_i].key == (_k)) #define NODE_CHILD_CAN_CONTAIN_ITEM(_n, _i, _k)\ ((_i) == D_RO(_n)->n || D_RO(_n)->items[_i].key > (_k)) &&\ !TOID_IS_NULL(D_RO(_n)->slots[_i]) /* * btree_map_remove_item -- (internal) removes item from node */ static PMEMoid btree_map_remove_item(TOID(struct btree_map) map, TOID(struct tree_map_node) node, TOID(struct tree_map_node) parent, uint64_t key, int p) { PMEMoid ret = OID_NULL; for (int i = 0; i <= D_RO(node)->n; ++i) { if (NODE_CONTAINS_ITEM(node, i, key)) { ret = D_RO(node)->items[i].value; btree_map_remove_from_node(map, node, parent, i); break; } else if (NODE_CHILD_CAN_CONTAIN_ITEM(node, i, key)) { ret = btree_map_remove_item(map, D_RO(node)->slots[i], node, key, i); break; } } /* check for deficient nodes walking up */ if (!TOID_IS_NULL(parent) && D_RO(node)->n < BTREE_MIN) btree_map_rebalance(map, node, parent, p); return ret; } /* * btree_map_remove -- removes key-value pair from the map */ PMEMoid btree_map_remove(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key) { PMEMoid ret = OID_NULL; TX_BEGIN(pop) { ret = btree_map_remove_item(map, D_RW(map)->root, TOID_NULL(struct tree_map_node), key, 0); } TX_END return ret; } /* * btree_map_get_in_node -- (internal) searches for a value in the node */ static PMEMoid btree_map_get_in_node(TOID(struct tree_map_node) node, uint64_t key) { for (int i = 0; i <= D_RO(node)->n; ++i) { if (NODE_CONTAINS_ITEM(node, i, key)) return D_RO(node)->items[i].value; else if (NODE_CHILD_CAN_CONTAIN_ITEM(node, i, key)) return btree_map_get_in_node(D_RO(node)->slots[i], key); } return OID_NULL; } /* * btree_map_get -- searches for a value of the key */ PMEMoid btree_map_get(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key) { if (TOID_IS_NULL(D_RO(map)->root)) return OID_NULL; return btree_map_get_in_node(D_RO(map)->root, key); } /* * btree_map_lookup_in_node -- (internal) searches for key if exists */ static int btree_map_lookup_in_node(TOID(struct tree_map_node) node, uint64_t key) { for (int i = 0; i <= D_RO(node)->n; ++i) { if (NODE_CONTAINS_ITEM(node, i, key)) return 1; else if (NODE_CHILD_CAN_CONTAIN_ITEM(node, i, key)) return btree_map_lookup_in_node( D_RO(node)->slots[i], key); } return 0; } /* * btree_map_lookup -- searches if key exists */ int btree_map_lookup(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key) { if (TOID_IS_NULL(D_RO(map)->root)) return 0; return btree_map_lookup_in_node(D_RO(map)->root, key); } /* * btree_map_foreach_node -- (internal) recursively traverses tree */ static int btree_map_foreach_node(const TOID(struct tree_map_node) p, int (*cb)(uint64_t key, PMEMoid, void *arg), void *arg) { if (TOID_IS_NULL(p)) return 0; for (int i = 0; i <= D_RO(p)->n; ++i) { if (btree_map_foreach_node(D_RO(p)->slots[i], cb, arg) != 0) return 1; if (i != D_RO(p)->n && D_RO(p)->items[i].key != 0) { if (cb(D_RO(p)->items[i].key, D_RO(p)->items[i].value, arg) != 0) return 1; } } return 0; } /* * btree_map_foreach -- initiates recursive traversal */ int btree_map_foreach(PMEMobjpool *pop, TOID(struct btree_map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { return btree_map_foreach_node(D_RO(map)->root, cb, arg); } /* * ctree_map_check -- check if given persistent object is a tree map */ int btree_map_check(PMEMobjpool *pop, TOID(struct btree_map) map) { return TOID_IS_NULL(map) || !TOID_VALID(map); } /* * btree_map_insert_new -- allocates a new object and inserts it into the tree */ int btree_map_insert_new(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { int ret = 0; TX_BEGIN(pop) { PMEMoid n = pmemobj_tx_alloc(size, type_num); constructor(pop, pmemobj_direct(n), arg); btree_map_insert(pop, map, key, n); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * btree_map_remove_free -- removes and frees an object from the tree */ int btree_map_remove_free(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key) { int ret = 0; TX_BEGIN(pop) { PMEMoid val = btree_map_remove(pop, map, key); pmemobj_tx_free(val); } TX_ONABORT { ret = 1; } TX_END return ret; } pmdk-1.8/src/examples/libpmemobj/tree_map/rbtree_map.h0000664000000000000000000000600013615011243021617 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rbtree_map.h -- TreeMap sorted collection implementation */ #ifndef RBTREE_MAP_H #define RBTREE_MAP_H #include #ifndef RBTREE_MAP_TYPE_OFFSET #define RBTREE_MAP_TYPE_OFFSET 1016 #endif struct rbtree_map; TOID_DECLARE(struct rbtree_map, RBTREE_MAP_TYPE_OFFSET + 0); int rbtree_map_check(PMEMobjpool *pop, TOID(struct rbtree_map) map); int rbtree_map_create(PMEMobjpool *pop, TOID(struct rbtree_map) *map, void *arg); int rbtree_map_destroy(PMEMobjpool *pop, TOID(struct rbtree_map) *map); int rbtree_map_insert(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key, PMEMoid value); int rbtree_map_insert_new(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg); PMEMoid rbtree_map_remove(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key); int rbtree_map_remove_free(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key); int rbtree_map_clear(PMEMobjpool *pop, TOID(struct rbtree_map) map); PMEMoid rbtree_map_get(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key); int rbtree_map_lookup(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key); int rbtree_map_foreach(PMEMobjpool *pop, TOID(struct rbtree_map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg); int rbtree_map_is_empty(PMEMobjpool *pop, TOID(struct rbtree_map) map); #endif /* RBTREE_MAP_H */ pmdk-1.8/src/examples/libpmemobj/tree_map/rbtree_map.vcxproj.filters0000664000000000000000000000124413615011243024537 0ustar rootroot {6b3d53f9-3187-4f70-9d79-a6c29e123af5} {d5387ced-fc6e-4980-a4ba-5112aca37cc0} Source Files Header Files pmdk-1.8/src/examples/libpmemobj/tree_map/btree_map.vcxproj0000664000000000000000000000523013615011243022705 0ustar rootroot Debug x64 Release x64 {79D37FFE-FF76-44B3-BB27-3DCAEFF2EBE9} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} StaticLibrary true v140 StaticLibrary false v140 true ..\..\..\LongPath.manifest CompileAsCpp pmdk-1.8/src/examples/libpmemobj/tree_map/btree_map.h0000664000000000000000000000573613615011243021454 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * btree_map.h -- TreeMap sorted collection implementation */ #ifndef BTREE_MAP_H #define BTREE_MAP_H #include #ifndef BTREE_MAP_TYPE_OFFSET #define BTREE_MAP_TYPE_OFFSET 1012 #endif struct btree_map; TOID_DECLARE(struct btree_map, BTREE_MAP_TYPE_OFFSET + 0); int btree_map_check(PMEMobjpool *pop, TOID(struct btree_map) map); int btree_map_create(PMEMobjpool *pop, TOID(struct btree_map) *map, void *arg); int btree_map_destroy(PMEMobjpool *pop, TOID(struct btree_map) *map); int btree_map_insert(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key, PMEMoid value); int btree_map_insert_new(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg); PMEMoid btree_map_remove(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key); int btree_map_remove_free(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key); int btree_map_clear(PMEMobjpool *pop, TOID(struct btree_map) map); PMEMoid btree_map_get(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key); int btree_map_lookup(PMEMobjpool *pop, TOID(struct btree_map) map, uint64_t key); int btree_map_foreach(PMEMobjpool *pop, TOID(struct btree_map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg); int btree_map_is_empty(PMEMobjpool *pop, TOID(struct btree_map) map); #endif /* BTREE_MAP_H */ pmdk-1.8/src/examples/libpmemobj/tree_map/rbtree_map.vcxproj0000664000000000000000000000523213615011243023071 0ustar rootroot Debug x64 Release x64 {17A4B817-68B1-4719-A9EF-BD8FAB747DE6} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} StaticLibrary true v140 StaticLibrary false v140 true ..\..\..\LongPath.manifest CompileAsCpp pmdk-1.8/src/examples/libpmemobj/tree_map/ctree_map.h0000664000000000000000000000573613615011243021455 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * ctree_map.h -- TreeMap sorted collection implementation */ #ifndef CTREE_MAP_H #define CTREE_MAP_H #include #ifndef CTREE_MAP_TYPE_OFFSET #define CTREE_MAP_TYPE_OFFSET 1008 #endif struct ctree_map; TOID_DECLARE(struct ctree_map, CTREE_MAP_TYPE_OFFSET + 0); int ctree_map_check(PMEMobjpool *pop, TOID(struct ctree_map) map); int ctree_map_create(PMEMobjpool *pop, TOID(struct ctree_map) *map, void *arg); int ctree_map_destroy(PMEMobjpool *pop, TOID(struct ctree_map) *map); int ctree_map_insert(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key, PMEMoid value); int ctree_map_insert_new(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg); PMEMoid ctree_map_remove(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key); int ctree_map_remove_free(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key); int ctree_map_clear(PMEMobjpool *pop, TOID(struct ctree_map) map); PMEMoid ctree_map_get(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key); int ctree_map_lookup(PMEMobjpool *pop, TOID(struct ctree_map) map, uint64_t key); int ctree_map_foreach(PMEMobjpool *pop, TOID(struct ctree_map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg); int ctree_map_is_empty(PMEMobjpool *pop, TOID(struct ctree_map) map); #endif /* CTREE_MAP_H */ pmdk-1.8/src/examples/libpmemobj/tree_map/ctree_map.vcxproj.filters0000664000000000000000000000124213615011243024354 0ustar rootroot {ae1612b3-efcb-4228-8946-071051e0395a} {4dbe7d5b-3e5f-42cb-a55a-eba2f613e9c5} Source Files Header Files pmdk-1.8/src/examples/libpmemobj/tree_map/rtree_map.vcxproj0000664000000000000000000000532613615011243022733 0ustar rootroot Debug x64 Release x64 {3ED56E55-84A6-422C-A8D4-A8439FB8F245} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} StaticLibrary true v140 StaticLibrary false v140 true ..\..\..\LongPath.manifest CompileAsCpp 4200 pmdk-1.8/src/examples/libpmemobj/tree_map/rtree_map.h0000664000000000000000000000626613615011243021473 0ustar rootroot/* * Copyright 2016, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rtree_map.h -- Radix TreeMap collection implementation */ #ifndef RTREE_MAP_H #define RTREE_MAP_H #include #ifndef RTREE_MAP_TYPE_OFFSET #define RTREE_MAP_TYPE_OFFSET 1020 #endif struct rtree_map; TOID_DECLARE(struct rtree_map, RTREE_MAP_TYPE_OFFSET + 0); int rtree_map_check(PMEMobjpool *pop, TOID(struct rtree_map) map); int rtree_map_create(PMEMobjpool *pop, TOID(struct rtree_map) *map, void *arg); int rtree_map_destroy(PMEMobjpool *pop, TOID(struct rtree_map) *map); int rtree_map_insert(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size, PMEMoid value); int rtree_map_insert_new(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg); PMEMoid rtree_map_remove(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size); int rtree_map_remove_free(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size); int rtree_map_clear(PMEMobjpool *pop, TOID(struct rtree_map) map); PMEMoid rtree_map_get(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size); int rtree_map_lookup(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size); int rtree_map_foreach(PMEMobjpool *pop, TOID(struct rtree_map) map, int (*cb)(const unsigned char *key, uint64_t key_size, PMEMoid value, void *arg), void *arg); int rtree_map_is_empty(PMEMobjpool *pop, TOID(struct rtree_map) map); #endif /* RTREE_MAP_H */ pmdk-1.8/src/examples/libpmemobj/tree_map/rtree_map.c0000664000000000000000000003363113615011243021462 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rtree_map.c -- implementation of rtree */ #include #include #include #include #include #include "rtree_map.h" TOID_DECLARE(struct tree_map_node, RTREE_MAP_TYPE_OFFSET + 1); /* Good values: 0x10 an 0x100, but implementation is bound to 0x100 */ #ifndef ALPHABET_SIZE #define ALPHABET_SIZE 0x100 #endif struct tree_map_node { TOID(struct tree_map_node) slots[ALPHABET_SIZE]; unsigned has_value; PMEMoid value; uint64_t key_size; unsigned char key[]; }; struct rtree_map { TOID(struct tree_map_node) root; }; /* * rtree_map_create -- allocates a new rtree instance */ int rtree_map_create(PMEMobjpool *pop, TOID(struct rtree_map) *map, void *arg) { int ret = 0; TX_BEGIN(pop) { TX_ADD_DIRECT(map); *map = TX_ZNEW(struct rtree_map); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * rtree_map_clear_node -- (internal) removes all elements from the node */ static void rtree_map_clear_node(TOID(struct tree_map_node) node) { for (unsigned i = 0; i < ALPHABET_SIZE; i++) { rtree_map_clear_node(D_RO(node)->slots[i]); } pmemobj_tx_add_range(node.oid, 0, sizeof(struct tree_map_node) + D_RO(node)->key_size); TX_FREE(node); } /* * rtree_map_clear -- removes all elements from the map */ int rtree_map_clear(PMEMobjpool *pop, TOID(struct rtree_map) map) { int ret = 0; TX_BEGIN(pop) { rtree_map_clear_node(D_RO(map)->root); TX_ADD_FIELD(map, root); D_RW(map)->root = TOID_NULL(struct tree_map_node); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * rtree_map_destroy -- cleanups and frees rtree instance */ int rtree_map_destroy(PMEMobjpool *pop, TOID(struct rtree_map) *map) { int ret = 0; TX_BEGIN(pop) { rtree_map_clear(pop, *map); TX_ADD_DIRECT(map); TX_FREE(*map); *map = TOID_NULL(struct rtree_map); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * rtree_new_node -- (internal) inserts a node into an empty map */ static TOID(struct tree_map_node) rtree_new_node(const unsigned char *key, uint64_t key_size, PMEMoid value, unsigned has_value) { TOID(struct tree_map_node) node; node = TX_ZALLOC(struct tree_map_node, sizeof(struct tree_map_node) + key_size); /* * !!! Here should be: D_RO(node)->value * ... because we don't change map */ D_RW(node)->value = value; D_RW(node)->has_value = has_value; D_RW(node)->key_size = key_size; memcpy(D_RW(node)->key, key, key_size); return node; } /* * rtree_map_insert_empty -- (internal) inserts a node into an empty map */ static void rtree_map_insert_empty(TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size, PMEMoid value) { TX_ADD_FIELD(map, root); D_RW(map)->root = rtree_new_node(key, key_size, value, 1); } /* * key_comm_len -- (internal) calculate the len of common part of keys */ static unsigned key_comm_len(TOID(struct tree_map_node) node, const unsigned char *key, uint64_t key_size) { unsigned i; for (i = 0; i < MIN(key_size, D_RO(node)->key_size) && key[i] == D_RO(node)->key[i]; i++) ; return i; } /* * rtree_map_insert_value -- (internal) inserts a pair into a tree */ static void rtree_map_insert_value(TOID(struct tree_map_node) *node, const unsigned char *key, uint64_t key_size, PMEMoid value) { unsigned i; if (TOID_IS_NULL(*node)) { TX_ADD_DIRECT(node); *node = rtree_new_node(key, key_size, value, 1); return; } i = key_comm_len(*node, key, key_size); if (i != D_RO(*node)->key_size) { /* Node does not exist. Let's add. */ TOID(struct tree_map_node) orig_node = *node; TX_ADD_DIRECT(node); if (i != key_size) { *node = rtree_new_node(D_RO(orig_node)->key, i, OID_NULL, 0); } else { *node = rtree_new_node(D_RO(orig_node)->key, i, value, 1); } D_RW(*node)->slots[D_RO(orig_node)->key[i]] = orig_node; TX_ADD_FIELD(orig_node, key_size); D_RW(orig_node)->key_size -= i; pmemobj_tx_add_range_direct(D_RW(orig_node)->key, D_RO(orig_node)->key_size); memmove(D_RW(orig_node)->key, D_RO(orig_node)->key + i, D_RO(orig_node)->key_size); if (i != key_size) { D_RW(*node)->slots[key[i]] = rtree_new_node(key + i, key_size - i, value, 1); } return; } if (i == key_size) { if (OID_IS_NULL(D_RO(*node)->value) || D_RO(*node)->has_value) { /* Just replace old value with new */ TX_ADD_FIELD(*node, value); TX_ADD_FIELD(*node, has_value); D_RW(*node)->value = value; D_RW(*node)->has_value = 1; } else { /* * Ignore. By the fact current value should be * removed in advance, or handled in a different way. */ } } else { /* Recurse deeply */ return rtree_map_insert_value(&D_RW(*node)->slots[key[i]], key + i, key_size - i, value); } } /* * rtree_map_is_empty -- checks whether the tree map is empty */ int rtree_map_is_empty(PMEMobjpool *pop, TOID(struct rtree_map) map) { return TOID_IS_NULL(D_RO(map)->root); } /* * rtree_map_insert -- inserts a new key-value pair into the map */ int rtree_map_insert(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size, PMEMoid value) { int ret = 0; TX_BEGIN(pop) { if (rtree_map_is_empty(pop, map)) { rtree_map_insert_empty(map, key, key_size, value); } else { rtree_map_insert_value(&D_RW(map)->root, key, key_size, value); } } TX_ONABORT { ret = 1; } TX_END return ret; } /* * rtree_map_insert_new -- allocates a new object and inserts it into the tree */ int rtree_map_insert_new(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { int ret = 0; TX_BEGIN(pop) { PMEMoid n = pmemobj_tx_alloc(size, type_num); constructor(pop, pmemobj_direct(n), arg); rtree_map_insert(pop, map, key, key_size, n); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * is_leaf -- (internal) check a node for zero qty of children */ static bool is_leaf(TOID(struct tree_map_node) node) { unsigned j; for (j = 0; j < ALPHABET_SIZE && TOID_IS_NULL(D_RO(node)->slots[j]); j++) ; return (j == ALPHABET_SIZE); } /* * has_only_one_child -- (internal) check a node for qty of children */ static bool has_only_one_child(TOID(struct tree_map_node) node, unsigned *child_idx) { unsigned j, child_qty; for (j = 0, child_qty = 0; j < ALPHABET_SIZE; j++) if (!TOID_IS_NULL(D_RO(node)->slots[j])) { child_qty++; *child_idx = j; } return (1 == child_qty); } /* * remove_extra_node -- (internal) remove unneeded extra node */ static void remove_extra_node(TOID(struct tree_map_node) *node) { unsigned child_idx; TOID(struct tree_map_node) tmp, tmp_child; /* Our node has child with only one child. */ tmp = *node; has_only_one_child(tmp, &child_idx); tmp_child = D_RO(tmp)->slots[child_idx]; /* * That child's incoming label is appended to the ours incoming label * and the child is removed. */ uint64_t new_key_size = D_RO(tmp)->key_size + D_RO(tmp_child)->key_size; unsigned char *new_key = (unsigned char *)malloc(new_key_size); assert(new_key != NULL); memcpy(new_key, D_RO(tmp)->key, D_RO(tmp)->key_size); memcpy(new_key + D_RO(tmp)->key_size, D_RO(tmp_child)->key, D_RO(tmp_child)->key_size); TX_ADD_DIRECT(node); *node = rtree_new_node(new_key, new_key_size, D_RO(tmp_child)->value, D_RO(tmp_child)->has_value); free(new_key); TX_FREE(tmp); memcpy(D_RW(*node)->slots, D_RO(tmp_child)->slots, sizeof(D_RO(tmp_child)->slots)); TX_FREE(tmp_child); } /* * rtree_map_remove_node -- (internal) removes node from tree */ static PMEMoid rtree_map_remove_node(TOID(struct rtree_map) map, TOID(struct tree_map_node) *node, const unsigned char *key, uint64_t key_size, bool *check_for_child) { bool c4c; unsigned i, child_idx; PMEMoid ret = OID_NULL; *check_for_child = false; if (TOID_IS_NULL(*node)) return OID_NULL; i = key_comm_len(*node, key, key_size); if (i != D_RO(*node)->key_size) /* Node does not exist */ return OID_NULL; if (i == key_size) { if (0 == D_RO(*node)->has_value) return OID_NULL; /* Node is found */ ret = D_RO(*node)->value; /* delete node from tree */ TX_ADD_FIELD((*node), value); TX_ADD_FIELD((*node), has_value); D_RW(*node)->value = OID_NULL; D_RW(*node)->has_value = 0; if (is_leaf(*node)) { pmemobj_tx_add_range(node->oid, 0, sizeof(*node) + D_RO(*node)->key_size); TX_FREE(*node); TX_ADD_DIRECT(node); (*node) = TOID_NULL(struct tree_map_node); } return ret; } /* Recurse deeply */ ret = rtree_map_remove_node(map, &D_RW(*node)->slots[key[i]], key + i, key_size - i, &c4c); if (c4c) { /* Our node has child with only one child. Remove. */ remove_extra_node(&D_RW(*node)->slots[key[i]]); return ret; } if (has_only_one_child(*node, &child_idx) && (0 == D_RO(*node)->has_value)) { *check_for_child = true; } return ret; } /* * rtree_map_remove -- removes key-value pair from the map */ PMEMoid rtree_map_remove(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size) { PMEMoid ret = OID_NULL; bool check_for_child; if (TOID_IS_NULL(map)) return OID_NULL; TX_BEGIN(pop) { ret = rtree_map_remove_node(map, &D_RW(map)->root, key, key_size, &check_for_child); if (check_for_child) { /* Our root node has only one child. Remove. */ remove_extra_node(&D_RW(map)->root); } } TX_END return ret; } /* * rtree_map_remove_free -- removes and frees an object from the tree */ int rtree_map_remove_free(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size) { int ret = 0; if (TOID_IS_NULL(map)) return 1; TX_BEGIN(pop) { pmemobj_tx_free(rtree_map_remove(pop, map, key, key_size)); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * rtree_map_get_in_node -- (internal) searches for a value in the node */ static PMEMoid rtree_map_get_in_node(TOID(struct tree_map_node) node, const unsigned char *key, uint64_t key_size) { unsigned i; if (TOID_IS_NULL(node)) return OID_NULL; i = key_comm_len(node, key, key_size); if (i != D_RO(node)->key_size) /* Node does not exist */ return OID_NULL; if (i == key_size) { /* Node is found */ return D_RO(node)->value; } else { /* Recurse deeply */ return rtree_map_get_in_node(D_RO(node)->slots[key[i]], key + i, key_size - i); } } /* * rtree_map_get -- searches for a value of the key */ PMEMoid rtree_map_get(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size) { if (TOID_IS_NULL(D_RO(map)->root)) return OID_NULL; return rtree_map_get_in_node(D_RO(map)->root, key, key_size); } /* * rtree_map_lookup_in_node -- (internal) searches for key if exists */ static int rtree_map_lookup_in_node(TOID(struct tree_map_node) node, const unsigned char *key, uint64_t key_size) { unsigned i; if (TOID_IS_NULL(node)) return 0; i = key_comm_len(node, key, key_size); if (i != D_RO(node)->key_size) /* Node does not exist */ return 0; if (i == key_size) { /* Node is found */ return 1; } /* Recurse deeply */ return rtree_map_lookup_in_node(D_RO(node)->slots[key[i]], key + i, key_size - i); } /* * rtree_map_lookup -- searches if key exists */ int rtree_map_lookup(PMEMobjpool *pop, TOID(struct rtree_map) map, const unsigned char *key, uint64_t key_size) { if (TOID_IS_NULL(D_RO(map)->root)) return 0; return rtree_map_lookup_in_node(D_RO(map)->root, key, key_size); } /* * rtree_map_foreach_node -- (internal) recursively traverses tree */ static int rtree_map_foreach_node(const TOID(struct tree_map_node) node, int (*cb)(const unsigned char *key, uint64_t key_size, PMEMoid, void *arg), void *arg) { unsigned i; if (TOID_IS_NULL(node)) return 0; for (i = 0; i < ALPHABET_SIZE; i++) { if (rtree_map_foreach_node(D_RO(node)->slots[i], cb, arg) != 0) return 1; } if (NULL != cb) { if (cb(D_RO(node)->key, D_RO(node)->key_size, D_RO(node)->value, arg) != 0) return 1; } return 0; } /* * rtree_map_foreach -- initiates recursive traversal */ int rtree_map_foreach(PMEMobjpool *pop, TOID(struct rtree_map) map, int (*cb)(const unsigned char *key, uint64_t key_size, PMEMoid value, void *arg), void *arg) { return rtree_map_foreach_node(D_RO(map)->root, cb, arg); } /* * ctree_map_check -- check if given persistent object is a tree map */ int rtree_map_check(PMEMobjpool *pop, TOID(struct rtree_map) map) { return TOID_IS_NULL(map) || !TOID_VALID(map); } pmdk-1.8/src/examples/libpmemobj/tree_map/Makefile0000664000000000000000000000350213615011243020772 0ustar rootroot# # Copyright 2015-2017, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/tree_map/Makefile -- build the tree map example # LIBRARIES = ctree_map btree_map rtree_map rbtree_map LIBS = -lpmemobj include ../../Makefile.inc libctree_map.o: ctree_map.o libbtree_map.o: btree_map.o librtree_map.o: rtree_map.o librbtree_map.o: rbtree_map.o pmdk-1.8/src/examples/libpmemobj/tree_map/rtree_map.vcxproj.filters0000664000000000000000000000124213615011243024373 0ustar rootroot {662f1b6f-2944-4a24-867d-1945f972ad8b} {ba8f3f5c-98e6-4dba-ac8f-fc44e1a45af8} Source Files Header Files pmdk-1.8/src/examples/libpmemobj/tree_map/rbtree_map.c0000664000000000000000000003273513615011243021630 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * rbtree.c -- red-black tree implementation /w sentinel nodes */ #include #include #include "rbtree_map.h" TOID_DECLARE(struct tree_map_node, RBTREE_MAP_TYPE_OFFSET + 1); #define NODE_P(_n)\ D_RW(_n)->parent #define NODE_GRANDP(_n)\ NODE_P(NODE_P(_n)) #define NODE_PARENT_AT(_n, _rbc)\ D_RW(NODE_P(_n))->slots[_rbc] #define NODE_PARENT_RIGHT(_n)\ NODE_PARENT_AT(_n, RB_RIGHT) #define NODE_IS(_n, _rbc)\ TOID_EQUALS(_n, NODE_PARENT_AT(_n, _rbc)) #define NODE_IS_RIGHT(_n)\ TOID_EQUALS(_n, NODE_PARENT_RIGHT(_n)) #define NODE_LOCATION(_n)\ NODE_IS_RIGHT(_n) #define RB_FIRST(_m)\ D_RW(D_RW(_m)->root)->slots[RB_LEFT] #define NODE_IS_NULL(_n)\ TOID_EQUALS(_n, s) enum rb_color { COLOR_BLACK, COLOR_RED, MAX_COLOR }; enum rb_children { RB_LEFT, RB_RIGHT, MAX_RB }; struct tree_map_node { uint64_t key; PMEMoid value; enum rb_color color; TOID(struct tree_map_node) parent; TOID(struct tree_map_node) slots[MAX_RB]; }; struct rbtree_map { TOID(struct tree_map_node) sentinel; TOID(struct tree_map_node) root; }; /* * rbtree_map_create -- allocates a new red-black tree instance */ int rbtree_map_create(PMEMobjpool *pop, TOID(struct rbtree_map) *map, void *arg) { int ret = 0; TX_BEGIN(pop) { pmemobj_tx_add_range_direct(map, sizeof(*map)); *map = TX_ZNEW(struct rbtree_map); TOID(struct tree_map_node) s = TX_ZNEW(struct tree_map_node); D_RW(s)->color = COLOR_BLACK; D_RW(s)->parent = s; D_RW(s)->slots[RB_LEFT] = s; D_RW(s)->slots[RB_RIGHT] = s; TOID(struct tree_map_node) r = TX_ZNEW(struct tree_map_node); D_RW(r)->color = COLOR_BLACK; D_RW(r)->parent = s; D_RW(r)->slots[RB_LEFT] = s; D_RW(r)->slots[RB_RIGHT] = s; D_RW(*map)->sentinel = s; D_RW(*map)->root = r; } TX_ONABORT { ret = 1; } TX_END return ret; } /* * rbtree_map_clear_node -- (internal) clears this node and its children */ static void rbtree_map_clear_node(TOID(struct rbtree_map) map, TOID(struct tree_map_node) p) { TOID(struct tree_map_node) s = D_RO(map)->sentinel; if (!NODE_IS_NULL(D_RO(p)->slots[RB_LEFT])) rbtree_map_clear_node(map, D_RO(p)->slots[RB_LEFT]); if (!NODE_IS_NULL(D_RO(p)->slots[RB_RIGHT])) rbtree_map_clear_node(map, D_RO(p)->slots[RB_RIGHT]); TX_FREE(p); } /* * rbtree_map_clear -- removes all elements from the map */ int rbtree_map_clear(PMEMobjpool *pop, TOID(struct rbtree_map) map) { TX_BEGIN(pop) { rbtree_map_clear_node(map, D_RW(map)->root); TX_ADD_FIELD(map, root); TX_ADD_FIELD(map, sentinel); TX_FREE(D_RW(map)->sentinel); D_RW(map)->root = TOID_NULL(struct tree_map_node); D_RW(map)->sentinel = TOID_NULL(struct tree_map_node); } TX_END return 0; } /* * rbtree_map_destroy -- cleanups and frees red-black tree instance */ int rbtree_map_destroy(PMEMobjpool *pop, TOID(struct rbtree_map) *map) { int ret = 0; TX_BEGIN(pop) { rbtree_map_clear(pop, *map); pmemobj_tx_add_range_direct(map, sizeof(*map)); TX_FREE(*map); *map = TOID_NULL(struct rbtree_map); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * rbtree_map_rotate -- (internal) performs a left/right rotation around a node */ static void rbtree_map_rotate(TOID(struct rbtree_map) map, TOID(struct tree_map_node) node, enum rb_children c) { TOID(struct tree_map_node) child = D_RO(node)->slots[!c]; TOID(struct tree_map_node) s = D_RO(map)->sentinel; TX_ADD(node); TX_ADD(child); D_RW(node)->slots[!c] = D_RO(child)->slots[c]; if (!TOID_EQUALS(D_RO(child)->slots[c], s)) TX_SET(D_RW(child)->slots[c], parent, node); NODE_P(child) = NODE_P(node); TX_SET(NODE_P(node), slots[NODE_LOCATION(node)], child); D_RW(child)->slots[c] = node; D_RW(node)->parent = child; } /* * rbtree_map_insert_bst -- (internal) inserts a node in regular BST fashion */ static void rbtree_map_insert_bst(TOID(struct rbtree_map) map, TOID(struct tree_map_node) n) { TOID(struct tree_map_node) parent = D_RO(map)->root; TOID(struct tree_map_node) *dst = &RB_FIRST(map); TOID(struct tree_map_node) s = D_RO(map)->sentinel; D_RW(n)->slots[RB_LEFT] = s; D_RW(n)->slots[RB_RIGHT] = s; while (!NODE_IS_NULL(*dst)) { parent = *dst; dst = &D_RW(*dst)->slots[D_RO(n)->key > D_RO(*dst)->key]; } TX_SET(n, parent, parent); pmemobj_tx_add_range_direct(dst, sizeof(*dst)); *dst = n; } /* * rbtree_map_recolor -- (internal) restores red-black tree properties */ static TOID(struct tree_map_node) rbtree_map_recolor(TOID(struct rbtree_map) map, TOID(struct tree_map_node) n, enum rb_children c) { TOID(struct tree_map_node) uncle = D_RO(NODE_GRANDP(n))->slots[!c]; if (D_RO(uncle)->color == COLOR_RED) { TX_SET(uncle, color, COLOR_BLACK); TX_SET(NODE_P(n), color, COLOR_BLACK); TX_SET(NODE_GRANDP(n), color, COLOR_RED); return NODE_GRANDP(n); } else { if (NODE_IS(n, !c)) { n = NODE_P(n); rbtree_map_rotate(map, n, c); } TX_SET(NODE_P(n), color, COLOR_BLACK); TX_SET(NODE_GRANDP(n), color, COLOR_RED); rbtree_map_rotate(map, NODE_GRANDP(n), (enum rb_children)!c); } return n; } /* * rbtree_map_insert -- inserts a new key-value pair into the map */ int rbtree_map_insert(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key, PMEMoid value) { int ret = 0; TX_BEGIN(pop) { TOID(struct tree_map_node) n = TX_ZNEW(struct tree_map_node); D_RW(n)->key = key; D_RW(n)->value = value; rbtree_map_insert_bst(map, n); D_RW(n)->color = COLOR_RED; while (D_RO(NODE_P(n))->color == COLOR_RED) n = rbtree_map_recolor(map, n, (enum rb_children) NODE_LOCATION(NODE_P(n))); TX_SET(RB_FIRST(map), color, COLOR_BLACK); } TX_END return ret; } /* * rbtree_map_successor -- (internal) returns the successor of a node */ static TOID(struct tree_map_node) rbtree_map_successor(TOID(struct rbtree_map) map, TOID(struct tree_map_node) n) { TOID(struct tree_map_node) dst = D_RO(n)->slots[RB_RIGHT]; TOID(struct tree_map_node) s = D_RO(map)->sentinel; if (!TOID_EQUALS(s, dst)) { while (!NODE_IS_NULL(D_RO(dst)->slots[RB_LEFT])) dst = D_RO(dst)->slots[RB_LEFT]; } else { dst = D_RO(n)->parent; while (TOID_EQUALS(n, D_RO(dst)->slots[RB_RIGHT])) { n = dst; dst = NODE_P(dst); } if (TOID_EQUALS(dst, D_RO(map)->root)) return s; } return dst; } /* * rbtree_map_find_node -- (internal) returns the node that contains the key */ static TOID(struct tree_map_node) rbtree_map_find_node(TOID(struct rbtree_map) map, uint64_t key) { TOID(struct tree_map_node) dst = RB_FIRST(map); TOID(struct tree_map_node) s = D_RO(map)->sentinel; while (!NODE_IS_NULL(dst)) { if (D_RO(dst)->key == key) return dst; dst = D_RO(dst)->slots[key > D_RO(dst)->key]; } return TOID_NULL(struct tree_map_node); } /* * rbtree_map_repair_branch -- (internal) restores red-black tree in one branch */ static TOID(struct tree_map_node) rbtree_map_repair_branch(TOID(struct rbtree_map) map, TOID(struct tree_map_node) n, enum rb_children c) { TOID(struct tree_map_node) sb = NODE_PARENT_AT(n, !c); /* sibling */ if (D_RO(sb)->color == COLOR_RED) { TX_SET(sb, color, COLOR_BLACK); TX_SET(NODE_P(n), color, COLOR_RED); rbtree_map_rotate(map, NODE_P(n), c); sb = NODE_PARENT_AT(n, !c); } if (D_RO(D_RO(sb)->slots[RB_RIGHT])->color == COLOR_BLACK && D_RO(D_RO(sb)->slots[RB_LEFT])->color == COLOR_BLACK) { TX_SET(sb, color, COLOR_RED); return D_RO(n)->parent; } else { if (D_RO(D_RO(sb)->slots[!c])->color == COLOR_BLACK) { TX_SET(D_RW(sb)->slots[c], color, COLOR_BLACK); TX_SET(sb, color, COLOR_RED); rbtree_map_rotate(map, sb, (enum rb_children)!c); sb = NODE_PARENT_AT(n, !c); } TX_SET(sb, color, D_RO(NODE_P(n))->color); TX_SET(NODE_P(n), color, COLOR_BLACK); TX_SET(D_RW(sb)->slots[!c], color, COLOR_BLACK); rbtree_map_rotate(map, NODE_P(n), c); return RB_FIRST(map); } return n; } /* * rbtree_map_repair -- (internal) restores red-black tree properties * after remove */ static void rbtree_map_repair(TOID(struct rbtree_map) map, TOID(struct tree_map_node) n) { /* if left, repair right sibling, otherwise repair left sibling. */ while (!TOID_EQUALS(n, RB_FIRST(map)) && D_RO(n)->color == COLOR_BLACK) n = rbtree_map_repair_branch(map, n, (enum rb_children) NODE_LOCATION(n)); TX_SET(n, color, COLOR_BLACK); } /* * rbtree_map_remove -- removes key-value pair from the map */ PMEMoid rbtree_map_remove(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key) { PMEMoid ret = OID_NULL; TOID(struct tree_map_node) n = rbtree_map_find_node(map, key); if (TOID_IS_NULL(n)) return ret; ret = D_RO(n)->value; TOID(struct tree_map_node) s = D_RO(map)->sentinel; TOID(struct tree_map_node) r = D_RO(map)->root; TOID(struct tree_map_node) y = (NODE_IS_NULL(D_RO(n)->slots[RB_LEFT]) || NODE_IS_NULL(D_RO(n)->slots[RB_RIGHT])) ? n : rbtree_map_successor(map, n); TOID(struct tree_map_node) x = NODE_IS_NULL(D_RO(y)->slots[RB_LEFT]) ? D_RO(y)->slots[RB_RIGHT] : D_RO(y)->slots[RB_LEFT]; TX_BEGIN(pop) { TX_SET(x, parent, NODE_P(y)); if (TOID_EQUALS(NODE_P(x), r)) { TX_SET(r, slots[RB_LEFT], x); } else { TX_SET(NODE_P(y), slots[NODE_LOCATION(y)], x); } if (D_RO(y)->color == COLOR_BLACK) rbtree_map_repair(map, x); if (!TOID_EQUALS(y, n)) { TX_ADD(y); D_RW(y)->slots[RB_LEFT] = D_RO(n)->slots[RB_LEFT]; D_RW(y)->slots[RB_RIGHT] = D_RO(n)->slots[RB_RIGHT]; D_RW(y)->parent = D_RO(n)->parent; D_RW(y)->color = D_RO(n)->color; TX_SET(D_RW(n)->slots[RB_LEFT], parent, y); TX_SET(D_RW(n)->slots[RB_RIGHT], parent, y); TX_SET(NODE_P(n), slots[NODE_LOCATION(n)], y); } TX_FREE(n); } TX_END return ret; } /* * rbtree_map_get -- searches for a value of the key */ PMEMoid rbtree_map_get(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key) { TOID(struct tree_map_node) node = rbtree_map_find_node(map, key); if (TOID_IS_NULL(node)) return OID_NULL; return D_RO(node)->value; } /* * rbtree_map_lookup -- searches if key exists */ int rbtree_map_lookup(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key) { TOID(struct tree_map_node) node = rbtree_map_find_node(map, key); if (TOID_IS_NULL(node)) return 0; return 1; } /* * rbtree_map_foreach_node -- (internal) recursively traverses tree */ static int rbtree_map_foreach_node(TOID(struct rbtree_map) map, TOID(struct tree_map_node) p, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { int ret = 0; if (TOID_EQUALS(p, D_RO(map)->sentinel)) return 0; if ((ret = rbtree_map_foreach_node(map, D_RO(p)->slots[RB_LEFT], cb, arg)) == 0) { if ((ret = cb(D_RO(p)->key, D_RO(p)->value, arg)) == 0) rbtree_map_foreach_node(map, D_RO(p)->slots[RB_RIGHT], cb, arg); } return ret; } /* * rbtree_map_foreach -- initiates recursive traversal */ int rbtree_map_foreach(PMEMobjpool *pop, TOID(struct rbtree_map) map, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { return rbtree_map_foreach_node(map, RB_FIRST(map), cb, arg); } /* * rbtree_map_is_empty -- checks whether the tree map is empty */ int rbtree_map_is_empty(PMEMobjpool *pop, TOID(struct rbtree_map) map) { return TOID_IS_NULL(RB_FIRST(map)); } /* * rbtree_map_check -- check if given persistent object is a tree map */ int rbtree_map_check(PMEMobjpool *pop, TOID(struct rbtree_map) map) { return TOID_IS_NULL(map) || !TOID_VALID(map); } /* * rbtree_map_insert_new -- allocates a new object and inserts it into the tree */ int rbtree_map_insert_new(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key, size_t size, unsigned type_num, void (*constructor)(PMEMobjpool *pop, void *ptr, void *arg), void *arg) { int ret = 0; TX_BEGIN(pop) { PMEMoid n = pmemobj_tx_alloc(size, type_num); constructor(pop, pmemobj_direct(n), arg); rbtree_map_insert(pop, map, key, n); } TX_ONABORT { ret = 1; } TX_END return ret; } /* * rbtree_map_remove_free -- removes and frees an object from the tree */ int rbtree_map_remove_free(PMEMobjpool *pop, TOID(struct rbtree_map) map, uint64_t key) { int ret = 0; TX_BEGIN(pop) { PMEMoid val = rbtree_map_remove(pop, map, key); pmemobj_tx_free(val); } TX_ONABORT { ret = 1; } TX_END return ret; } pmdk-1.8/src/examples/libpmemobj/setjmp.c0000664000000000000000000000624613615011243017214 0ustar rootroot/* * Copyright 2016-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * setjmp.c -- example illustrating an issue with indeterminate value * of non-volatile automatic variables after transaction abort. * See libpmemobj(7) for details. * * NOTE: To observe the problem (likely segfault on a second call to free()), * the example program should be compiled with optimizations enabled (-O2). */ #include #include #include /* name of our layout in the pool */ #define LAYOUT_NAME "setjmp_example" int main(int argc, const char *argv[]) { const char path[] = "/pmem-fs/myfile"; PMEMobjpool *pop; /* create the pmemobj pool */ pop = pmemobj_create(path, LAYOUT_NAME, PMEMOBJ_MIN_POOL, 0666); if (pop == NULL) { perror(path); exit(1); } /* initialize pointer variables with invalid addresses */ int *bad_example_1 = (int *)0xBAADF00D; int *bad_example_2 = (int *)0xBAADF00D; int *bad_example_3 = (int *)0xBAADF00D; int *volatile good_example = (int *)0xBAADF00D; TX_BEGIN(pop) { bad_example_1 = malloc(sizeof(int)); bad_example_2 = malloc(sizeof(int)); bad_example_3 = malloc(sizeof(int)); good_example = malloc(sizeof(int)); /* manual or library abort called here */ pmemobj_tx_abort(EINVAL); } TX_ONCOMMIT { /* * This section is longjmp-safe */ } TX_ONABORT { /* * This section is not longjmp-safe */ free(good_example); /* OK */ free(bad_example_1); /* undefined behavior */ } TX_FINALLY { /* * This section is not longjmp-safe on transaction abort only */ free(bad_example_2); /* undefined behavior */ } TX_END free(bad_example_3); /* undefined behavior */ pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/pmemobjfs/0000775000000000000000000000000013615011243017520 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/pmemobjfs/README0000664000000000000000000000361513615011243020405 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemobj/pmemobjfs/README. This directory contains an example application implemented using libpmemobj. The pmemobjfs is a simple implementation of FUSE file system using the transactional API of libpmemobj. To create a file system layout run the following command: $ mkfs.pmemobjfs [-s ] [-b ] To mount the filesystem run the following command: $ pmemobjfs -s -odefault_permissions,hard_remove,allow_other The -o default_permissions option enables permission checking by the FUSE implementation. The -s option runs FUSE in single threaded mode. The current implementation does not use any synchronization mechanisms. The -o hard_remove forces FUSE to not use the .fuse_hiddenXXX files when deleting a file. The -o allow_other option allows other users to access the file system. To begin a transaction run the following command: $ pmemobjfs.tx_begin To commit a transaction run the following command: $ pmemobjfs.tx_commit To abort a transaction run the following command: $ pmemobjfs.tx_abort The transactions works across multiple operations performed on pmemobjfs file system including modification of files. Example usage of transactions: $ mkfs.pmemobjfs /mnt/pmem/pmemobjfs.obj $ pmemobjfs -s -odefault_permissions,hard_remove,allow_other\ /mnt/pmem/pmemobjfs.obj /mnt/pmemobjfs $ ls /mnt/pmemobjfs $ pmemobjfs.tx_begin /mnt/pmemobjfs $ mkdir /mnt/pmemobjfs/dir1 $ echo "The pmemobjfs filesystem" > /mnt/pmemobjfs/dir1/file1 $ pmemobjfs.tx_commit $ ls /mnt/pmemobjfs/dir1 file1 $ pmemobjfs.tx_begin /mnt/pmemobjfs $ rm /mnt/pmemobjfs/dir1/file1 $ ls /mnt/pmemobjfs/dir1 $ pmemobjfs.tx_abort $ ls /mnt/pmemobjfs/dir1 file1 ** DEPENDENCIES: ** In order to build pmemobjfs you need to install fuse (version >= 2.9.1) development package. rpm-based systems : fuse-devel dpkg-based systems: libfuse-dev pmdk-1.8/src/examples/libpmemobj/pmemobjfs/.gitignore0000664000000000000000000000014113615011243021504 0ustar rootrootpmemobjfs pmemobjfs.tx_begin pmemobjfs.tx_commit pmemobjfs.tx_abort mkfs.pmemobjfs pmemobjfs.log pmdk-1.8/src/examples/libpmemobj/pmemobjfs/pmemobjfs.c0000664000000000000000000015502013615011243021651 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pmemobjfs.c -- simple filesystem based on libpmemobj tx API */ #include #include #include #include #include #include #include #include #include #include #include #include #include #include #ifndef PMEMOBJFS_TRACK_BLOCKS #define PMEMOBJFS_TRACK_BLOCKS 1 #endif #if DEBUG static FILE *log_fh; static uint64_t log_cnt; #define log(fmt, args...) do {\ if (log_fh) {\ fprintf(log_fh, "[%016lx] %s: " fmt "\n", log_cnt, __func__, ## args);\ log_cnt++;\ fflush(log_fh);\ }\ } while (0) #else #define log(fmt, args...) do {} while (0) #endif #define PMEMOBJFS_MOUNT "pmemobjfs" #define PMEMOBJFS_MKFS "mkfs.pmemobjfs" #define PMEMOBJFS_TX_BEGIN "pmemobjfs.tx_begin" #define PMEMOBJFS_TX_COMMIT "pmemobjfs.tx_commit" #define PMEMOBJFS_TX_ABORT "pmemobjfs.tx_abort" #define PMEMOBJFS_TMP_TEMPLATE "/.tx_XXXXXX" #define PMEMOBJFS_CTL 'I' #define PMEMOBJFS_CTL_TX_BEGIN _IO(PMEMOBJFS_CTL, 1) #define PMEMOBJFS_CTL_TX_COMMIT _IO(PMEMOBJFS_CTL, 2) #define PMEMOBJFS_CTL_TX_ABORT _IO(PMEMOBJFS_CTL, 3) /* * struct pmemobjfs -- volatile state of pmemobjfs */ struct pmemobjfs { PMEMobjpool *pop; struct map_ctx *mapc; uint64_t pool_uuid_lo; int ioctl_cmd; uint64_t ioctl_off; uint64_t block_size; uint64_t max_name; }; #define PMEMOBJFS (struct pmemobjfs *)fuse_get_context()->private_data #define PDLL_ENTRY(type)\ struct {\ TOID(type) next;\ TOID(type) prev;\ } #define PDLL_HEAD(type)\ struct {\ TOID(type) first;\ TOID(type) last;\ } #define PDLL_HEAD_INIT(head) do {\ ((head)).first = ((typeof((head).first))OID_NULL);\ ((head)).last = ((typeof((head).first))OID_NULL);\ } while (0) #define PDLL_FOREACH(entry, head, field)\ for ((entry) = ((head)).first; !TOID_IS_NULL(entry);\ (entry) = D_RO(entry)->field.next) #define PDLL_FOREACH_SAFE(entry, next, head, field)\ for ((entry) = ((head)).first; !TOID_IS_NULL(entry) &&\ ((next) = D_RO(entry)->field.next, 1);\ (entry) = (next)) #define PDLL_INSERT_HEAD(head, entry, field) do {\ pmemobj_tx_add_range_direct(&(head).first, sizeof((head).first));\ TX_ADD_FIELD(entry, field);\ D_RW(entry)->field.next = (head).first;\ D_RW(entry)->field.prev =\ (typeof(D_RW(entry)->field.prev))OID_NULL;\ (head).first = entry;\ if (TOID_IS_NULL((head).last)) {\ pmemobj_tx_add_range_direct(&(head).last, sizeof((head).last));\ (head).last = entry;\ }\ typeof(entry) next = D_RO(entry)->field.next;\ if (!TOID_IS_NULL(next)) {\ pmemobj_tx_add_range_direct(&D_RW(next)->field.prev,\ sizeof(D_RW(next)->field.prev));\ D_RW(next)->field.prev = entry;\ }\ } while (0) #define PDLL_REMOVE(head, entry, field) do {\ if (TOID_EQUALS((head).first, entry) &&\ TOID_EQUALS((head).last, entry)) {\ pmemobj_tx_add_range_direct(&(head).first,\ sizeof((head).first));\ pmemobj_tx_add_range_direct(&(head).last, sizeof((head).last));\ (head).first = (typeof(D_RW(entry)->field.prev))OID_NULL;\ (head).last = (typeof(D_RW(entry)->field.prev))OID_NULL;\ } else if (TOID_EQUALS((head).first, entry)) {\ typeof(entry) next = D_RW(entry)->field.next;\ pmemobj_tx_add_range_direct(&D_RW(next)->field.prev,\ sizeof(D_RW(next)->field.prev));\ pmemobj_tx_add_range_direct(&(head).first,\ sizeof((head).first));\ (head).first = D_RO(entry)->field.next;\ D_RW(next)->field.prev.oid = OID_NULL;\ } else if (TOID_EQUALS((head).last, entry)) {\ typeof(entry) prev = D_RW(entry)->field.prev;\ pmemobj_tx_add_range_direct(&D_RW(prev)->field.next,\ sizeof(D_RW(prev)->field.next));\ pmemobj_tx_add_range_direct(&(head).last, sizeof((head).last));\ (head).last = D_RO(entry)->field.prev;\ D_RW(prev)->field.next.oid = OID_NULL;\ } else {\ typeof(entry) prev = D_RW(entry)->field.prev;\ typeof(entry) next = D_RW(entry)->field.next;\ pmemobj_tx_add_range_direct(&D_RW(prev)->field.next,\ sizeof(D_RW(prev)->field.next));\ pmemobj_tx_add_range_direct(&D_RW(next)->field.prev,\ sizeof(D_RW(next)->field.prev));\ D_RW(prev)->field.next = D_RO(entry)->field.next;\ D_RW(next)->field.prev = D_RO(entry)->field.prev;\ }\ } while (0) typedef uint8_t objfs_block_t; /* * pmemobjfs persistent layout */ POBJ_LAYOUT_BEGIN(pmemobjfs); POBJ_LAYOUT_ROOT(pmemobjfs, struct objfs_super); POBJ_LAYOUT_TOID(pmemobjfs, struct objfs_inode); POBJ_LAYOUT_TOID(pmemobjfs, struct objfs_dir_entry); POBJ_LAYOUT_TOID(pmemobjfs, objfs_block_t); POBJ_LAYOUT_TOID(pmemobjfs, char); POBJ_LAYOUT_END(pmemobjfs); #define PMEMOBJFS_MIN_BLOCK_SIZE ((size_t)(512 - 64)) /* * struct objfs_super -- pmemobjfs super (root) object */ struct objfs_super { TOID(struct objfs_inode) root_inode; /* root dir inode */ TOID(struct map) opened; /* map of opened files / dirs */ uint64_t block_size; /* size of data block */ }; /* * struct objfs_dir_entry -- pmemobjfs directory entry structure */ struct objfs_dir_entry { PDLL_ENTRY(struct objfs_dir_entry) pdll; /* list entry */ TOID(struct objfs_inode) inode; /* pointer to inode */ char name[]; /* name */ }; /* * struct objfs_dir -- pmemobjfs directory structure */ struct objfs_dir { PDLL_HEAD(struct objfs_dir_entry) entries; /* directory entries */ }; /* * key == 0 for ctree_map is not allowed */ #define GET_KEY(off) ((off) + 1) /* * struct objfs_file -- pmemobjfs file structure */ struct objfs_file { TOID(struct map) blocks; /* blocks map */ }; /* * struct objfs_symlink -- symbolic link */ struct objfs_symlink { uint64_t len; /* length of symbolic link */ TOID(char) name; /* symbolic link data */ }; /* * struct objfs_inode -- pmemobjfs inode structure */ struct objfs_inode { uint64_t size; /* size of file */ uint64_t flags; /* file flags */ uint64_t dev; /* device info */ uint32_t ctime; /* time of last status change */ uint32_t mtime; /* time of last modification */ uint32_t atime; /* time of last access */ uint32_t uid; /* user ID */ uint32_t gid; /* group ID */ uint32_t ref; /* reference counter */ struct objfs_file file; /* file specific data */ struct objfs_dir dir; /* directory specific data */ struct objfs_symlink symlink; /* symlink specific data */ }; /* * pmemobjfs_ioctl -- do the ioctl command */ static void pmemobjfs_ioctl(struct pmemobjfs *objfs) { switch (objfs->ioctl_cmd) { case PMEMOBJFS_CTL_TX_BEGIN: (void) pmemobj_tx_begin(objfs->pop, NULL, TX_PARAM_NONE); break; case PMEMOBJFS_CTL_TX_ABORT: pmemobj_tx_abort(-1); (void) pmemobj_tx_end(); break; case PMEMOBJFS_CTL_TX_COMMIT: pmemobj_tx_commit(); (void) pmemobj_tx_end(); break; default: break; } /* clear deferred inode offset and command */ objfs->ioctl_cmd = 0; objfs->ioctl_off = 0; } /* * pmemobjfs_inode_alloc -- allocate inode structure */ static TOID(struct objfs_inode) pmemobjfs_inode_alloc(struct pmemobjfs *objfs, uint64_t flags, uint32_t uid, uint32_t gid, uint64_t dev) { TOID(struct objfs_inode) inode = TOID_NULL(struct objfs_inode); TX_BEGIN(objfs->pop) { inode = TX_ZNEW(struct objfs_inode); time_t cur_time = time(NULL); D_RW(inode)->flags = flags; D_RW(inode)->dev = dev; D_RW(inode)->ctime = cur_time; D_RW(inode)->mtime = cur_time; D_RW(inode)->atime = cur_time; D_RW(inode)->uid = uid; D_RW(inode)->gid = gid; D_RW(inode)->ref = 0; } TX_ONABORT { inode = TOID_NULL(struct objfs_inode); } TX_END return inode; } /* * pmemobjfs_inode_init_dir -- initialize directory in inode */ static void pmemobjfs_inode_init_dir(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { TX_BEGIN(objfs->pop) { PDLL_HEAD_INIT(D_RW(inode)->dir.entries); } TX_END; } /* * pmemobjfs_inode_destroy_dir -- destroy directory from inode */ static void pmemobjfs_inode_destroy_dir(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { /* nothing to do */ } /* * pmemobjfs_file_alloc -- allocate file structure */ static void pmemobjfs_inode_init_file(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { TX_BEGIN(objfs->pop) { map_create(objfs->mapc, &D_RW(inode)->file.blocks, NULL); } TX_END } /* * pmemobjfs_file_free -- free file structure */ static void pmemobjfs_inode_destroy_file(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { TX_BEGIN(objfs->pop) { map_destroy(objfs->mapc, &D_RW(inode)->file.blocks); } TX_END } /* * pmemobjfs_inode_hold -- increase reference counter of inode */ static void pmemobjfs_inode_hold(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { if (TOID_IS_NULL(inode)) return; TX_BEGIN(objfs->pop) { /* update number of references */ TX_ADD_FIELD(inode, ref); D_RW(inode)->ref++; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = time(NULL); } TX_END } /* * pmemobjfs_dir_entry_alloc -- allocate directory entry structure */ static TOID(struct objfs_dir_entry) pmemobjfs_dir_entry_alloc(struct pmemobjfs *objfs, const char *name, TOID(struct objfs_inode) inode) { TOID(struct objfs_dir_entry) entry = TOID_NULL(struct objfs_dir_entry); TX_BEGIN(objfs->pop) { size_t len = strlen(name) + 1; entry = TX_ALLOC(struct objfs_dir_entry, objfs->block_size); memcpy(D_RW(entry)->name, name, len); D_RW(entry)->inode = inode; pmemobjfs_inode_hold(objfs, inode); } TX_ONABORT { entry = TOID_NULL(struct objfs_dir_entry); } TX_END return entry; } /* * pmemobjfs_dir_entry_free -- free dir entry structure */ static void pmemobjfs_dir_entry_free(struct pmemobjfs *objfs, TOID(struct objfs_dir_entry) entry) { TX_BEGIN(objfs->pop) { TX_FREE(entry); } TX_END } /* * pmemobjfs_inode_init_symlink -- initialize symbolic link */ static void pmemobjfs_inode_init_symlink(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *name) { TX_BEGIN(objfs->pop) { size_t len = strlen(name) + 1; D_RW(inode)->symlink.len = len; TOID_ASSIGN(D_RW(inode)->symlink.name, TX_STRDUP(name, TOID_TYPE_NUM(char))); } TX_END } /* * pmemobjfs_inode_destroy_symlink -- destroy symbolic link */ static void pmemobjfs_inode_destroy_symlink(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { TX_BEGIN(objfs->pop) { TX_FREE(D_RO(inode)->symlink.name); } TX_END } /* * pmemobjfs_symlink_read -- read symlink to buffer */ static int pmemobjfs_symlink_read(TOID(struct objfs_inode) inode, char *buff, size_t size) { /* check inode type */ switch (D_RO(inode)->flags & S_IFMT) { case S_IFLNK: break; case S_IFDIR: return -EISDIR; case S_IFREG: default: return -EINVAL; } char *name = D_RW(D_RW(inode)->symlink.name); strncpy(buff, name, size); return 0; } /* * pmemobjfs_symlink_size -- get size of symlink */ static size_t pmemobjfs_symlink_size(TOID(struct objfs_inode) inode) { return D_RO(inode)->symlink.len - 1; } /* * pmemobjfs_inode_free -- free inode structure */ static void pmemobjfs_inode_free(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { TX_BEGIN(objfs->pop) { /* release data specific for inode type */ if (S_ISREG(D_RO(inode)->flags)) { pmemobjfs_inode_destroy_file(objfs, inode); } else if (S_ISDIR(D_RO(inode)->flags)) { pmemobjfs_inode_destroy_dir(objfs, inode); } else if (S_ISLNK(D_RO(inode)->flags)) { pmemobjfs_inode_destroy_symlink(objfs, inode); } TX_FREE(inode); } TX_END } /* * pmemobjfs_inode_put -- decrease reference counter of inode and free */ static void pmemobjfs_inode_put(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { if (TOID_IS_NULL(inode)) return; TX_BEGIN(objfs->pop) { /* update number of references */ TX_ADD_FIELD(inode, ref); D_RW(inode)->ref--; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = time(NULL); if (!D_RO(inode)->ref) pmemobjfs_inode_free(objfs, inode); } TX_END } /* * pmemobjfs_dir_get_inode -- get inode from dir of given name */ static TOID(struct objfs_inode) pmemobjfs_dir_get_inode(TOID(struct objfs_inode) inode, const char *name) { log("%s", name); TOID(struct objfs_dir_entry) entry; PDLL_FOREACH(entry, D_RW(inode)->dir.entries, pdll) { if (strcmp(name, D_RO(entry)->name) == 0) return D_RO(entry)->inode; } return TOID_NULL(struct objfs_inode); } /* * pmemobjfs_get_dir_entry -- get dir entry from dir of given name */ static TOID(struct objfs_dir_entry) pmemobjfs_get_dir_entry(TOID(struct objfs_inode) inode, const char *name) { log("%s", name); TOID(struct objfs_dir_entry) entry; PDLL_FOREACH(entry, D_RW(inode)->dir.entries, pdll) { if (strcmp(name, D_RO(entry)->name) == 0) return entry; } return TOID_NULL(struct objfs_dir_entry); } /* * pmemobjfs_inode_lookup_parent -- lookup for parent inode and child name */ static int pmemobjfs_inode_lookup_parent(struct pmemobjfs *objfs, const char *path, TOID(struct objfs_inode) *inodep, const char **child) { log("%s", path); TOID(struct objfs_super) super = POBJ_ROOT(objfs->pop, struct objfs_super); TOID(struct objfs_inode) cur = D_RO(super)->root_inode; TOID(struct objfs_inode) par = TOID_NULL(struct objfs_inode); if (path[0] == '/') path++; int ret = 0; char *p = strdup(path); char *name = p; char *ch = NULL; while (name && *name != '\0' && !TOID_IS_NULL(cur)) { char *slash = strchr(name, '/'); if (slash) { *slash = '\0'; slash++; } if (!S_ISDIR(D_RO(cur)->flags)) { ret = -ENOTDIR; goto out; } if (strlen(name) > objfs->max_name) { ret = -ENAMETOOLONG; goto out; } par = cur; cur = pmemobjfs_dir_get_inode(cur, name); ch = name; name = slash; } if (child) { if (strchr(ch, '/')) { ret = -ENOENT; goto out; } if (TOID_IS_NULL(par)) { ret = -ENOENT; goto out; } cur = par; size_t parent_len = ch - p; *child = path + parent_len; } else { if (TOID_IS_NULL(cur)) ret = -ENOENT; } if (inodep) *inodep = cur; out: free(p); return ret; } /* * pmemobjfs_inode_lookup -- get inode for given path */ static int pmemobjfs_inode_lookup(struct pmemobjfs *objfs, const char *path, TOID(struct objfs_inode) *inodep) { log("%s", path); return pmemobjfs_inode_lookup_parent(objfs, path, inodep, NULL); } /* * pmemobjfs_file_get_block -- get block at given offset */ static TOID(objfs_block_t) pmemobjfs_file_get_block(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, uint64_t offset) { TOID(objfs_block_t) block; PMEMoid block_oid = map_get(objfs->mapc, D_RO(inode)->file.blocks, GET_KEY(offset)); TOID_ASSIGN(block, block_oid); return block; } /* * pmemobjfs_file_get_block_for_write -- get or allocate block at given offset */ static TOID(objfs_block_t) pmemobjfs_file_get_block_for_write(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, uint64_t offset) { TOID(objfs_block_t) block = pmemobjfs_file_get_block(objfs, inode, offset); if (TOID_IS_NULL(block)) { TX_BEGIN(objfs->pop) { block = TX_ALLOC(objfs_block_t, objfs->block_size); map_insert(objfs->mapc, D_RW(inode)->file.blocks, GET_KEY(offset), block.oid); } TX_ONABORT { block = TOID_NULL(objfs_block_t); } TX_END } else { #if PMEMOBJFS_TRACK_BLOCKS TX_ADD(block); #endif } return block; } /* * pmemobjfs_truncate -- truncate file */ static int pmemobjfs_truncate(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, off_t off) { /* check inode type */ switch (D_RO(inode)->flags & S_IFMT) { case S_IFREG: break; case S_IFDIR: return -EISDIR; default: return -EINVAL; } int ret = 0; TX_BEGIN(objfs->pop) { uint64_t old_off = D_RO(inode)->size; if (old_off > off) { /* release blocks */ uint64_t old_boff = (old_off - 1) / objfs->block_size; uint64_t boff = (off + 1) / objfs->block_size; for (uint64_t o = boff; o <= old_boff; o++) { map_remove_free(objfs->mapc, D_RW(inode)->file.blocks, GET_KEY(o)); } } time_t t = time(NULL); /* update modification time */ TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = t; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = t; /* update size */ TX_ADD_FIELD(inode, size); D_RW(inode)->size = off; } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_read -- read from file */ static int pmemobjfs_read(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, char *buff, size_t size, off_t offset) { /* check inode type */ switch (D_RO(inode)->flags & S_IFMT) { case S_IFREG: break; case S_IFDIR: return -EISDIR; default: return -EINVAL; } uint64_t fsize = D_RO(inode)->size; size_t sz = size; size_t off = offset; while (sz > 0) { if (off >= fsize) break; uint64_t block_id = off / objfs->block_size; uint64_t block_off = off % objfs->block_size; uint64_t block_size = sz < objfs->block_size ? sz : objfs->block_size; TOID(objfs_block_t) block = pmemobjfs_file_get_block(objfs, inode, block_id); if (block_off + block_size > objfs->block_size) block_size = objfs->block_size - block_off; if (TOID_IS_NULL(block)) { memset(buff, 0, block_size); } else { memcpy(buff, &D_RW(block)[block_off], block_size); } buff += block_size; off += block_size; sz -= block_size; } return size - sz; } /* * pmemobjfs_write -- write to file */ static int pmemobjfs_write(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *buff, size_t size, off_t offset) { /* check inode type */ switch (D_RO(inode)->flags & S_IFMT) { case S_IFREG: break; case S_IFDIR: return -EISDIR; default: return -EINVAL; } int ret = 0; TX_BEGIN(objfs->pop) { size_t sz = size; off_t off = offset; while (sz > 0) { uint64_t block_id = off / objfs->block_size; uint64_t block_off = off % objfs->block_size; uint64_t block_size = sz < objfs->block_size ? sz : objfs->block_size; TOID(objfs_block_t) block = pmemobjfs_file_get_block_for_write(objfs, inode, block_id); if (TOID_IS_NULL(block)) return -ENOSPC; if (block_off + block_size > objfs->block_size) block_size = objfs->block_size - block_off; memcpy(&D_RW(block)[block_off], buff, block_size); buff += block_size; off += block_size; sz -= block_size; } time_t t = time(NULL); if (offset + size > D_RO(inode)->size) { /* update size */ TX_ADD_FIELD(inode, size); D_RW(inode)->size = offset + size; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = t; } /* update modification time */ TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = t; } TX_ONCOMMIT { ret = size; } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_fallocate -- allocate blocks for file */ static int pmemobjfs_fallocate(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, off_t offset, off_t size) { /* check inode type */ switch (D_RO(inode)->flags & S_IFMT) { case S_IFREG: break; case S_IFDIR: return -EISDIR; default: return -EINVAL; } int ret = 0; TX_BEGIN(objfs->pop) { /* allocate blocks from requested range */ uint64_t b_off = offset / objfs->block_size; uint64_t e_off = (offset + size) / objfs->block_size; for (uint64_t off = b_off; off <= e_off; off++) pmemobjfs_file_get_block_for_write(objfs, inode, off); time_t t = time(NULL); /* update modification time */ TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = t; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = t; /* update inode size */ D_RW(inode)->size = offset + size; TX_ADD_FIELD(inode, size); } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_remove_dir_entry -- remove dir entry from directory */ static void pmemobjfs_remove_dir_entry(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, TOID(struct objfs_dir_entry) entry) { TX_BEGIN(objfs->pop) { pmemobjfs_inode_put(objfs, D_RO(entry)->inode); PDLL_REMOVE(D_RW(inode)->dir.entries, entry, pdll); pmemobjfs_dir_entry_free(objfs, entry); } TX_END } /* * pmemobjfs_remove_dir_entry_name -- remove dir entry of given name */ static void pmemobjfs_remove_dir_entry_name(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *name) { TX_BEGIN(objfs->pop) { TOID(struct objfs_dir_entry) entry = pmemobjfs_get_dir_entry(inode, name); pmemobjfs_remove_dir_entry(objfs, inode, entry); } TX_END } /* * pmemobjfs_add_dir_entry -- add new directory entry */ static int pmemobjfs_add_dir_entry(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, TOID(struct objfs_dir_entry) entry) { /* check inode type */ if (!S_ISDIR(D_RO(inode)->flags)) return -ENOTDIR; int ret = 0; TX_BEGIN(objfs->pop) { /* insert new dir entry to list */ PDLL_INSERT_HEAD(D_RW(inode)->dir.entries, entry, pdll); /* update dir size */ TX_ADD_FIELD(inode, size); D_RW(inode)->size++; } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_unlink_dir_entry -- unlink directory entry */ static int pmemobjfs_unlink_dir_entry(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, TOID(struct objfs_dir_entry) entry) { /* check inode type */ if (!S_ISDIR(D_RO(inode)->flags)) return -ENOTDIR; int ret = 0; TX_BEGIN(objfs->pop) { pmemobjfs_remove_dir_entry(objfs, inode, entry); /* update dir size */ TX_ADD_FIELD(inode, size); D_RW(inode)->size--; } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_new_dir -- create new directory */ static TOID(struct objfs_inode) pmemobjfs_new_dir(struct pmemobjfs *objfs, TOID(struct objfs_inode) parent, const char *name, uint64_t flags, uint32_t uid, uint32_t gid) { TOID(struct objfs_inode) inode = TOID_NULL(struct objfs_inode); TX_BEGIN(objfs->pop) { inode = pmemobjfs_inode_alloc(objfs, flags, uid, gid, 0); pmemobjfs_inode_init_dir(objfs, inode); /* add . and .. to new directory */ TOID(struct objfs_dir_entry) dot = pmemobjfs_dir_entry_alloc(objfs, ".", inode); TOID(struct objfs_dir_entry) dotdot = pmemobjfs_dir_entry_alloc(objfs, "..", parent); pmemobjfs_add_dir_entry(objfs, inode, dot); pmemobjfs_add_dir_entry(objfs, inode, dotdot); } TX_ONABORT { inode = TOID_NULL(struct objfs_inode); } TX_END return inode; } /* * pmemobjfs_mkdir -- make new directory */ static int pmemobjfs_mkdir(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *name, uint64_t flags, uint32_t uid, uint32_t gid) { /* check inode type */ if (!S_ISDIR(D_RO(inode)->flags)) return -ENOTDIR; int ret = 0; TX_BEGIN(objfs->pop) { TOID(struct objfs_inode) new_inode = pmemobjfs_new_dir(objfs, inode, name, flags, uid, gid); TOID(struct objfs_dir_entry) entry = pmemobjfs_dir_entry_alloc(objfs, name, new_inode); pmemobjfs_add_dir_entry(objfs, inode, entry); /* update modification time */ TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = time(NULL); } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_remove_dir -- remove directory from directory */ static void pmemobjfs_remove_dir(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, TOID(struct objfs_dir_entry) entry) { /* removing entry inode */ TOID(struct objfs_inode) rinode = D_RO(entry)->inode; TX_BEGIN(objfs->pop) { /* remove . and .. from removing dir */ pmemobjfs_remove_dir_entry_name(objfs, rinode, "."); pmemobjfs_remove_dir_entry_name(objfs, rinode, ".."); /* remove dir entry from parent */ pmemobjfs_remove_dir_entry(objfs, inode, entry); } TX_END } /* * pmemobjfs_rmdir -- remove directory of given name */ static int pmemobjfs_rmdir(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *name) { /* check parent inode type */ if (!S_ISDIR(D_RO(inode)->flags)) return -ENOTDIR; TOID(struct objfs_dir_entry) entry = pmemobjfs_get_dir_entry(inode, name); if (TOID_IS_NULL(entry)) return -ENOENT; TOID(struct objfs_inode) entry_inode = D_RO(entry)->inode; /* check removing dir type */ if (!S_ISDIR(D_RO(entry_inode)->flags)) return -ENOTDIR; /* check if dir is empty (contains only . and ..) */ if (D_RO(entry_inode)->size > 2) return -ENOTEMPTY; int ret = 0; TX_BEGIN(objfs->pop) { pmemobjfs_remove_dir(objfs, inode, entry); /* update dir size */ TX_ADD_FIELD(inode, size); D_RW(inode)->size--; /* update modification time */ TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = time(NULL); } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_create -- create new file in directory */ static int pmemobjfs_create(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *name, mode_t mode, uid_t uid, gid_t gid, TOID(struct objfs_inode) *inodep) { int ret = 0; uint64_t flags = mode | S_IFREG; TOID(struct objfs_dir_entry) entry = TOID_NULL(struct objfs_dir_entry); TX_BEGIN(objfs->pop) { TOID(struct objfs_inode) new_file= pmemobjfs_inode_alloc(objfs, flags, uid, gid, 0); pmemobjfs_inode_init_file(objfs, new_file); entry = pmemobjfs_dir_entry_alloc(objfs, name, new_file); pmemobjfs_add_dir_entry(objfs, inode, entry); time_t t = time(NULL); /* update modification time */ TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = t; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = t; } TX_ONABORT { ret = -ECANCELED; } TX_ONCOMMIT { if (inodep) *inodep = D_RO(entry)->inode; } TX_END return ret; } /* * pmemobjfs_open -- open inode */ static int pmemobjfs_open(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { TOID(struct objfs_super) super = POBJ_ROOT(objfs->pop, struct objfs_super); int ret = 0; TX_BEGIN(objfs->pop) { /* insert inode to opened inodes map */ map_insert(objfs->mapc, D_RW(super)->opened, inode.oid.off, inode.oid); /* hold inode */ pmemobjfs_inode_hold(objfs, inode); } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_close -- release inode */ static int pmemobjfs_close(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode) { TOID(struct objfs_super) super = POBJ_ROOT(objfs->pop, struct objfs_super); int ret = 0; TX_BEGIN(objfs->pop) { /* remove inode from opened inodes map */ map_remove(objfs->mapc, D_RW(super)->opened, inode.oid.off); /* release inode */ pmemobjfs_inode_put(objfs, inode); } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_rename -- rename/move inode */ static int pmemobjfs_rename(struct pmemobjfs *objfs, TOID(struct objfs_inode) src_parent, const char *src_name, TOID(struct objfs_inode) dst_parent, const char *dst_name) { /* check source and destination inodes type */ if (!S_ISDIR(D_RO(src_parent)->flags)) return -ENOTDIR; if (!S_ISDIR(D_RO(dst_parent)->flags)) return -ENOTDIR; /* get source dir entry */ TOID(struct objfs_dir_entry) src_entry = pmemobjfs_get_dir_entry(src_parent, src_name); TOID(struct objfs_inode) src_inode = D_RO(src_entry)->inode; if (TOID_IS_NULL(src_entry)) return -ENOENT; int ret = 0; TX_BEGIN(objfs->pop) { /* * Allocate new dir entry with destination name * and source inode. * NOTE: * This *must* be called before removing dir entry from * source directory because otherwise the source inode * could be released before inserting to new dir entry. */ TOID(struct objfs_dir_entry) dst_entry = pmemobjfs_dir_entry_alloc(objfs, dst_name, src_inode); /* remove old dir entry from source */ pmemobjfs_unlink_dir_entry(objfs, src_parent, src_entry); /* add new dir entry to destination */ pmemobjfs_add_dir_entry(objfs, dst_parent, dst_entry); } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_symlink -- create symbolic link */ static int pmemobjfs_symlink(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *name, const char *path, uid_t uid, gid_t gid) { /* check inode type */ if (!S_ISDIR(D_RO(inode)->flags)) return -ENOTDIR; /* set 0777 permissions for symbolic links */ uint64_t flags = 0777 | S_IFLNK; int ret = 0; TX_BEGIN(objfs->pop) { TOID(struct objfs_inode) symlink = pmemobjfs_inode_alloc(objfs, flags, uid, gid, 0); pmemobjfs_inode_init_symlink(objfs, symlink, path); D_RW(symlink)->size = pmemobjfs_symlink_size(symlink); TOID(struct objfs_dir_entry) entry = pmemobjfs_dir_entry_alloc(objfs, name, symlink); pmemobjfs_add_dir_entry(objfs, inode, entry); time_t t = time(NULL); /* update modification time */ TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = t; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = t; } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_mknod -- create node */ static int pmemobjfs_mknod(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *name, mode_t mode, uid_t uid, gid_t gid, dev_t dev) { /* check inode type */ if (!S_ISDIR(D_RO(inode)->flags)) return -ENOTDIR; int ret = 0; TX_BEGIN(objfs->pop) { TOID(struct objfs_inode) node = pmemobjfs_inode_alloc(objfs, mode, uid, gid, dev); D_RW(node)->size = 0; TOID(struct objfs_dir_entry) entry = pmemobjfs_dir_entry_alloc(objfs, name, node); pmemobjfs_add_dir_entry(objfs, inode, entry); time_t t = time(NULL); /* update modification time */ TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = t; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = t; } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_chmod -- change mode of inode */ static int pmemobjfs_chmod(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, mode_t mode) { int ret = 0; TX_BEGIN(objfs->pop) { TX_ADD_FIELD(inode, flags); /* mask file type bit fields */ uint64_t flags = D_RO(inode)->flags; flags = flags & S_IFMT; D_RW(inode)->flags = flags | (mode & ~S_IFMT); /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = time(NULL); } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_chown -- change owner and group of inode */ static int pmemobjfs_chown(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, uid_t uid, gid_t gid) { int ret = 0; TX_BEGIN(objfs->pop) { TX_ADD_FIELD(inode, uid); D_RW(inode)->uid = uid; TX_ADD_FIELD(inode, gid); D_RW(inode)->gid = gid; /* update status change time */ TX_ADD_FIELD(inode, ctime); D_RW(inode)->ctime = time(NULL); } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_getattr -- get inode's attributes */ static int pmemobjfs_getattr(TOID(struct objfs_inode) inode, struct stat *statbuf) { memset(statbuf, 0, sizeof(*statbuf)); statbuf->st_size = D_RO(inode)->size; statbuf->st_ctime = D_RO(inode)->ctime; statbuf->st_mtime = D_RO(inode)->mtime; statbuf->st_atime = D_RO(inode)->atime; statbuf->st_mode = D_RO(inode)->flags; statbuf->st_uid = D_RO(inode)->uid; statbuf->st_gid = D_RO(inode)->gid; statbuf->st_rdev = D_RO(inode)->dev; return 0; } /* * pmemobjfs_utimens -- set atime and mtime */ static int pmemobjfs_utimens(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const struct timespec tv[2]) { int ret = 0; TX_BEGIN(objfs->pop) { TX_ADD_FIELD(inode, atime); D_RW(inode)->atime = tv[0].tv_sec; TX_ADD_FIELD(inode, mtime); D_RW(inode)->mtime = tv[0].tv_sec; } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_unlink -- unlink file from inode */ static int pmemobjfs_unlink(struct pmemobjfs *objfs, TOID(struct objfs_inode) inode, const char *name) { /* check inode type */ if (!S_ISDIR(D_RO(inode)->flags)) return -ENOTDIR; TOID(struct objfs_dir_entry) entry = pmemobjfs_get_dir_entry(inode, name); if (TOID_IS_NULL(entry)) return -ENOENT; TOID(struct objfs_inode) entry_inode = D_RO(entry)->inode; /* check unlinking inode type */ if (S_ISDIR(D_RO(entry_inode)->flags)) return -EISDIR; int ret = 0; TX_BEGIN(objfs->pop) { pmemobjfs_remove_dir_entry(objfs, inode, entry); TX_ADD_FIELD(inode, size); D_RW(inode)->size--; } TX_ONABORT { ret = -ECANCELED; } TX_END return ret; } /* * pmemobjfs_put_opened_cb -- release all opened inodes */ static int pmemobjfs_put_opened_cb(uint64_t key, PMEMoid value, void *arg) { struct pmemobjfs *objfs = arg; TOID(struct objfs_inode) inode; TOID_ASSIGN(inode, value); TOID(struct objfs_super) super = POBJ_ROOT(objfs->pop, struct objfs_super); /* * Set current value to OID_NULL so the tree_map_clear won't * free this inode and release the inode. */ map_insert(objfs->mapc, D_RW(super)->opened, key, OID_NULL); pmemobjfs_inode_put(objfs, inode); return 0; } /* * pmemobjfs_fuse_getattr -- (FUSE) get file attributes */ static int pmemobjfs_fuse_getattr(const char *path, struct stat *statbuf) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; int ret = pmemobjfs_inode_lookup(objfs, path, &inode); if (ret) return ret; return pmemobjfs_getattr(inode, statbuf); } /* * pmemobjfs_fuse_opendir -- (FUSE) open directory */ static int pmemobjfs_fuse_opendir(const char *path, struct fuse_file_info *fi) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; int ret = pmemobjfs_inode_lookup(objfs, path, &inode); if (ret) return ret; /* check inode type */ switch (D_RO(inode)->flags & S_IFMT) { case S_IFDIR: break; case S_IFREG: return -ENOTDIR; default: return -EINVAL; } /* add inode to opened inodes map */ ret = pmemobjfs_open(objfs, inode); if (!ret) fi->fh = inode.oid.off; return ret; } /* * pmemobjfs_fuse_releasedir -- (FUSE) release opened dir */ static int pmemobjfs_fuse_releasedir(const char *path, struct fuse_file_info *fi) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; if (!fi->fh) return -EINVAL; TOID(struct objfs_inode) inode; inode.oid.off = fi->fh; inode.oid.pool_uuid_lo = objfs->pool_uuid_lo; /* remove inode from opened inodes map */ int ret = pmemobjfs_close(objfs, inode); fi->fh = 0; return ret; } /* * pmemobjfs_fuse_readdir -- (FUSE) read directory entries */ static int pmemobjfs_fuse_readdir(const char *path, void *buff, fuse_fill_dir_t fill, off_t off, struct fuse_file_info *fi) { log("%s off = %lu", path, off); struct pmemobjfs *objfs = PMEMOBJFS; if (!fi->fh) return -EINVAL; TOID(struct objfs_inode) inode; inode.oid.off = fi->fh; inode.oid.pool_uuid_lo = objfs->pool_uuid_lo; if (!TOID_VALID(inode)) return -EINVAL; /* check inode type */ if (!S_ISDIR(D_RO(inode)->flags)) return -ENOTDIR; /* walk through all dir entries and fill fuse buffer */ int ret; TOID(struct objfs_dir_entry) entry; PDLL_FOREACH(entry, D_RW(inode)->dir.entries, pdll) { ret = fill(buff, D_RW(entry)->name, NULL, 0); if (ret) return ret; } return 0; } /* * pmemobjfs_fuse_mkdir -- (FUSE) create directory */ static int pmemobjfs_fuse_mkdir(const char *path, mode_t mode) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; const char *name; int ret = pmemobjfs_inode_lookup_parent(objfs, path, &inode, &name); if (ret) return ret; uid_t uid = fuse_get_context()->uid; uid_t gid = fuse_get_context()->gid; return pmemobjfs_mkdir(objfs, inode, name, mode | S_IFDIR, uid, gid); } /* * pmemobjfs_fuse_rmdir -- (FUSE) remove directory */ static int pmemobjfs_fuse_rmdir(const char *path) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; const char *name; int ret = pmemobjfs_inode_lookup_parent(objfs, path, &inode, &name); if (ret) return ret; return pmemobjfs_rmdir(objfs, inode, name); } /* * pmemobjfs_fuse_chmod -- (FUSE) change file permissions */ static int pmemobjfs_fuse_chmod(const char *path, mode_t mode) { log("%s 0%o", path, mode); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; int ret = pmemobjfs_inode_lookup(objfs, path, &inode); if (ret) return ret; return pmemobjfs_chmod(objfs, inode, mode); } /* * pmemobjfs_fuse_chown -- (FUSE) change owner */ static int pmemobjfs_fuse_chown(const char *path, uid_t uid, gid_t gid) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; int ret = pmemobjfs_inode_lookup(objfs, path, &inode); if (ret) return ret; return pmemobjfs_chown(objfs, inode, uid, gid); } /* * pmemobjfs_fuse_create -- (FUSE) create file */ static int pmemobjfs_fuse_create(const char *path, mode_t mode, struct fuse_file_info *fi) { log("%s mode %o", path, mode); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; const char *name; int ret = pmemobjfs_inode_lookup_parent(objfs, path, &inode, &name); if (ret) return ret; if (!S_ISDIR(D_RO(inode)->flags)) return -EINVAL; uid_t uid = fuse_get_context()->uid; uid_t gid = fuse_get_context()->gid; TOID(struct objfs_inode) new_file; ret = pmemobjfs_create(objfs, inode, name, mode, uid, gid, &new_file); if (ret) return ret; /* add new inode to opened inodes */ ret = pmemobjfs_open(objfs, new_file); if (ret) return ret; fi->fh = new_file.oid.off; return 0; } /* * pmemobjfs_fuse_utimens -- (FUSE) update access and modification times */ static int pmemobjfs_fuse_utimens(const char *path, const struct timespec tv[2]) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; int ret = pmemobjfs_inode_lookup(objfs, path, &inode); if (ret) return ret; return pmemobjfs_utimens(objfs, inode, tv); } /* * pmemobjfs_fuse_open -- (FUSE) open file */ static int pmemobjfs_fuse_open(const char *path, struct fuse_file_info *fi) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; int ret = pmemobjfs_inode_lookup(objfs, path, &inode); if (ret) return ret; /* check inode type */ switch (D_RO(inode)->flags & S_IFMT) { case S_IFREG: break; case S_IFDIR: return -EISDIR; default: return -EINVAL; } ret = pmemobjfs_open(objfs, inode); if (!ret) fi->fh = inode.oid.off; return ret; } /* * pmemobjfs_fuse_release -- (FUSE) release opened file */ static int pmemobjfs_fuse_release(const char *path, struct fuse_file_info *fi) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; if (!fi->fh) return -EINVAL; TOID(struct objfs_inode) inode; inode.oid.off = fi->fh; inode.oid.pool_uuid_lo = objfs->pool_uuid_lo; int ret = pmemobjfs_close(objfs, inode); /* perform deferred ioctl operation */ if (!ret && objfs->ioctl_off && objfs->ioctl_off == fi->fh) pmemobjfs_ioctl(objfs); fi->fh = 0; return ret; } /* * pmemobjfs_fuse_write -- (FUSE) write to file */ static int pmemobjfs_fuse_write(const char *path, const char *buff, size_t size, off_t offset, struct fuse_file_info *fi) { log("%s size = %zu off = %lu", path, size, offset); struct pmemobjfs *objfs = PMEMOBJFS; if (!fi->fh) return -EINVAL; TOID(struct objfs_inode) inode; inode.oid.off = fi->fh; inode.oid.pool_uuid_lo = objfs->pool_uuid_lo; if (!TOID_VALID(inode)) return -EINVAL; return pmemobjfs_write(objfs, inode, buff, size, offset); } /* * pmemobjfs_fuse_read -- (FUSE) read from file */ static int pmemobjfs_fuse_read(const char *path, char *buff, size_t size, off_t off, struct fuse_file_info *fi) { log("%s size = %zu off = %lu", path, size, off); struct pmemobjfs *objfs = PMEMOBJFS; if (!fi->fh) return -EINVAL; TOID(struct objfs_inode) inode; inode.oid.off = fi->fh; inode.oid.pool_uuid_lo = objfs->pool_uuid_lo; if (!TOID_VALID(inode)) return -EINVAL; return pmemobjfs_read(objfs, inode, buff, size, off); } /* * pmemobjfs_fuse_truncate -- (FUSE) truncate file */ static int pmemobjfs_fuse_truncate(const char *path, off_t off) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; int ret = pmemobjfs_inode_lookup(objfs, path, &inode); if (ret) return ret; return pmemobjfs_truncate(objfs, inode, off); } /* * pmemobjfs_fuse_ftruncate -- (FUSE) truncate file */ static int pmemobjfs_fuse_ftruncate(const char *path, off_t off, struct fuse_file_info *fi) { log("%s off = %lu", path, off); struct pmemobjfs *objfs = PMEMOBJFS; if (!fi->fh) return -EINVAL; TOID(struct objfs_inode) inode; inode.oid.off = fi->fh; inode.oid.pool_uuid_lo = objfs->pool_uuid_lo; if (!TOID_VALID(inode)) return -EINVAL; return pmemobjfs_truncate(objfs, inode, off); } /* * pmemobjfs_fuse_unlink -- (FUSE) unlink inode */ static int pmemobjfs_fuse_unlink(const char *path) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; const char *name; int ret = pmemobjfs_inode_lookup_parent(objfs, path, &inode, &name); if (ret) return ret; return pmemobjfs_unlink(objfs, inode, name); } /* * pmemobjfs_fuse_flush -- (FUSE) flush file */ static int pmemobjfs_fuse_flush(const char *path, struct fuse_file_info *fi) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; if (!fi->fh) return -EINVAL; TOID(struct objfs_inode) inode; inode.oid.off = fi->fh; inode.oid.pool_uuid_lo = objfs->pool_uuid_lo; if (!TOID_VALID(inode)) return -EINVAL; /* check inode type */ switch (D_RO(inode)->flags & S_IFMT) { case S_IFREG: break; case S_IFDIR: return -EISDIR; default: return -EINVAL; } /* nothing to do */ return 0; } /* * pmemobjfs_fuse_ioctl -- (FUSE) ioctl for file */ #ifdef __FreeBSD__ #define EBADFD EBADF /* XXX */ #endif static int pmemobjfs_fuse_ioctl(const char *path, int cmd, void *arg, struct fuse_file_info *fi, unsigned flags, void *data) { log("%s cmd %d", path, _IOC_NR(cmd)); struct pmemobjfs *objfs = PMEMOBJFS; /* check transaction stage */ switch (cmd) { case PMEMOBJFS_CTL_TX_BEGIN: if (pmemobj_tx_stage() != TX_STAGE_NONE) return -EINPROGRESS; break; case PMEMOBJFS_CTL_TX_ABORT: if (pmemobj_tx_stage() != TX_STAGE_WORK) return -EBADFD; break; case PMEMOBJFS_CTL_TX_COMMIT: if (pmemobj_tx_stage() != TX_STAGE_WORK) return -EBADFD; break; default: return -EINVAL; } /* * Store the inode offset and command and defer ioctl * execution to releasing the file. This is required * to avoid unlinking .tx_XXXXXX file inside transaction * because one would be rolled back if transaction abort * would occur. */ objfs->ioctl_off = fi->fh; objfs->ioctl_cmd = cmd; return 0; } /* * pmemobjfs_fuse_rename -- (FUSE) rename file or directory */ static int pmemobjfs_fuse_rename(const char *path, const char *dest) { log("%s dest %s\n", path, dest); struct pmemobjfs *objfs = PMEMOBJFS; int ret; /* get source inode's parent and name */ TOID(struct objfs_inode) src_parent; const char *src_name; ret = pmemobjfs_inode_lookup_parent(objfs, path, &src_parent, &src_name); if (ret) return ret; /* get destination inode's parent and name */ TOID(struct objfs_inode) dst_parent; const char *dst_name; ret = pmemobjfs_inode_lookup_parent(objfs, dest, &dst_parent, &dst_name); if (ret) return ret; return pmemobjfs_rename(objfs, src_parent, src_name, dst_parent, dst_name); } /* * pmemobjfs_fuse_symlink -- (FUSE) create symbolic link */ static int pmemobjfs_fuse_symlink(const char *path, const char *link) { log("%s link %s", path, link); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; const char *name; int ret = pmemobjfs_inode_lookup_parent(objfs, link, &inode, &name); if (ret) return ret; uid_t uid = fuse_get_context()->uid; uid_t gid = fuse_get_context()->gid; return pmemobjfs_symlink(objfs, inode, name, path, uid, gid); } /* * pmemobjfs_fuse_readlink -- (FUSE) read symbolic link */ static int pmemobjfs_fuse_readlink(const char *path, char *buff, size_t size) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; int ret = pmemobjfs_inode_lookup(objfs, path, &inode); if (ret) return ret; return pmemobjfs_symlink_read(inode, buff, size); } /* * pmemobjfs_fuse_mknod -- (FUSE) create node */ static int pmemobjfs_fuse_mknod(const char *path, mode_t mode, dev_t dev) { log("%s mode %o major %u minor %u", path, mode, (unsigned)MAJOR(dev), (unsigned)MINOR(dev)); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_inode) inode; const char *name; int ret = pmemobjfs_inode_lookup_parent(objfs, path, &inode, &name); if (ret) return ret; uid_t uid = fuse_get_context()->uid; uid_t gid = fuse_get_context()->gid; return pmemobjfs_mknod(objfs, inode, name, mode, uid, gid, dev); } /* * pmemobjfs_fuse_fallocate -- (FUSE) allocate blocks for file */ static int pmemobjfs_fuse_fallocate(const char *path, int mode, off_t offset, off_t size, struct fuse_file_info *fi) { log("%s mode %d offset %lu size %lu", path, mode, offset, size); struct pmemobjfs *objfs = PMEMOBJFS; if (!fi->fh) return -EINVAL; TOID(struct objfs_inode) inode; inode.oid.off = fi->fh; inode.oid.pool_uuid_lo = objfs->pool_uuid_lo; if (!TOID_VALID(inode)) return -EINVAL; return pmemobjfs_fallocate(objfs, inode, offset, size); } /* * pmemobjfs_fuse_statvfs -- (FUSE) get filesystem info */ static int pmemobjfs_fuse_statvfs(const char *path, struct statvfs *buff) { log("%s", path); struct pmemobjfs *objfs = PMEMOBJFS; memset(buff, 0, sizeof(*buff)); /* * Some fields are ignored by FUSE. * Some fields cannot be set due to the nature of pmemobjfs. */ buff->f_bsize = objfs->block_size; /* ignored buff->f_frsize */ /* unknown buff->f_blocks */ /* unknown buff->f_bfree */ /* unknown buff->f_bavail */ /* unknown buff->f_files */ /* unknown buff->f_ffree */ /* ignored buff->f_favail */ /* ignored buff->f_fsid */ /* ignored buff->f_flag */ buff->f_namemax = objfs->max_name; return 0; } /* * pmemobjfs_fuse_init -- (FUSE) initialization */ static void * pmemobjfs_fuse_init(struct fuse_conn_info *conn) { log(""); struct pmemobjfs *objfs = PMEMOBJFS; TOID(struct objfs_super) super = POBJ_ROOT(objfs->pop, struct objfs_super); /* fill some runtime information */ objfs->block_size = D_RO(super)->block_size; objfs->max_name = objfs->block_size - sizeof(struct objfs_dir_entry); objfs->pool_uuid_lo = super.oid.pool_uuid_lo; TX_BEGIN(objfs->pop) { /* release all opened inodes */ map_foreach(objfs->mapc, D_RW(super)->opened, pmemobjfs_put_opened_cb, objfs); /* clear opened inodes map */ map_clear(objfs->mapc, D_RW(super)->opened); } TX_ONABORT { objfs = NULL; } TX_END return objfs; } /* * pmemobjfs_ops -- fuse operations */ static struct fuse_operations pmemobjfs_ops = { /* filesystem operations */ .init = pmemobjfs_fuse_init, .statfs = pmemobjfs_fuse_statvfs, /* inode operations */ .getattr = pmemobjfs_fuse_getattr, .chmod = pmemobjfs_fuse_chmod, .chown = pmemobjfs_fuse_chown, .utimens = pmemobjfs_fuse_utimens, .ioctl = pmemobjfs_fuse_ioctl, /* directory operations */ .opendir = pmemobjfs_fuse_opendir, .releasedir = pmemobjfs_fuse_releasedir, .readdir = pmemobjfs_fuse_readdir, .mkdir = pmemobjfs_fuse_mkdir, .rmdir = pmemobjfs_fuse_rmdir, .rename = pmemobjfs_fuse_rename, .mknod = pmemobjfs_fuse_mknod, .symlink = pmemobjfs_fuse_symlink, .create = pmemobjfs_fuse_create, .unlink = pmemobjfs_fuse_unlink, /* regular file operations */ .open = pmemobjfs_fuse_open, .release = pmemobjfs_fuse_release, .write = pmemobjfs_fuse_write, .read = pmemobjfs_fuse_read, .flush = pmemobjfs_fuse_flush, .truncate = pmemobjfs_fuse_truncate, .ftruncate = pmemobjfs_fuse_ftruncate, .fallocate = pmemobjfs_fuse_fallocate, /* symlink operations */ .readlink = pmemobjfs_fuse_readlink, }; /* * pmemobjfs_mkfs -- create pmemobjfs filesystem */ static int pmemobjfs_mkfs(const char *fname, size_t size, size_t bsize, mode_t mode) { struct pmemobjfs *objfs = calloc(1, sizeof(*objfs)); if (!objfs) return -1; int ret = 0; objfs->block_size = bsize; objfs->pop = pmemobj_create(fname, POBJ_LAYOUT_NAME(pmemobjfs), size, mode); if (!objfs->pop) { fprintf(stderr, "error: %s\n", pmemobj_errormsg()); ret = -1; goto out_free_objfs; } objfs->mapc = map_ctx_init(MAP_CTREE, objfs->pop); if (!objfs->mapc) { perror("map_ctx_init"); ret = -1; goto out_close_pop; } TOID(struct objfs_super) super = POBJ_ROOT(objfs->pop, struct objfs_super); uid_t uid = getuid(); gid_t gid = getgid(); mode_t mask = umask(0); umask(mask); TX_BEGIN(objfs->pop) { /* inherit permissions from umask */ uint64_t root_flags = S_IFDIR | (~mask & 0777); TX_ADD(super); /* create an opened files map */ map_create(objfs->mapc, &D_RW(super)->opened, NULL); /* create root inode, inherit uid and gid from current user */ D_RW(super)->root_inode = pmemobjfs_new_dir(objfs, TOID_NULL(struct objfs_inode), "/", root_flags, uid, gid); D_RW(super)->block_size = bsize; } TX_ONABORT { fprintf(stderr, "error: creating pmemobjfs aborted\n"); ret = -ECANCELED; } TX_END map_ctx_free(objfs->mapc); out_close_pop: pmemobj_close(objfs->pop); out_free_objfs: free(objfs); return ret; } /* * parse_size -- parse size from string */ static int parse_size(const char *str, uint64_t *sizep) { uint64_t size = 0; int shift = 0; char unit[3] = {0}; int ret = sscanf(str, "%lu%3s", &size, unit); if (ret <= 0) return -1; if (ret == 2) { if ((unit[1] != '\0' && unit[1] != 'B') || unit[2] != '\0') return -1; switch (unit[0]) { case 'K': case 'k': shift = 10; break; case 'M': shift = 20; break; case 'G': shift = 30; break; case 'T': shift = 40; break; case 'P': shift = 50; break; default: return -1; } } if (sizep) *sizep = size << shift; return 0; } /* * pmemobjfs_mkfs_main -- parse arguments and create pmemobjfs */ static int pmemobjfs_mkfs_main(int argc, char *argv[]) { static const char *usage_str = "usage: %s " "[-h] " "[-s ] " "[-b ] " "\n"; if (argc < 2) { fprintf(stderr, usage_str, argv[0]); return -1; } uint64_t size = PMEMOBJ_MIN_POOL; uint64_t bsize = PMEMOBJFS_MIN_BLOCK_SIZE; int opt; const char *optstr = "hs:b:"; int size_used = 0; while ((opt = getopt(argc, argv, optstr)) != -1) { switch (opt) { case 'h': printf(usage_str, argv[0]); return 0; case 'b': if (parse_size(optarg, &bsize)) { fprintf(stderr, "error: invalid block size " "value specified -- '%s'\n", optarg); return -1; } break; case 's': if (parse_size(optarg, &size)) { fprintf(stderr, "error: invalid size " "value specified -- '%s'\n", optarg); return -1; } size_used = 1; break; } } if (optind >= argc) { fprintf(stderr, usage_str, argv[0]); return -1; } const char *path = argv[optind]; if (!access(path, F_OK)) { if (size_used) { fprintf(stderr, "error: cannot use size option " "for existing file\n"); return -1; } size = 0; } else { if (size < PMEMOBJ_MIN_POOL) { fprintf(stderr, "error: minimum size is %lu\n", PMEMOBJ_MIN_POOL); return -1; } } if (bsize < PMEMOBJFS_MIN_BLOCK_SIZE) { fprintf(stderr, "error: minimum block size is %zu\n", PMEMOBJFS_MIN_BLOCK_SIZE); return -1; } return pmemobjfs_mkfs(path, size, bsize, 0777); } /* * pmemobjfs_tx_ioctl -- transaction ioctl * * In order to call the ioctl we need to create a temporary file in * specified directory and call the ioctl on that file. After calling the * ioctl the file is unlinked. The actual action is performed after unlinking * the file so if the operation was to start a transaction the temporary file * won't be unlinked within the transaction. */ static int pmemobjfs_tx_ioctl(const char *dir, int req) { int ret = 0; /* append temporary file template to specified path */ size_t dirlen = strlen(dir); size_t tmpllen = strlen(PMEMOBJFS_TMP_TEMPLATE); char *path = malloc(dirlen + tmpllen + 1); if (!path) return -1; memcpy(path, dir, dirlen); strcpy(path + dirlen, PMEMOBJFS_TMP_TEMPLATE); /* create temporary file */ mode_t prev_umask = umask(S_IRWXG | S_IRWXO); int fd = mkstemp(path); umask(prev_umask); if (fd < 0) { perror(path); ret = -1; goto out_free; } /* perform specified ioctl command */ ret = ioctl(fd, req); if (ret) { perror(path); goto out_unlink; } out_unlink: /* unlink temporary file */ ret = unlink(path); if (ret) perror(path); close(fd); out_free: free(path); return ret; } int main(int argc, char *argv[]) { char *bname = basename(argv[0]); if (strcmp(PMEMOBJFS_MKFS, bname) == 0) { return pmemobjfs_mkfs_main(argc, argv); } else if (strcmp(PMEMOBJFS_TX_BEGIN, bname) == 0) { if (argc != 2) { fprintf(stderr, "usage: %s \n", bname); return -1; } char *arg = argv[1]; return pmemobjfs_tx_ioctl(arg, PMEMOBJFS_CTL_TX_BEGIN); } else if (strcmp(PMEMOBJFS_TX_COMMIT, bname) == 0) { if (argc != 2) { fprintf(stderr, "usage: %s \n", bname); return -1; } char *arg = argv[1]; return pmemobjfs_tx_ioctl(arg, PMEMOBJFS_CTL_TX_COMMIT); } else if (strcmp(PMEMOBJFS_TX_ABORT, bname) == 0) { if (argc != 2) { fprintf(stderr, "usage: %s \n", bname); return -1; } char *arg = argv[1]; return pmemobjfs_tx_ioctl(arg, PMEMOBJFS_CTL_TX_ABORT); } #if DEBUG log_fh = fopen("pmemobjfs.log", "w+"); if (!log_fh) err(-1, "pmemobjfs.log"); log("\n\n\nPMEMOBJFS\n"); #endif const char *fname = argv[argc - 2]; struct pmemobjfs *objfs = calloc(1, sizeof(*objfs)); if (!objfs) { perror("malloc"); return -1; } int ret = 0; objfs->pop = pmemobj_open(fname, POBJ_LAYOUT_NAME(pmemobjfs)); if (objfs->pop == NULL) { perror("pmemobj_open"); ret = -1; goto out; } objfs->mapc = map_ctx_init(MAP_CTREE, objfs->pop); if (!objfs->mapc) { perror("map_ctx_init"); ret = -1; goto out; } argv[argc - 2] = argv[argc - 1]; argv[argc - 1] = NULL; argc--; ret = fuse_main(argc, argv, &pmemobjfs_ops, objfs); pmemobj_close(objfs->pop); out: free(objfs); log("ret = %d", ret); return ret; } pmdk-1.8/src/examples/libpmemobj/pmemobjfs/Makefile0000664000000000000000000000520413615011243021161 0ustar rootroot# # Copyright 2015-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/pmemobjfs/Makefile -- build simple fuse base on libpmemobj # TOP := $(dir $(lastword $(MAKEFILE_LIST)))../../../../ include $(TOP)/src/common.inc vpath %.c ../tree_map/ vpath %.c ../map/ FUSE_REQ_VER = 2.9.1 FUSE := $(call check_package, fuse --atleast-version $(FUSE_REQ_VER)) ifeq ($(FUSE), y) LINKS = mkfs.pmemobjfs\ pmemobjfs.tx_begin\ pmemobjfs.tx_commit\ pmemobjfs.tx_abort PROGS = pmemobjfs else $(info NOTE: Skipping pmemobjfs because fuse (version >= $(FUSE_REQ_VER)) is missing \ -- see src/examples/libpmemobj/pmemobjfs/README for details.) endif all-progs: $(LINKS) clean-progs: clean-links include ../../Makefile.inc LIBS += -lpmemobj -lpmem -pthread ifeq ($(FUSE), y) LIBS += $(shell $(PKG_CONFIG) --libs fuse) CFLAGS += $(shell $(PKG_CONFIG) --cflags fuse) endif CFLAGS += -I../tree_map CFLAGS += -I../map CFLAGS += -DFUSE_USE_VERSION=28 CFLAGS += -DUSE_CTREE $(LINKS): pmemobjfs $(LN) -sf pmemobjfs $@ clean-links: $(RM) $(LINKS) pmemobjfs: pmemobjfs.o ../map/libmap_ctree.a ../map/libmap_ctree.a: $(MAKE) -C ../map map_ctree .PHONY: clean-links pmdk-1.8/src/examples/libpmemobj/pminvaders/0000775000000000000000000000000013615011243017706 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/pminvaders/README0000664000000000000000000000247113615011243020572 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemobj/pminvaders/README. This directory contains two variants of an example application implemented using libpmemobj, it's a simple game in which all the objects are stored on persistent memory. This means that the game process can be safely killed and then resumed. The first implementation of the game demonstrates the usage of the very basics of the pmemobj library, that includes the pool management (open/create/close), type-safety macros, atomic (non-transactional) allocations and internal collections. The second version is fully based on the transactional API and does not use internal pmemobj collections. To launch the game: ./pminvaders [game session file] or: ./pminvaders2 [game session file] The file with the game session will either be created if it doesn't exist or opened if it contains a valid pool. Controls: move (left/right) - 'o' / 'p' or arrow keys shoot - spacebar quit - 'q' If you want to run this demo on non-pmem storage, set the PMEM_IS_PMEM_FORCE environment variable, so that the library does not issue msync calls. ** DEPENDENCIES: ** In order to build both variants of the game you need to install ncurses development package. rpm-based systems : ncurses-devel dpkg-based systems: libncursesX-dev (where X is the API/ABI version) pmdk-1.8/src/examples/libpmemobj/pminvaders/pminvaders.c0000664000000000000000000002207613615011243022231 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pminvaders.c -- example usage of non-tx allocations */ #include #ifdef __FreeBSD__ #include /* Need pkg, not system, version */ #else #include #endif #include #include #include #include #include #include #include #define LAYOUT_NAME "pminvaders" #define PMINVADERS_POOL_SIZE (100 * 1024 * 1024) /* 100 megabytes */ #define GAME_WIDTH 30 #define GAME_HEIGHT 30 #define RRAND(min, max) (rand() % ((max) - (min) + 1) + (min)) #define STEP 50 #define PLAYER_Y (GAME_HEIGHT - 1) #define MAX_GSTATE_TIMER 10000 #define MIN_GSTATE_TIMER 5000 #define MAX_ALIEN_TIMER 1000 #define MAX_PLAYER_TIMER 1000 #define MAX_BULLET_TIMER 500 enum colors { C_UNKNOWN, C_PLAYER, C_ALIEN, C_BULLET, MAX_C }; struct game_state { uint32_t timer; /* alien spawn timer */ uint16_t score; uint16_t high_score; }; struct alien { uint16_t x; uint16_t y; uint32_t timer; /* movement timer */ }; struct player { uint16_t x; uint16_t padding; /* to 8 bytes */ uint32_t timer; /* weapon cooldown */ }; struct bullet { uint16_t x; uint16_t y; uint32_t timer; /* movement timer */ }; /* * Layout definition */ POBJ_LAYOUT_BEGIN(pminvaders); POBJ_LAYOUT_ROOT(pminvaders, struct game_state); POBJ_LAYOUT_TOID(pminvaders, struct player); POBJ_LAYOUT_TOID(pminvaders, struct alien); POBJ_LAYOUT_TOID(pminvaders, struct bullet); POBJ_LAYOUT_END(pminvaders); static PMEMobjpool *pop; static struct game_state *gstate; /* * create_alien -- constructor for aliens, spawn at random position */ static int create_alien(PMEMobjpool *pop, void *ptr, void *arg) { struct alien *a = ptr; a->y = 1; a->x = RRAND(2, GAME_WIDTH - 2); a->timer = 1; pmemobj_persist(pop, a, sizeof(*a)); return 0; } /* * create_player -- constructor for the player, spawn in the middle of the map */ static int create_player(PMEMobjpool *pop, void *ptr, void *arg) { struct player *p = ptr; p->x = GAME_WIDTH / 2; p->timer = 1; pmemobj_persist(pop, p, sizeof(*p)); return 0; } /* * create_bullet -- constructor for bullets, spawn at the position of the player */ static int create_bullet(PMEMobjpool *pop, void *ptr, void *arg) { struct bullet *b = ptr; struct player *p = arg; b->x = p->x; b->y = PLAYER_Y - 1; b->timer = 1; pmemobj_persist(pop, b, sizeof(*b)); return 0; } static void draw_border(void) { for (int x = 0; x <= GAME_WIDTH; ++x) { mvaddch(0, x, ACS_HLINE); mvaddch(GAME_HEIGHT, x, ACS_HLINE); } for (int y = 0; y <= GAME_HEIGHT; ++y) { mvaddch(y, 0, ACS_VLINE); mvaddch(y, GAME_WIDTH, ACS_VLINE); } mvaddch(0, 0, ACS_ULCORNER); mvaddch(GAME_HEIGHT, 0, ACS_LLCORNER); mvaddch(0, GAME_WIDTH, ACS_URCORNER); mvaddch(GAME_HEIGHT, GAME_WIDTH, ACS_LRCORNER); } static void draw_alien(const TOID(struct alien) a) { mvaddch(D_RO(a)->y, D_RO(a)->x, ACS_DIAMOND|COLOR_PAIR(C_ALIEN)); } static void draw_player(const TOID(struct player) p) { mvaddch(PLAYER_Y, D_RO(p)->x, ACS_DIAMOND|COLOR_PAIR(C_PLAYER)); } static void draw_bullet(const TOID(struct bullet) b) { mvaddch(D_RO(b)->y, D_RO(b)->x, ACS_BULLET|COLOR_PAIR(C_BULLET)); } static void draw_score(void) { mvprintw(1, 1, "Score: %u | %u\n", gstate->score, gstate->high_score); } /* * timer_tick -- very simple persistent timer */ static int timer_tick(uint32_t *timer) { int ret = *timer == 0 || ((*timer)--) == 0; pmemobj_persist(pop, timer, sizeof(*timer)); return ret; } /* * update_score -- change player score and global high score */ static void update_score(int m) { if (m < 0 && gstate->score == 0) return; uint16_t score = gstate->score + m; uint16_t highscore = score > gstate->high_score ? score : gstate->high_score; struct game_state s = { .timer = gstate->timer, .score = score, .high_score = highscore }; *gstate = s; pmemobj_persist(pop, gstate, sizeof(*gstate)); } /* * process_aliens -- process spawn and movement of the aliens */ static void process_aliens(void) { /* alien spawn timer */ if (timer_tick(&gstate->timer)) { gstate->timer = RRAND(MIN_GSTATE_TIMER, MAX_GSTATE_TIMER); pmemobj_persist(pop, gstate, sizeof(*gstate)); POBJ_NEW(pop, NULL, struct alien, create_alien, NULL); } TOID(struct alien) iter, next; POBJ_FOREACH_SAFE_TYPE(pop, iter, next) { if (timer_tick(&D_RW(iter)->timer)) { D_RW(iter)->timer = MAX_ALIEN_TIMER; D_RW(iter)->y++; } pmemobj_persist(pop, D_RW(iter), sizeof(struct alien)); draw_alien(iter); /* decrease the score if the ship wasn't intercepted */ if (D_RO(iter)->y > GAME_HEIGHT - 1) { POBJ_FREE(&iter); update_score(-1); pmemobj_persist(pop, gstate, sizeof(*gstate)); } } } /* * process_collision -- search for any aliens on the position of the bullet */ static int process_collision(const TOID(struct bullet) b) { TOID(struct alien) iter; POBJ_FOREACH_TYPE(pop, iter) { if (D_RO(b)->x == D_RO(iter)->x && D_RO(b)->y == D_RO(iter)->y) { update_score(1); POBJ_FREE(&iter); return 1; } } return 0; } /* * process_bullets -- process bullets movement and collision */ static void process_bullets(void) { TOID(struct bullet) iter, next; POBJ_FOREACH_SAFE_TYPE(pop, iter, next) { /* bullet movement timer */ if (timer_tick(&D_RW(iter)->timer)) { D_RW(iter)->timer = MAX_BULLET_TIMER; D_RW(iter)->y--; } pmemobj_persist(pop, D_RW(iter), sizeof(struct bullet)); draw_bullet(iter); if (D_RO(iter)->y == 0 || process_collision(iter)) POBJ_FREE(&iter); } } /* * process_player -- handle player actions */ static void process_player(int input) { TOID(struct player) plr = POBJ_FIRST(pop, struct player); /* weapon cooldown tick */ timer_tick(&D_RW(plr)->timer); switch (input) { case KEY_LEFT: case 'o': { uint16_t dstx = D_RO(plr)->x - 1; if (dstx != 0) D_RW(plr)->x = dstx; } break; case KEY_RIGHT: case 'p': { uint16_t dstx = D_RO(plr)->x + 1; if (dstx != GAME_WIDTH - 1) D_RW(plr)->x = dstx; } break; case ' ': if (D_RO(plr)->timer == 0) { D_RW(plr)->timer = MAX_PLAYER_TIMER; POBJ_NEW(pop, NULL, struct bullet, create_bullet, D_RW(plr)); } break; default: break; } pmemobj_persist(pop, D_RW(plr), sizeof(struct player)); draw_player(plr); } /* * game_loop -- process drawing and logic of the game */ static void game_loop(int input) { erase(); draw_score(); draw_border(); process_aliens(); process_bullets(); process_player(input); usleep(STEP); refresh(); } int main(int argc, char *argv[]) { if (argc != 2) { printf("usage: %s file-name\n", argv[0]); return 1; } const char *path = argv[1]; pop = NULL; srand(time(NULL)); if (access(path, F_OK) != 0) { if ((pop = pmemobj_create(path, POBJ_LAYOUT_NAME(pminvaders), PMINVADERS_POOL_SIZE, S_IWUSR | S_IRUSR)) == NULL) { printf("failed to create pool\n"); return 1; } /* create the player and initialize with a constructor */ POBJ_NEW(pop, NULL, struct player, create_player, NULL); } else { if ((pop = pmemobj_open(path, LAYOUT_NAME)) == NULL) { printf("failed to open pool\n"); return 1; } } /* global state of the game is kept in the root object */ TOID(struct game_state) game_state = POBJ_ROOT(pop, struct game_state); gstate = D_RW(game_state); initscr(); start_color(); init_pair(C_PLAYER, COLOR_GREEN, COLOR_BLACK); init_pair(C_ALIEN, COLOR_RED, COLOR_BLACK); init_pair(C_BULLET, COLOR_YELLOW, COLOR_BLACK); nodelay(stdscr, true); curs_set(0); keypad(stdscr, true); int in; while ((in = getch()) != 'q') { game_loop(in); } pmemobj_close(pop); endwin(); return 0; } pmdk-1.8/src/examples/libpmemobj/pminvaders/.gitignore0000664000000000000000000000003013615011243021667 0ustar rootrootpminvaders pminvaders2 pmdk-1.8/src/examples/libpmemobj/pminvaders/pminvaders2.c0000664000000000000000000003773513615011243022323 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * pminvaders2.c -- PMEM-based clone of space invaders (version 2.0) * * RULES: * +1 point for each alien destroyed (72 per level) * -100 points and move to a lower level when killed */ #include #ifdef __FreeBSD__ #include /* Need pkg, not system, version */ #else #include #endif #include #include #include #include #include #include /* * Layout definition */ POBJ_LAYOUT_BEGIN(pminvaders2); POBJ_LAYOUT_ROOT(pminvaders2, struct root); POBJ_LAYOUT_TOID(pminvaders2, struct state); POBJ_LAYOUT_TOID(pminvaders2, struct alien); POBJ_LAYOUT_TOID(pminvaders2, struct player); POBJ_LAYOUT_TOID(pminvaders2, struct bullet); POBJ_LAYOUT_TOID(pminvaders2, struct star); POBJ_LAYOUT_END(pminvaders2); #define POOL_SIZE (100 * 1024 * 1024) /* 100 megabytes */ #define GAME_WIDTH 50 #define GAME_HEIGHT 25 #define ALIENS_ROW 4 #define ALIENS_COL 18 #define RRAND(min, max) (rand() % ((max) - (min) + 1) + (min)) #define STEP 50 #define PLAYER_Y (GAME_HEIGHT - 1) #define MAX_GSTATE_TIMER 10000 #define MIN_GSTATE_TIMER 5000 #define MAX_ALIEN_TIMER 1000 #define MAX_PLAYER_TIMER 1000 #define MAX_BULLET_TIMER 500 #define MAX_STAR1_TIMER 200 #define MAX_STAR2_TIMER 100 enum game_event { EVENT_NOP, EVENT_BOUNCE, EVENT_PLAYER_KILLED, EVENT_ALIENS_KILLED }; enum colors { C_UNKNOWN, C_PLAYER, C_ALIEN, C_BULLET, C_STAR, C_INTRO }; struct state { unsigned timer; int score; unsigned high_score; int level; int new_level; int dx; int dy; }; struct player { unsigned x; unsigned timer; }; struct alien { unsigned x; unsigned y; TOID(struct alien) prev; TOID(struct alien) next; }; struct star { unsigned x; unsigned y; int c; unsigned timer; TOID(struct star) prev; TOID(struct star) next; }; struct bullet { unsigned x; unsigned y; unsigned timer; TOID(struct bullet) prev; TOID(struct bullet) next; }; struct root { TOID(struct state) state; TOID(struct player) player; TOID(struct alien) aliens; TOID(struct bullet) bullets; TOID(struct star) stars; }; /* * draw_star -- draw a star */ static void draw_star(const struct star *s) { mvaddch(s->y, s->x, s->c | COLOR_PAIR(C_STAR)); } /* * draw_alien -- draw an alien */ static void draw_alien(const struct alien *a) { mvaddch(a->y, a->x, ACS_DIAMOND | COLOR_PAIR(C_ALIEN)); } /* * draw_player -- draw a player */ static void draw_player(const struct player *p) { mvaddch(PLAYER_Y, p->x, ACS_DIAMOND | COLOR_PAIR(C_PLAYER)); } /* * draw_bullet -- draw a bullet */ static void draw_bullet(const struct bullet *b) { mvaddch(b->y, b->x, ACS_BULLET | COLOR_PAIR(C_BULLET)); } /* * draw_score -- draw the game score and the global highest score */ static void draw_score(const struct state *s) { mvprintw(1, 1, "Level: %u Score: %u | %u\n", s->level, s->score, s->high_score); } /* * draw_title -- draw the game title and menu */ static void draw_title(void) { int x = (GAME_WIDTH - 40) / 2; int y = GAME_HEIGHT / 2 - 2; attron(COLOR_PAIR(C_INTRO)); mvprintw(y + 0, x, "#### # # ### # # # # ### ###"); mvprintw(y + 1, x, "# # ## ## # ## # # # # # #"); mvprintw(y + 2, x, "#### # # # # # # # # # ### # #"); mvprintw(y + 3, x, "# # # # # # ## # # # # #"); mvprintw(y + 4, x, "# # # ### # # # ### # ###"); attroff(COLOR_PAIR(C_INTRO)); mvprintw(y + 6, x, " Press 'space' to resume "); mvprintw(y + 7, x, " Press 'q' to quit "); } /* * draw_border -- draw a frame around the map */ static void draw_border(void) { for (int x = 0; x <= GAME_WIDTH; ++x) { mvaddch(0, x, ACS_HLINE); mvaddch(GAME_HEIGHT, x, ACS_HLINE); } for (int y = 0; y <= GAME_HEIGHT; ++y) { mvaddch(y, 0, ACS_VLINE); mvaddch(y, GAME_WIDTH, ACS_VLINE); } mvaddch(0, 0, ACS_ULCORNER); mvaddch(GAME_HEIGHT, 0, ACS_LLCORNER); mvaddch(0, GAME_WIDTH, ACS_URCORNER); mvaddch(GAME_HEIGHT, GAME_WIDTH, ACS_LRCORNER); } /* * timer_tick -- very simple persistent timer */ static int timer_tick(unsigned *timer) { return *timer == 0 || ((*timer)--) == 0; } /* * create_star -- create a single star at random position */ static TOID(struct star) create_star(unsigned x, unsigned y, TOID(struct star) next) { TOID(struct star) s = TX_ZNEW(struct star); struct star *sp = D_RW(s); sp->x = x; sp->y = y; sp->c = rand() % 2 ? '*' : '.'; sp->timer = sp->c == '.' ? MAX_STAR1_TIMER : MAX_STAR2_TIMER; sp->prev = TOID_NULL(struct star); sp->next = next; if (!TOID_IS_NULL(next)) D_RW(next)->prev = s; return s; } /* * create_stars -- create a new set of stars at random positions */ static void create_stars(TOID(struct root) r) { for (int x = 1; x < GAME_WIDTH; x++) { if (rand() % 100 < 4) D_RW(r)->stars = create_star(x, 1, D_RW(r)->stars); } } /* * process_stars -- process creation and movement of the stars */ static void process_stars(PMEMobjpool *pop, TOID(struct root) r) { int new_line = 0; TX_BEGIN(pop) { TOID(struct star) s = D_RO(r)->stars; while (!TOID_IS_NULL(s)) { TX_ADD(s); struct star *sptr = D_RW(s); TOID(struct star) sp = sptr->prev; TOID(struct star) sn = sptr->next; if (timer_tick(&sptr->timer)) { sptr->timer = sptr->c == '.' ? MAX_STAR1_TIMER : MAX_STAR2_TIMER; sptr->y++; if (sptr->c == '.') new_line = 1; } draw_star(sptr); if (sptr->y >= GAME_HEIGHT) { if (!TOID_IS_NULL(sp)) { TX_ADD(sp); D_RW(sp)->next = sn; } else { TX_ADD(r); D_RW(r)->stars = sn; } if (!TOID_IS_NULL(sn)) { TX_ADD(sn); D_RW(sn)->prev = sp; } TX_FREE(s); } s = sn; } if (new_line) create_stars(r); } TX_END; } /* * create_alien -- create an alien at given position */ static TOID(struct alien) create_alien(unsigned x, unsigned y, TOID(struct alien) next) { TOID(struct alien) a = TX_ZNEW(struct alien); struct alien *ap = D_RW(a); ap->x = x; ap->y = y; ap->prev = TOID_NULL(struct alien); ap->next = next; if (!TOID_IS_NULL(next)) D_RW(next)->prev = a; return a; } /* * create_aliens -- create new set of aliens */ static void create_aliens(TOID(struct root) r) { for (int x = 0; x < ALIENS_COL; x++) { for (int y = 0; y < ALIENS_ROW; y++) { unsigned pos_x = (GAME_WIDTH / 2) - (ALIENS_COL) + (x * 2); unsigned pos_y = y + 3; D_RW(r)->aliens = create_alien(pos_x, pos_y, D_RW(r)->aliens); } } } /* * remove_aliens -- remove all the aliens from the map */ static void remove_aliens(TOID(struct alien) *ah) { while (!TOID_IS_NULL(*ah)) { TOID(struct alien) an = D_RW(*ah)->next; TX_FREE(*ah); *ah = an; } } /* * move_aliens -- process movement of the aliens */ static int move_aliens(PMEMobjpool *pop, TOID(struct root) r, int dx, int dy) { int ret = EVENT_NOP; int cnt = 0; TOID(struct alien) a = D_RO(r)->aliens; while (!TOID_IS_NULL(a)) { TX_ADD(a); struct alien *ap = D_RW(a); cnt++; if (dy) ap->y += dy; else ap->x += dx; if (ap->y >= PLAYER_Y) ret = EVENT_PLAYER_KILLED; else if (dy == 0 && (ap->x >= GAME_WIDTH - 2 || ap->x <= 2)) ret = EVENT_BOUNCE; a = ap->next; } if (cnt == 0) ret = EVENT_ALIENS_KILLED; /* all killed */ return ret; } /* * create_player -- spawn the player in the middle of the map */ static TOID(struct player) create_player(void) { TOID(struct player) p = TX_ZNEW(struct player); struct player *pp = D_RW(p); pp->x = GAME_WIDTH / 2; pp->timer = 1; return p; } /* * create_bullet -- spawn the bullet at the position of the player */ static TOID(struct bullet) create_bullet(unsigned x, TOID(struct bullet) next) { TOID(struct bullet) b = TX_ZNEW(struct bullet); struct bullet *bp = D_RW(b); bp->x = x; bp->y = PLAYER_Y - 1; bp->timer = 1; bp->prev = TOID_NULL(struct bullet); bp->next = next; if (!TOID_IS_NULL(next)) D_RW(next)->prev = b; return b; } /* * create_state -- create game state */ static TOID(struct state) create_state(void) { TOID(struct state) s = TX_ZNEW(struct state); struct state *sp = D_RW(s); sp->timer = 1; sp->score = 0; sp->high_score = 0; sp->level = 0; sp->new_level = 1; sp->dx = 1; sp->dy = 0; return s; } /* * new_level -- prepare map for the new game level */ static void new_level(PMEMobjpool *pop, TOID(struct root) r) { TX_BEGIN(pop) { TX_ADD(r); struct root *rp = D_RW(r); remove_aliens(&rp->aliens); create_aliens(r); TX_ADD(rp->state); struct state *sp = D_RW(rp->state); if (sp->new_level > 0 || sp->level > 1) sp->level += sp->new_level; sp->new_level = 0; sp->dx = 1; sp->dy = 0; sp->timer = MAX_ALIEN_TIMER - 100 * (sp->level - 1); } TX_END; } /* * update_score -- change player score and global high score */ static void update_score(struct state *sp, int m) { if (m < 0 && sp->score == 0) return; sp->score += m; if (sp->score < 0) sp->score = 0; if (sp->score > sp->high_score) sp->high_score = sp->score; } /* * process_aliens -- process movement of the aliens and game events */ static void process_aliens(PMEMobjpool *pop, TOID(struct root) r) { TX_BEGIN(pop) { TOID(struct state) s = D_RO(r)->state; TX_ADD(s); struct state *sp = D_RW(s); if (timer_tick(&sp->timer)) { sp->timer = MAX_ALIEN_TIMER - 50 * (sp->level - 1); switch (move_aliens(pop, r, sp->dx, sp->dy)) { case EVENT_ALIENS_KILLED: /* all aliens killed */ sp->new_level = 1; break; case EVENT_PLAYER_KILLED: /* player killed */ flash(); beep(); sp->new_level = -1; update_score(sp, -100); break; case EVENT_BOUNCE: /* move down + change direction */ sp->dy = 1; sp->dx = -sp->dx; break; case EVENT_NOP: /* do nothing */ sp->dy = 0; break; } } } TX_END; TOID(struct alien) a = D_RO(r)->aliens; while (!TOID_IS_NULL(a)) { const struct alien *ap = D_RO(a); draw_alien(ap); a = ap->next; } } /* * process_collision -- search for any aliens on the position of the bullet */ static int process_collision(PMEMobjpool *pop, TOID(struct root) r, struct state *sp, const struct bullet *bp) { int ret = 0; TX_BEGIN(pop) { TOID(struct alien) a = D_RO(r)->aliens; while (!TOID_IS_NULL(a)) { struct alien *aptr = D_RW(a); TOID(struct alien) ap = aptr->prev; TOID(struct alien) an = aptr->next; if (bp->x == aptr->x && bp->y == aptr->y) { update_score(sp, 1); if (!TOID_IS_NULL(ap)) { TX_ADD(ap); D_RW(ap)->next = an; } else { TX_ADD(r); D_RW(r)->aliens = an; } if (!TOID_IS_NULL(an)) { TX_ADD(an); D_RW(an)->prev = ap; } TX_FREE(a); ret = 1; break; } a = an; } } TX_END; return ret; } /* * process_bullets -- process bullets movement and collision */ static void process_bullets(PMEMobjpool *pop, TOID(struct root) r, struct state *sp) { TX_BEGIN(pop) { TOID(struct bullet) b = D_RO(r)->bullets; while (!TOID_IS_NULL(b)) { TX_ADD(b); struct bullet *bptr = D_RW(b); TOID(struct bullet) bp = bptr->prev; TOID(struct bullet) bn = bptr->next; if (timer_tick(&bptr->timer)) { bptr->timer = MAX_BULLET_TIMER; bptr->y--; } draw_bullet(bptr); if (bptr->y <= 0 || process_collision(pop, r, sp, bptr)) { if (!TOID_IS_NULL(bp)) { TX_ADD(bp); D_RW(bp)->next = bn; } else { TX_ADD(r); D_RW(r)->bullets = bn; } if (!TOID_IS_NULL(bn)) { TX_ADD(bn); D_RW(bn)->prev = bp; } TX_FREE(b); } b = bn; } } TX_END; } /* * process_player -- handle player actions */ static void process_player(PMEMobjpool *pop, TOID(struct root) r, int input) { TOID(struct player) p = D_RO(r)->player; TX_BEGIN(pop) { TX_ADD(r); TX_ADD(p); struct player *pp = D_RW(p); timer_tick(&pp->timer); if (input == KEY_LEFT || input == 'o') { unsigned dstx = pp->x - 1; if (dstx != 0) pp->x = dstx; } else if (input == KEY_RIGHT || input == 'p') { unsigned dstx = pp->x + 1; if (dstx != GAME_WIDTH) pp->x = dstx; } else if (input == ' ' && pp->timer == 0) { pp->timer = MAX_PLAYER_TIMER; D_RW(r)->bullets = create_bullet(pp->x, D_RW(r)->bullets); } } TX_END; draw_player(D_RO(p)); } /* * game_init -- create and initialize game state and the player */ static TOID(struct root) game_init(PMEMobjpool *pop) { TOID(struct root) r = POBJ_ROOT(pop, struct root); TX_BEGIN(pop) { TX_ADD(r); struct root *rp = D_RW(r); if (TOID_IS_NULL(rp->state)) rp->state = create_state(); if (TOID_IS_NULL(rp->player)) rp->player = create_player(); } TX_END; return r; } /* * game_loop -- process drawing and logic of the game */ static int game_loop(PMEMobjpool *pop, TOID(struct root) r) { int input = getch(); TOID(struct state) s = D_RO(r)->state; struct state *sp = D_RW(s); erase(); draw_score(sp); draw_border(); TX_BEGIN(pop) { TX_ADD(r); TX_ADD(s); if (sp->new_level != 0) new_level(pop, r); process_aliens(pop, r); process_bullets(pop, r, sp); process_player(pop, r, input); } TX_END; usleep(STEP); refresh(); if (input == 'q') return -1; else return 0; } /* * intro_loop -- process drawing of the intro animation */ static int intro_loop(PMEMobjpool *pop, TOID(struct root) r) { int input = getch(); erase(); draw_border(); TX_BEGIN(pop) { TX_ADD(r); struct root *rp = D_RW(r); if (TOID_IS_NULL(rp->stars)) create_stars(r); process_stars(pop, r); } TX_END; draw_title(); usleep(STEP); refresh(); switch (input) { case ' ': return 1; case 'q': return -1; default: return 0; } } int main(int argc, char *argv[]) { static PMEMobjpool *pop; int in; if (argc != 2) exit(1); srand(time(NULL)); if (access(argv[1], F_OK)) { if ((pop = pmemobj_create(argv[1], POBJ_LAYOUT_NAME(pminvaders2), POOL_SIZE, S_IRUSR | S_IWUSR)) == NULL) { fprintf(stderr, "%s", pmemobj_errormsg()); exit(1); } } else { if ((pop = pmemobj_open(argv[1], POBJ_LAYOUT_NAME(pminvaders2))) == NULL) { fprintf(stderr, "%s", pmemobj_errormsg()); exit(1); } } initscr(); start_color(); init_pair(C_PLAYER, COLOR_GREEN, COLOR_BLACK); init_pair(C_ALIEN, COLOR_RED, COLOR_BLACK); init_pair(C_BULLET, COLOR_YELLOW, COLOR_BLACK); init_pair(C_STAR, COLOR_WHITE, COLOR_BLACK); init_pair(C_INTRO, COLOR_BLUE, COLOR_BLACK); nodelay(stdscr, true); curs_set(0); keypad(stdscr, true); TOID(struct root) r = game_init(pop); while ((in = intro_loop(pop, r)) == 0) ; if (in == -1) goto end; while ((in = game_loop(pop, r)) == 0) ; end: endwin(); pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/pminvaders/Makefile0000664000000000000000000000417113615011243021351 0ustar rootroot# # Copyright 2015-2019, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/pminvaders/Makefile -- build the pminvaders examples # TOP := $(dir $(lastword $(MAKEFILE_LIST)))../../../../ include $(TOP)/src/common.inc NCURSES := $(call check_package, ncurses) ifeq ($(NCURSES),y) PROGS = pminvaders pminvaders2 else $(info NOTE: Skipping pminvaders and pminvaders2 because ncurses is missing \ -- see src/examples/libpmemobj/pminvaders/README for details.) endif LIBS = -lpmemobj -lpmem -pthread ifeq ($(NCURSES),y) LIBS += $(shell $(PKG_CONFIG) --libs ncurses) endif include ../../Makefile.inc pminvaders: pminvaders.o pminvaders2: pminvaders2.o pmdk-1.8/src/examples/libpmemobj/slab_allocator/0000775000000000000000000000000013615011243020517 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/slab_allocator/slab_allocator.h0000664000000000000000000000401113615011243023645 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * slab_allocator.h -- slab-like mechanism for libpmemobj */ #ifndef SLAB_ALLOCATOR_H #define SLAB_ALLOCATOR_H #include struct slab_allocator; struct slab_allocator *slab_new(PMEMobjpool *pop, size_t size); void slab_delete(struct slab_allocator *slab); int slab_alloc(struct slab_allocator *slab, PMEMoid *oid, pmemobj_constr constructor, void *arg); PMEMoid slab_tx_alloc(struct slab_allocator *slab); #endif /* SLAB_ALLOCATOR_H */ pmdk-1.8/src/examples/libpmemobj/slab_allocator/slab_allocator.c0000664000000000000000000000605513615011243023652 0ustar rootroot/* * Copyright 2017-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * slab_allocator.c -- slab-like mechanism for libpmemobj */ #include "slab_allocator.h" #include struct slab_allocator { PMEMobjpool *pop; struct pobj_alloc_class_desc class; }; /* * slab_new -- creates a new slab allocator instance */ struct slab_allocator * slab_new(PMEMobjpool *pop, size_t size) { struct slab_allocator *slab = malloc(sizeof(struct slab_allocator)); if (slab == NULL) return NULL; slab->pop = pop; slab->class.header_type = POBJ_HEADER_NONE; slab->class.unit_size = size; slab->class.alignment = 0; /* should be a reasonably high number, but not too crazy */ slab->class.units_per_block = 1000; if (pmemobj_ctl_set(pop, "heap.alloc_class.new.desc", &slab->class) != 0) goto error; return slab; error: free(slab); return NULL; } /* * slab_delete -- deletes an existing slab allocator instance */ void slab_delete(struct slab_allocator *slab) { free(slab); } /* * slab_alloc -- works just like pmemobj_alloc but uses the predefined * blocks from the slab */ int slab_alloc(struct slab_allocator *slab, PMEMoid *oid, pmemobj_constr constructor, void *arg) { return pmemobj_xalloc(slab->pop, oid, slab->class.unit_size, 0, POBJ_CLASS_ID(slab->class.class_id), constructor, arg); } /* * slab_tx_alloc -- works just like pmemobj_tx_alloc but uses the predefined * blocks from the slab */ PMEMoid slab_tx_alloc(struct slab_allocator *slab) { return pmemobj_tx_xalloc(slab->class.unit_size, 0, POBJ_CLASS_ID(slab->class.class_id)); } pmdk-1.8/src/examples/libpmemobj/slab_allocator/.gitignore0000664000000000000000000000003313615011243022503 0ustar rootrootmain libslab_allocator.so pmdk-1.8/src/examples/libpmemobj/slab_allocator/main.c0000664000000000000000000000677513615011243021626 0ustar rootroot/* * Copyright 2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * main.c -- example usage of a slab-like mechanism implemented in libpmemobj * * This application does nothing besides demonstrating the example slab * allocator mechanism. * * By using the CTL alloc class API we can instrument libpmemobj to optimally * manage memory for the pool. */ #include #include #include #include "slab_allocator.h" POBJ_LAYOUT_BEGIN(slab_allocator); POBJ_LAYOUT_ROOT(slab_allocator, struct root); POBJ_LAYOUT_TOID(slab_allocator, struct bar); POBJ_LAYOUT_TOID(slab_allocator, struct foo); POBJ_LAYOUT_END(slab_allocator); struct foo { char data[100]; }; struct bar { char data[500]; }; struct root { TOID(struct foo) foop; TOID(struct bar) barp; }; int main(int argc, char *argv[]) { if (argc < 2) { printf("usage: %s file-name\n", argv[0]); return 1; } const char *path = argv[1]; PMEMobjpool *pop; if (file_exists(path) != 0) { if ((pop = pmemobj_create(path, POBJ_LAYOUT_NAME(btree), PMEMOBJ_MIN_POOL, 0666)) == NULL) { perror("failed to create pool\n"); return 1; } } else { if ((pop = pmemobj_open(path, POBJ_LAYOUT_NAME(btree))) == NULL) { perror("failed to open pool\n"); return 1; } } struct slab_allocator *foo_producer = slab_new(pop, sizeof(struct foo)); assert(foo_producer != NULL); struct slab_allocator *bar_producer = slab_new(pop, sizeof(struct bar)); assert(bar_producer != NULL); TOID(struct root) root = POBJ_ROOT(pop, struct root); if (TOID_IS_NULL(D_RO(root)->foop)) { TX_BEGIN(pop) { TX_SET(root, foop.oid, slab_tx_alloc(foo_producer)); } TX_END } if (TOID_IS_NULL(D_RO(root)->barp)) { slab_alloc(bar_producer, &D_RW(root)->barp.oid, NULL, NULL); } assert(pmemobj_alloc_usable_size(D_RO(root)->foop.oid) == sizeof(struct foo)); assert(pmemobj_alloc_usable_size(D_RO(root)->barp.oid) == sizeof(struct bar)); slab_delete(foo_producer); slab_delete(bar_producer); pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/slab_allocator/Makefile0000664000000000000000000000354513615011243022166 0ustar rootroot# # Copyright 2017, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/slab_allocator/Makefile -- build the slab_allocator example # TOP := $(dir $(lastword $(MAKEFILE_LIST)))../../../../ include $(TOP)/src/common.inc PROGS = main LIBRARIES = slab_allocator LIBS = -lpmemobj include ../../Makefile.inc libslab_allocator.o: slab_allocator.o main: main.o slab_allocator.o pmdk-1.8/src/examples/libpmemobj/hashmap/0000775000000000000000000000000013615011243017157 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_atomic.vcxproj0000664000000000000000000000547213615011243023561 0ustar rootroot Debug x64 Release x64 {F5E2F6C4-19BA-497A-B754-232E469BE647} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} StaticLibrary true v140 StaticLibrary false v140 true ..\..\..\LongPath.manifest CompileAsCpp 4200;4996 pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_rp.vcxproj0000664000000000000000000000546213615011243022725 0ustar rootroot Debug x64 Release x64 {F5E2F6C4-19BA-497A-B754-232E4666E647} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} StaticLibrary true v140 StaticLibrary false v140 true ..\..\..\LongPath.manifest CompileAsCpp 4200;4996 pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap.h0000664000000000000000000000350413615011243020753 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef HASHMAP_H #define HASHMAP_H /* common API provided by both implementations */ #include #include struct hashmap_args { uint32_t seed; }; enum hashmap_cmd { HASHMAP_CMD_REBUILD, HASHMAP_CMD_DEBUG, }; #endif /* HASHMAP_H */ pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_atomic.h0000664000000000000000000000552313615011243022312 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef HASHMAP_ATOMIC_H #define HASHMAP_ATOMIC_H #include #include #include #include #ifndef HASHMAP_ATOMIC_TYPE_OFFSET #define HASHMAP_ATOMIC_TYPE_OFFSET 1000 #endif struct hashmap_atomic; TOID_DECLARE(struct hashmap_atomic, HASHMAP_ATOMIC_TYPE_OFFSET + 0); int hm_atomic_check(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap); int hm_atomic_create(PMEMobjpool *pop, TOID(struct hashmap_atomic) *map, void *arg); int hm_atomic_init(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap); int hm_atomic_insert(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint64_t key, PMEMoid value); PMEMoid hm_atomic_remove(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint64_t key); PMEMoid hm_atomic_get(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint64_t key); int hm_atomic_lookup(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint64_t key); int hm_atomic_foreach(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg); size_t hm_atomic_count(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap); int hm_atomic_cmd(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, unsigned cmd, uint64_t arg); #endif /* HASHMAP_ATOMIC_H */ pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_rp.h0000664000000000000000000000633713615011243021463 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef HASHMAP_RP_H #define HASHMAP_RP_H #include #include #include #include #ifndef HASHMAP_RP_TYPE_OFFSET #define HASHMAP_RP_TYPE_OFFSET 1008 #endif /* Flags to indicate if insertion is being made during rebuild process */ #define HASHMAP_RP_REBUILD 1 #define HASHMAP_RP_NO_REBUILD 0 /* Initial number of entries for hashamap_rp */ #define INIT_ENTRIES_NUM_RP 16 /* Load factor to indicate resize threshold */ #define HASHMAP_RP_LOAD_FACTOR 0.5f /* Maximum number of swaps allowed during single insertion */ #define HASHMAP_RP_MAX_SWAPS 150 /* Size of an action array used during single insertion */ #define HASHMAP_RP_MAX_ACTIONS (4 * HASHMAP_RP_MAX_SWAPS + 5) struct hashmap_rp; TOID_DECLARE(struct hashmap_rp, HASHMAP_RP_TYPE_OFFSET + 0); int hm_rp_check(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap); int hm_rp_create(PMEMobjpool *pop, TOID(struct hashmap_rp) *map, void *arg); int hm_rp_init(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap); int hm_rp_insert(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, uint64_t key, PMEMoid value); PMEMoid hm_rp_remove(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, uint64_t key); PMEMoid hm_rp_get(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, uint64_t key); int hm_rp_lookup(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, uint64_t key); int hm_rp_foreach(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg); size_t hm_rp_count(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap); int hm_rp_cmd(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, unsigned cmd, uint64_t arg); #endif /* HASHMAP_RP_H */ pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_tx.vcxproj.filters0000664000000000000000000000153713615011243024405 0ustar rootroot Header Files Header Files Header Files {b0b832cc-e298-40a8-aa01-9c935ebf8393} {62513d1d-c5c7-4d25-9ecb-60cf10a23132} Source Files pmdk-1.8/src/examples/libpmemobj/hashmap/README0000664000000000000000000000270413615011243020042 0ustar rootrootThis directory contains an example libraries implemented using libpmemobj. To build examples with debug parameters run "make debug". The *hashmap_tx*, *hashmap_atomic* and *hashmap_rp* libraries are three implementations of hashmap which utilizes transactional, atomic and reserve/publish API of libpmemobj respectively. Libraries may be used through *mapcli* application located in examples/libpmemobj/map directory. Atomic version, while simpler on the surface than transactional one, has 2 significant drawbacks: - libpmemobj's list API implements only double-linked lists, which wastes memory here (we don't need to traverse backward) and what is more important: - it needs to handle recovery process after crash/interruption Transactional version implements its own single-linked lists (it's still not memory efficient, because every allocation has significant overhead, so to mitigate it application would have to keep more values in one node) and can get away without any recovery process - every memory transaction is either done in 0% or 100%. Hashmap_rp version uses libpmemobj action interface for reserve/publish mechanism. While hashmap_tx and hashmap_atomic container implementations are using separate chaining with linked lists for collisions resolving, hashmap_rp provides open addressing with Robin Hood collision resolution. Hashmap_rp built with debug parameter monitors number of swaps performed for single insertion and calls additional asserts. pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_rp.vcxproj.filters0000664000000000000000000000153713615011243024373 0ustar rootroot {58d06973-b76f-41b5-ab03-4e55598436be} {5e7e56b2-b3d3-4d53-bbd1-95e666111077} Source Files Header Files Header Files Header Files pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_rp.c0000664000000000000000000004371613615011243021460 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * Integer hash set implementation with open addressing Robin Hood collision * resolution which uses action.h reserve/publish API. */ #include #include #include #include #include #include #include "hashmap_rp.h" #define TOMBSTONE_MASK (1ULL << 63) #ifdef DEBUG #define HM_ASSERT(cnd) assert(cnd) #else #define HM_ASSERT(cnd) #endif /* layout definition */ TOID_DECLARE(struct entry, HASHMAP_RP_TYPE_OFFSET + 1); struct entry { uint64_t key; PMEMoid value; uint64_t hash; }; struct add_entry { struct entry data; /* position index in hashmap, where data should be inserted/updated */ size_t pos; /* Action array to perform addition in set of actions */ struct pobj_action *actv; /* Action array index counter */ size_t actv_cnt; #ifdef DEBUG /* Swaps counter for current insertion. Enabled in debug mode */ int swaps; #endif }; struct hashmap_rp { /* number of values inserted */ uint64_t count; /* container capacity */ uint64_t capacity; /* resize threshold */ uint64_t resize_threshold; /* entries */ TOID(struct entry) entries; }; int *swaps_array = NULL; #ifdef DEBUG static inline int is_power_of_2(uint64_t v) { return v && !(v & (v - 1)); } #endif /* * entry_is_deleted -- checks 'tombstone' bit if hash is deleted */ static inline int entry_is_deleted(uint64_t hash) { return (hash & TOMBSTONE_MASK) > 0; } /* * entry_is_empty -- checks if entry is empty */ static inline int entry_is_empty(uint64_t hash) { return hash == 0 || entry_is_deleted(hash); } /* * increment_pos -- increment position index, skip 0 */ static uint64_t increment_pos(const struct hashmap_rp *hashmap, uint64_t pos) { HM_ASSERT(is_power_of_2(hashmap->capacity)); pos = (pos + 1) & (hashmap->capacity - 1); return pos == 0 ? 1 : pos; } /* * probe_distance -- returns probe number, an indicator how far from * desired position given hash is stored in hashmap */ static int probe_distance(const struct hashmap_rp *hashmap, uint64_t hash_key, uint64_t slot_index) { uint64_t capacity = hashmap->capacity; HM_ASSERT(is_power_of_2(hashmap->capacity)); return (int)(slot_index + capacity - hash_key) & (capacity - 1); } /* * hash -- hash function based on Austin Appleby MurmurHash3 64-bit finalizer. * Returned value is modified to work with special values for unused and * and deleted hashes. */ static uint64_t hash(const struct hashmap_rp *hashmap, uint64_t key) { key ^= key >> 33; key *= 0xff51afd7ed558ccd; key ^= key >> 33; key *= 0xc4ceb9fe1a85ec53; key ^= key >> 33; HM_ASSERT(is_power_of_2(hashmap->capacity)); key &= hashmap->capacity - 1; /* first, 'tombstone' bit is used to indicate deleted item */ key &= ~TOMBSTONE_MASK; /* * Ensure that we never return 0 as a hash, since we use 0 to * indicate that element has never been used at all. */ return key == 0 ? 1 : key; } /* * hashmap_create -- hashmap initializer */ static void hashmap_create(PMEMobjpool *pop, TOID(struct hashmap_rp) *hashmap_p, uint32_t seed) { struct pobj_action actv[4]; size_t actv_cnt = 0; TOID(struct hashmap_rp) hashmap = POBJ_RESERVE_NEW(pop, struct hashmap_rp, &actv[actv_cnt]); if (TOID_IS_NULL(hashmap)) goto reserve_err; actv_cnt++; D_RW(hashmap)->count = 0; D_RW(hashmap)->capacity = INIT_ENTRIES_NUM_RP; D_RW(hashmap)->resize_threshold = (uint64_t)(INIT_ENTRIES_NUM_RP * HASHMAP_RP_LOAD_FACTOR); size_t sz = sizeof(struct entry) * D_RO(hashmap)->capacity; /* init entries with zero in order to track unused hashes */ D_RW(hashmap)->entries = POBJ_XRESERVE_ALLOC(pop, struct entry, sz, &actv[actv_cnt], POBJ_XALLOC_ZERO); if (TOID_IS_NULL(D_RO(hashmap)->entries)) goto reserve_err; actv_cnt++; pmemobj_persist(pop, D_RW(hashmap), sizeof(struct hashmap_rp)); pmemobj_set_value(pop, &actv[actv_cnt++], &hashmap_p->oid.pool_uuid_lo, hashmap.oid.pool_uuid_lo); pmemobj_set_value(pop, &actv[actv_cnt++], &hashmap_p->oid.off, hashmap.oid.off); pmemobj_publish(pop, actv, actv_cnt); #ifdef DEBUG swaps_array = (int *)calloc(INIT_ENTRIES_NUM_RP, sizeof(int)); if (!swaps_array) abort(); #endif return; reserve_err: fprintf(stderr, "hashmap alloc failed: %s\n", pmemobj_errormsg()); pmemobj_cancel(pop, actv, actv_cnt); abort(); } /* * entry_update -- updates entry in given hashmap with given arguments */ static void entry_update(PMEMobjpool *pop, struct hashmap_rp *hashmap, struct add_entry *args, int rebuild) { HM_ASSERT(HASHMAP_RP_MAX_ACTIONS > args->actv_cnt + 4); struct entry *entry_p = D_RW(hashmap->entries); entry_p += args->pos; if (rebuild == HASHMAP_RP_REBUILD) { entry_p->key = args->data.key; entry_p->value = args->data.value; entry_p->hash = args->data.hash; } else { pmemobj_set_value(pop, args->actv + args->actv_cnt++, &entry_p->key, args->data.key); pmemobj_set_value(pop, args->actv + args->actv_cnt++, &entry_p->value.pool_uuid_lo, args->data.value.pool_uuid_lo); pmemobj_set_value(pop, args->actv + args->actv_cnt++, &entry_p->value.off, args->data.value.off); pmemobj_set_value(pop, args->actv + args->actv_cnt++, &entry_p->hash, args->data.hash); } #ifdef DEBUG assert(sizeof(swaps_array) / sizeof(swaps_array[0]) > args->pos); swaps_array[args->pos] = args->swaps; #endif } /* * entry_add -- increments given hashmap's elements counter and calls * entry_update */ static void entry_add(PMEMobjpool *pop, struct hashmap_rp *hashmap, struct add_entry *args, int rebuild) { HM_ASSERT(HASHMAP_RP_MAX_ACTIONS > args->actv_cnt + 1); if (rebuild == HASHMAP_RP_REBUILD) hashmap->count++; else { pmemobj_set_value(pop, args->actv + args->actv_cnt++, &hashmap->count, hashmap->count + 1); } entry_update(pop, hashmap, args, rebuild); } /* * insert_helper -- inserts specified value into the hashmap * If function was called during rebuild process, no redo logs will be used. * returns: * - 0 if successful, * - 1 if value already existed * - -1 on error */ static int insert_helper(PMEMobjpool *pop, struct hashmap_rp *hashmap, uint64_t key, PMEMoid value, int rebuild) { HM_ASSERT(hashmap->count + 1 < hashmap->resize_threshold); struct pobj_action actv[HASHMAP_RP_MAX_ACTIONS]; struct add_entry args; args.data.key = key; args.data.value = value; args.data.hash = hash(hashmap, key); args.pos = args.data.hash; if (rebuild != HASHMAP_RP_REBUILD) { args.actv = actv; args.actv_cnt = 0; } int dist = 0; struct entry *entry_p = NULL; #ifdef DEBUG int swaps = 0; #endif for (int n = 0; n < HASHMAP_RP_MAX_SWAPS; ++n) { entry_p = D_RW(hashmap->entries); entry_p += args.pos; #ifdef DEBUG args.swaps = swaps; #endif /* Case 1: key already exists, override value */ if (!entry_is_empty(entry_p->hash) && entry_p->key == args.data.key) { entry_update(pop, hashmap, &args, rebuild); if (rebuild != HASHMAP_RP_REBUILD) pmemobj_publish(pop, args.actv, args.actv_cnt); return 1; } /* Case 2: slot is empty from the beginning */ if (entry_p->hash == 0) { entry_add(pop, hashmap, &args, rebuild); if (rebuild != HASHMAP_RP_REBUILD) pmemobj_publish(pop, args.actv, args.actv_cnt); return 0; } /* * Case 3: existing element (or tombstone) has probed less than * current element. Swap them (or put into tombstone slot) and * keep going to find another slot for that element. */ int existing_dist = probe_distance(hashmap, entry_p->hash, args.pos); if (existing_dist < dist) { if (entry_is_deleted(entry_p->hash)) { entry_add(pop, hashmap, &args, rebuild); if (rebuild != HASHMAP_RP_REBUILD) pmemobj_publish(pop, args.actv, args.actv_cnt); return 0; } struct entry temp = *entry_p; entry_update(pop, hashmap, &args, rebuild); args.data = temp; #ifdef DEBUG swaps++; #endif dist = existing_dist; } /* * Case 4: increment slot number and probe counter, keep going * to find free slot */ args.pos = increment_pos(hashmap, args.pos); dist += 1; } fprintf(stderr, "insertion requires too many swaps\n"); if (rebuild != HASHMAP_RP_REBUILD) pmemobj_cancel(pop, args.actv, args.actv_cnt); return -1; } /* * index_lookup -- checks if given key exists in hashmap. * Returns index number if key was found, 0 otherwise. */ static uint64_t index_lookup(const struct hashmap_rp *hashmap, uint64_t key) { const uint64_t hash_lookup = hash(hashmap, key); uint64_t pos = hash_lookup; uint64_t dist = 0; const struct entry *entry_p = NULL; do { entry_p = D_RO(hashmap->entries); entry_p += pos; if (entry_p->hash == hash_lookup && entry_p->key == key) return pos; pos = increment_pos(hashmap, pos); } while (entry_p->hash != 0 && (dist++) <= probe_distance(hashmap, entry_p->hash, pos) - 1); return 0; } /* * entries_cache -- cache entries from second argument in entries from first * argument */ static int entries_cache(PMEMobjpool *pop, struct hashmap_rp *dest, const struct hashmap_rp *src) { const struct entry *e_begin = D_RO(src->entries); const struct entry *e_end = e_begin + src->capacity; for (const struct entry *e = e_begin; e != e_end; ++e) { if (entry_is_empty(e->hash)) continue; if (insert_helper(pop, dest, e->key, e->value, HASHMAP_RP_REBUILD) == -1) return -1; } HM_ASSERT(src->count == dest->count); return 0; } /* * hm_rp_rebuild -- rebuilds the hashmap with a new capacity. * Returns 0 on success, -1 otherwise. */ static int hm_rp_rebuild(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, size_t capacity_new) { /* * We will need 6 actions: * - 1 action to set new capacity * - 1 action to set new resize threshold * - 1 action to alloc memory for new entries * - 1 action to free old entries * - 2 actions to set new oid pointing to new entries */ struct pobj_action actv[6]; size_t actv_cnt = 0; size_t sz_alloc = sizeof(struct entry) * capacity_new; uint64_t resize_threshold_new = (uint64_t)(capacity_new * HASHMAP_RP_LOAD_FACTOR); pmemobj_set_value(pop, &actv[actv_cnt++], &D_RW(hashmap)->capacity, capacity_new); pmemobj_set_value(pop, &actv[actv_cnt++], &D_RW(hashmap)->resize_threshold, resize_threshold_new); struct hashmap_rp hashmap_rebuild; hashmap_rebuild.count = 0; hashmap_rebuild.capacity = capacity_new; hashmap_rebuild.resize_threshold = resize_threshold_new; hashmap_rebuild.entries = POBJ_XRESERVE_ALLOC(pop, struct entry, sz_alloc, &actv[actv_cnt], POBJ_XALLOC_ZERO); if (TOID_IS_NULL(hashmap_rebuild.entries)) { fprintf(stderr, "hashmap rebuild failed: %s\n", pmemobj_errormsg()); goto rebuild_err; } actv_cnt++; #ifdef DEBUG free(swaps_array); swaps_array = (int *)calloc(capacity_new, sizeof(int)); if (!swaps_array) goto rebuild_err; #endif if (entries_cache(pop, &hashmap_rebuild, D_RW(hashmap)) == -1) goto rebuild_err; pmemobj_persist(pop, D_RW(hashmap_rebuild.entries), sz_alloc); pmemobj_defer_free(pop, D_RW(hashmap)->entries.oid, &actv[actv_cnt++]); pmemobj_set_value(pop, &actv[actv_cnt++], &D_RW(hashmap)->entries.oid.pool_uuid_lo, hashmap_rebuild.entries.oid.pool_uuid_lo); pmemobj_set_value(pop, &actv[actv_cnt++], &D_RW(hashmap)->entries.oid.off, hashmap_rebuild.entries.oid.off); HM_ASSERT(sizeof(actv) / sizeof(actv[0]) >= actv_cnt); pmemobj_publish(pop, actv, actv_cnt); return 0; rebuild_err: pmemobj_cancel(pop, actv, actv_cnt); #ifdef DEBUG free(swaps_array); #endif return -1; } /* * hm_rp_create -- initializes hashmap state, called after pmemobj_create */ int hm_rp_create(PMEMobjpool *pop, TOID(struct hashmap_rp) *map, void *arg) { struct hashmap_args *args = (struct hashmap_args *)arg; uint32_t seed = args ? args->seed : 0; hashmap_create(pop, map, seed); return 0; } /* * hm_rp_check -- checks if specified persistent object is an instance of * hashmap */ int hm_rp_check(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap) { return TOID_IS_NULL(hashmap) || !TOID_VALID(hashmap); } /* * hm_rp_init -- recovers hashmap state, called after pmemobj_open. * Since hashmap_rp is performing rebuild/insertion completely or not at all, * function is dummy and simply returns 0. */ int hm_rp_init(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap) { return 0; } /* * hm_rp_insert -- rebuilds hashmap if necessary and wraps insert_helper. * returns: * - 0 if successful, * - 1 if value already existed * - -1 if something bad happened */ int hm_rp_insert(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, uint64_t key, PMEMoid value) { if (D_RO(hashmap)->count + 1 >= D_RO(hashmap)->resize_threshold) { uint64_t capacity_new = D_RO(hashmap)->capacity * 2; if (hm_rp_rebuild(pop, hashmap, capacity_new) != 0) return -1; } return insert_helper(pop, D_RW(hashmap), key, value, HASHMAP_RP_NO_REBUILD); } /* * hm_rp_remove -- removes specified key from the hashmap, * returns: * - key's value if successful, * - OID_NULL if value didn't exist or if something bad happened */ PMEMoid hm_rp_remove(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, uint64_t key) { const uint64_t pos = index_lookup(D_RO(hashmap), key); if (pos == 0) return OID_NULL; struct entry *entry_p = D_RW(D_RW(hashmap)->entries); entry_p += pos; PMEMoid ret = entry_p->value; size_t actvcnt = 0; struct pobj_action actv[5]; pmemobj_set_value(pop, &actv[actvcnt++], &entry_p->hash, entry_p->hash | TOMBSTONE_MASK); pmemobj_set_value(pop, &actv[actvcnt++], &entry_p->value.pool_uuid_lo, 0); pmemobj_set_value(pop, &actv[actvcnt++], &entry_p->value.off, 0); pmemobj_set_value(pop, &actv[actvcnt++], &entry_p->key, 0); pmemobj_set_value(pop, &actv[actvcnt++], &D_RW(hashmap)->count, D_RW(hashmap)->count - 1); HM_ASSERT(sizeof(actv) / sizeof(actv[0]) >= actvcnt); pmemobj_publish(pop, actv, actvcnt); uint64_t reduced_threshold = (uint64_t) (((uint64_t)(D_RO(hashmap)->capacity / 2)) * HASHMAP_RP_LOAD_FACTOR); if (reduced_threshold >= INIT_ENTRIES_NUM_RP && D_RW(hashmap)->count < reduced_threshold && hm_rp_rebuild(pop, hashmap, D_RO(hashmap)->capacity / 2)) return OID_NULL; return ret; } /* * hm_rp_get -- checks whether specified key is in the hashmap. * Returns associated value if key exists, OID_NULL otherwise. */ PMEMoid hm_rp_get(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, uint64_t key) { struct entry *entry_p = (struct entry *)pmemobj_direct(D_RW(hashmap)->entries.oid); uint64_t pos = index_lookup(D_RO(hashmap), key); return pos == 0 ? OID_NULL : (entry_p + pos)->value; } /* * hm_rp_lookup -- checks whether specified key is in the hashmap. * Returns 1 if key was found, 0 otherwise. */ int hm_rp_lookup(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, uint64_t key) { return index_lookup(D_RO(hashmap), key) != 0; } /* * hm_rp_foreach -- calls cb for all values from the hashmap */ int hm_rp_foreach(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { struct entry *entry_p = (struct entry *)pmemobj_direct(D_RO(hashmap)->entries.oid); int ret = 0; for (size_t i = 0; i < D_RO(hashmap)->capacity; ++i, ++entry_p) { uint64_t hash = entry_p->hash; if (entry_is_empty(hash)) continue; ret = cb(entry_p->key, entry_p->value, arg); if (ret) return ret; } return 0; } /* * hm_rp_debug -- prints complete hashmap state */ static void hm_rp_debug(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, FILE *out) { #ifdef DEBUG fprintf(out, "debug: true, "); #endif fprintf(out, "capacity: %" PRIu64 ", count: %" PRIu64 "\n", D_RO(hashmap)->capacity, D_RO(hashmap)->count); struct entry *entry_p = D_RW((D_RW(hashmap)->entries)); for (size_t i = 0; i < D_RO(hashmap)->capacity; ++i, ++entry_p) { uint64_t hash = entry_p->hash; if (entry_is_empty(hash)) continue; uint64_t key = entry_p->key; #ifdef DEBUG fprintf(out, "%zu: %" PRIu64 " hash: %" PRIu64 " dist:%" PRIu32 " swaps:%u\n", i, key, hash, probe_distance(D_RO(hashmap), hash, i), swaps_array[i]); #else fprintf(out, "%zu: %" PRIu64 " dist:%u \n", i, key, probe_distance(D_RO(hashmap), hash, i)); #endif } } /* * hm_rp_count -- returns number of elements */ size_t hm_rp_count(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap) { return D_RO(hashmap)->count; } /* * hm_rp_cmd -- execute cmd for hashmap */ int hm_rp_cmd(PMEMobjpool *pop, TOID(struct hashmap_rp) hashmap, unsigned cmd, uint64_t arg) { switch (cmd) { case HASHMAP_CMD_REBUILD: hm_rp_rebuild(pop, hashmap, D_RO(hashmap)->capacity); return 0; case HASHMAP_CMD_DEBUG: if (!arg) return -EINVAL; hm_rp_debug(pop, hashmap, (FILE *)arg); return 0; default: return -EINVAL; } } pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_tx.h0000664000000000000000000000534113615011243021467 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef HASHMAP_TX_H #define HASHMAP_TX_H #include #include #include #include #ifndef HASHMAP_TX_TYPE_OFFSET #define HASHMAP_TX_TYPE_OFFSET 1004 #endif struct hashmap_tx; TOID_DECLARE(struct hashmap_tx, HASHMAP_TX_TYPE_OFFSET + 0); int hm_tx_check(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap); int hm_tx_create(PMEMobjpool *pop, TOID(struct hashmap_tx) *map, void *arg); int hm_tx_init(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap); int hm_tx_insert(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint64_t key, PMEMoid value); PMEMoid hm_tx_remove(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint64_t key); PMEMoid hm_tx_get(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint64_t key); int hm_tx_lookup(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint64_t key); int hm_tx_foreach(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg); size_t hm_tx_count(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap); int hm_tx_cmd(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, unsigned cmd, uint64_t arg); #endif /* HASHMAP_TX_H */ pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_internal.h0000664000000000000000000000376413615011243022657 0ustar rootroot/* * Copyright 2015-2017, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ #ifndef HASHSET_INTERNAL_H #define HASHSET_INTERNAL_H /* large prime number used as a hashing function coefficient */ #define HASH_FUNC_COEFF_P 32212254719ULL /* initial number of buckets */ #define INIT_BUCKETS_NUM 10 /* number of values in a bucket which trigger hashtable rebuild check */ #define MIN_HASHSET_THRESHOLD 5 /* number of values in a bucket which force hashtable rebuild */ #define MAX_HASHSET_THRESHOLD 10 #endif pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_atomic.c0000664000000000000000000003400413615011243022301 0ustar rootroot/* * Copyright 2015-2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* integer hash set implementation which uses only atomic APIs */ #include #include #include #include #include #include #include "hashmap_atomic.h" #include "hashmap_internal.h" /* layout definition */ TOID_DECLARE(struct buckets, HASHMAP_ATOMIC_TYPE_OFFSET + 1); TOID_DECLARE(struct entry, HASHMAP_ATOMIC_TYPE_OFFSET + 2); struct entry { uint64_t key; PMEMoid value; /* list pointer */ POBJ_LIST_ENTRY(struct entry) list; }; struct entry_args { uint64_t key; PMEMoid value; }; POBJ_LIST_HEAD(entries_head, struct entry); struct buckets { /* number of buckets */ size_t nbuckets; /* array of lists */ struct entries_head bucket[]; }; struct hashmap_atomic { /* random number generator seed */ uint32_t seed; /* hash function coefficients */ uint32_t hash_fun_a; uint32_t hash_fun_b; uint64_t hash_fun_p; /* number of values inserted */ uint64_t count; /* whether "count" should be updated */ uint32_t count_dirty; /* buckets */ TOID(struct buckets) buckets; /* buckets, used during rehashing, null otherwise */ TOID(struct buckets) buckets_tmp; }; /* * create_entry -- entry initializer */ static int create_entry(PMEMobjpool *pop, void *ptr, void *arg) { struct entry *e = (struct entry *)ptr; struct entry_args *args = (struct entry_args *)arg; e->key = args->key; e->value = args->value; memset(&e->list, 0, sizeof(e->list)); pmemobj_persist(pop, e, sizeof(*e)); return 0; } /* * create_buckets -- buckets initializer */ static int create_buckets(PMEMobjpool *pop, void *ptr, void *arg) { struct buckets *b = (struct buckets *)ptr; b->nbuckets = *((size_t *)arg); pmemobj_memset_persist(pop, &b->bucket, 0, b->nbuckets * sizeof(b->bucket[0])); pmemobj_persist(pop, &b->nbuckets, sizeof(b->nbuckets)); return 0; } /* * create_hashmap -- hashmap initializer */ static void create_hashmap(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint32_t seed) { D_RW(hashmap)->seed = seed; do { D_RW(hashmap)->hash_fun_a = (uint32_t)rand(); } while (D_RW(hashmap)->hash_fun_a == 0); D_RW(hashmap)->hash_fun_b = (uint32_t)rand(); D_RW(hashmap)->hash_fun_p = HASH_FUNC_COEFF_P; size_t len = INIT_BUCKETS_NUM; size_t sz = sizeof(struct buckets) + len * sizeof(struct entries_head); if (POBJ_ALLOC(pop, &D_RW(hashmap)->buckets, struct buckets, sz, create_buckets, &len)) { fprintf(stderr, "root alloc failed: %s\n", pmemobj_errormsg()); abort(); } pmemobj_persist(pop, D_RW(hashmap), sizeof(*D_RW(hashmap))); } /* * hash -- the simplest hashing function, * see https://en.wikipedia.org/wiki/Universal_hashing#Hashing_integers */ static uint64_t hash(const TOID(struct hashmap_atomic) *hashmap, const TOID(struct buckets) *buckets, uint64_t value) { uint32_t a = D_RO(*hashmap)->hash_fun_a; uint32_t b = D_RO(*hashmap)->hash_fun_b; uint64_t p = D_RO(*hashmap)->hash_fun_p; size_t len = D_RO(*buckets)->nbuckets; return ((a * value + b) % p) % len; } /* * hm_atomic_rebuild_finish -- finishes rebuild, assumes buckets_tmp is not null */ static void hm_atomic_rebuild_finish(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap) { TOID(struct buckets) cur = D_RO(hashmap)->buckets; TOID(struct buckets) tmp = D_RO(hashmap)->buckets_tmp; for (size_t i = 0; i < D_RO(cur)->nbuckets; ++i) { while (!POBJ_LIST_EMPTY(&D_RO(cur)->bucket[i])) { TOID(struct entry) en = POBJ_LIST_FIRST(&D_RO(cur)->bucket[i]); uint64_t h = hash(&hashmap, &tmp, D_RO(en)->key); if (POBJ_LIST_MOVE_ELEMENT_HEAD(pop, &D_RW(cur)->bucket[i], &D_RW(tmp)->bucket[h], en, list, list)) { fprintf(stderr, "move failed: %s\n", pmemobj_errormsg()); abort(); } } } POBJ_FREE(&D_RO(hashmap)->buckets); D_RW(hashmap)->buckets = D_RO(hashmap)->buckets_tmp; pmemobj_persist(pop, &D_RW(hashmap)->buckets, sizeof(D_RW(hashmap)->buckets)); /* * We have to set offset manually instead of substituting OID_NULL, * because we won't be able to recover easily if crash happens after * pool_uuid_lo, but before offset is set. Another reason why everyone * should use transaction API. * See recovery process in hm_init and TOID_IS_NULL macro definition. */ D_RW(hashmap)->buckets_tmp.oid.off = 0; pmemobj_persist(pop, &D_RW(hashmap)->buckets_tmp, sizeof(D_RW(hashmap)->buckets_tmp)); } /* * hm_atomic_rebuild -- rebuilds the hashmap with a new number of buckets */ static void hm_atomic_rebuild(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, size_t new_len) { if (new_len == 0) new_len = D_RO(D_RO(hashmap)->buckets)->nbuckets; size_t sz = sizeof(struct buckets) + new_len * sizeof(struct entries_head); POBJ_ALLOC(pop, &D_RW(hashmap)->buckets_tmp, struct buckets, sz, create_buckets, &new_len); if (TOID_IS_NULL(D_RO(hashmap)->buckets_tmp)) { fprintf(stderr, "failed to allocate temporary space of size: %zu" ", %s\n", new_len, pmemobj_errormsg()); return; } hm_atomic_rebuild_finish(pop, hashmap); } /* * hm_atomic_insert -- inserts specified value into the hashmap, * returns: * - 0 if successful, * - 1 if value already existed, * - -1 if something bad happened */ int hm_atomic_insert(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint64_t key, PMEMoid value) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; uint64_t h = hash(&hashmap, &buckets, key); int num = 0; POBJ_LIST_FOREACH(var, &D_RO(buckets)->bucket[h], list) { if (D_RO(var)->key == key) return 1; num++; } D_RW(hashmap)->count_dirty = 1; pmemobj_persist(pop, &D_RW(hashmap)->count_dirty, sizeof(D_RW(hashmap)->count_dirty)); struct entry_args args; args.key = key; args.value = value; PMEMoid oid = POBJ_LIST_INSERT_NEW_HEAD(pop, &D_RW(buckets)->bucket[h], list, sizeof(struct entry), create_entry, &args); if (OID_IS_NULL(oid)) { fprintf(stderr, "failed to allocate entry: %s\n", pmemobj_errormsg()); return -1; } D_RW(hashmap)->count++; pmemobj_persist(pop, &D_RW(hashmap)->count, sizeof(D_RW(hashmap)->count)); D_RW(hashmap)->count_dirty = 0; pmemobj_persist(pop, &D_RW(hashmap)->count_dirty, sizeof(D_RW(hashmap)->count_dirty)); num++; if (num > MAX_HASHSET_THRESHOLD || (num > MIN_HASHSET_THRESHOLD && D_RO(hashmap)->count > 2 * D_RO(buckets)->nbuckets)) hm_atomic_rebuild(pop, hashmap, D_RW(buckets)->nbuckets * 2); return 0; } /* * hm_atomic_remove -- removes specified value from the hashmap, * returns: * - key's value if successful, * - OID_NULL if value didn't exist or if something bad happened */ PMEMoid hm_atomic_remove(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint64_t key) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; uint64_t h = hash(&hashmap, &buckets, key); POBJ_LIST_FOREACH(var, &D_RW(buckets)->bucket[h], list) { if (D_RO(var)->key == key) break; } if (TOID_IS_NULL(var)) return OID_NULL; D_RW(hashmap)->count_dirty = 1; pmemobj_persist(pop, &D_RW(hashmap)->count_dirty, sizeof(D_RW(hashmap)->count_dirty)); if (POBJ_LIST_REMOVE_FREE(pop, &D_RW(buckets)->bucket[h], var, list)) { fprintf(stderr, "list remove failed: %s\n", pmemobj_errormsg()); return OID_NULL; } D_RW(hashmap)->count--; pmemobj_persist(pop, &D_RW(hashmap)->count, sizeof(D_RW(hashmap)->count)); D_RW(hashmap)->count_dirty = 0; pmemobj_persist(pop, &D_RW(hashmap)->count_dirty, sizeof(D_RW(hashmap)->count_dirty)); if (D_RO(hashmap)->count < D_RO(buckets)->nbuckets) hm_atomic_rebuild(pop, hashmap, D_RO(buckets)->nbuckets / 2); return D_RO(var)->value; } /* * hm_atomic_foreach -- prints all values from the hashmap */ int hm_atomic_foreach(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; int ret = 0; for (size_t i = 0; i < D_RO(buckets)->nbuckets; ++i) POBJ_LIST_FOREACH(var, &D_RO(buckets)->bucket[i], list) { ret = cb(D_RO(var)->key, D_RO(var)->value, arg); if (ret) return ret; } return 0; } /* * hm_atomic_debug -- prints complete hashmap state */ static void hm_atomic_debug(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, FILE *out) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; fprintf(out, "a: %u b: %u p: %" PRIu64 "\n", D_RO(hashmap)->hash_fun_a, D_RO(hashmap)->hash_fun_b, D_RO(hashmap)->hash_fun_p); fprintf(out, "count: %" PRIu64 ", buckets: %zu\n", D_RO(hashmap)->count, D_RO(buckets)->nbuckets); for (size_t i = 0; i < D_RO(buckets)->nbuckets; ++i) { if (POBJ_LIST_EMPTY(&D_RO(buckets)->bucket[i])) continue; int num = 0; fprintf(out, "%zu: ", i); POBJ_LIST_FOREACH(var, &D_RO(buckets)->bucket[i], list) { fprintf(out, "%" PRIu64 " ", D_RO(var)->key); num++; } fprintf(out, "(%d)\n", num); } } /* * hm_atomic_get -- checks whether specified value is in the hashmap */ PMEMoid hm_atomic_get(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint64_t key) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; uint64_t h = hash(&hashmap, &buckets, key); POBJ_LIST_FOREACH(var, &D_RO(buckets)->bucket[h], list) if (D_RO(var)->key == key) return D_RO(var)->value; return OID_NULL; } /* * hm_atomic_lookup -- checks whether specified value is in the hashmap */ int hm_atomic_lookup(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, uint64_t key) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; uint64_t h = hash(&hashmap, &buckets, key); POBJ_LIST_FOREACH(var, &D_RO(buckets)->bucket[h], list) if (D_RO(var)->key == key) return 1; return 0; } /* * hm_atomic_create -- initializes hashmap state, called after pmemobj_create */ int hm_atomic_create(PMEMobjpool *pop, TOID(struct hashmap_atomic) *map, void *arg) { struct hashmap_args *args = (struct hashmap_args *)arg; uint32_t seed = args ? args->seed : 0; POBJ_ZNEW(pop, map, struct hashmap_atomic); create_hashmap(pop, *map, seed); return 0; } /* * hm_atomic_init -- recovers hashmap state, called after pmemobj_open */ int hm_atomic_init(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap) { srand(D_RO(hashmap)->seed); /* handle rebuild interruption */ if (!TOID_IS_NULL(D_RO(hashmap)->buckets_tmp)) { printf("rebuild, previous attempt crashed\n"); if (TOID_EQUALS(D_RO(hashmap)->buckets, D_RO(hashmap)->buckets_tmp)) { /* see comment in hm_rebuild_finish */ D_RW(hashmap)->buckets_tmp.oid.off = 0; pmemobj_persist(pop, &D_RW(hashmap)->buckets_tmp, sizeof(D_RW(hashmap)->buckets_tmp)); } else if (TOID_IS_NULL(D_RW(hashmap)->buckets)) { D_RW(hashmap)->buckets = D_RW(hashmap)->buckets_tmp; pmemobj_persist(pop, &D_RW(hashmap)->buckets, sizeof(D_RW(hashmap)->buckets)); /* see comment in hm_rebuild_finish */ D_RW(hashmap)->buckets_tmp.oid.off = 0; pmemobj_persist(pop, &D_RW(hashmap)->buckets_tmp, sizeof(D_RW(hashmap)->buckets_tmp)); } else { hm_atomic_rebuild_finish(pop, hashmap); } } /* handle insert or remove interruption */ if (D_RO(hashmap)->count_dirty) { printf("count dirty, recalculating\n"); TOID(struct entry) var; TOID(struct buckets) buckets = D_RO(hashmap)->buckets; uint64_t cnt = 0; for (size_t i = 0; i < D_RO(buckets)->nbuckets; ++i) POBJ_LIST_FOREACH(var, &D_RO(buckets)->bucket[i], list) cnt++; printf("old count: %" PRIu64 ", new count: %" PRIu64 "\n", D_RO(hashmap)->count, cnt); D_RW(hashmap)->count = cnt; pmemobj_persist(pop, &D_RW(hashmap)->count, sizeof(D_RW(hashmap)->count)); D_RW(hashmap)->count_dirty = 0; pmemobj_persist(pop, &D_RW(hashmap)->count_dirty, sizeof(D_RW(hashmap)->count_dirty)); } return 0; } /* * hm_atomic_check -- checks if specified persistent object is an * instance of hashmap */ int hm_atomic_check(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap) { return TOID_IS_NULL(hashmap) || !TOID_VALID(hashmap); } /* * hm_atomic_count -- returns number of elements */ size_t hm_atomic_count(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap) { return D_RO(hashmap)->count; } /* * hm_atomic_cmd -- execute cmd for hashmap */ int hm_atomic_cmd(PMEMobjpool *pop, TOID(struct hashmap_atomic) hashmap, unsigned cmd, uint64_t arg) { switch (cmd) { case HASHMAP_CMD_REBUILD: hm_atomic_rebuild(pop, hashmap, arg); return 0; case HASHMAP_CMD_DEBUG: if (!arg) return -EINVAL; hm_atomic_debug(pop, hashmap, (FILE *)arg); return 0; default: return -EINVAL; } } pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_atomic.vcxproj.filters0000664000000000000000000000154713615011243025227 0ustar rootroot {58d06973-b76f-41b5-ab03-4e50a98436be} {5e7e56b2-b3d3-4d53-bbd1-95e962111077} Source Files Header Files Header Files Header Files pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_tx.vcxproj0000664000000000000000000000546213615011243022737 0ustar rootroot Debug x64 Release x64 {D93A2683-6D99-4F18-B378-91195D23E007} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} StaticLibrary true v140 StaticLibrary false v140 true ..\..\..\LongPath.manifest CompileAsCpp 4200;4996 pmdk-1.8/src/examples/libpmemobj/hashmap/hashmap_tx.c0000664000000000000000000002570713615011243021472 0ustar rootroot/* * Copyright 2015-2019, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* integer hash set implementation which uses only transaction APIs */ #include #include #include #include #include #include "hashmap_tx.h" #include "hashmap_internal.h" /* layout definition */ TOID_DECLARE(struct buckets, HASHMAP_TX_TYPE_OFFSET + 1); TOID_DECLARE(struct entry, HASHMAP_TX_TYPE_OFFSET + 2); struct entry { uint64_t key; PMEMoid value; /* next entry list pointer */ TOID(struct entry) next; }; struct buckets { /* number of buckets */ size_t nbuckets; /* array of lists */ TOID(struct entry) bucket[]; }; struct hashmap_tx { /* random number generator seed */ uint32_t seed; /* hash function coefficients */ uint32_t hash_fun_a; uint32_t hash_fun_b; uint64_t hash_fun_p; /* number of values inserted */ uint64_t count; /* buckets */ TOID(struct buckets) buckets; }; /* * create_hashmap -- hashmap initializer */ static void create_hashmap(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint32_t seed) { size_t len = INIT_BUCKETS_NUM; size_t sz = sizeof(struct buckets) + len * sizeof(TOID(struct entry)); TX_BEGIN(pop) { TX_ADD(hashmap); D_RW(hashmap)->seed = seed; do { D_RW(hashmap)->hash_fun_a = (uint32_t)rand(); } while (D_RW(hashmap)->hash_fun_a == 0); D_RW(hashmap)->hash_fun_b = (uint32_t)rand(); D_RW(hashmap)->hash_fun_p = HASH_FUNC_COEFF_P; D_RW(hashmap)->buckets = TX_ZALLOC(struct buckets, sz); D_RW(D_RW(hashmap)->buckets)->nbuckets = len; } TX_ONABORT { fprintf(stderr, "%s: transaction aborted: %s\n", __func__, pmemobj_errormsg()); abort(); } TX_END } /* * hash -- the simplest hashing function, * see https://en.wikipedia.org/wiki/Universal_hashing#Hashing_integers */ static uint64_t hash(const TOID(struct hashmap_tx) *hashmap, const TOID(struct buckets) *buckets, uint64_t value) { uint32_t a = D_RO(*hashmap)->hash_fun_a; uint32_t b = D_RO(*hashmap)->hash_fun_b; uint64_t p = D_RO(*hashmap)->hash_fun_p; size_t len = D_RO(*buckets)->nbuckets; return ((a * value + b) % p) % len; } /* * hm_tx_rebuild -- rebuilds the hashmap with a new number of buckets */ static void hm_tx_rebuild(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, size_t new_len) { TOID(struct buckets) buckets_old = D_RO(hashmap)->buckets; if (new_len == 0) new_len = D_RO(buckets_old)->nbuckets; size_t sz_old = sizeof(struct buckets) + D_RO(buckets_old)->nbuckets * sizeof(TOID(struct entry)); size_t sz_new = sizeof(struct buckets) + new_len * sizeof(TOID(struct entry)); TX_BEGIN(pop) { TX_ADD_FIELD(hashmap, buckets); TOID(struct buckets) buckets_new = TX_ZALLOC(struct buckets, sz_new); D_RW(buckets_new)->nbuckets = new_len; pmemobj_tx_add_range(buckets_old.oid, 0, sz_old); for (size_t i = 0; i < D_RO(buckets_old)->nbuckets; ++i) { while (!TOID_IS_NULL(D_RO(buckets_old)->bucket[i])) { TOID(struct entry) en = D_RO(buckets_old)->bucket[i]; uint64_t h = hash(&hashmap, &buckets_new, D_RO(en)->key); D_RW(buckets_old)->bucket[i] = D_RO(en)->next; TX_ADD_FIELD(en, next); D_RW(en)->next = D_RO(buckets_new)->bucket[h]; D_RW(buckets_new)->bucket[h] = en; } } D_RW(hashmap)->buckets = buckets_new; TX_FREE(buckets_old); } TX_ONABORT { fprintf(stderr, "%s: transaction aborted: %s\n", __func__, pmemobj_errormsg()); /* * We don't need to do anything here, because everything is * consistent. The only thing affected is performance. */ } TX_END } /* * hm_tx_insert -- inserts specified value into the hashmap, * returns: * - 0 if successful, * - 1 if value already existed, * - -1 if something bad happened */ int hm_tx_insert(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint64_t key, PMEMoid value) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; uint64_t h = hash(&hashmap, &buckets, key); int num = 0; for (var = D_RO(buckets)->bucket[h]; !TOID_IS_NULL(var); var = D_RO(var)->next) { if (D_RO(var)->key == key) return 1; num++; } int ret = 0; TX_BEGIN(pop) { TX_ADD_FIELD(D_RO(hashmap)->buckets, bucket[h]); TX_ADD_FIELD(hashmap, count); TOID(struct entry) e = TX_NEW(struct entry); D_RW(e)->key = key; D_RW(e)->value = value; D_RW(e)->next = D_RO(buckets)->bucket[h]; D_RW(buckets)->bucket[h] = e; D_RW(hashmap)->count++; num++; } TX_ONABORT { fprintf(stderr, "transaction aborted: %s\n", pmemobj_errormsg()); ret = -1; } TX_END if (ret) return ret; if (num > MAX_HASHSET_THRESHOLD || (num > MIN_HASHSET_THRESHOLD && D_RO(hashmap)->count > 2 * D_RO(buckets)->nbuckets)) hm_tx_rebuild(pop, hashmap, D_RO(buckets)->nbuckets * 2); return 0; } /* * hm_tx_remove -- removes specified value from the hashmap, * returns: * - key's value if successful, * - OID_NULL if value didn't exist or if something bad happened */ PMEMoid hm_tx_remove(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint64_t key) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var, prev = TOID_NULL(struct entry); uint64_t h = hash(&hashmap, &buckets, key); for (var = D_RO(buckets)->bucket[h]; !TOID_IS_NULL(var); prev = var, var = D_RO(var)->next) { if (D_RO(var)->key == key) break; } if (TOID_IS_NULL(var)) return OID_NULL; int ret = 0; PMEMoid retoid = D_RO(var)->value; TX_BEGIN(pop) { if (TOID_IS_NULL(prev)) TX_ADD_FIELD(D_RO(hashmap)->buckets, bucket[h]); else TX_ADD_FIELD(prev, next); TX_ADD_FIELD(hashmap, count); if (TOID_IS_NULL(prev)) D_RW(buckets)->bucket[h] = D_RO(var)->next; else D_RW(prev)->next = D_RO(var)->next; D_RW(hashmap)->count--; TX_FREE(var); } TX_ONABORT { fprintf(stderr, "transaction aborted: %s\n", pmemobj_errormsg()); ret = -1; } TX_END if (ret) return OID_NULL; if (D_RO(hashmap)->count < D_RO(buckets)->nbuckets) hm_tx_rebuild(pop, hashmap, D_RO(buckets)->nbuckets / 2); return retoid; } /* * hm_tx_foreach -- prints all values from the hashmap */ int hm_tx_foreach(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, int (*cb)(uint64_t key, PMEMoid value, void *arg), void *arg) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; int ret = 0; for (size_t i = 0; i < D_RO(buckets)->nbuckets; ++i) { if (TOID_IS_NULL(D_RO(buckets)->bucket[i])) continue; for (var = D_RO(buckets)->bucket[i]; !TOID_IS_NULL(var); var = D_RO(var)->next) { ret = cb(D_RO(var)->key, D_RO(var)->value, arg); if (ret) break; } } return ret; } /* * hm_tx_debug -- prints complete hashmap state */ static void hm_tx_debug(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, FILE *out) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; fprintf(out, "a: %u b: %u p: %" PRIu64 "\n", D_RO(hashmap)->hash_fun_a, D_RO(hashmap)->hash_fun_b, D_RO(hashmap)->hash_fun_p); fprintf(out, "count: %" PRIu64 ", buckets: %zu\n", D_RO(hashmap)->count, D_RO(buckets)->nbuckets); for (size_t i = 0; i < D_RO(buckets)->nbuckets; ++i) { if (TOID_IS_NULL(D_RO(buckets)->bucket[i])) continue; int num = 0; fprintf(out, "%zu: ", i); for (var = D_RO(buckets)->bucket[i]; !TOID_IS_NULL(var); var = D_RO(var)->next) { fprintf(out, "%" PRIu64 " ", D_RO(var)->key); num++; } fprintf(out, "(%d)\n", num); } } /* * hm_tx_get -- checks whether specified value is in the hashmap */ PMEMoid hm_tx_get(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint64_t key) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; uint64_t h = hash(&hashmap, &buckets, key); for (var = D_RO(buckets)->bucket[h]; !TOID_IS_NULL(var); var = D_RO(var)->next) if (D_RO(var)->key == key) return D_RO(var)->value; return OID_NULL; } /* * hm_tx_lookup -- checks whether specified value exists */ int hm_tx_lookup(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, uint64_t key) { TOID(struct buckets) buckets = D_RO(hashmap)->buckets; TOID(struct entry) var; uint64_t h = hash(&hashmap, &buckets, key); for (var = D_RO(buckets)->bucket[h]; !TOID_IS_NULL(var); var = D_RO(var)->next) if (D_RO(var)->key == key) return 1; return 0; } /* * hm_tx_count -- returns number of elements */ size_t hm_tx_count(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap) { return D_RO(hashmap)->count; } /* * hm_tx_init -- recovers hashmap state, called after pmemobj_open */ int hm_tx_init(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap) { srand(D_RO(hashmap)->seed); return 0; } /* * hm_tx_create -- allocates new hashmap */ int hm_tx_create(PMEMobjpool *pop, TOID(struct hashmap_tx) *map, void *arg) { struct hashmap_args *args = (struct hashmap_args *)arg; int ret = 0; TX_BEGIN(pop) { TX_ADD_DIRECT(map); *map = TX_ZNEW(struct hashmap_tx); uint32_t seed = args ? args->seed : 0; create_hashmap(pop, *map, seed); } TX_ONABORT { ret = -1; } TX_END return ret; } /* * hm_tx_check -- checks if specified persistent object is an * instance of hashmap */ int hm_tx_check(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap) { return TOID_IS_NULL(hashmap) || !TOID_VALID(hashmap); } /* * hm_tx_cmd -- execute cmd for hashmap */ int hm_tx_cmd(PMEMobjpool *pop, TOID(struct hashmap_tx) hashmap, unsigned cmd, uint64_t arg) { switch (cmd) { case HASHMAP_CMD_REBUILD: hm_tx_rebuild(pop, hashmap, arg); return 0; case HASHMAP_CMD_DEBUG: if (!arg) return -EINVAL; hm_tx_debug(pop, hashmap, (FILE *)arg); return 0; default: return -EINVAL; } } pmdk-1.8/src/examples/libpmemobj/hashmap/Makefile0000664000000000000000000000334413615011243020623 0ustar rootroot# # Copyright 2015-2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # LIBRARIES = hashmap_atomic hashmap_tx hashmap_rp LIBS = -lpmemobj include ../../Makefile.inc libhashmap_atomic.o: hashmap_atomic.o libhashmap_tx.o: hashmap_tx.o libhashmap_rp.o: hashmap_rp.o pmdk-1.8/src/examples/libpmemobj/btree.vcxproj0000664000000000000000000000561713615011243020265 0ustar rootroot Debug x64 Release x64 {0FB8F0FD-276C-413B-97A8-67ABE0C9043B} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\LongPath.manifest 4996;4200 CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/pi.vcxproj0000664000000000000000000000551113615011243017565 0ustar rootroot Debug x64 Release x64 {11D76FBC-DFAA-4B31-9DB0-206E171E3F94} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\LongPath.manifest CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/queue/0000775000000000000000000000000013615011243016662 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/queue/queue.c0000664000000000000000000001640013615011243020153 0ustar rootroot/* * Copyright 2018, Intel Corporation * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: * * * Redistributions of source code must retain the above copyright * notice, this list of conditions and the following disclaimer. * * * Redistributions in binary form must reproduce the above copyright * notice, this list of conditions and the following disclaimer in * the documentation and/or other materials provided with the * distribution. * * * Neither the name of the copyright holder nor the names of its * contributors may be used to endorse or promote products derived * from this software without specific prior written permission. * * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. */ /* * queue.c -- array based queue example */ #include #include #include #include #include POBJ_LAYOUT_BEGIN(queue); POBJ_LAYOUT_ROOT(queue, struct root); POBJ_LAYOUT_TOID(queue, struct entry); POBJ_LAYOUT_TOID(queue, struct queue); POBJ_LAYOUT_END(queue); struct entry { /* queue entry that contains arbitrary data */ size_t len; /* length of the data buffer */ char data[]; }; struct queue { /* array-based queue container */ size_t front; /* position of the first entry */ size_t back; /* position of the last entry */ size_t capacity; /* size of the entries array */ TOID(struct entry) entries[]; }; struct root { TOID(struct queue) queue; }; /* * queue_constructor -- constructor of the queue container */ static int queue_constructor(PMEMobjpool *pop, void *ptr, void *arg) { struct queue *q = ptr; size_t *capacity = arg; q->front = 0; q->back = 0; q->capacity = *capacity; /* atomic API requires that objects are persisted in the constructor */ pmemobj_persist(pop, q, sizeof(*q)); return 0; } /* * queue_new -- allocates a new queue container using the atomic API */ static int queue_new(PMEMobjpool *pop, TOID(struct queue) *q, size_t nentries) { return POBJ_ALLOC(pop, q, struct queue, sizeof(struct queue) + sizeof(TOID(struct entry)) * nentries, queue_constructor, &nentries); } /* * queue_nentries -- returns the number of entries */ static size_t queue_nentries(struct queue *queue) { return queue->back - queue->front; } /* * queue_enqueue -- allocates and inserts a new entry into the queue */ static int queue_enqueue(PMEMobjpool *pop, struct queue *queue, const char *data, size_t len) { if (queue->capacity - queue_nentries(queue) == 0) return -1; /* at capacity */ /* back is never decreased, need to calculate the real position */ size_t pos = queue->back % queue->capacity; int ret = 0; printf("inserting %zu: %s\n", pos, data); TX_BEGIN(pop) { /* let's first reserve the space at the end of the queue */ TX_ADD_DIRECT(&queue->back); queue->back += 1; /* now we can safely allocate and initialize the new entry */ TOID(struct entry) entry = TX_ALLOC(struct entry, sizeof(struct entry) + len); D_RW(entry)->len = len; memcpy(D_RW(entry)->data, data, len); /* and then snapshot the queue entry that we will modify */ TX_ADD_DIRECT(&queue->entries[pos]); queue->entries[pos] = entry; } TX_ONABORT { /* don't forget about error handling! ;) */ ret = -1; } TX_END return ret; } /* * queue_dequeue - removes and frees the first element from the queue */ static int queue_dequeue(PMEMobjpool *pop, struct queue *queue) { if (queue_nentries(queue) == 0) return -1; /* no entries to remove */ /* front is also never decreased */ size_t pos = queue->front % queue->capacity; int ret = 0; printf("removing %zu: %s\n", pos, D_RO(queue->entries[pos])->data); TX_BEGIN(pop) { /* move the queue forward */ TX_ADD_DIRECT(&queue->front); queue->front += 1; /* and since this entry is now unreachable, free it */ TX_FREE(queue->entries[pos]); /* notice that we do not change the PMEMoid itself */ } TX_ONABORT { ret = -1; } TX_END return ret; } /* * queue_show -- prints all queue entries */ static void queue_show(PMEMobjpool *pop, struct queue *queue) { size_t nentries = queue_nentries(queue); printf("Entries %zu/%zu\n", nentries, queue->capacity); for (size_t i = queue->front; i < queue->back; ++i) { size_t pos = i % queue->capacity; printf("%zu: %s\n", pos, D_RO(queue->entries[pos])->data); } } /* available queue operations */ enum queue_op { UNKNOWN_QUEUE_OP, QUEUE_NEW, QUEUE_ENQUEUE, QUEUE_DEQUEUE, QUEUE_SHOW, MAX_QUEUE_OP, }; /* queue operations strings */ static const char *ops_str[MAX_QUEUE_OP] = {"", "new", "enqueue", "dequeue", "show"}; /* * parse_queue_op -- parses the operation string and returns matching queue_op */ static enum queue_op queue_op_parse(const char *str) { for (int i = 0; i < MAX_QUEUE_OP; ++i) if (strcmp(str, ops_str[i]) == 0) return (enum queue_op)i; return UNKNOWN_QUEUE_OP; } /* * fail -- helper function to exit the application in the event of an error */ static void __attribute__((noreturn)) /* this function terminates */ fail(const char *msg) { fprintf(stderr, "%s\n", msg); exit(EXIT_FAILURE); } int main(int argc, char *argv[]) { enum queue_op op; if (argc < 3 || (op = queue_op_parse(argv[2])) == UNKNOWN_QUEUE_OP) fail("usage: file-name [new |show|enqueue |dequeue]"); PMEMobjpool *pop = pmemobj_open(argv[1], POBJ_LAYOUT_NAME(queue)); if (pop == NULL) fail("failed to open the pool"); TOID(struct root) root = POBJ_ROOT(pop, struct root); struct root *rootp = D_RW(root); size_t capacity; switch (op) { case QUEUE_NEW: if (argc != 4) fail("missing size of the queue"); char *end; errno = 0; capacity = strtoull(argv[3], &end, 0); if (errno == ERANGE || *end != '\0') fail("invalid size of the queue"); if (queue_new(pop, &rootp->queue, capacity) != 0) fail("failed to create a new queue"); break; case QUEUE_ENQUEUE: if (argc != 4) fail("missing new entry data"); if (D_RW(rootp->queue) == NULL) fail("queue must exist"); if (queue_enqueue(pop, D_RW(rootp->queue), argv[3], strlen(argv[3]) + 1) != 0) fail("failed to insert new entry"); break; case QUEUE_DEQUEUE: if (D_RW(rootp->queue) == NULL) fail("queue must exist"); if (queue_dequeue(pop, D_RW(rootp->queue)) != 0) fail("failed to remove entry"); break; case QUEUE_SHOW: if (D_RW(rootp->queue) == NULL) fail("queue must exist"); queue_show(pop, D_RW(rootp->queue)); break; default: assert(0); /* unreachable */ break; } pmemobj_close(pop); return 0; } pmdk-1.8/src/examples/libpmemobj/queue/README0000664000000000000000000000221513615011243017542 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemobj/queue/README. This directory contains an array-based implementation of a persistent queue. Usage: ./queue The file must exist and must be a libpmemobj pool with a layout name "queue". This file can be created using pmempool: $ pmempool create obj --layout queue The queue supports 4 operations: * new - create a new queue with the given capacity. * enqueue - inserts a new entry into the queue with the given data string * dequeue - removes the entry at the front of the queue * show - lists all entries in the queue For example, the following series of commands will create a new queue in the pool, and then will push and pop two entries. $ ./queue /mnt/pmem/queue.pool new 16 $ ./queue /mnt/pmem/queue.pool enqueue hello inserting 0: hello $ ./queue /mnt/pmem/queue.pool enqueue world inserting 0: world $ ./queue /mnt/pmem/queue.pool show Entries 2/16 0: hello 1: world $ ./queue /mnt/pmem/queue.pool dequeue removing 0: hello $ ./queue /mnt/pmem/queue.pool dequeue removing 1: world $ ./queue /mnt/pmem/queue.pool show Entries 0/16 pmdk-1.8/src/examples/libpmemobj/queue/.gitignore0000664000000000000000000000000613615011243020646 0ustar rootrootqueue pmdk-1.8/src/examples/libpmemobj/queue/Makefile0000664000000000000000000000325513615011243020327 0ustar rootroot# # Copyright 2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/queue/Makefile -- build the queue example # PROGS = queue LIBS = -lpmemobj include ../../Makefile.inc queue: queue.o pmdk-1.8/src/examples/libpmemobj/Makefile0000664000000000000000000000401213615011243017173 0ustar rootroot# # Copyright 2014-2018, Intel Corporation # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # * Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # * Redistributions in binary form must reproduce the above copyright # notice, this list of conditions and the following disclaimer in # the documentation and/or other materials provided with the # distribution. # # * Neither the name of the copyright holder nor the names of its # contributors may be used to endorse or promote products derived # from this software without specific prior written permission. # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT # OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # # # examples/libpmemobj/Makefile -- build the libpmemobj examples # PROGS = manpage btree pi lists setjmp DIRS = pminvaders pmemlog pmemblk string_store string_store_tx\ string_store_tx_type hashmap tree_map pmemobjfs map libart array\ linkedlist list_map slab_allocator queue LIBS = -lpmemobj -lpmem -pthread -lm -pthread include ../Makefile.inc map: hashmap tree_map pmemobjfs: map manpage: manpage.o btree: btree.o pi: pi.o lists: lists.o setjmp: CFLAGS += -O2 setjmp: setjmp.o pmdk-1.8/src/examples/libpmemobj/linkedlist/0000775000000000000000000000000013615011243017700 5ustar rootrootpmdk-1.8/src/examples/libpmemobj/linkedlist/fifo.vcxproj0000664000000000000000000000564213615011243022247 0ustar rootroot Debug x64 Release x64 {D3A99F36-4B72-4766-ABCD-CCEDC26DD139} pmemobj 10.0.16299.0 {1baa1617-93ae-4196-8a1a-bd492fb18aef} {9e9e3d25-2139-4a5d-9200-18148ddead45} Application true v140 Application false v140 true ..\..\..\LongPath.manifest CompileAsCpp libpmem.lib;libpmemobj.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies) pmdk-1.8/src/examples/libpmemobj/linkedlist/README0000664000000000000000000000100613615011243020555 0ustar rootrootPersistent Memory Development Kit This is examples/libpmemobj/linkedlist/README. This directory contains an implementation of singly linked list and tail queue in pmem_list.h. fifo.c contains an usage example of persistent tail queue. Syntax of example usage is given below. fifo